[Bug regression/102513] New: False positive -Wstringop-overflow= or -Warray-bounds warning with recursive function

[andreas.rheinhardt at outlook dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102513

            Bug ID: 102513
           Summary: False positive -Wstringop-overflow= or -Warray-bounds
                    warning with recursive function
           Product: gcc
           Version: 10.0
            Status: UNCONFIRMED
          Keywords: diagnostic
          Severity: normal
          Priority: P3
         Component: regression
          Assignee: unassigned at gcc dot gnu.org
          Reporter: andreas.rheinhardt at outlook dot com
  Target Milestone: ---

Created attachment 51514
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51514&action=edit
A file that produces false warnings when compiled with -O3.

extern int block2[7][256];

static int encode_block(int block2[7][256], unsigned level)
{
    int best_score = 0;

    for (unsigned x = 0; x < level; x++) {
        int v = block2[1][x];
        block2[level][x] = 0;
        best_score      += v * v;
    }

    if (level > 0 && best_score > 64) {
        int score = 0;

        score += encode_block(block2, level - 1);
        score += encode_block(block2, level - 1);

        if (score < best_score) {
            best_score = score;
        }
    }

    return best_score;
}

int foo(void)
{
    return encode_block(block2, 5);
}

As is easily seen, level in the above code is always in the range 0..5 and
therefore all accesses are safe. Yet when one compiles the above with -O3 (-O2
is quiet), one receives several warnings:

../gcc_bug.c: In function 'encode_block.constprop':
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: iteration 256 invokes undefined behavior
[-Waggressive-loop-optimizations]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:7:5: note: within this loop
    7 |     for (unsigned x = 0; x < level; x++) {
      |     ^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869176 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869172 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869168 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869168 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
In function 'encode_block',
    inlined from 'encode_block.constprop' at ../gcc_bug.c:17:18:
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869172 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869168 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~
../gcc_bug.c:9:26: warning: '__builtin_memset' writing 17179869168 bytes into a
region of size 0 overflows the destination [-Wstringop-overflow=]
    9 |         block2[level][x] = 0;
      |         ~~~~~~~~~~~~~~~~~^~~

If one adds -Wall, one gets -Warray-bounds warnings instead of
-Wstringop-overflow= warnings.
This happens in latest git master; I bisected it to
9b14fc3326e087975653b1af8ac54114041cde51. None of the other open
-Warray-bounds/-Wstringop-overflow= warnings have been traced back to this
commit, so I believe this bug report not to be a duplicate of one of the other
open bugs.

PS: The above is based upon
https://github.com/FFmpeg/FFmpeg/blob/56e9e0273a79fb4ff4276503e69981f8e383f22b/libavcodec/svq1enc.c#L91
This code is covered by our regression tests and none of them fail, so GCC does
not seem to miscompile it (despite the warnings indicating GCC's internal state
to be garbage).
(Somehow the warning is not triggered by
9b14fc3326e087975653b1af8ac54114041cde51 for this file: The warnings only
appeared with GCC 11, GCC 10 is fine. I did not bisect this further. For
another one of our files
(https://github.com/FFmpeg/FFmpeg/blob/8be701d9f7f77ff2282cc7fe6e0791ca5419de70/libavfilter/vf_find_rect.c#L157)
the warnings appeared with 9b14fc3326e087975653b1af8ac54114041cde51 and are
also in GCC 10, yet somehow not in GCC 11 and also not in GCC master. I also
did not bisect this further, as the underlying bug is probably the same for
all, so fixing the testcase should fix all of it.)