[Bug middle-end/78174] out of bounds array subscript in rtl.h NOTE_DATA macro

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78174

--- Comment #2 from Martin Sebor <msebor at gcc dot gnu.org> ---
The problem can be reduced to the following test case that triggers the warning
even with unpatched GCC (and prior releases).

$ cat b.c && g++ -O2 -S -Wall b.c
struct A { int i, j; };
struct B { int i0, j0, i1, j1, i2, j2, i3, j3, i4, j4; };

struct C {
  union {
    struct A a[1];
    struct B b;
  } u;
};

struct D: C { };

void f (struct D *d)
{
  struct A *p = &d->u.a[3];
  __builtin___memset_chk (p, 0, sizeof *p, __builtin_object_size (p, 1));
}
b.c: In function ‘void f(D*)’:
b.c:17:73: warning: call to void* __builtin___memset_chk(void*, int, long
unsigned int, long unsigned int) will always overflow destination buffer
   __builtin___memset_chk (p, 0, sizeof *p, __builtin_object_size (p, 1));
                                                                         ^
Such code is undefined under the C++ (and C) standard rules.  (Curiously, GCC
doesn't issue the warning for an equivalent function that takes struct C* as an
argument.)

The same warning is also emitted when the memset call in emit-rtl.c is replaced
with a __builtin___memset_chk call corresponding to the one above.  The patch
that triggers the warning simply enables the same compile-time checking for
memset as is done by __builtin___memset_chk with type-1 object size checking.

I see that Glibc uses type-0 checking for memset even with _FORTIFY_SOURCE=2,
so the patch could either be changed to do the same or enhanced to let users
specify the __builtin_object_size checking type, and perhaps even a different
type for raw memory functions like memset and for string functions, to make it
possible to fine-tune the checking to the state of the code base and avoid
issuing warnings for this kind of code in legacy software.  I think those would
be enhancements worth considering, but I also think that GCC itself should try
to avoid relying on undefined behavior.

GCC provides zero-length arrays as a substitute for flexible array members.  In
this case, the array has one (not zero) element.  It would be nice if changing
it to a zero-element array would let GCC treat it as a flexible array member on
steroids so that users would have to opt in to relying on an extension to avoid
the warning here, but that's not the case.  I realize that GCC sometimes treats
even one element arrays (and even bigger if they're last) as zero-length even
though that's not documented anywhere (AFAIK), but it would be nice to get away
from that.