[Bug target/61656] New: Undefined behavior in classify_argument

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61656

            Bug ID: 61656
           Summary: Undefined behavior in classify_argument
           Product: gcc
           Version: 4.10.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jakub at gcc dot gnu.org
                CC: hjl.tools at gmail dot com, hubicka at gcc dot gnu.org,
                    uros at gcc dot gnu.org
            Target: x86_64-linux

/usr/src/gcc/obj052/gcc/xgcc -B/usr/src/gcc/obj052/gcc/
-fno-diagnostics-show-caret -fdiagnostics-color=never -O0 -w -c -o pr4
2025-2.o /usr/src/gcc/gcc/testsuite/gcc.c-torture/compile/pr42025-2.c
../../gcc/config/i386/i386.c:6583:60: runtime error: mload of value 32669,
which is not a valid value for type 'x86_64_reg_class'

This is on passing
typedef struct { void *p; } Ptr;
struct A { int i; union { Ptr p; char *q; } u; };
by value and the problem is that when processing the union with bit_offset 64,
words is 1 (u is DImode 64-bit field), but when we recurse, we are called with
64-bit scalar DImode q and bit_offset 64, that is size (128-1)&0x7f and so it
is the size < 64+64 case where we return 2 and { X86_64_INTEGER_CLASS,
X86_64_INTEGER_CLASS }; in subclauses.  But words is 1 and we merge classes up
to num (2).  A simple fix could be:
   if (!num)
     return 0;
-  for (i = 0; i < num; i++)
+  for (i = 0; i < num && i < words; i++)
     classes[i] = merge_classes (subclasses[i], classes[i]);
in the UNION_TYPE case, as it seems the caller will not care about classes
above returned number (words).  I'd hope such a patch should not change the ABI
even.
I don't know whether there isn't an ABI problem related to this though, say
if at bit_offset 64 we have just SImode field in the union rather than DImode,
then I'd guess the recursive call would give us { X86_64_INTEGER_CLASS,
X86_64_INTEGERSI_CLASS }; but we'd use X86_64_INTEGER_CLASS anyway, as we are
looking at position 0, not 1.