[Bug c/56344] New: ICE for program with larger automatic structs

["georggcc at googlemail dot com" <gcc-bugzilla at gcc dot gnu dot org>]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56344

             Bug #: 56344
           Summary: ICE for program with larger automatic structs
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: georggcc@googlemail.com


Created attachment 29467
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29467
artificial test program, COLS set suitably

The program attached makes the compiler fail with ICE or otherwise misbehave in
a few ways. The behavior depends at least on the value of COLS and optimization
being on (-O2) or off. The program's arrays would be huge, but the compiler's
behavior does not vary "monotonically" with the value defined for COLS.

At 24000L, I get, with or without -O2,

Compilation started at Fri Feb 15 14:45:59

gcc -std=c99 -pedantic  -W -c big.c
big.c: In function 'main':
big.c:36:10: internal compiler error: Segmentation fault: 11
   result = mulm(m1, m2);
          ^

big.c:36:10: internal compiler error: Abort trap: 6
gcc: internal compiler error: Abort trap: 6 (program cc1)

Compilation abort trap: 6 at Fri Feb 15 14:46:02

At 22000L (i.e., smaller), or 100000L (i.e., larger), the
compiler's memory consumption stays at ~14MB and ~80MB of
physical and virtual memory, resp; kernel_task and cc1
seem busy, but nothing happens for several minutes.

At 20000L, and with -O2, I get

Compilation started at Fri Feb 15 15:37:28

gcc -pedantic  -W -c -O2 big.c
In file included from big.c:1:0:

gcc: internal compiler error: Abort trap: 6 (program cc1)

Compilation abort trap: 6 at Fri Feb 15 15:37:28

Dropping -O2 results in the same apparent behavior as
for 22000L and 100000L.

With some other sizes, like 15000L, cc1 uses around
1.6 GB and 3.0 GB of physical and virtual memory,
respectively, and occasionally some CPU; after 5 min(!),
and not too much VM swapping,

Compilation started at Fri Feb 15 15:41:03

gcc -pedantic  -W -c big.c

Compilation finished at Fri Feb 15 15:46:11

(What is the compiler doing with all this memory?)

At 150000L, the compiler finishes in no time and everything
is fine, with or without -O2. (Of course, the resulting program
will not run without segfault.) I noticed that in these large
value of COLS, manipulation of %rsp changes, from

  subq    $0x35a4e928,%rsp    ; COLS 15000L

in _mulm, to

  movq    $0xffffffeb0b94fbe0,%r11  ; COLS 150000L
  addq    %r11,%rsp

(As an aside, a gcc-avr in a different environment says this:
error: size of array 'data' is too large.)

(I notices #46119 but don't know whether it is related.)

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/Users/bauhaus/mine/libexec/gcc/x86_64-apple-darwin11.4.2/4.8.0/lto-wrapper
Target: x86_64-apple-darwin11.4.2
Configured with: /Users/bauhaus/src/gcc/configure --prefix=/Users/bauhaus/mine
--disable-nls --disable-multilib --disable-libstdcxx-pch
--enable-languages=c,ada,c++ CC=gcc
Thread model: posix
gcc version 4.8.0 20130212 (experimental) [trunk revision 195968] (GCC) 

4 GiB of physical RAM.