When compiling the attached module with -O3, gcc 3.4 uses an excessive amount of memory as shown in the table below. This problem is possibly related to bug report 9805 (although that one uses g++). The table below shows the maximum size of cc1 during compilation (as reported by top): options | gcc -c -O2 | gcc -c -O3 ------------------------------------------ gcc 3.2.2 | 19 Mb | 16 Mb gcc 3.3 | 113 Mb | 120 Mb gcc 3.4 | 120 Mb | 1772 Mb Needless to say that compilation with gcc 3.4 takes a very long time as well... Release: 3.4 20030315 (experimental) Environment: System: SunOS scooby 5.9 Generic_112233-04 sun4u sparc SUNW,Sun-Blade-100 Architecture: sun4 host: sparc-sun-solaris2.9 build: sparc-sun-solaris2.9 target: sparc-sun-solaris2.9 Configured with: /scratch/temp/cvs/gcc/configure --prefix=/opt/local/gcc340 --enable-languages=c,c++,f77 How-To-Repeat: scooby> gcc -v -c -O3 phfit.c Reading specs from /opt/local/gcc340/lib/gcc-lib/sparc-sun-solaris2.9/3.4/specs Configured with: /scratch/temp/cvs/gcc/configure --prefix=/opt/local/gcc340 --enable-languages=c,c++,f77 Thread model: posix gcc version 3.4 20030315 (experimental) /opt/local/gcc340/lib/gcc-lib/sparc-sun-solaris2.9/3.4/cc1 -quiet -v -Dsparc -D__sparc__ -D__sparc -D__GCC_NEW_VARARGS__ -Acpu=sparc -Amachine=sparc phfit.c -quiet -dumpbase phfit.c -auxbase phfit -O3 -version -o /var/tmp//ccvvR7Vi.s ignoring nonexistent directory "/opt/local/gcc340/sparc-sun-solaris2.9/include" #include "..." search starts here: #include <...> search starts here: /usr/local/include /opt/local/gcc340/include /opt/local/gcc340/lib/gcc-lib/sparc-sun-solaris2.9/3.4/include /usr/include End of search list. GNU C version 3.4 20030315 (experimental) (sparc-sun-solaris2.9) compiled by GNU C version 3.4 20030315 (experimental). GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 /usr/ccs/bin/as -V -Qy -s -o phfit.o /var/tmp//ccvvR7Vi.s /usr/ccs/bin/as: Sun WorkShop 6 update 2 Compiler Common 6.2 Solaris_9_CBE 2001/04/02
Fix: Workaround: use lower level of optimization.
State-Changed-From-To: open->analyzed State-Changed-Why: This is not specific to Sparc. Here are my results on x86: options | gcc -S -O2 | gcc -S -O3 ------------------------------------------ gcc 3.2.3 | 13 MB | 14 MB gcc 3.3 | 44 MB | 48 MB gcc 3.4 | 52 MB | > 400 MB
From: Steven Bosscher <s.bosscher@student.tudelft.nl> To: p.van-hoof@qub.ac.uk, gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org, jh@suse.de Cc: Subject: Re: optimization/10155: [3.3/3.4 regression] gcc -O2/-O3 uses excessive amount of memory Date: Fri, 02 May 2003 12:58:47 +0200 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10155 How much of this can be explained with Kaveh's physmem patch? IIRC that patch is not in 3.2, and the increase in memory consumption at -O2 may be a result of that patch. It looks like it's related to the amount of RAM in your box. For me, GCC -O2 never consumes more than 40MB, and my machine has 192MB. If your system pulls it off to keep cc1 alive up to 1772MB, then I suppose you have at least four times as much, so your GC param settings will be much larger than mine --> GCC eats more memory. That _should_ give you a nice speedup over older GCC 3.x... So perhaps we can change the synopsis to reflect that his really only is a bug with -O3. Paul, if you compile with "gcc -v", you can see your garbage collector parameter settings. For me they are: GGC heuristics: --param ggc-min-expand=42 --param ggc-min-heapsize=23891 Can you show us your settings? The increase in memory at -O3 is a result of unit at a time compilation (which is why I CC you, Honza). You can check that by compiling with -O2 + all flags enabled at -O3 except -funit-at-a-time: ./cc1 10155.c -quiet -ftime-report -O2 TOTAL : 24.74 0.74 26.24 ./cc1 10155.c -quiet -ftime-report -O2 -funswitch-loops -frename-registers -finline-functions TOTAL : 31.49 0.59 33.87 Loop unswitching is responsible for most of the compile time increase. Now add -funit-at-a-time, and kabooooom! you lose. Apparently unit-at-a-time should still honor some size constraints, and it does not in its current form. Greetz Steven
From: Jan Hubicka <jh@suse.cz> To: Steven Bosscher <s.bosscher@student.tudelft.nl> Cc: p.van-hoof@qub.ac.uk, gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org, jh@suse.de, rakdver@atrey.karlin.mff.cuni.cz Subject: Re: optimization/10155: [3.3/3.4 regression] gcc -O2/-O3 uses excessive amount of memory Date: Fri, 2 May 2003 15:25:16 +0200 > http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10155 > > How much of this can be explained with Kaveh's physmem > patch? IIRC that patch is not in 3.2, and the increase > in memory consumption at -O2 may be a result of that > patch. > > The increase in memory at -O3 is a result of unit at a > time compilation (which is why I CC you, Honza). You > can check that by compiling with -O2 + all flags enabled > at -O3 except -funit-at-a-time: > > ./cc1 10155.c -quiet -ftime-report -O2 > TOTAL : 24.74 0.74 26.24 > > ./cc1 10155.c -quiet -ftime-report -O2 -funswitch-loops > -frename-registers -finline-functions > TOTAL : 31.49 0.59 33.87 > > Loop unswitching is responsible for most of the compile Zdenek, this really ought not to happen, what is going on? > time increase. > Now add -funit-at-a-time, and kabooooom! you lose. > > Apparently unit-at-a-time should still honor some size > constraints, and it does not in its current form. It should be more problem of inlining heuristics, than unit-at-a-time (ie unit-at-a-time enables more inlining oppurtunities but it is inlining heuristic mistake to take so many of them). Or perhaps we manage to flatten functions called once into one extraordinarily large function body and give up on it. I will try to investigate it, but my current priority is to get unit-at-a-time working on C++. Fixing this testcase should be easy then :) Honza > > Greetz > Steven > >
From: Zdenek Dvorak <rakdver@atrey.karlin.mff.cuni.cz> To: Jan Hubicka <jh@suse.cz> Cc: Steven Bosscher <s.bosscher@student.tudelft.nl>, p.van-hoof@qub.ac.uk, gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org, jh@suse.de Subject: Re: optimization/10155: [3.3/3.4 regression] gcc -O2/-O3 uses excessive amount of memory Date: Fri, 2 May 2003 19:38:39 +0200 Hello, > > http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10155 > > > > How much of this can be explained with Kaveh's physmem > > patch? IIRC that patch is not in 3.2, and the increase > > in memory consumption at -O2 may be a result of that > > patch. > > > > The increase in memory at -O3 is a result of unit at a > > time compilation (which is why I CC you, Honza). You > > can check that by compiling with -O2 + all flags enabled > > at -O3 except -funit-at-a-time: > > > > ./cc1 10155.c -quiet -ftime-report -O2 > > TOTAL : 24.74 0.74 26.24 > > > > ./cc1 10155.c -quiet -ftime-report -O2 -funswitch-loops > > -frename-registers -finline-functions > > TOTAL : 31.49 0.59 33.87 > > > > Loop unswitching is responsible for most of the compile > Zdenek, this really ought not to happen, what is going on? I haven't tested the loop optimizer against a program consisting from several thousand 3-line loops, so I am not that much surprised that something went wrong. I will check where is the problem. Zdenek > > time increase. > > Now add -funit-at-a-time, and kabooooom! you lose. > > > > Apparently unit-at-a-time should still honor some size > > constraints, and it does not in its current form. > > It should be more problem of inlining heuristics, than unit-at-a-time > (ie unit-at-a-time enables more inlining oppurtunities but it is > inlining heuristic mistake to take so many of them). > Or perhaps we manage to flatten functions called once into one > extraordinarily large function body and give up on it. I will try to > investigate it, but my current priority is to get unit-at-a-time working > on C++. Fixing this testcase should be easy then :) > > Honza > > > > Greetz > > Steven > > > >
From: Zdenek Dvorak <rakdver@atrey.karlin.mff.cuni.cz> To: Jan Hubicka <jh@suse.cz> Cc: Steven Bosscher <s.bosscher@student.tudelft.nl>, p.van-hoof@qub.ac.uk, gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org, jh@suse.de Subject: Re: optimization/10155: [3.3/3.4 regression] gcc -O2/-O3 uses excessive amount of memory Date: Fri, 2 May 2003 21:08:49 +0200 Hello, > > http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10155 > > > > How much of this can be explained with Kaveh's physmem > > patch? IIRC that patch is not in 3.2, and the increase > > in memory consumption at -O2 may be a result of that > > patch. > > > > The increase in memory at -O3 is a result of unit at a > > time compilation (which is why I CC you, Honza). You > > can check that by compiling with -O2 + all flags enabled > > at -O3 except -funit-at-a-time: > > > > ./cc1 10155.c -quiet -ftime-report -O2 > > TOTAL : 24.74 0.74 26.24 > > > > ./cc1 10155.c -quiet -ftime-report -O2 -funswitch-loops > > -frename-registers -finline-functions > > TOTAL : 31.49 0.59 33.87 > > > > Loop unswitching is responsible for most of the compile > Zdenek, this really ought not to happen, what is going on? it is just too many loops; the unswitching takes in fact almost no time (there is nothing to unswitch at all) -- all the time increase you see is in verify_loop_structure (it exhibits quadratic behavior on this kind of tests) and verify_dominators, so I guess there is not much to worry about. Zdenek
From: Peter van Hoof <p.van-hoof@qub.ac.uk> To: Steven Bosscher <s.bosscher@student.tudelft.nl> Cc: gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, jh@suse.de, rakdver@atrey.karlin.mff.cuni.cz Subject: Re: optimization/10155: [3.3/3.4 regression] gcc -O2/-O3 uses excessive amount of memory Date: Tue, 06 May 2003 21:41:00 +0100 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10155 > It looks like it's related to the amount of RAM in your > box. For me, GCC -O2 never consumes more than 40MB, and > my machine has 192MB. If your system pulls it off to > keep cc1 alive up to 1772MB, then I suppose you have at > least four times as much, so your GC param settings will > be much larger than mine --> GCC eats more memory. That > _should_ give you a nice speedup over older GCC 3.x... > So perhaps we can change the synopsis to reflect that his > really only is a bug with -O3. > > Peter, if you compile with "gcc -v", you can see your > garbage collector parameter settings. For me they are: > GGC heuristics: --param ggc-min-expand=42 --param ggc-min-heapsize=23891 > > Can you show us your settings? Now that my computer is back in the land of the living, I can finally answer this (in case it is still relevant)... I ran the initial test case on a Sun Blade 100 with 2GB of RAM, these are the settings with this machine: GNU C version 3.4 20030504 (experimental) (sparc-sun-solaris2.9) compiled by GNU C version 3.4 20030504 (experimental). GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 The compiler reaches a maximum size of 1894MB, which means that the machine has to swap out some other processes. I also have an Ultra 5 with 640MB of RAM, these are the settings on that machine GGC heuristics: --param ggc-min-expand=73 --param ggc-min-heapsize=81920 I killed the compilation when the compiler grew to >1700MB because swap space was running dangerously low, i.e. on this machine the compiler would use up all the swap space it could get its hands on. Hope this helps... Peter.
Responsible-Changed-From-To: unassigned->hubicka Responsible-Changed-Why: Unit-at-a-time related, Honza's work. Just as a reminder and to get rid of at least one of those way to many "unassigned" PRs.
Jan, is this one fixed with your new unit-at-a-time inline heuristics? It looks like another good candidate to validate your approach.
Subject: Re: [3.4 regression] gcc uses excessive amount of memory with -funit-at-a-time > PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org. > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=10155 > > > steven at gcc dot gnu dot org changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC|s dot bosscher at student | > |dot tudelft dot nl | > > > ------- Additional Comments From steven at gcc dot gnu dot org 2003-07-04 10:28 ------- > Jan, is this one fixed with your new unit-at-a-time inline heuristics? It looks > like another good candidate to validate your approach. Yes, this is why I added the limit for maximal function body growth. Honza > > > > ------- You are receiving this mail because: ------- > You are on the CC list for the bug, or are watching someone who is.
So Jan, is this really, really fixed? If so, can you close the bug?
Subject: Re: [3.4 regression] gcc uses excessive amount of memory with -funit-at-a-time > PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org. > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=10155 > > > > ------- Additional Comments From steven at gcc dot gnu dot org 2003-07-29 08:41 ------- > So Jan, is this really, really fixed? If so, can you close the bug? Hmm, kind of. The memory consumption goes down to 300MB and you have knob (--param large-function-growth) to control how much of bloat you want. Now we really should to figure out why RTL part of optimizer is taking that much memory and time and avoid it/make it to give up. So I think it should stay open until then. I will try to do some testing in foreseable future, but the problem of unlimited function growth due to inlining of functions called once is in general solved. Honza > > > > ------- You are receiving this mail because: ------- > You are on the CC list for the bug, or are watching someone who is.
The scheduler uses large amounts of memory for some reason so really it is not Jan's problem at all as just more functions are being inlined. It still happnes on the mainline: on powerpc-apple-darwin7.0.0: -O0 -O2 -O3 40 44 >500
I know there are many regressions for memory problems which are caused by the scheduler. Who owns the scheduler?
Now the scheduler does not take that much memory but the loop optimzer phase does so assigning it to Zdenek.
Fixed by the patch by Zdenek for the loop, note this test only now uses about 43M at -O3.