Bug 40735 - memory hog compiling big functions with -fPIE
memory hog compiling big functions with -fPIE
Status: NEW
Product: gcc
Classification: Unclassified
Component: target
4.4.1
: P2 normal
: ---
Assigned To: Not yet assigned to anyone
: memory-hog
Depends on:
Blocks: 47344
  Show dependency treegraph
 
Reported: 2009-07-13 19:37 UTC by Matthias Klose
Modified: 2012-08-28 11:29 UTC (History)
4 users (show)

See Also:
Host:
Target: arm-linux-gnueabi
Build:
Known to work: 4.2.4
Known to fail: 4.3.3, 4.4.0, 4.6.2, 4.7.0
Last reconfirmed: 2009-07-15 07:34:32


Attachments
preprocessed source (79.23 KB, application/x-lzma)
2009-07-13 19:37 UTC, Matthias Klose
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Klose 2009-07-13 19:37:16 UTC
[forwarded from https://launchpad.net/bugs/398403]

Building the attached file with gcc from the 4.4 branch with -g -fstack-protector -fPIE -Os, the build fails (killed by oom), last info seen is memory usage of about 1500mb. Building without -fPIE memory usage is limited around 700MB.

Same behaviour for 4.3. With gcc-4.2 peak memory usage is with/without -fPIE below 400MB. Not checked on trunk, currently fails to build.
Comment 1 Matthias Klose 2009-07-13 19:37:36 UTC
Created attachment 18188 [details]
preprocessed source
Comment 2 Richard Biener 2009-07-14 09:20:58 UTC
I cannot compile the attached testcase.

gcc-4.2 -S -o /dev/null -g -fstack-protector -fPIE -Os test_node.i -std=c99
In file included from ../nih/test_alloc.h:32,
                 from ../nih/test.h:36,
                 from tests/test_node.c:23:
../nih/list.h:110: warning: declaration does not declare anything
In file included from tests/test_node.c:40:
./parse.h:71: warning: declaration does not declare anything
tests/test_node.c: In function ‘test_start_tag’:
tests/test_node.c:211: error: ‘ParseStack’ has no member named ‘node’
tests/test_node.c:251: error: ‘ParseStack’ has no member named ‘node’
tests/test_node.c:297: error: ‘ParseStack’ has no member named ‘data’
tests/test_node.c:297: error: ‘ParseStack’ has no member named ‘data’
tests/test_node.c:365: error: ‘ParseStack’ has no member named ‘node’
tests/test_node.c:422: error: ‘ParseStack’ has no member named ‘data’
tests/test_node.c:422: error: ‘ParseStack’ has no member named ‘data’
tests/test_node.c: In function ‘test_object_functions’:
tests/test_node.c:1256: error: ‘NihListEntry’ has no member named ‘str’
tests/test_node.c:1256: error: ‘NihListEntry’ has no member named ‘str’
...
Comment 3 Matthias Klose 2009-07-14 10:31:19 UTC
my bad, should be -std=gnu99 instead: gcc-4.2 -S -o /dev/null -g -fstack-protector -fPIE -Os test_node.i -std=gnu99
Comment 4 Richard Biener 2009-07-14 11:31:00 UTC
This is likely caused by the DF merge.  There are numerous bugs about this
already and nothing really can be done here.

Btw, my numbers are

rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.4 -S -o /dev/null -g -fstack-protector -fPIE -Os test_node.i -std=gnu99
total: 744108 kB
rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.3 -S -o /dev/null -g -fstack-protector -fPIE -Os test_node.i -std=gnu99
total: 719836 kB
rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.2 -S -o /dev/null -g -fstack-protector -fPIE -Os test_node.i -std=gnu99
total: 459757 kB

Thus not 1.5GB but 750MB vs 450MB.

rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.4 -S -o /dev/null -fPIE -Os test_node.i -std=gnu99
total: 743380 kB
rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.4 -S -o /dev/null -Os test_node.i -std=gnu99
total: 630756 kB

the -fPIE effect itself is even less recognizable.
Comment 5 Mikael Pettersson 2009-07-14 23:07:07 UTC
Seems heavily target-dependent. With 4.3.4 I see peak usage of 640M for i686, just under 1.2G for powerpc, and 1.6G for arm.
Comment 6 Andrew Pinski 2009-07-14 23:10:36 UTC
Hmm, maybe scheduling is causing it. Does -fno-schedule-insns -fno-schedule-insns2 cause the memory usage to go down?
Comment 7 Mikael Pettersson 2009-07-14 23:33:27 UTC
(In reply to comment #6)
> Hmm, maybe scheduling is causing it. Does -fno-schedule-insns
> -fno-schedule-insns2 cause the memory usage to go down?

Nope, memory usage patterns stayed the same.


Comment 8 Steven Bosscher 2009-07-15 07:34:32 UTC
Does seem to be a real issue, somewhere.
When trunk builds again, can you please give it a try too?
Comment 9 Richard Biener 2009-07-15 09:44:41 UTC
DF time on this testcase is already big, and the testcase has lots of function
calls which would explain the difference between targets (DF needs to track all
call-used/clobbered regs).
Comment 10 Mikael Pettersson 2009-07-16 18:33:18 UTC
More memory usage numbers on this test case:

With 4.4.1-RC-20090715: i686 peaks at 616M, powerpc at 799M, and arm at 1211M.

With 4.5.0-20090709: i686 peaks at 530M, powerpc at 707M, and arm at 933M.
Comment 11 Richard Biener 2009-08-04 12:30:16 UTC
GCC 4.3.4 is being released, adjusting target milestone.
Comment 12 Richard Biener 2010-05-22 18:13:40 UTC
GCC 4.3.5 is being released, adjusting target milestone.
Comment 13 Richard Biener 2011-06-27 12:14:51 UTC
4.3 branch is being closed, moving to 4.4.7 target.
Comment 14 Richard Biener 2012-01-16 14:16:09 UTC
(In reply to comment #4)
> This is likely caused by the DF merge.  There are numerous bugs about this
> already and nothing really can be done here.
> 
> Btw, my numbers are
> 
> rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.4 -S -o /dev/null -g
> -fstack-protector -fPIE -Os test_node.i -std=gnu99
> total: 744108 kB
> rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.3 -S -o /dev/null -g
> -fstack-protector -fPIE -Os test_node.i -std=gnu99
> total: 719836 kB
> rguenther@murzim:/tmp> ~/bin/maxmem2.sh gcc-4.2 -S -o /dev/null -g
> -fstack-protector -fPIE -Os test_node.i -std=gnu99
> total: 459757 kB

> ~/bin/maxmem2.sh gcc-4.6 -S -o /dev/null t.i -v -std=gnu99 -Os -fPIE -fstack-protector -g
total: 772239 kB
> ~/bin/maxmem2.sh gcc-4.7 -S -o /dev/null t.i -v -std=gnu99 -Os -fPIE -fstack-protector -g --param ggc-min-expand=100 --param ggc-min-heapsize=131072
total: 794883 kB

we're continuing to increase, thus re-confirmed.
Comment 15 Richard Biener 2012-01-16 14:29:20 UTC
Blocks the memory-hog meta-bug which is marked as regression.
Comment 16 xiaoyuanbo 2012-02-22 12:52:59 UTC
obtain head
Comment 17 Steven Bosscher 2012-08-28 08:43:58 UTC
(In reply to comment #4)
> ~/bin/maxmem2.sh gcc-4.4 -S -o /dev/null -g
> -fstack-protector -fPIE -Os test_node.i -std=gnu99
> total: 744108 kB

richi, can you share this maxmem2 script?
Comment 18 Alexander Monakov 2012-08-28 08:48:28 UTC
(In reply to comment #17)
> 
> richi, can you share this maxmem2 script?

It's available on the wiki: http://gcc.gnu.org/wiki/PerformanceTesting
Comment 19 Steven Bosscher 2012-08-28 09:34:10 UTC
With r190665, compile time is mostly spent in:
 loop doloop             : 130.15 ( 8%) usr
 variable tracking       : 990.72 (60%) usr
 var-tracking dataflow   :  76.34 ( 5%) usr

So this isn't a DF issue in GCC 4.8 at least, but yet another var-tracking issue (but probably not related to bug 53958, because var-tracking dataflow time is almost reasonable here).

I'll have a look at the doloop issue, but the var-tracking issue is for Alex.
Comment 20 Steven Bosscher 2012-08-28 10:49:19 UTC
With release checking, the loop-doloop time is almost zero (so something is slow in doloop checking). The var-tracking slowness does not disappear, and dominates the compile time even more strongly:

 variable tracking       : 530.57 (84%) usr
Comment 21 Steven Bosscher 2012-08-28 11:29:43 UTC
(In reply to comment #14)
> > ~/bin/maxmem2.sh gcc-4.7 -S -o /dev/null t.i -v -std=gnu99 -Os -fPIE -fstack-protector -g --param ggc-min-expand=100 --param ggc-min-heapsize=131072
> total: 794883 kB
> 
> we're continuing to increase, thus re-confirmed.

At r190665, with the same -min-expand=100 --param ggc-min-heapsize=131072 :

tests/test_node.c:6115:1: note: variable tracking size limit exceeded with -fvar-tracking-assignments, retrying without
$ total: 1443401 kB