Bug 54056 - Fast run out of memory compiling template specialization
Summary: Fast run out of memory compiling template specialization
Status: UNCONFIRMED
Alias: None
Product: gcc
Classification: Unclassified
Component: c++ (show other bugs)
Version: 4.7.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-07-20 20:25 UTC by Sergey Prokhorenko
Modified: 2012-11-23 14:11 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments
Test case to reproduce. (10.35 KB, text/x-c++src)
2012-07-20 20:25 UTC, Sergey Prokhorenko
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sergey Prokhorenko 2012-07-20 20:25:58 UTC
Created attachment 27848 [details]
Test case to reproduce.

The attached file, compiled with both g++ 4.6.3 and g++ 4.7.0, quickly eats up all memory and swap space.

The similar templates source in a real project quickly (in 10-20 seconds) eat 16GB of memory, swaps out and die. But when compiled with g++ 4.5.2 it does not run out of memory. We found this trying to move to 4.6 or 4.7.

Also, the bug does not happen if the attached test case is compiled with -DNOSWAPBUG. In this case a template without extra parameter is used. This was found in the real project, where I compiled 30MB preprocessed file and it did not eat memory if the "buggy" templates form (with 2 parameters) was replaced with the NOSWAPBUG one.

I used a generator to produce test case. On my simple machine that I use now to test with 1GB of memory, max number of templates it can handle is 200. With 300 it quickly takes up all memory and swap space.

Here's -ftime-report for 200 structs:

Execution times (seconds)
 phase setup             :   0.01 ( 1%) usr   0.00 ( 0%) sys   0.02 ( 1%) wall    1438 kB ( 1%) ggc
 phase parsing           :   1.36 (98%) usr   0.16 (94%) sys   1.52 (81%) wall  234458 kB (99%) ggc
 phase lang. deferred    :   0.02 ( 1%) usr   0.00 ( 0%) sys   0.17 ( 9%) wall       0 kB ( 0%) ggc
 phase cgraph            :   0.00 ( 0%) usr   0.01 ( 6%) sys   0.13 ( 7%) wall     131 kB ( 0%) ggc
 phase generate          :   0.02 ( 1%) usr   0.01 ( 6%) sys   0.33 (18%) wall     132 kB ( 0%) ggc
 |name lookup            :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 1%) wall     307 kB ( 0%) ggc
 garbage collection      :   0.02 ( 1%) usr   0.00 ( 0%) sys   0.15 ( 8%) wall       0 kB ( 0%) ggc
 callgraph construction  :   0.00 ( 0%) usr   0.01 ( 6%) sys   0.04 ( 2%) wall     131 kB ( 0%) ggc
 callgraph optimization  :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.06 ( 3%) wall       0 kB ( 0%) ggc
 varpool construction    :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.03 ( 2%) wall       0 kB ( 0%) ggc
 preprocessing           :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 1%) wall      29 kB ( 0%) ggc
 parser (global)         :   0.00 ( 0%) usr   0.02 (12%) sys   0.02 ( 1%) wall     767 kB ( 0%) ggc
 parser struct body      :   1.31 (94%) usr   0.14 (82%) sys   1.42 (76%) wall  231197 kB (98%) ggc
 template instantiation  :   0.05 ( 4%) usr   0.00 ( 0%) sys   0.07 ( 4%) wall    2442 kB ( 1%) ggc
 TOTAL                 :   1.39             0.17             1.87             236046 kB

And to compare here's exactly same 200 struct report but with -DNOSWAPBUG:

 phase setup             :   0.01 (20%) usr   0.00 ( 0%) sys   0.02 (33%) wall    1438 kB (55%) ggc
 phase parsing           :   0.04 (80%) usr   0.01 (100%) sys   0.04 (67%) wall    1024 kB (39%) ggc
 |name lookup            :   0.02 (40%) usr   0.00 ( 0%) sys   0.01 (17%) wall     216 kB ( 8%) ggc
 preprocessing           :   0.00 ( 0%) usr   0.01 (100%) sys   0.01 (17%) wall      29 kB ( 1%) ggc
 parser (global)         :   0.02 (40%) usr   0.00 ( 0%) sys   0.02 (33%) wall     551 kB (21%) ggc
 parser struct body      :   0.02 (40%) usr   0.00 ( 0%) sys   0.01 (17%) wall     282 kB (11%) ggc
 TOTAL                 :   0.05             0.01             0.06               2607 kB

Not only memory 230KB vs 2KB, but compilation time is 1.39s vs 0.05s.

My environment at work is Ubuntu 11.10 x86_64, so 64 bit compiler, and Ubuntu 12.04 at home, also 64 bit.
Comment 1 Sergey Prokhorenko 2012-07-21 10:58:04 UTC
I also asked few people to test, this could be reproduced on:
g++ (GCC) 4.7.0 20120507 (Red Hat 4.7.0-5)
g++-mp-4.6 (MacPorts gcc46 4.6.3_3) 4.6.3
g++-mp-4.7 (MacPorts gcc47 4.7.1_1) 4.7.1
CentOS 6.3 / GCC 4.7.1 custom build

They also tested on gcc 4.4.6, 4.2.1, 4.5.3 on the same systems, and with these earlier gcc versions the bug does not happen.