Bug 114578 - [13/14 Regression] memory hog, virtual memory exhausted, building a c++ file on arm-linux-gnueabihf
Summary: [13/14 Regression] memory hog, virtual memory exhausted, building a c++ file ...
Status: NEW
Alias: None
Product: gcc
Classification: Unclassified
Component: c++ (show other bugs)
Version: 13.2.1
: P2 normal
Target Milestone: 13.5
Assignee: Not yet assigned to anyone
URL:
Keywords: compile-time-hog, memory-hog
Depends on:
Blocks:
 
Reported: 2024-04-04 07:37 UTC by Matthias Klose
Modified: 2025-06-05 17:10 UTC (History)
2 users (show)

See Also:
Host: arm-linux-gnueabihf
Target:
Build:
Known to work: 12.3.1
Known to fail: 13.2.1, 14.0
Last reconfirmed: 2025-02-04 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Klose 2024-04-04 07:37:27 UTC
seen with the 13 branch and trunk, building on arm-linux-gnueabihf.

$ g++ -g -O0 -fstack-protector-strong -fno-stack-clash-protection -fno-exceptions -std=gnu++17 -fvisibility=hidden -fvisibility-inlines-hidden -fPIC -c qrc_assets.ii
virtual memory exhausted: Cannot allocate memory

the gcc-12 branch is able to build that with -g -O2.

preprocessed source (120MB uncompressed) at
https://people.debian.org/~doko/tmp/qrc_assets.ii.xz
Comment 1 Drea Pinski 2024-04-04 21:03:21 UTC
Time variable                                   usr           sys          wall           GGC
 phase setup                        :   0.00 (  0%)   0.03 (  0%)   0.10 (  0%)  1903k (  0%)
 phase parsing                      : 114.74 ( 95%)  45.79 (100%) 160.53 ( 95%)  6159M (100%)
 phase lang. deferred               :   0.07 (  0%)   0.00 (  0%)   0.07 (  0%)  9312  (  0%)
 phase opt and generate             :   6.36 (  5%)   0.10 (  0%)   6.46 (  4%)   293k (  0%)
 phase finalize                     :   0.00 (  0%)   0.06 (  0%)   1.72 (  1%)     0  (  0%)
 |name lookup                       :   0.00 (  0%)   0.01 (  0%)   0.00 (  0%)   109k (  0%)
 garbage collection                 :   1.89 (  2%)   0.00 (  0%)   1.89 (  1%)     0  (  0%)
 callgraph construction             :   6.35 (  5%)   0.09 (  0%)   6.44 (  4%)    13k (  0%)
 preprocessing                      :  30.74 ( 25%)  24.93 ( 54%)  55.53 ( 33%)  2047M ( 33%)
 parser (global)                    :  81.10 ( 67%)  20.86 ( 45%) 102.10 ( 60%)  4111M ( 67%)
 constant expression evaluation     :   1.08 (  1%)   0.00 (  0%)   1.08 (  1%)   304  (  0%)
 post expand cleanups               :   0.00 (  0%)   0.00 (  0%)   0.01 (  0%)  2728  (  0%)
 initialize rtl                     :   0.00 (  0%)   0.00 (  0%)   0.01 (  0%)  9032  (  0%)
 rest of compilation                :   0.01 (  0%)   0.01 (  0%)   0.00 (  0%)    16k (  0%)
 TOTAL                              : 121.17         45.98        168.88         6161M
Extra diagnostic checks enabled; compiler may run slowly.


For x86_64-linux-gnu (but with checking enabled). Definitely a front-end issue. Preprocessing takes 2G is even more shocking.
Comment 2 Drea Pinski 2024-04-04 21:06:40 UTC
Though 111MB preprocessed source makes this a huge one in general but if we say on average a token is 3 characters and a token takes ~16 bytes each. that does expand to over 500 MB of memory since we do up front lexing for C++.
Comment 3 Drea Pinski 2024-04-04 21:09:05 UTC
The real way to fix this is to have `#embed` supported for both C++ and C and move this code over to use that instead.
Comment 4 Drea Pinski 2024-04-04 21:10:04 UTC
The code has 3 huge arrays which is definitely a canidate for  `#embed` usage.
Comment 5 Drea Pinski 2024-04-04 21:17:15 UTC
Someone should report this issue to the QT folks to see if they could reimplement this to use `#embed` (though GCC needs to add #embed support still but that is PR 105863)
Comment 6 Jakub Jelinek 2024-05-21 09:19:51 UTC
GCC 13.3 is being released, retargeting bugs to GCC 13.4.
Comment 7 Richard Biener 2025-02-04 08:31:05 UTC
I'll note that 32bit host systems are not the focus for (memory) optimizations.
I can confirm a slight memory usage regression on a 64bit host from 12.4 to 13.3, from

27.09user 1.25system 0:28.42elapsed 99%CPU (0avgtext+0avgdata 3644708maxresident)k
3616inputs+468408outputs (14major+781348minor)pagefaults 0swaps

to

27.95user 1.27system 0:29.28elapsed 99%CPU (0avgtext+0avgdata 3808172maxresident)k
1816inputs+468504outputs (8major+829796minor)pagefaults 0swaps

GCC 14.2 using

26.18user 1.24system 0:27.49elapsed 99%CPU (0avgtext+0avgdata 3810184maxresident)k
3784inputs+468504outputs (12major+800842minor)pagefaults 0swaps

and trunk improving to the following - likely due to automagic #embed-ification.

2.03user 0.09system 0:02.13elapsed 99%CPU (0avgtext+0avgdata 267264maxresident)k
0inputs+62184outputs (0major+34537minor)pagefaults 0swaps

So - fixed on trunk.
Comment 8 Jakub Jelinek 2025-06-05 17:10:57 UTC
GCC 13.4 is being released, retargeting bugs to GCC 13.5.