Bug 98431 - Gcc uses a lot of compile-time and memory
Summary: Gcc uses a lot of compile-time and memory
Status: NEW
Alias: None
Product: gcc
Classification: Unclassified
Component: middle-end (show other bugs)
Version: 7.3.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords: compile-time-hog, memory-hog
Depends on:
Blocks:
 
Reported: 2020-12-23 20:35 UTC by Farshid Shadpey
Modified: 2021-01-06 14:47 UTC (History)
5 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail: 10.2.0, 7.5.0
Last reconfirmed: 2021-01-05 00:00:00


Attachments
preprocessed file zipped (967.87 KB, application/x-7z-compressed)
2020-12-24 22:07 UTC, Farshid Shadpey
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Farshid Shadpey 2020-12-23 20:35:18 UTC
# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04) 


#gcc -save-temps -c -m64 -fwrapv  -O0 -DCLASSIC_INTERFACE=0 -DALLOCATIONFCN=0 -DMAT_FILE=1 -DONESTEPFCN=0 -DTERMFCN=1 -DMULTI_INSTANCE_CODE=0 -DINTEGER_CODE=0 -DMT=0 -DNI_ROOTMODEL_TMS_Build -DTID01EQ=1 -DMODEL=TMS_Build -DNUMST=4 -DNCSTATES=0 -DHAVESTDIO -DRT -DUSE_RTMODEL  @TMS_Build_comp.rsp -o"TMS_Build_106165d6_1_ds_dxf.obj" "../TMS_Build_veristand_rtw/TMS_Build_106165d6_1_ds_dxf.c"
gcc: fatal error: Killed signal terminated program cc1
compilation terminated.
Comment 1 Farshid Shadpey 2020-12-23 20:46:02 UTC
cannot attach the preprocessed file .i since it is too big (14 MB)
Please advise.
Comment 2 Farshid Shadpey 2020-12-23 20:55:30 UTC
Also it is intersting to know that the same file compiles with the following gcc compiler on Windows:

c:\Farshid\Matlab\Veristand_issue_models\TMS_Repro\TMS_Build_veristand_rtw>"C:\PROGRA~3\MATLAB\SUPPOR~1\R2019b\3P778C~1.INS\MINGW_~1.INS\bin/gcc" -v
Using built-in specs.
COLLECT_GCC=C:\PROGRA~3\MATLAB\SUPPOR~1\R2019b\3P778C~1.INS\MINGW_~1.INS\bin/gcc
COLLECT_LTO_WRAPPER=C:/PROGRA~3/MATLAB/SUPPOR~1/R2019b/3P778C~1.INS/MINGW_~1.INS/bin/../libexec/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-6.3.0/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --with-sysroot=/c/mingw630/x86_64-630-posix-seh-rt_v5-rev2/mingw64 --enable-shared --enable-static --disable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/c/mingw630/prerequisites/x86_64-w64-mingw32-static --with-mpfr=/c/mingw630/prerequisites/x86_64-w64-mingw32-static --with-mpc=/c/mingw630/prerequisites/x86_64-w64-mingw32-static --with-isl=/c/mingw630/prerequisites/x86_64-w64-mingw32-static --with-pkgversion='x86_64-posix-seh-rev2, Built by MinGW-W64 project' --with-bugurl=http://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -fno-ident -I/c/mingw630/x86_64-630-posix-seh-rt_v5-rev2/mingw64/opt/include -I/c/mingw630/prerequisites/x86_64-zlib-static/include -I/c/mingw630/prerequisites/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -fno-ident -I/c/mingw630/x86_64-630-posix-seh-rt_v5-rev2/mingw64/opt/include -I/c/mingw630/prerequisites/x86_64-zlib-static/include -I/c/mingw630/prerequisites/x86_64-w64-mingw32-static/include' CPPFLAGS=' -I/c/mingw630/x86_64-630-posix-seh-rt_v5-rev2/mingw64/opt/include -I/c/mingw630/prerequisites/x86_64-zlib-static/include -I/c/mingw630/prerequisites/x86_64-w64-mingw32-static/include' LDFLAGS='-pipe -fno-ident -L/c/mingw630/x86_64-630-posix-seh-rt_v5-rev2/mingw64/opt/lib -L/c/mingw630/prerequisites/x86_64-zlib-static/lib -L/c/mingw630/prerequisites/x86_64-w64-mingw32-static/lib '
Thread model: posix
gcc version 6.3.0 (x86_64-posix-seh-rev2, Built by MinGW-W64 project)
Comment 3 Farshid Shadpey 2020-12-24 02:08:19 UTC
Sorry i appended a wrong output from a machine without enough ram. The correct report with segmentation fault is:

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-nilrt-linux/7.3.0/lto-wrapper
Target: x86_64-nilrt-linux
Configured with: ../../../../../../work-shared/gcc-7.3.0-r0/gcc-7.3.0/configure --build=x86_64-linux --host=x86_64-nilrt-linux --target=x86_64-nilrt-linux --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib --includedir=/usr/include --oldincludedir=/usr/include --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/srv/jenkins/perforce/ThirdPartyExports/NIOpenEmbedded/trunk/8.5/objects/build.x64/tmp-glibc/work/core2-64-nilrt-linux/gcc/7.3.0-r0/recipe-sysroot --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --enable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=x86_64-nilrt-linux- --without-local-prefix --enable-lto --enable-libssp --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=gnu --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-sysroot=/ --with-build-sysroot=/srv/jenkins/perforce/ThirdPartyExports/NIOpenEmbedded/trunk/8.5/objects/build.x64/tmp-glibc/work/core2-64-nilrt-linux/gcc/7.3.0-r0/recipe-sysroot --with-gxx-include-dir=/usr/include/c++/7.3.0 --without-long-double-128 --enable-nls --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
gcc version 7.3.0 (GCC)

# gcc   -c -m64 -fPIC -fdollars-in-identifiers -DkNIOSLinux  -O0 -DCLASSIC_INTERFACE=1 -DALLOCATIONFCN=0 -DMAT_FILE=1 -DONESTEPFCN=0 -DTERMFCN=1 -DMULTI_INSTANCE_CODE=0 -DINTEGER_CODE=0 -DMT=0 -DNI_ROOTMODEL_TMS_Build -DTID01EQ=1 -DMODEL=TMS_Build -DNUMST=4 -DNCSTATES=0 -DHAVESTDIO -DRT -DUSE_RTMODEL  @TMS_Build_comp.rsp -o"TMS_Build_106165d6_1_ds_dxf.obj" "../TMS_Build_veristand_rtw/TMS_Build_106165d6_1_ds_dxf.c" -save-temps
gcc: internal compiler error: Segmentation fault (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
Comment 4 Farshid Shadpey 2020-12-24 02:09:52 UTC
the gzip version of preprocessed file is 1.4 MB and cannot be attached.
please advise
Comment 5 Arseny Solokha 2020-12-24 18:37:20 UTC
Try reducing your testcase. You can use C-Vise[1] or C-Reduce[2] for that (these are not the only tools of this kind, though).

[1] https://github.com/marxin/cvise
[2] https://embed.cs.utah.edu/creduce
Comment 6 Farshid Shadpey 2020-12-24 22:07:00 UTC
Created attachment 49842 [details]
preprocessed file zipped
Comment 7 Farshid Shadpey 2020-12-24 22:08:56 UTC
something went wrong i cut and paste between different computers, remote session, etc. The output below should be the correct file, plesae ignore the previous ones. 

Thanks
Farshid



# gcc   -c -m64 -fPIC -fdollars-in-identifiers -DkNIOSLinux  -O0 -DCLASSIC_INTERFACE=1 -DALLOCATIONFCN=0 -DMAT_FILE=1 -DONESTEPFCN=0 -DTERMFCN=1 -DMULTI_INSTANCE_CODE=0 -DINTEGER_CODE=0 -DMT=0 -DNI_ROOTMODEL_TMS_Build -DTID01EQ=1 -DMODEL=TMS_Build -DNUMST=4 -DNCSTATES=0 -DHAVESTDIO -DRT -DUSE_RTMODEL  @TMS_Build_comp.rsp -o"TMS_Build_106165d6_1_ds_dxf.obj" "../TMS_Build_veristand_rtw/TMS_Build_106165d6_1_ds_dxf.c" -save-temps -v
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-nilrt-linux
Configured with: ../../../../../../work-shared/gcc-7.3.0-r0/gcc-7.3.0/configure --build=x86_64-linux --host=x86_64-nilrt-linux --target=x86_64-nilrt-linux --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib --includedir=/usr/include --oldincludedir=/usr/include --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/srv/jenkins/perforce/ThirdPartyExports/NIOpenEmbedded/trunk/8.5/objects/build.x64/tmp-glibc/work/core2-64-nilrt-linux/gcc/7.3.0-r0/recipe-sysroot --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --enable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=x86_64-nilrt-linux- --without-local-prefix --enable-lto --enable-libssp --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=gnu --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-sysroot=/ --with-build-sysroot=/srv/jenkins/perforce/ThirdPartyExports/NIOpenEmbedded/trunk/8.5/objects/build.x64/tmp-glibc/work/core2-64-nilrt-linux/gcc/7.3.0-r0/recipe-sysroot --with-gxx-include-dir=/usr/include/c++/7.3.0 --without-long-double-128 --enable-nls --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
gcc version 7.3.0 (GCC)
COLLECT_GCC_OPTIONS='-c' '-m64' '-fPIC' '-fdollars-in-identifiers' '-D' 'kNIOSLinux' '-O0' '-D' 'CLASSIC_INTERFACE=1' '-D' 'ALLOCATIONFCN=0' '-D' 'MAT_FILE=1' '-D' 'ONESTEPFCN=0' '-D' 'TERMFCN=1' '-D' 'MULTI_INSTANCE_CODE=0' '-D' 'INTEGER_CODE=0' '-D' 'MT=0' '-D' 'NI_ROOTMODEL_TMS_Build' '-D' 'TID01EQ=1' '-D' 'MODEL=TMS_Build' '-D' 'NUMST=4' '-D' 'NCSTATES=0' '-D' 'HAVESTDIO' '-D' 'RT' '-D' 'USE_RTMODEL' '-I' '.' '-I' './R2019b/extern/include' '-I' './R2019b/simulink/include' '-I' './R2019b/rtw/c/src' '-I' './R2019b/rtw/c/src/ext_mode/common' '-I' './R2019b/toolbox/physmod/simscape/engine/sli/c/win64' '-I' './R2019b/toolbox/physmod/simscape/engine/core/c/win64' '-I' './R2019b/toolbox/physmod/simscape/compiler/core/c/win64' '-I' './R2019b/toolbox/physmod/network_engine/c/win64' '-I' './R2019b/toolbox/physmod/common/math/core/c/win64' '-I' './R2019b/toolbox/physmod/common/lang/core/c/win64' '-I' './R2019b/toolbox/physmod/common/external/library/c/win64' '-I' './R2019b/toolbox/physmod/common/foundation/core/c/win64' '-I' '.' '-o' 'TMS_Build_106165d6_1_ds_dxf.obj' '-save-temps' '-v' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-nilrt-linux/7.3.0/cc1 -E -quiet -v -I . -I ./R2019b/extern/include -I ./R2019b/simulink/include -I ./R2019b/rtw/c/src -I ./R2019b/rtw/c/src/ext_mode/common -I ./R2019b/toolbox/physmod/simscape/engine/sli/c/win64 -I ./R2019b/toolbox/physmod/simscape/engine/core/c/win64 -I ./R2019b/toolbox/physmod/simscape/compiler/core/c/win64 -I ./R2019b/toolbox/physmod/network_engine/c/win64 -I ./R2019b/toolbox/physmod/common/math/core/c/win64 -I ./R2019b/toolbox/physmod/common/lang/core/c/win64 -I ./R2019b/toolbox/physmod/common/external/library/c/win64 -I ./R2019b/toolbox/physmod/common/foundation/core/c/win64 -I . -D kNIOSLinux -D CLASSIC_INTERFACE=1 -D ALLOCATIONFCN=0 -D MAT_FILE=1 -D ONESTEPFCN=0 -D TERMFCN=1 -D MULTI_INSTANCE_CODE=0 -D INTEGER_CODE=0 -D MT=0 -D NI_ROOTMODEL_TMS_Build -D TID01EQ=1 -D MODEL=TMS_Build -D NUMST=4 -D NCSTATES=0 -D HAVESTDIO -D RT -D USE_RTMODEL ../TMS_Build_veristand_rtw/TMS_Build_106165d6_1_ds_dxf.c -m64 -mtune=generic -march=x86-64 -fPIC -fdollars-in-identifiers -O0 -fpch-preprocess -o TMS_Build_106165d6_1_ds_dxf.i
ignoring duplicate directory "/usr/lib/gcc/x86_64-nilrt-linux/7.3.0/include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-nilrt-linux/7.3.0/../../../../x86_64-nilrt-linux/include"
ignoring nonexistent directory "./R2019b/rtw/c/src"
ignoring nonexistent directory "./R2019b/rtw/c/src/ext_mode/common"
ignoring duplicate directory "."
#include "..." search starts here:
#include <...> search starts here:
 .
 ./R2019b/extern/include
 ./R2019b/simulink/include
 ./R2019b/toolbox/physmod/simscape/engine/sli/c/win64
 ./R2019b/toolbox/physmod/simscape/engine/core/c/win64
 ./R2019b/toolbox/physmod/simscape/compiler/core/c/win64
 ./R2019b/toolbox/physmod/network_engine/c/win64
 ./R2019b/toolbox/physmod/common/math/core/c/win64
 ./R2019b/toolbox/physmod/common/lang/core/c/win64
 ./R2019b/toolbox/physmod/common/external/library/c/win64
 ./R2019b/toolbox/physmod/common/foundation/core/c/win64
 /usr/lib/gcc/x86_64-nilrt-linux/7.3.0/include
 /usr/lib/gcc/x86_64-nilrt-linux/7.3.0/include-fixed
 /usr/include
End of search list.
COLLECT_GCC_OPTIONS='-c' '-m64' '-fPIC' '-fdollars-in-identifiers' '-D' 'kNIOSLinux' '-O0' '-D' 'CLASSIC_INTERFACE=1' '-D' 'ALLOCATIONFCN=0' '-D' 'MAT_FILE=1' '-D' 'ONESTEPFCN=0' '-D' 'TERMFCN=1' '-D' 'MULTI_INSTANCE_CODE=0' '-D' 'INTEGER_CODE=0' '-D' 'MT=0' '-D' 'NI_ROOTMODEL_TMS_Build' '-D' 'TID01EQ=1' '-D' 'MODEL=TMS_Build' '-D' 'NUMST=4' '-D' 'NCSTATES=0' '-D' 'HAVESTDIO' '-D' 'RT' '-D' 'USE_RTMODEL' '-I' '.' '-I' './R2019b/extern/include' '-I' './R2019b/simulink/include' '-I' './R2019b/rtw/c/src' '-I' './R2019b/rtw/c/src/ext_mode/common' '-I' './R2019b/toolbox/physmod/simscape/engine/sli/c/win64' '-I' './R2019b/toolbox/physmod/simscape/engine/core/c/win64' '-I' './R2019b/toolbox/physmod/simscape/compiler/core/c/win64' '-I' './R2019b/toolbox/physmod/network_engine/c/win64' '-I' './R2019b/toolbox/physmod/common/math/core/c/win64' '-I' './R2019b/toolbox/physmod/common/lang/core/c/win64' '-I' './R2019b/toolbox/physmod/common/external/library/c/win64' '-I' './R2019b/toolbox/physmod/common/foundation/core/c/win64' '-I' '.' '-o' 'TMS_Build_106165d6_1_ds_dxf.obj' '-save-temps' '-v' '-mtune=generic' '-march=x86-64'
 /usr/libexec/gcc/x86_64-nilrt-linux/7.3.0/cc1 -fpreprocessed TMS_Build_106165d6_1_ds_dxf.i -quiet -dumpbase TMS_Build_106165d6_1_ds_dxf.c -m64 -mtune=generic -march=x86-64 -auxbase-strip TMS_Build_106165d6_1_ds_dxf.obj -O0 -version -fPIC -fdollars-in-identifiers -o TMS_Build_106165d6_1_ds_dxf.s
GNU C11 (GCC) version 7.3.0 (x86_64-nilrt-linux)
        compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.5, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU C11 (GCC) version 7.3.0 (x86_64-nilrt-linux)
        compiled by GNU C version 7.3.0, GMP version 6.1.2, MPFR version 3.1.5, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: cb0fdc91d63702a7162bd0d5e74d1bfb
gcc: internal compiler error: Segmentation fault (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
Comment 8 Richard Biener 2021-01-05 10:07:40 UTC
It compiles successfully for me with the FSF 7.3.0 release and -O0 -fPIC -fdollars-in-identifiers - it does use a lot of memory though (~6GB).

Note that GCC 7 is no longer supported.

With GCC 10.2 I see

 out of ssa                         :  22.14 ( 24%)   1.07 ( 24%)  23.21 ( 24%)      63 kB (  0%)
 LRA non-specific                   :  39.41 ( 43%)   0.00 (  0%)  39.41 ( 41%)   19259 kB (  1%)
 TOTAL                              :  91.38          4.53         95.92        1534299 kB
91.38user 4.73system 1:36.12elapsed 99%CPU (0avgtext+0avgdata 5768460maxresident)k
0inputs+76184outputs (0major+1393158minor)pagefaults 0swaps

Confirmed as memory/compile-time hog.
Comment 9 Martin Liška 2021-01-05 10:16:36 UTC
Just for the record, GCC 10 without any options takes 108s, which 4.8.0 74s.
So not a rapid difference.