| Summary: | gcc 4.4.0 20090102 - jc1: out of memory allocating ... (with 1 G of RAM) | ||
|---|---|---|---|
| Product: | gcc | Reporter: | Rob <rob1weld> |
| Component: | java | Assignee: | Not yet assigned to anyone <unassigned> |
| Status: | RESOLVED WONTFIX | ||
| Severity: | normal | CC: | gcc-bugs, java-prs |
| Priority: | P3 | Keywords: | memory-hog |
| Version: | 4.4.0 | ||
| Target Milestone: | --- | ||
| Host: | i386-pc-solaris2.11 | Target: | i386-pc-solaris2.11 |
| Build: | i386-pc-solaris2.11 | Known to work: | |
| Known to fail: | Last reconfirmed: | ||
| Attachments: |
Memory Usage Report for "classpath/tools/.libs/libgcj_tools_la-tools.o"
Screenshot of build shows libgcj_tools crashing (hogging memory) Screenshot of build shows libgcj_tools building (after reboot) Screenshot of build shows libgcj_tools building (after patch from 38924) |
||
|
Description
Rob
2009-01-03 17:13:03 UTC
Do you have virtual memory turned on because it sounds like you don't. (In reply to comment #1) > Do you have virtual memory turned on because it sounds like you don't. I do have it turned on. This OS uses a swap file that is by default 50% of the size of the RAM. I installed on a 1024M Virtual Machine and thus have 512M of swap. It looks like jc1 is grabbing a big chunk of ~900M. After my change the rest of the compilation worked, as did check and install. Rob With 1400M (and 512M swap) it dies here: gmake[5]: Entering directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava' if /bin/sh ./libtool --tag=GCJ --mode=compile /usr/share/src/gcc_build/gcc/gcj -B/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/ -B/usr/share/src/gcc_build/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fclasspath= -fbootclasspath=../../../../gcc_trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -findirect-dispatch -fno-indirect-classes -fsource-filename=/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/classpath/tools/all-classes.lst -g -O2 -m64 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF "classpath/tools/.deps/libgcj_tools_la-tools.Tpo" -c -o classpath/tools/libgcj_tools_la-tools.lo `test -f 'classpath/tools/tools.zip' || echo '../../../../gcc_trunk/libjava/'`classpath/tools/tools.zip; \ then mv -f "classpath/tools/.deps/libgcj_tools_la-tools.Tpo" "classpath/tools/.deps/libgcj_tools_la-tools.Plo"; else rm -f "classpath/tools/.deps/libgcj_tools_la-tools.Tpo"; exit 1; fi libtool: compile: /usr/share/src/gcc_build/gcc/gcj -B/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/ -B/usr/share/src/gcc_build/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fclasspath= -fbootclasspath=../../../../gcc_trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -findirect-dispatch -fno-indirect-classes -fsource-filename=/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/classpath/tools/all-classes.lst -g -O2 -m64 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF classpath/tools/.deps/libgcj_tools_la-tools.Tpo -c classpath/tools/tools.zip -fPIC -o classpath/tools/.libs/libgcj_tools_la-tools.o jc1: out of memory allocating 4072 bytes after a total of 380526592 bytes gmake[5]: *** [classpath/tools/libgcj_tools_la-tools.lo] Error 1 gmake[5]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava' gmake[4]: *** [all-recursive] Error 1 gmake[4]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava' gmake[3]: *** [multi-do] Error 1 gmake[3]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava' gmake[2]: *** [all-multi] Error 2 gmake[2]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava' gmake[1]: *** [all-target-libjava] Error 2 gmake[1]: Leaving directory `/usr/share/src/gcc_build' gmake: *** [all] Error 2 (In reply to comment #3) > With 1400M (and 512M swap) it dies here: >... > -g -O2 -m64 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF > classpath/tools/.deps/libgcj_tools_la-tools.Tpo -c classpath/tools/tools.zip > -fPIC -o classpath/tools/.libs/libgcj_tools_la-tools.o > jc1: out of memory allocating 4072 bytes after a total of 380526592 bytes > gmake[5]: *** [classpath/tools/libgcj_tools_la-tools.lo] Error 1 > gmake[5]: Leaving directory > `/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava' > gmake[4]: *** [all-recursive] Error 1 > gmake[4]: Leaving directory > `/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava' > gmake[3]: *** [multi-do] Error 1 > gmake[3]: Leaving directory > `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava' > gmake[2]: *** [all-multi] Error 2 > gmake[2]: Leaving directory > `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava' > gmake[1]: *** [all-target-libjava] Error 2 > gmake[1]: Leaving directory `/usr/share/src/gcc_build' > gmake: *** [all] Error 2 I increased my memory to 1900MB (to leave 1G for WinXP) in VirtualBox but it crashed soonafter complaining that WinXP had too little memory. I decreased to 1800MB and tried again, VirtualBox is stable now. When running the build I watched the "System Performance Monitor" and monitored the amount of memory used. With the ./configure options I am using "gnu/javax/swing/text/html/parser/.libs/HTML_401F.o" was the least of our problems. Compiling "classpath/tools/.libs/libgcj_tools_la-tools.o" takes up a much greater amount of memory (as does another section shortly after "HTML_401F.o". # gcc/xgcc -v Using built-in specs. Target: i386-pc-solaris2.11 Configured with: ../gcc_trunk/configure --enable-languages=ada,c,c++,fortran,java,objc,obj-c++ --enable-shared --disable-static --enable-decimal-float --with-long-double-128 --enable-nls --with-included-gettext --enable-gather-detailed-mem-stats --with-stabs --enable-debug -enable-largefile --enable-symvers --without-system-zlib --enable-gtk-cairo --enable-qt-peer --enable-xmlj --enable-gconf-peer --enable-gjdoc --enable-java-awt=gtk,xlib,qt,x --enable-gc-debug --enable-libgcj-debug --enable-objc-gc --enable-libstdcxx-debug --disable-stage1-checking --enable-checking=release --without-system-libunwind --with-gnu-as --with-as=/usr/local/bin/as --with-gnu-ld --with-ld=/usr/local/bin/ld Thread model: posix gcc version 4.4.0 20090111 (experimental) [trunk revision 143259] (GCC) The final result is that on OpenSolaris 2008.11 with 512M of swap you __MUST__ HAVE A MINIMUM of 1750M to compile gcc rev. 143259 (if you use the same options that I did). More is always better but I doubt you can go over 1850MB without VirtualBox becoming unstable. This is a problem from a standpoint of performance of the build (speed) and the restriction of some peoples ability to build the Toolchain on some Platforms. We can just barely compile gcc because of a couple of humps (under certain conditions). Last I checked we were a few versions behind in our Boehm and there were issues dropping the newest rev. into the dirs due to Java, fixed? Thanks, Rob Created attachment 17099 [details] Memory Usage Report for "classpath/tools/.libs/libgcj_tools_la-tools.o" (In reply to comment #4) > (In reply to comment #3) > > With 1400M (and 512M swap) it dies here: > >... > > -g -O2 -m64 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF > > classpath/tools/.deps/libgcj_tools_la-tools.Tpo -c classpath/tools/tools.zip > > -fPIC -o classpath/tools/.libs/libgcj_tools_la-tools.o > > jc1: out of memory allocating 4072 bytes after a total of 380526592 bytes > > ... I manually compiled the one file and added these commands: -v -fmem-report -ftime-report -fpre-ipa-mem-report -fpost-ipa-mem-report # cd /usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava # /usr/share/src/gcc_build/gcc/gcj -B/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/ -B/usr/share/src/gcc_build/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fclasspath= -fbootclasspath=../../../../gcc_trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -findirect-dispatch -fno-indirect-classes -fsource-filename=/usr/share/src/gcc_build/i386-pc-solaris2.11/amd64/libjava/classpath/tools/all-classes.lst -g -O2 -v -fmem-report -ftime-report -fpre-ipa-mem-report -fpost-ipa-mem-report -m64 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF classpath/tools/.deps/libgcj_tools_la-tools.Tpo -c classpath/tools/tools.zip -fPIC -o classpath/tools/.libs/libgcj_tools_la-tools.o 2>&1 | tee /usr/share/src/gcc_build/test_gcc_memory_for_libgcj_tools_compile.result.txt Notice: 1. The report of the "Memory still allocated at the end of the compilation process". 2. The line "??? tree nodes created" 3. Notice this section, bad news followed by an empty section: Heap vectors: source location Leak Peak Times ------------------------------------------------------- gimplify.c:1593 (gimplify_switch_expr) 0: 0.0% 40 123: 0.0% gimplify.c:238 (gimple_push_bind_expr) 0: 0.0% 40 4250: 1.6% tree-eh.c:637 (record_in_goto_queue_label) 0: 0.0% 48 13: 0.0% gimple-low.c:113 (lower_function_body) 0: 0.0% 72 4251: 1.6% gimplify.c:1951 (gimplify_compound_lval) 0: 0.0% 144 260891:96.5% gimplify.c:239 (gimple_push_bind_expr) 0: 0.0% 400 139: 0.1% gimplify.c:1670 (gimplify_case_label_expr) 0: 0.0% 1984 128: 0.0% function.c:4107 (push_struct_function) 24: 0.2% 24 1: 0.0% java/class.c:1797 (make_class_data) 15068:99.8% 16696 598: 0.2% Total 15092 270394 source location Leak Peak Times ------------------------------------------------------- ------------------------------------------------------- 4. Lots in the "Garbage", "Leak" and "Times" piles: source location Garbage Freed Leak Overhead Times ------------------------------------------------------- ... source location Garbage Freed Leak Overhead Times ------------------------------------------------------- ... tree-ssa-operands.c:499 (ssa_operand_alloc) 0: 0.0% 85135158:32.8% 0: 0.0% 2324278: 7.2% 11654 ... gimplify.c:522 (create_tmp_var_raw) 43432400: 6.9% 0: 0.0% 1482184: 2.1% 0: 0.0% 510393 tree-ssanames.c:141 (make_ssa_name_fn) 57218392: 9.1% 0: 0.0% 1459640: 2.0% 0: 0.0% 1047822 Total 631644212 259687118 72135253 32455003 24761794 source location Garbage Freed Leak Overhead Times ------------------------------------------------------- 5. Slow too: Execution times (seconds) ... TOTAL : 581.18 56.35 707.22 909163 kB Thanks Rob Created attachment 17126 [details]
Screenshot of build shows libgcj_tools crashing (hogging memory)
Here we have a screenshot of gcc building 'libgcj_tools' (the two humps) and
crashing on the second one.
The next attachment (a reboot was necessary to continue the build) shows that
the second hump completes and the build finished shortly thereafter.
I configured gcc using:
# gcc/xgcc -v
Using built-in specs.
Target: i386-pc-solaris2.11
Configured with: ../gcc_trunk/configure --enable-languages=ada,c,c++,fortran,java,objc,obj-c++ --enable-shared --disable-static --enable-decimal-float --with-long-double-128 --enable-nls --with-included-gettext --enable-gather-detailed-mem-stats --with-stabs --enable-debug --enable-largefile --enable-symvers --without-system-zlib --enable-gtk-cairo --enable-gconf-peer --enable-xmlj --enable-gtk-peer --enable-qt-peer --enable-plugin --enable-tool-wrappers --enable-local-sockets --enable-gjdoc --enable-java-awt=gtk,xlib,qt,x --enable-gc-debug --enable-libgcj-debug --enable-objc-gc --enable-libstdcxx-debug --disable-stage1-checking --enable-checking=release --without-system-libunwind --with-gnu-as --with-as=/usr/local/bin/as --with-gnu-ld --with-ld=/usr/local/bin/ld
Thread model: posix
gcc version 4.4.0 20090117 (experimental) [trunk revision 143454] (GCC)
It does matter which options you use since I had 'little trouble' a week
ago with few additional ./configure options and 'some trouble' with a few
more options added. Now that I've added all these extra options I am at
a point where gcc will barely compile with all my available memory.
Rob
Created attachment 17127 [details]
Screenshot of build shows libgcj_tools building (after reboot)
Before reboot:
# gmake
...
gmake[3]: Entering directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava'
if /bin/sh ./libtool --tag=GCJ --mode=compile /usr/share/src/gcc_build/gcc/gcj -B/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava/ -B/usr/share/src/gcc_build/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fclasspath= -fbootclasspath=../../../gcc_trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -findirect-dispatch -fno-indirect-classes -fsource-filename=/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava/classpath/tools/all-classes.lst -g -O2 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF "classpath/tools/.deps/libgcj_tools_la-tools.Tpo" -c -o classpath/tools/libgcj_tools_la-tools.lo `test -f 'classpath/tools/tools.zip' || echo '../../../gcc_trunk/libjava/'`classpath/tools/tools.zip; \
then mv -f "classpath/tools/.deps/libgcj_tools_la-tools.Tpo" "classpath/tools/.deps/libgcj_tools_la-tools.Plo"; else rm -f "classpath/tools/.deps/libgcj_tools_la-tools.Tpo"; exit 1; fi
libtool: compile: /usr/share/src/gcc_build/gcc/gcj -B/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava/ -B/usr/share/src/gcc_build/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fclasspath= -fbootclasspath=../../../gcc_trunk/libjava/classpath/lib --encoding=UTF-8 -Wno-deprecated -fbootstrap-classes -findirect-dispatch -fno-indirect-classes -fsource-filename=/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava/classpath/tools/all-classes.lst -g -O2 -MT classpath/tools/libgcj_tools_la-tools.lo -MD -MP -MF classpath/tools/.deps/libgcj_tools_la-tools.Tpo -c classpath/tools/tools.zip -fPIC -o classpath/tools/.libs/libgcj_tools_la-tools.o
jc1: out of memory allocating 4072 bytes after a total of 688709632 bytes
gmake[3]: *** [classpath/tools/libgcj_tools_la-tools.lo] Error 1
gmake[3]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava'
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory `/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava'
gmake[1]: *** [all-target-libjava] Error 2
gmake[1]: Leaving directory `/usr/share/src/gcc_build'
After reboot:
# gmake
...
libtool: link: /usr/share/src/gcc_build/./gcc/xgcc -shared-libgcc -B/usr/share/src/gcc_build/./gcc -nostdinc++ -L/usr/share/src/gcc_build/i386-pc-solaris2.11/libstdc++-v3/src -L/usr/share/src/gcc_build/i386-pc-solaris2.11/libstdc++-v3/src/.libs -B/usr/local/i386-pc-solaris2.11/bin/ -B/usr/local/i386-pc-solaris2.11/lib/ -isystem /usr/local/i386-pc-solaris2.11/include -isystem /usr/local/i386-pc-solaris2.11/sys-include -shared -nostdlib /usr/lib/crti.o /usr/lib/values-Xa.o /usr/share/src/gcc_build/./gcc/crtbegin.o classpath/tools/.libs/libgcj_tools_la-tools.o -L/usr/share/src/gcc_build/i386-pc-solaris2.11/libstdc++-v3/src -L/usr/share/src/gcc_build/i386-pc-solaris2.11/libstdc++-v3/src/.libs -L/usr/share/src/gcc_build/i386-pc-solaris2.11/libjava -L/usr/share/src/gcc_build/./gcc -L/usr/local/i386-pc-solaris2.11/bin -L/usr/local/i386-pc-solaris2.11/lib -lgcc_s /usr/share/src/gcc_build/./gcc/crtend.o /usr/lib/crtn.o -Wl,--version-script=../../../gcc_trunk/libjava/libgcj.ver -Wl,-Bsymbolic-functions -Wl,-soname -Wl,libgcj-tools.so.10 -o .libs/libgcj-tools.so.10.0.0
libtool: link: (cd ".libs" && rm -f "libgcj-tools.so.10" && ln -s "libgcj-tools.so.10.0.0" "libgcj-tools.so.10")
libtool: link: (cd ".libs" && rm -f "libgcj-tools.so" && ln -s "libgcj-tools.so.10.0.0" "libgcj-tools.so")
libtool: link: ( cd ".libs" && rm -f "libgcj-tools.la" && ln -s "../libgcj-tools.la" "libgcj-tools.la" )
...
I was at a point were the size of gcc's build was very near all the memory
that I had available and I needed to reboot my OS to ensure I had every
last byte available. With the reboot I was just barely able to compile
the second 'libgcj_tools' using the maximum available under VirtualBox.
Applying "gcc_trunk/gcc/config/sol2.h" hack from this post: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38924 That is confirmed to "workaround" the problem but not "fix" it. Afterwards it makes Bug 38728 by _far_ much worse, greatly insreasing the running time of "make install" by causing multiple re-linking (ONE file relinking was not a Bug in 38728, now we have many). Screenshot coming, Rob Created attachment 17162 [details]
Screenshot of build shows libgcj_tools building (after patch from 38924)
Description of the Screenshot:
At the far left is a tiny blip (in the "Memory and Swap History" Display) that
shows the memory usage of "gnu/javax/swing/text/html/parser/.libs/HTML_401F.o"
has been _very_ strongly reduced.
A _short_ distance to the right are _two_ humps that represent the 32 and 64
bit portions of linking the ".libs/libgcj-tools.so.10.0.0" libraries. Notice
that this time the 64 bit library actually takes up _slightly_ _less_ memory
than the linking of the 32 bit library as opposed to it's previous operation
without the patch (and it is nowhere close to bringing the OS to it's knees).
Finally to the far right we see the build complete successfully.
Notice that the red line (indicates cache usage) shows only about 50M used.
The "workaround" from the other post has an enormous effect towards
alleviating the memory hogging. It seems like we might be trying to
get collect to drive 'ld' in a (italics) "--enable-intermodule" (end-italics)
(IE: not using, but 'sort of' equivalent to the principle of) manner.
Please would a gcc/binutils Guru comment for us?
Rob
Closing as won't fix as the Java front-end has been removed from the trunk. |