| Summary: | When attempting to compile pdftk-1.41 my machine comes to a grind | ||
|---|---|---|---|
| Product: | gcc | Reporter: | tink <tink.lq> |
| Component: | java | Assignee: | Not yet assigned to anyone <unassigned> |
| Status: | RESOLVED WORKSFORME | ||
| Severity: | normal | CC: | gcc-bugs, gcc, java-prs, mikpelinux, tromey |
| Priority: | P3 | ||
| Version: | 4.3.3 | ||
| Target Milestone: | --- | ||
| Host: | Target: | x86_64-slackware-linux | |
| Build: | Known to work: | ||
| Known to fail: | Last reconfirmed: | ||
I confirm the bug is present in gcj 4.3.3 and 4.4.3. Only x86-64 port is affected (x86-32 works fine); gcj 4.2.4 works fine also for 64bit, so it seems a regression. ecj1 is really the Eclipse frontend which we just inherit, the GCC java frontend (which just handles bytecode) is called jc1. This bug also misses a testcase to reproduce the problem. You can a more recent ecj version, like that which is downloaded by the contrib/download_ecj. Well - not really a GCC bug. (In reply to comment #2) > ecj1 is really the Eclipse frontend which we just inherit, the GCC java > frontend (which just handles bytecode) is called jc1. Why starting from gcc 4.3 jc1 handles only bytecode and no more java source? > This bug also misses a testcase to reproduce the problem. You can reproduce the problem only into x86_64 platform: 1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2 2. cd pdftk-1.41/java_libs/com/lowagie/text 3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o 4. ecj1 allocates about 4.2GB of memory 5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends successfully, even if after many time due to swapping. ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc 4.2.4) require only about 85 MB of memory. > You can a more recent ecj version, like that which is downloaded by > the contrib/download_ecj. ecj-4.5.jar (actually the latest ecj) does not resolve the problem. > Well - not really a GCC bug. Ok, but we have no other options to compile java sources with gcc 4.3 (only downgrade to 4.2)... (In reply to comment #3) > (In reply to comment #2) > > ecj1 is really the Eclipse frontend which we just inherit, the GCC java > > frontend (which just handles bytecode) is called jc1. > > Why starting from gcc 4.3 jc1 handles only bytecode and no more java source? > > > This bug also misses a testcase to reproduce the problem. > > You can reproduce the problem only into x86_64 platform: > 1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2 > 2. cd pdftk-1.41/java_libs/com/lowagie/text > 3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o > 4. ecj1 allocates about 4.2GB of memory > 5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends > successfully, even if after many time due to swapping. > > ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc > 4.2.4) require only about 85 MB of memory. > > > You can a more recent ecj version, like that which is downloaded by > > the contrib/download_ecj. > > ecj-4.5.jar (actually the latest ecj) does not resolve the problem. > > > Well - not really a GCC bug. > > Ok, but we have no other options to compile java sources with gcc 4.3 (only > downgrade to 4.2)... Use another Java-to-bytecode compiler and feed gcc with bytecode. Richard. (In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > ecj1 is really the Eclipse frontend which we just inherit, the GCC java > > > frontend (which just handles bytecode) is called jc1. > > > > Why starting from gcc 4.3 jc1 handles only bytecode and no more java source? > > > > > This bug also misses a testcase to reproduce the problem. > > > > You can reproduce the problem only into x86_64 platform: > > 1. Download pdftk http://www.pdfhacks.com/pdftk/pdftk-1.41.tar.bz2 > > 2. cd pdftk-1.41/java_libs/com/lowagie/text > > 3. gcj -O2 -w --classpath="../../../" -c Anchor.java -o Anchor.o > > 4. ecj1 allocates about 4.2GB of memory > > 5. If you have enough memory (I have 2GB RAM + 3GB swap), compilation ends > > successfully, even if after many time due to swapping. > > > > ecj1 on x86_32 (same gcc version and build options) or jc1 on x86_64 (gcc > > 4.2.4) require only about 85 MB of memory. > > > > > You can a more recent ecj version, like that which is downloaded by > > > the contrib/download_ecj. > > > > ecj-4.5.jar (actually the latest ecj) does not resolve the problem. > > > > > Well - not really a GCC bug. > > > > Ok, but we have no other options to compile java sources with gcc 4.3 (only > > downgrade to 4.2)... > > Use another Java-to-bytecode compiler and feed gcc with bytecode. Btw, the symptoms you are seeing hint at not working garbage collection. You might want to report this to your operating system vendor. Richard. > Richard. > Even if you fix this issue, I don't believe you will be able to compile pdftk.cc with gcc 4.3 or later... http://gcc.gnu.org/ml/java/2008-03/msg00028.html due to the code incorrectly mixing c++ and java exceptions which are no longer allowed under FSF gcc. (In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > > Use another Java-to-bytecode compiler and feed gcc with bytecode. I'd like to use only free software... There is another free (GPL-compatible) java-to-bytecode compiler? > Btw, the symptoms you are seeing hint at not working garbage collection. > You might want to report this to your operating system vendor. Sorry, but I think the garbage collector is from gcc (libgcj), not from operating system. I try to substitute ecj1 with a shell script: #!/usr/bin/sh gij -cp /usr/share/java/ecj.jar org.eclipse.jdt.internal.compiler.batch.GCCMain "$@" And the simptom are the same, so I think the problem is inside gcc java interpreter. If I substitute "gij" with Sun "java", the compilation ends with "only" about 400MB memory used. Go deeply, gcj call ecj1 with the following arguments: ecj1 Anchor.java -w -fbootclasspath=../../../:/usr/share/java/libgcj-4.3.3.jar -fencoding=utf8 -fsource=1.5 -ftarget=1.5 -fzip-dependency /tmp/ccAgHLvl.zip -fzip-target /tmp/cccrxtPw.jar If I remove the last argument, class files are generated in the file system and ecj1/gij uses "only" about 130MB of memory. If I re-add the -fzip-target parameter with .class files into the file system, compilation is really fast. So, I think the problem is in gcc 64bit java interpreter when it stores the .class files in memory instead of file system. (In reply to comment #6) > Even if you fix this issue, I don't believe you will be able to compile > pdftk.cc with gcc 4.3 or later... > > http://gcc.gnu.org/ml/java/2008-03/msg00028.html > > due to the code incorrectly mixing c++ and java exceptions which are no longer > allowed under FSF gcc. pdftk will finally compile correctly (I have the executable), but it takes many hours to compile... I have only to apply a little patch to remove local java.security.MessageDigest. Luca >There is another free (GPL-compatible) java-to-bytecode compiler?
Sun's OpenJDK (which includes javac) is GPLv2 IIRC.
I fully compiled ecj1 with: gcj --main=org.eclipse.jdt.internal.compiler.batch.GCCMain -o ecj1 ecj.jar and now it works fine (and very quickly!) So, the problem seems to be in 64bit gij (java bytecode interpreter), used to run ecj.jar (java-to-bytecode compiler). How could I help to debug gij? Does slackware provide packaging files to show how they build their gcc? Normally all you need to do is download ftp://sourceware.org/pub/java/ecj-latest.jar and place it as ecj.jar in the top of the gcc source tree for the build to properly utilize it. Does your stock gcj compile simple java code like... public class testme { public static void main(String args[]){ System.out.println("Hello"); } } with just 'gcj --main=testme -O testme.java'? You shouldn't need to pass more than that for gcj to work if it is using ecj.jar. (In reply to comment #10) > Does slackware provide packaging files to show how they build their gcc? Yes, sure. It is a "standard" build, with ecj.jar placed in the top source tree, and it is exactly the same build script between 32 and 64 bits (always x86 architecture), but only 64 bits has the problem. I could try another x86_64 distribution, but I think the problem is into gcc java interpreter... > Does your stock gcj compile simple java code like... > with just 'gcj --main=testme -O testme.java'? Yes, the problem seems to be only if ecj.jar (run by ecj1/gij interpreter) has to compile many .java source files and in the same time it has to keep in memory many bytecode .class file (option -fzip-target), like in pdftk. If you run ecj.jar by Sun java interpreter or if you fully compile ecj.jar with gcj to build a complete ecj1 (not only a wrapper for gij), pdftk compilation works fine. I upgraded all gcc components to 4.4.3 (before I upgraded only java parts) and it works fine. I downgraded to 4.3.3 and put only libgcc_s.so.1 from gcc 4.4.3 and it works fine. So, the problem was in libgcc_s and it was fixed in gcc 4.4. (In reply to comment #12) > I upgraded all gcc components to 4.4.3 (before I upgraded only java parts) and > it works fine. I downgraded to 4.3.3 and put only libgcc_s.so.1 from gcc 4.4.3 > and it works fine. So, the problem was in libgcc_s and it was fixed in gcc 4.4. Exactly as I thought. There is/was an unwinding problem with made garbage collection break, there is a glibc fix available or an unwinder fix in gcc 4.4. Your distributor didn't pick up the glibc fix appearantly. (In reply to comment #13) > Exactly as I thought. There is/was an unwinding problem with made garbage > collection break, there is a glibc fix available or an unwinder fix in gcc 4.4. > Your distributor didn't pick up the glibc fix appearantly. Do you have the PR numbers or commit dates for the glibc and unwinder fixes? (In reply to comment #14) > (In reply to comment #13) > > Exactly as I thought. There is/was an unwinding problem with made garbage > > collection break, there is a glibc fix available or an unwinder fix in gcc 4.4. > > Your distributor didn't pick up the glibc fix appearantly. > > Do you have the PR numbers or commit dates for the glibc and unwinder fixes? I think this was fixed in the unwinder by 2009-04-04 Jakub Jelinek <jakub@redhat.com> * unwind-dw2.h (_Unwind_FrameState): Add REG_UNDEFINED enum value. * unwind-dw2.c (execute_cfa_program): Set how to REG_UNDEFINED instead of REG_UNSAVED for DW_CFA_undefined. (uw_update_context_1): Handle REG_UNDEFINED the same as REG_UNSAVED. (uw_update_context): If RA column is REG_UNDEFINED, mark it as outermost frame. and the bug can be avoided in glibc by Wed Jun 25 13:30:42 CEST 2008 - rguenther@suse.de - Add patch to fix unwinding through clone for x86_64 and i386. [bnc#290807, bnc#403464] bnc#403464 contains useful links (that's bugzilla.novell.com), one reference to PR36568 which contains a glibc patch. |
make -C ../java_libs make[1]: Entering directory `/home/p679492/src/pdftk-1.41/java_libs' make -C "/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text"; make[2]: Entering directory `/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text' gcj -O2 -w --encoding=UTF-8 --classpath="/home/p679492/src/pdftk-1.41/java_libs" -c Anchor.java -o Anchor.o GC Warning: Repeated allocation of very large block (appr. size 2097152000): May lead to memory leak and poor performance. gcj: Internal error: Killed (program ecj1) Please submit a full bug report. See <http://gcc.gnu.org/bugs.html> for instructions. make[2]: *** [Anchor.o] Error 1 make[2]: Leaving directory `/home/p679492/src/pdftk-1.41/java_libs/com/lowagie/text' make[1]: *** [itext] Error 2 make[1]: Leaving directory `/home/p679492/src/pdftk-1.41/java_libs' make: *** [java_libs] Error 2 This bumps swap usage up to ~ 3G on a machine w/ 4GB of RAM ... the box becomes unresponsive until ecj1 dies.