gcj performance: benchmarks

[Bryce McKinlay <bryce@albatross.co.nz>]

Continuing with the current performance theme, I thought I'd post a few
benchmark results of my own:

                        IBM116  GCJ     JDK12   KAFFE   JDK117  GIJ
                        --------------------------------------------
Sieve.java              1287    2002    548     367     76      21
ByteArrays.java         10.9s   12.6s   17.8s   130.4s  36.9s   16.9s
BigHeap.java            4.7s+   5.8s    !       4.3s+   1.5s+   6.1s

ShellSort 100000:
 generate               0.98s   0.79s   1.21s   0.99s   2.52s   1.69s
 sort                   3.93s   2.65s   2.39s   4.80s   8.33s   25.55s
 check                  0.05s   0.08s   0.12s   0.22s   0.26s   0.74s

jBYTEmark:
 Numeric Sort           56.25   59.71   34.11   16.64   2.89    0.94
 String Sort            14.76   13.99   7.45    10.80   3.76    1.08
 Bitfield Operations    64.29   48.24   37.62   16.74   3.03    1.15
 FP Emulation           56.45   41.89   19.31   10.05   2.56    0.88
 Fourier                9.11    5.34    1.64    4.83    2.70    1.10
 Assignment             63.31   39.19   18.84   16.70   4.04    1.05
 IDEA Encryption        23.00   12.36   8.23    7.87    2.97    1.18
 Huffman Compression    50.73   37.79   20.01   13.10   3.13    0.97
 Neural Net             51.74   37.32   12.44   11.99   2.60    *
 LU Decomposition       76.59   29.65   16.45   10.76   3.27    *
 ---
 Integer Index:         41.79   31.52   17.83   12.66   3.16    *
 FP Index:              33.05   18.08   6.95    8.54    2.84    *

+ = kaffe and the 1.1 JDKs perform slowly on the first iteration and then much
more quickly on subsequent iterations (the best scores are given here).
GCJ/GIJ performance is fairly constant on each iteration. I think this has more
to do with the GC implementation rather than the JITs.
! = jdk 1.2 locks up and does not complete this test
* = gij gets a NullPointerException and doesn't complete this test.

IBM116: "JDK 1.1.6 IBM alpha build l116-199906300917 (JIT enabled: jitc)"
GCJ:   gcj -O2 --no-bounds-check (compiled from .java source where possible).
JDK12:  Blackdown "Linux_JDK_1.2_pre-release-v2" w/ sunwjit
KAFFE:  kaffe 1.0b4 w/built-in JIT
JDK117: Blackdown "Linux_JDK_1.1.7B_v3_green_threads" (no JIT)
GIJ:    gcj interpreter

Sieve.java is the Eratosthene's Sieve prime-number benchmark that comes with
TYA.
ByteArrays.java allocates an array of 20,000 short byte-arrays 10 times.
BigHeap.java allocates 2,000,000 Byte objects
ShellSort.java as posted to the list, with timers added.
jBYTEmark is an old java computational benchmark I found and modified slightly
so that it works with modern (1.1-level) compilers. gcj can't compile it from
source but it works fine from javac-generated class files.

All benchmarking was done on my P2-233mhz machine running Redhat Linux 6.0

Conclusions: 1) gcj is generally competitive with the fastest java 1.1
implementation available on Linux, although it lags a little in computational
performance. It probibly could/should be significantly faster. However, the
benchmarks don't show the huge benefits of greatly reduced startup/class-loading
time, and the elimination of JIT compilation overhead that gcj gives us.
2) the interpreter is rather slow ;-)

regards

  [ bryce ]