Some benchmark results

[Terry Sikes <tlsikes at yahoo dot com>]

I did some benchmarking, and things look quite good for gcj and high
performance computing. All of my testing is on an Athlon 2600+ (stock
clock), 512 MB RAM, Fedora Core 2.

I started with a couple of codes, one is "almabench" from Scott Robert
Ladd's website (www.coyotegulch.com, which appears to be down right
now), and the other is "Scimark" from NIST. Gcj beat JDK 1.5 very
handily in both. "Almabench" does quite a bit of trig, with which the
last couple of Sun JVMs have had problems. Who knows when the 2x
slowdown for trig in those JVMs will be fixed...regardless gcj did even
better than 2x faster. There is a C++ version of almabench also, but I
didn't have time to instrument it for the "per loop timings" I put in
for the Java code (I will later).

To summarize the results, on these benchmarks gcj beat JDK 1.5 by 64% to
~300% overall. gcj was also competitive with gcc, scoring about 76% of
the composite score on scimark, and actually slightly beating it on
almabench. I'm going to look into scimark a bit more to see if I can
understand some of the discrepancies there. I think these results show
the potential of gcj for HPC. I'd like to get gcj 3.4 set up as well as
the latest CVS head...could anyone point me to a good document for
installing multiple gcc versions on the same Linux instance?

(Bryce, thanks for the link to Scimark on your benchmarks page.)

Hope it was interesting,

Terry

---benchmark results follow---
The numbers for the gcj version of almabench vs. the C++ version were
almost identical using the time shell command:
gcj:
real    0m20.449s
user    0m19.983s
sys     0m0.007s

gcc: (CPPFLAGS = -march=athlon -mcpu=athlon -O3 -funroll-loops)
real    0m21.430s
user    0m21.403s
sys     0m0.002s

gcj compile command was:
gcj -o almabench.o -march=athlon -mcpu=athlon -O3 -ffast-math
-funroll-loops -fno-bounds-check  -c almabench.java
(similar flags were used for scibench)

JDK compile command was:
javac -O *.java

[tsikes@photon java]$ gcj --version
gcj (GCC) 3.3.3 20040412 (Red Hat Linux 3.3.3-7)
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[tsikes@photon java]$ ./almabench
starting...
elapsed ms: 1014
elapsed ms: 1014
elapsed ms: 1014
elapsed ms: 1014
elapsed ms: 1014
elapsed ms: 1015
elapsed ms: 1015
elapsed ms: 1014
elapsed ms: 1015
elapsed ms: 1013

[tsikes@photon java]$ java -version
java version "1.5.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_01-b08)
Java HotSpot(TM) Client VM (build 1.5.0_01-b08, mixed mode, sharing)
[tsikes@photon java]$ java -server almabench
starting...
elapsed ms: 3212
elapsed ms: 3041
elapsed ms: 3029
elapsed ms: 3016
elapsed ms: 3018
elapsed ms: 3017
elapsed ms: 3014
elapsed ms: 3017
elapsed ms: 3019
elapsed ms: 3016

For Scimark, the numbers are:

gcj:
SciMark 2.0a

Composite Score: 397.56724916413043
FFT (1024): 412.54507384895
SOR (100x100):   468.38963398340786
Monte Carlo : 26.556732682957968
Sparse matmult (N=1000, nz=5000): 412.0465286948107
LU (100x100): 668.2982766105255

java.vendor: Free Software Foundation, Inc.
java.version: 3.3.3
os.arch: i686
os.name: Linux
os.version: 2.6.8-1.521

JDK 1.5:
SciMark 2.0a

Composite Score: 241.34133437710133
FFT (1024): 119.86260545032472
SOR (100x100):   483.0671768314782
Monte Carlo : 42.28661844722142
Sparse matmult (N=1000, nz=5000): 221.8551119994279
LU (100x100): 339.63515915705466

java.vendor: Sun Microsystems Inc.
java.version: 1.5.0_01
os.arch: i386
os.name: Linux
os.version: 2.6.8-1.521

gcc:
**                                                              **
** SciMark2 Numeric Benchmark, see http://math.nist.gov/scimark **
** for details. (Results can be submitted to pozo@nist.gov)     **
**                                                              **
Using       2.00 seconds min time per kenel.
Composite Score:          522.61
FFT             Mflops:   455.42    (N=1024)
SOR             Mflops:   646.63    (100 x 100)
MonteCarlo:     Mflops:   147.09
Sparse matmult  Mflops:   405.80    (N=1000, nz=5000)
LU              Mflops:   958.13    (M=100, N=100)