This is the mail archive of the
mailing list for the GCC project.
C Optimization Tests, 1 May 2004, tree-ssa/3.5/3.4/icc
- From: Scott Robert Ladd <coyote at coyotegulch dot com>
- To: gcc mailing list <gcc at gcc dot gnu dot org>
- Date: Sat, 01 May 2004 18:39:57 -0400
- Subject: C Optimization Tests, 1 May 2004, tree-ssa/3.5/3.4/icc
I've compared the speed of code generated by recent versions of GCC,
using a beta version of a C benchmark suite that I'm developing. The
benchmark suite is currently comprised of eight separate tests, each of
which times its inner loop and reports the result to a driver program.
In the following tables, times are in seconds, as computed for the inner
loop being tested. An asterisk marks the fastest time for a given piece
of benchmark and overall time. The benchmarks are described at the end
of this message.
AMD64/Opteron 240 (1.4GHz)
Gentoo AMD64 64-bit Linux 2.6.5
GCC options: -O3 -ffast-math -march=Opteron
test tree-ssa mainline 3.4.1
---- -------- -------- --------
alma 23.2* 53.5 70.1
evo 32.9 31.1* 32.0
fft 31.7 30.8 30.2*
huff 30.2 24.5 23.8*
lin 29.5* 29.6 30.2
mat1 30.0* 30.5 30.2
mole 30.0 26.3 12.2*
tree 30.0* 37.4 38.0
-------- -------- --------
total 237.7* 263.7 266.7
I am only testing 64-bit code generation on the AMD64.
Intel ia32/Pentium 4 Northwood 2.8GHz
Debian 32-bit Linux 2.6.5
GCC options: -O3 -ffast-math -march=pentium4
ICC options: -O3 -xN -ipo
test tree-ssa mainline 3.4.1 ICC 8.0
---- -------- -------- -------- --------
alma 64.6 64.7 66.0 22.3*
evo 53.5 52.8 52.5 37.7*
fft 28.2 27.2 27.2 30.6
huff 21.8 16.1* 18.2 16.4
lin 19.5 19.2* 19.2* 19.4
mat1 7.6 7.6 7.6 7.5*
mole 31.4 32.8 32.8 5.0*
tree 23.9* 24.7 24.7 27.2
-------- -------- -------- --------
total 250.6 244.8 246.8 166.1
I include the Intel C compiler (ICC) for comparison purposes, and
because so many people ask about it.
While tree-ssa is a clear overall winner overall on the Opteron, it
shows (as does mainline) a big regression on the huff and mole
benchmarks. On the Pentium 4, mainline is the winner on huff, while
tree-ssa still shows the regression; on the other hand, tree-ssa is a
clear winner for mole on the Pentium 4.
On the Opteron, tree-ssa is strong on alma, similar to Intel's dominance
on alma for the Pentium 4.
Intel's compiler turns in some incredibly fast times on mat1 and mole --
and the times appear legitimate. For example, I enabled verification
output for mole, and Intel is producing correct output six times faster
than any of the GCC versions.
I'll be adding more verification routines to future versions of the
benchmarks, to catch compilers that might be optimizing away parts of
the programs. At this point, however, I haven't seen that happen.
USUAL DISCLAIMER AND EXPLANATION STUFF:
The benchmark suite is *not* complete; I will be adding at least two
more tests, along with better automated reporting facilities. If you
would like a copy of the benchmark suite, please request it from me by
e-mail, as it's not ready for general distribution.
I am *not* testing compilation speed.
Please do not ask about other architectures; I don't have them, so I
can't test them. Well, I do have SPARC, but it's old, slow, and no one
asks me about SPARC anyway.
Most users will compile with the highest optimization level possible
under the assumption that it will produce the fastest code. Additional
options (e.g. -funroll-loops) may improve generated code speed; in fact,
it is almost always possible to find a "-O1 and other options" set that
produces faster code than -O3 (see my Acovea articles). HOWEVER, in this
comparison, I'm looking at how general users are going to use the tool
All GNU compilers were taken from anonymous CVS on 2004-05-01, and built
--disable-multilib (Opteron only)
A short description of the benchmarks:
alma -- Calculates the daily ephemeris (at noon) for the years
2000-2099; tests array handling, floating-point math, and mathematical
functions such as sin() and cos().
evo -- A simple genetic algorithm that maximizes a two-dimensional
function; tests 64-bit math, loop generation, and floating-point math.
fft -- Uses a Fast Fourier Transform to multiply two very (very) large
polynomials; tests the C99 _Complex type and basic floating-point math.
huff -- Compresses a large block of data using the Huffman algorithm;
tests string manipulation, bit twiddling, and the use of large memory
lin -- Solves a large linear equation via LUP decomposition; tests basic
floating-point math, two-dimensional array performance, and loop
mat1 -- Multiplies two very large matrices using the brute-force
algorithm; tests loop optimization.
mole -- A molecular dynamics simulation, with performance predicated on
matrix operations, loop efficiency, and sin() and cos(). I recently
added this test, which exhibits very different characteristics from alma
(even if they appear similar).
tree -- Creates and modifies a large B-tree in memory; tests integer
looping, and dynamic memory management.
That's all for now, folks.
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing