Results for g++ 3.1 application testing on i686-pc-linux-gnu

[Peter Schmid <schmid at snake dot iap dot physik dot tu-darmstadt dot de>]

I have mixed feelings with the present state of the gcc 3.1 release
candidate. Although there are no regressions with respect to previous
g++ releases concerning conformance to the c++ standard, there are
still, imho severe, regressions with respect to code quality and
the optimizer. Higher optimization levels lead to a pessimization with
respect to -O in some cases, confer the results for the Stepanov and
the oopack tests. This is a regression with respect to gcc 2.95.

Running the test script, tester.pl, from pooma-gcc takes 594.4
seconds user time for gcc 3.0 in contrast to 797.3 seconds for gcc
3.1. In my opinion these compile and code quality regressions should
be resolved before gcc 3.1 is released.

Additionally, I got no feedback on PR fortran/5900, concerning LAPACK
which could be a severe regression.

Here are the results in detail:

stepanov_v1p2.C:

-O0
Abstraction Penalty: 8.45
-O
Abstraction Penalty: 1.06
-O2
Abstraction Penalty: 1.27
-O3
Abstraction Penalty: 1.24
-O2  -finline-limit=10000
Abstraction Penalty: 1.24
-O2  -finline-limit=100000
Abstraction Penalty: 1.27

OOPACK Version 1.7:
Max=100000 Matrix=1000 Complex=100000 Iterator=100000

-O
                         Seconds       Mflops
Test       Iterations     C    OOP     C    OOP  Ratio
----       ----------  -----------  -----------  -----
Max            100000    1.7   2.1   59.9  48.3    1.2
Matrix           1000    3.5   3.9   70.6  63.6    1.1
Complex        100000    9.4  17.9   84.7  44.7    1.9
Iterator       100000    1.1   2.0  190.5 100.0    1.9

-O2
                         Seconds       Mflops
----       ----------  -----------  -----------  -----
Max            100000    2.0   2.0   50.8  50.8    1.0
Matrix           1000    2.0   2.1  125.0 119.6    1.0
Complex        100000    9.4  26.2   85.1  30.5    2.8
Iterator       100000    1.1   1.3  180.2 157.5    1.1

Max(C) and Complex(OOP) are slower at -O2; Complex is considerably
slower.

current boost cvs:
All tests pass.

blitz-20001213:
All tests pass.

root_v3.03.07:
Everything works, except for test/bench.cxx, as for gcc 3.0.

pooma-gcc:
No problems detected. All tests pass.

Code from Josuttis' Book "The C++ Standard Library":

Everything works fine, except for the unresolved state of PR libstc++/5133.

mtl-2.1.2-20:
No problems detected, after applying patches for missing typename
warnings.

STLport-4.5.3/STLport-5.0-0409
No problems detected. All tests pass.

FTensor--main--1.1--patch-16:
Problems with __restrict__, cf. PR. c++/6392, already fixed on the mainline.
Regression with respect to kcc and icc. Otherwise OK.

ACE 5.2.1:
No problems detected. All tests pass.

System setup:
SuSE 7.3
Glibc 2.2.4 + patches
Linux 2.4.18
GNU ld version 020428 20020428
g++ -v
Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/3.1/specs
Configured with: ../gcc/configure --enable-shared --disable-nls --enable-threads=posix --enable-languages=c,c++,f77,objc
Thread model: posix
gcc version 3.1 20020505 (prerelease)

Hope this helps,

Peter Schmid