Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[poschmid at lbl dot gov]

I ran many tests on my i686-pc-linux-gnu computer, the CPU is
a Pentium IV with a 1.8 GHz clock. 

I tested the following applications: Blitz++-0.7, FTensor-1.1pre25,
STLport-4.6.1, STLport-5.0-0409, qt-x11-free-3.3.0, root_v3.05.01,
root_v3.10.02, root_v4.00.02, cln-1.1.5, ACE-5.4+TAO-1.4+CIAO-0.4, stepanov,
oopack, mtl-2.1.2-21, estl, bench++, boost, the code examples that are
included in the books by Breymann, Josuttis and Vandevoorde and other 
applications.

After applying some patches of my own, the remaining problems are:

1)qt-x11-free-3.3.0 does not build (see PR 14400). This problem can be worked
around by configuring with -system-zlib -system-libpng enabled. Although the
build completes when configured this way, many applications crash on shutdown
with a segmentation fault.
2) pooma-gcc does not build (see PR 14545).  
3) There are 10 failures while running the current boost regression test
suite. All of them might be regressions since gcc 3.3 accepts the code which
is rejected by gcc 3.4.  
4) Some test examples of ACE and TAO fail.  
5) Because of recent changes in libstdc++, current root snapshots no longer
compile. root_v3.10.02 is the last release which can be build successfully.
6) The complex oop test included in oopack runs considerably slower when
compiled by gcc 3.4. It is 55.9 Mflops for gcc 3.3 but 40.7 Mflops for gcc
3.4. Additionally, there is still a substantial penalty for using oop methods
in contrast to c based ones (720 vs 40 Mflops).
  
Here are some of the results in detail:

qt-x11-free-3.3.0:
This is a problem with precompiled headers causing the C compiler to
crash. (See PR 14400)

pooma-gcc:
Since a week or so compiling most of the files included in the pooma-gcc
repository causes gcc to ice (see PR 14545).

Current boost cvs: 
Ten tests fail:
disjoint_sets/disjoint_set_test.cpp
graph/test/adj_list_cc.cpp
graph/test/subgraph.cpp
lambda/test/constructor_tests.cpp
lambda/test/operator_tests_simple.cpp
lambda/test/phoenix_control_structures.cpp
lambda/test/switch_construct.cpp
python/test/embedding.cpp
python/test/select_arg_to_python_test.cpp
thread/test/test_thread.cpp

Four of them are caused by recent changes in the libstdc++ implementation:
operator_tests_simple, phoenix_control_structures, switch_construct,
constructor_tests, all included in the lambda portion of boost, are rejected,
since the following code

 namespace std { 
 template <class T, class Allocator> class vector; 
} 
 
#include <vector> 
 using namespace std; 
 int main()  
{ 
    vector<int> v; 
} 

is rejected by gcc 3.4 but accepted by other compilers. I am not sure whether
Gaby's point of view (see PR 14370) is the way to proceed on this
subject. select_arg_to_python_test does not work because the linker cannot
find the symbol _Py_NoneStruct. This may be a local problem.

ACE-5.4+TAO-1.4+CIAO-0.4 :
The whole Ace, Tao and CIAO distribution compiles after applying some
patches. I ran all of the included tests which supplied an executable
run_test.pl. The Multicast_Test and the Proactor_Test of ACE fail. For the
tests Client_Leaks, Connection_Failure, BiDirectional_NestedUpcall,
Oneway_Buffering and Server_Leaks of TAO run_test did not return true.

root_v3.05.01:
Although gcc 3.4 compiles this code distribution, the stress test does not
pass. This is a regression with respect to gcc 3.3. All newer versions of root
do not compile. The file gcc3strm.cxx located in the root/cint/src directory
is rejected because of changes in the libstdc++ implementation.

gcc 3.4
OOPACK Version 1.7:
g++ -O3
Max=100000 Matrix=1000 Complex=100000 Iterator=100000

Max            100000    1.2   1.5   84.7  65.8    1.3
Matrix           1000    0.4   0.4  641.0 641.0    1.0
Complex        100000    1.1  19.7  720.7  40.7   17.7
Iterator       100000    0.3   0.3  689.7 645.2    1.1

gcc 3.3
OOPACK Version 1.7:
g++ -O3
Max=100000 Matrix=1000 Complex=100000 Iterator=100000

Max            100000    1.2   1.5   84.0  65.4    1.3
Matrix           1000    0.4   0.4  641.0 657.9    1.0
Complex        100000    1.1  14.3  733.9  55.9   13.1
Iterator       100000    0.3   0.3  666.7 666.7    1.0

FTensor--main--1.1pre25: No problems detected. All tests pass. On some
speed tests the performance of the optimizer is quite poor. Whereas
for the first, sixth, seventh, eighth and ninth tests the tensor based
implementation runs faster.

CXXFLAGS= -ftemplate-depth-1000 -Drestrict=__restrict__ -O3
-finline-functions -finline-limit-1000 -funroll-loops 

fast 1

real	0m0.901s
user	0m0.889s
sys	0m0.003s
Tensor 1

real	0m0.755s
user	0m0.742s
sys	0m0.006s
fast 2

real	0m1.956s
user	0m1.905s
sys	0m0.006s
Tensor 2

real	0m3.789s
user	0m3.103s
sys	0m0.021s
fast 3

real	0m0.456s
user	0m0.415s
sys	0m0.006s
Tensor 3

real	0m1.926s
user	0m1.896s
sys	0m0.006s
fast 4

real	0m0.644s
user	0m0.639s
sys	0m0.003s
Tensor 4

real	0m4.079s
user	0m4.007s
sys	0m0.018s
fast 5

real	0m0.962s
user	0m0.914s
sys	0m0.004s
Tensor 5

real	0m5.948s
user	0m5.646s
sys	0m0.024s
fast 6

real	0m6.680s
user	0m6.560s
sys	0m0.034s
Tensor 6

real	0m5.671s
user	0m5.640s
sys	0m0.023s
fast 7

real	0m13.301s
user	0m13.236s
sys	0m0.045s
Tensor 7

real	0m12.299s
user	0m12.245s
sys	0m0.039s
fast 8

real	0m28.533s
user	0m25.385s
sys	0m0.328s
Tensor 8

real	0m21.787s
user	0m20.313s
sys	0m0.218s
fast 9

real	0m37.726s
user	0m37.411s
sys	0m0.127s
Tensor 9

real	0m31.076s
user	0m30.941s
sys	0m0.101s


stepanov_v1p2.C: 

-O2
test      absolute   additions      ratio with
number    time       per second     test0

 0        0.14sec    357.14M         1.00
 1        0.14sec    357.14M         1.00
 2        0.16sec    312.50M         1.14
 3        0.14sec    357.14M         1.00
 4        0.14sec    357.14M         1.00
 5        0.15sec    333.33M         1.07
 6        0.14sec    357.14M         1.00
 7        0.14sec    357.14M         1.00
 8        0.15sec    333.33M         1.07
 9        0.15sec    333.33M         1.07
10        0.14sec    357.14M         1.00
11        0.14sec    357.14M         1.00
12        0.15sec    333.33M         1.07
mean:     0.14sec    346.07M         1.03

Total absolute time: 1.88 sec

Abstraction Penalty: 1.03

-O3

test      absolute   additions      ratio with
number    time       per second     test0

 0        0.14sec    357.14M         1.00
 1        0.14sec    357.14M         1.00
 2        0.14sec    357.14M         1.00
 3        0.14sec    357.14M         1.00
 4        0.14sec    357.14M         1.00
 5        0.15sec    333.33M         1.07
 6        0.14sec    357.14M         1.00
 7        0.14sec    357.14M         1.00
 8        0.14sec    357.14M         1.00
 9        0.14sec    357.14M         1.00
10        0.14sec    357.14M         1.00
11        0.14sec    357.14M         1.00
12        0.15sec    333.33M         1.07
mean:     0.14sec    353.37M         1.01

Total absolute time: 1.84 sec

Abstraction Penalty: 1.01


System setup:
SuSE 9
Glibc 2.3.2 
Linux 2.4.21
GNU ld version 2.14.90.0.5
g++ -v
Reading specs from /usr/local/lib/gcc/i686-pc-linux-gnu/3.4.0/specs
Configured with: ../gcc/configure --enable-threads=posix --enable-languages=c,c++,f77,objc --enable-__cxa_atexit --enable-libstdcxx-debug
Thread model: posix
gcc version 3.4.0 20040313 (prerelease)
Hope this helps,

Peter Schmid