This is the mail archive of the
libstdc++@gcc.gnu.org
mailing list for the libstdc++ project.
Some benchmarks for different g++ versions (i686-pc-linux-gnu)
- To: gcc at gcc dot gnu dot org
- Subject: Some benchmarks for different g++ versions (i686-pc-linux-gnu)
- From: Peter Schmid <schmid at snake dot iap dot physik dot tu-darmstadt dot de>
- Date: Thu, 22 Mar 2001 22:19:31 +0100 (CET)
- cc: libstdc++ at gcc dot gnu dot org
I ran some c++ benchmarks for different g++ versions on my system.
The system is based on a Pentium II running at 350 MHz, employs Linux 2.4.2
as operating system and utilizes Glibc 2.2 as C-library.
1) At first I used Bench++ Version 1.1 from
www.research.att.com/~orost/bench_plus_plus.html by Joseph Orost to
compare the performance of three different g++ compilers at different
optimization levels. I used gcc 3.0 20010321, gcc 3.1 20010318 and gcc
2.95.2, which is the system compiler on SuSE 7.1 (gpp-2.95.2-149) each
at "-O2" and "-O3 -funroll-loops". Here are the results for all runs
in comparision to the performance of gcc-2.95.2 at -O2:
RELATIVE TIMES ..........
TEST NAME gcc 2.95.2 gcc 2.95.2 gcc 3.0 gcc 3.0 gcc 3.1 gcc 3.1
19991024 19991024 20010321 20010321 20010318 20010318
-O2 -O3 -O2 -O3 -O2 -O3
-funroll- -funroll- -funroll-
loops loops loops
--------- ---------- ---------- ---------- ---------- ---------- ----------
A000091 1.00 0.97 0.97 0.96 0.98 0.99
A000092 1.00 1.00 1.06 1.04 1.05 1.04
A000094a 1.00 1.05 1.00 1.07 1.01 1.05
A000094b 1.00 1.00 0.89 0.90 0.91 0.89
A000094c 1.00 1.03 1.01 1.00 0.97 0.97
A000094d 1.00 0.65 0.96 0.67 0.95 0.66
A000094e 1.00 0.90 1.07 0.90 1.05 0.88
A000094f 1.00 0.92 0.98 0.98 1.02 1.02
A000094g 1.00 0.96 0.96 0.97 0.96 0.97
A000094h 1.00 0.91 0.90 0.91 0.90 1.41
A000094i 1.00 0.98 0.66 0.67 0.66 0.67
A000094j 1.00 1.14 1.34 1.27 1.42 1.27
A000094k 1.00 1.00 0.60 0.60 0.60 0.60
B000002b 1.00 1.01 1.00 0.96 1.06 1.05
B000003b 1.00 1.02 0.95 0.88 0.96 0.95
B000004b 1.00 0.99 1.00 0.95 0.98 0.98
B000010 1.00 0.97 1.01 0.99 1.00 1.00
B000011 1.00 0.99 0.95 0.91 0.99 0.90
B000013 1.00 0.45 0.93 0.47 0.93 0.50
D000001 1.00 1.00 0.99 1.08 0.99 1.05
D000002 1.00 1.10 0.58 0.39 0.59 0.39
D000003 1.00 1.01 1.05 1.19 1.06 1.07
D000004 1.00 0.99 0.54 0.39 0.54 0.39
D000005 1.00 1.00 0.28 0.19 0.27 0.18
D000006 1.00 1.00 0.50 0.38 0.50 0.38
E000001 1.00 0.99 0.96 0.97 0.88 0.88
E000002 1.00 1.01 0.51 0.52 0.49 0.49
E000003 1.00 0.60 0.47 0.47 0.44 0.44
E000004 1.00 0.50 0.40 0.40 0.37 0.37
E000006 1.00 0.98 -- -- -- --
E000007 1.00 0.94 1.15 1.11 1.12 1.08
E000008 1.00 1.31 1.26 1.20 1.25 1.16
F000001 1.00 0.87 0.14 0.14 0.14 0.14
F000002 1.00 0.90 0.67 0.66 0.67 0.66
F000003 1.00 0.78 0.95 0.92 0.95 0.92
F000004 1.00 1.28 1.00 0.98 1.01 0.97
F000005 1.00 1.05 0.77 0.77 0.77 0.78
F000006 1.00 1.03 0.32 0.62 0.32 0.62
F000007 1.00 1.29 0.78 1.30 0.78 1.31
F000008 1.00 0.90 0.33 0.56 0.33 0.57
G000001 1.00 0.98 2.98 3.04 2.89 2.95
G000002 1.00 0.98 0.84 0.85 0.75 0.76
G000003 1.00 1.05 1.27 1.29 1.21 1.21
G000004 1.00 0.98 2.00 1.98 1.75 1.76
G000005 1.00 1.07 4.51 4.33 3.91 3.82
G000006 1.00 1.01 1.33 1.31 1.30 1.35
G000007 1.00 0.99 3.04 3.02 2.72 2.81
H000001 1.00 1.00 0.97 0.97 0.96 0.96
H000002 1.00 0.99 1.01 1.01 1.01 1.01
H000003 1.00 1.28 0.59 0.63 -- --
H000004 1.00 0.92 0.42 0.41 0.43 0.39
H000005 1.00 -- 0.00 -- 0.00 --
H000006 1.00 1.09 1.14 1.11 0.22 0.19
H000007 1.00 0.88 2.08 2.12 2.08 2.12
H000008 1.00 1.02 0.37 0.40 0.37 0.44
H000009 1.00 0.98 0.58 0.64 0.62 0.64
L000001 1.00 0.29 2.70 0.44 2.35 0.16
L000002 1.00 0.29 2.72 0.44 2.33 0.17
L000003 1.00 0.98 2.17 0.15 2.17 0.16
L000004 1.00 0.24 2.94 0.69 2.33 0.00
O000001a 1.00 1.00 0.89 0.91 0.89 0.93
O000001b 1.00 0.99 0.89 0.92 0.89 0.92
O000002a 1.00 1.08 1.00 0.95 0.82 0.84
O000002b 1.00 0.96 0.97 0.94 0.97 0.94
O000003a 1.00 0.94 0.89 0.85 0.91 0.86
O000003b 1.00 1.02 0.97 0.96 0.83 0.86
O000004a 1.00 0.55 1.02 0.43 1.02 0.43
O000004b 1.00 0.55 1.02 0.43 1.02 0.43
O000005a 1.00 0.96 0.69 0.76 0.69 0.69
O000005b 1.00 1.00 0.41 0.24 0.41 0.23
O000006a 1.00 1.00 0.67 0.60 0.66 0.58
O000006b 1.00 0.98 0.74 0.63 0.74 0.60
O000007a 1.00 1.00 1.09 1.10 1.08 1.08
O000007b 1.00 0.98 1.01 1.00 1.01 1.00
O000008a 1.00 0.42 0.85 0.56 0.85 0.56
O000008b 1.00 0.31 0.78 0.13 0.78 0.13
O000009a 1.00 0.99 0.73 0.73 0.69 0.70
O000009b 1.00 1.00 0.77 0.78 0.74 0.75
O000010a 1.00 0.96 2.12 0.85 1.04 0.74
O000010b 1.00 0.13 0.94 0.03 0.94 0.03
O000011a 1.00 0.38 1.14 0.36 1.14 0.35
O000011b 1.00 0.13 0.83 0.09 0.82 0.09
O000012a 1.00 1.08 1.84 1.84 1.84 1.84
O000012b 1.00 0.99 0.99 0.98 0.99 0.98
P000001 1.00 0.00 0.62 0.00 0.61 0.00
P000002 1.00 0.98 0.62 0.60 0.61 0.61
P000003 1.00 0.98 0.87 0.90 0.87 0.90
P000004 1.00 inf nan nan nan nan
P000005 1.00 1.01 1.06 1.03 1.05 1.02
P000006 1.00 1.01 0.89 0.84 0.89 0.84
P000007 1.00 1.04 0.98 0.89 0.98 0.89
P000008 1.00 1.07 1.17 1.11 1.17 1.11
P000010 1.00 0.94 1.03 0.96 1.03 0.96
P000011 1.00 1.01 1.11 1.12 1.11 1.12
P000012 1.00 1.01 0.89 0.84 0.89 0.84
P000013 1.00 1.02 0.93 0.91 0.93 0.91
P000020 1.00 1.03 0.91 0.91 0.91 0.91
P000021 1.00 2.07 2.77 1.18 5.49 5.47
P000022 1.00 2.05 2.81 1.20 3.42 5.47
P000023 1.00 1.26 1.08 1.29 1.18 1.29
S000001a 1.00 0.97 0.76 0.64 0.76 0.66
S000001b 1.00 0.97 0.62 0.56 0.62 0.56
S000002a 1.00 0.93 0.90 0.86 0.90 0.88
S000002b 1.00 0.90 1.05 0.91 1.03 0.93
S000003a 1.00 0.91 0.58 0.66 0.57 0.65
S000003b 1.00 0.96 0.98 1.09 0.96 1.08
S000004a 1.00 0.99 1.38 1.30 1.39 1.44
S000004b 1.00 0.98 0.98 1.19 0.99 1.75
S000005a 1.00 1.06 1.32 1.32 1.32 1.32
S000005b 1.00 0.65 1.41 0.66 1.41 0.65
S000005c 1.00 0.78 0.99 1.00 1.00 1.00
S000005d 1.00 0.65 1.42 0.67 1.42 0.65
S000005e 1.00 0.99 1.30 1.31 1.29 1.30
S000005f 1.00 0.61 1.50 0.63 1.49 0.61
S000005g 1.00 0.99 1.30 1.31 1.29 1.30
S000005h 1.00 0.60 1.50 0.61 1.49 0.61
S000005i 1.00 0.77 0.99 1.00 0.99 1.00
S000005j 1.00 0.65 1.41 0.65 1.40 0.65
S000005k 1.00 1.05 1.08 0.97 1.07 0.97
S000005l 1.00 0.64 1.43 0.66 1.41 0.67
S000005m 1.00 0.66 1.43 0.67 1.44 0.67
Some remarks: Gcc 3.0 and gcc 3.1 do not compile e000006.cpp.
That is because of the class definition (simplified):
class exception_6 {
public:
bool flag;
void proc_1(void) throw(exception_6);
};,
a class that "throws itself" is rejected. I am not shure, whether this
is legal code or not. g++ 3.1 miscompiles h000003.cpp at optimization
levels "-O2" or higher. g++ 2.95 uses "CNTRL-@" as a fill character
for output although a space should be used.
_______________________________________________________________________________
2) As second test I employed Haney Kernels Version 1.4 from
http://www.kai.com/haney/haney.html. The configuration was the same
as for the previous test. As optimization level "-O2" was chosen.
g++ 2.95.2
Timing Complex matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (-674806,-8.91457e+06): 2.53 0.1012
Hand-coded C (-674806,-8.91457e+06): 1.39 0.0556
Fortran (-674806,-8.91457e+06): 1.33 0.0532
Timing Real matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (152005): 5.72 0.0572
Hand-coded C (152005): 2.02 0.0202
Fortran (152005): 2.15 0.0215
Timing Vector operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (0.000100917): 7.8 0.000156
Hand-coded C (0.000100917): 3.86 7.72e-05
Fortran (0.000100917): 4.03 8.06e-05
g++ 3.0 20010321
Timing Complex matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (-674806,-8.91457e+06): 1.91 0.0764
Hand-coded C (-674806,-8.91457e+06): 1.82 0.0728
Fortran (-674806,-8.91457e+06): 1.31 0.0524
Timing Real matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (152005): 2.46 0.0246
Hand-coded C (152005): 2.99 0.0299
Fortran (152005): 2.13 0.0213
Timing Vector operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (0.000100917): 8.21 0.0001642
Hand-coded C (0.000100917): 4.57 9.14e-05
Fortran (0.000100917): 4.36 8.72e-05
g++ 3.1 20010318
Timing Complex matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (-674806,-8.91457e+06): 1.84 0.0736
Hand-coded C (-674806,-8.91457e+06): 1.85 0.074
Fortran (-674806,-8.91457e+06): 1.37 0.0548
Timing Real matrix operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (152005): 2.51 0.0251
Hand-coded C (152005): 3.01 0.0301
Fortran (152005): 2.12 0.0212
Timing Vector operations for n = 100
Language Answer Check Time Iter Time
O-O C++ (0.000100917): 8.18 0.0001636
Hand-coded C (0.000100917): 4.49 8.98e-05
Fortran (0.000100917): 3.99 7.98e-05
_______________________________________________________________________________
3) As third benchmark I ran OOPACK Version 1.7 from
http://www.kai.com/oopack/oopack.html. The configuration was the same
as for the previous tests. As optimization level "-O2" was chosen. The
tests were run with the command line a.out Max=50000 Matrix=500
Complex=20000 Iterator=50000.
gcc 2.95.2
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Max 50000 1.3 1.4 38.2 34.5 1.1
Matrix 500 1.0 1.2 126.3 107.8 1.2
Complex 20000 1.4 4.7 115.1 34.0 3.4
Iterator 50000 0.6 1.1 181.8 93.5 1.9
gcc 3.0 20010321
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Max 50000 0.8 1.1 61.0 45.5 1.3
Matrix 500 1.0 1.1 120.2 117.9 1.0
Complex 20000 1.8 5.4 87.4 29.4 3.0
Iterator 50000 0.6 0.6 181.8 158.7 1.1
gcc 3.1 20010318
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Max 50000 0.8 1.1 61.0 45.5 1.3
Matrix 500 1.0 1.1 122.5 119.0 1.0
Complex 20000 1.8 5.7 87.4 28.1 3.1
Iterator 50000 0.5 0.6 185.2 158.7 1.2
-------------------------------------------------------------------------------
4) And finally KAI's version of Stepanov Benchmark -- Version 1.2,
obtained from http://www.kai.com/benchmarks/stepanov/, was chosen as a
benchmark. The configuration was the same as for the previous
tests. As optimization level "-O2" was chosen.
gcc 2.95.2
test absolute additions ratio with
number time per second test0
0 0.45sec 111.11M 1.00
1 0.46sec 108.70M 1.02
2 0.47sec 106.38M 1.04
3 0.46sec 108.70M 1.02
4 0.58sec 86.21M 1.29
5 0.46sec 108.70M 1.02
6 0.46sec 108.70M 1.02
7 0.46sec 108.70M 1.02
8 0.46sec 108.70M 1.02
9 0.45sec 111.11M 1.00
10 0.59sec 84.75M 1.31
11 0.46sec 108.70M 1.02
12 0.46sec 108.70M 1.02
mean: 0.48sec 104.93M 1.06
Total absolute time: 6.22 sec
Abstraction Penalty: 1.06
gcc 3.0 20010321
test absolute additions ratio with
number time per second test0
0 0.45sec 111.11M 1.00
1 0.50sec 100.00M 1.11
2 0.47sec 106.38M 1.04
3 0.58sec 86.21M 1.29
4 0.49sec 102.04M 1.09
5 0.58sec 86.21M 1.29
6 0.52sec 96.15M 1.16
7 0.58sec 86.21M 1.29
8 0.51sec 98.04M 1.13
9 0.59sec 84.75M 1.31
10 0.52sec 96.15M 1.16
11 0.59sec 84.75M 1.31
12 0.60sec 83.33M 1.33
mean: 0.53sec 93.53M 1.19
Total absolute time: 6.98 sec
Abstraction Penalty: 1.19
gcc 3.1 20010318 (experimental)
test absolute additions ratio with
number time per second test0
0 0.45sec 111.11M 1.00
1 0.50sec 100.00M 1.11
2 0.46sec 108.70M 1.02
3 0.58sec 86.21M 1.29
4 0.51sec 98.04M 1.13
5 0.58sec 86.21M 1.29
6 0.51sec 98.04M 1.13
7 0.58sec 86.21M 1.29
8 0.53sec 94.34M 1.18
9 0.58sec 86.21M 1.29
10 0.49sec 102.04M 1.09
11 0.59sec 84.75M 1.31
12 0.59sec 84.75M 1.31
mean: 0.53sec 93.93M 1.18
Total absolute time: 6.95 sec
Abstraction Penalty: 1.18
Futhermore, I note that gcc 2.95 compiles the code, subjectively, much
faster.
Hope this helps.
Peter Schmid