C++: EGCS perf. vs GCC

[Alex Maranda]

C++: EGCS performance vs. GCC - myth or reality?
------------------------------------------------

Hi,
Please do not take the title of this posting as a flame intent; I have conducted
yesterday a suite of tests involving gcc and egcs which led to somewhat
confusing results (at least for me). I do not imply in any manner that the
results have any consistency whatsoever for other platforms/setups than the one
described below. I would like to share these results with you and get an expert
oppinion on these figures.

PS: please use also "Reply to author" as I haven't subscribed to any of these
mailing lists.

Cheers,
Alex Maranda

================================================================================

1. Software used for testing purposes

The application is a network simulator I wrote a couple of years ago, for which
I produced benchmarks on various architectures/operating systems. It is a
compute-bound program, which uses heavily the basic semantics of C++ (virtual
functions, multiple inheritance), but not the later additions to the language
(i.e. templates, exceptions). The particular simulation involved is the model
of an ATM statistical multiplexer which does negligible I/O.
All simulation runs had identical conditions, the only variables being the
compilers and the compiler flags. The performance was evaluated after the number
of events per second (ev/sec) processed; obviously, the higher the number, the
better the performance (and the shorter the running time). Timing is as accurate
as it can be, as the program uses gettimeofday(). Multiple runs were performed
for each combination, to average out the variations induced by the OS scheduling
The best result was kept, as the standard deviation was very small. 

2. Hardware/OS

Processor: PII/350 (genuine Intel)
RAM: 64 Mo (not critical because the app is not memory intensive)
OS: RedHat Linux 5.1

3. Compilers used

gcc 2.8.1
egcs 1.1b (yes, I know 1.1.1 is out)

libstdc++ was 2.8.0 for gcc and whatever the version is for egcs (the default
distribution). Shouldn't make any difference though as I'm barely using stuff
from there.

4. Results

gcc 2.8.1
--------------------------------------------------------------------------------
Compiler flags                   Exec. size             Speed
                                 (bytes)  (%)           (ev/sec)  (%)
--------------------------------------------------------------------------------
-O3 -fno-rtti -mcpu=i386         46636    100          781263    100
-O3 -fno-rtti -mcpu=i486         50200    107.6        716301    91.6
-O3 -fno-rtti -mcpu=pentium      47392    101.6        777032    99.4
-O3 -fno-rtti -mcpu=pentiumpro   47344    101.5        725632    92.8
--------------------------------------------------------------------------------


egcs 1.1b
--------------------------------------------------------------------------------
Compiler flags                   Exec. size             Speed
                                 (bytes)  (%)           (ev/sec)  (%)
--------------------------------------------------------------------------------
-O3 -fno-rtti -mcpu=i386         44712    100           745837    100
-O3 -fno-rtti -mcpu=i486         46056    103           774631    103.8
-O3 -fno-rtti -mcpu=pentium      45352    101.4         719423    96.4
-O3 -fno-rtti -mcpu=pentiumpro   45532    101.4         768135    103
--------------------------------------------------------------------------------


Now a speed comparison having the gcc 2.8.1-i386 result as base:
--------------------------------------------------------------------------------
target arch.        gcc 2.8.1 (%)         egcs 1.1b (%)
--------------------------------------------------------------------------------
i386                100                   95.4
i486                94.3                  99.1
pentium             99.4                  92
pentiumpro          92.8                  98.3
--------------------------------------------------------------------------------


4. Conclusions (mine :-)

As a general comment, Pentium II is one hell of a processor; the same benchmark
run on a Sparc Ultra 30/SPARC Ultra II/248 mhz/Solaris 2.6 machine would not
yield more than 530000 ev/sec. More to the point, the above results are
worrying, at least for the Intel instructions scheduler (I understand the RISC
targets use in egcs the Haifa instructions scheduler).
I will state now the obvious, which is gcc 2.8.1 - i386 is best when run on PII!
I recall a posting which said that Pentium is better than Pentium II at running
i486 code, but other than that, the results are a complete surprise for me.
I've expected that the proper use of the PII pipeline, etc. would boost the
figures at least by 10%, and not degrade the performance!
Either I'm doing something terribly wrong (and if so please spot my error!) or
the Pentium scheduler is not doing the job properly.

5. Conclusions (yours :-)
...

Alex Maranda         mailto: amaranda@spider.com
Spider Software Ltd. Tel: +44 (0)131 4757036
Edinburgh, UK        http://members.xoom.com/Alex_Maranda
STREAMS based communications protocols for embedded systems