Why did performance plummet between egcs-1.1.2 and GCC-2.95?
James A. Bednar
jbednar@cs.utexas.edu
Sat Apr 1 23:26:00 GMT 2000
Howdy!
I've generally found that GCC gets even better with each new release,
but I was dismayed to discover today that performance on our main
research code benchmark dropped by nearly 70% somewhere between
egcs-1.1.2 and GCC-2.95. It remains that bad as of GCC-2.95-2. I see
a difference only when optimization is turned on (-O and higher), so
my best hypothesis is that some specific optimization was discovered
to be unsafe and was disabled in GCC-2.95. Does anyone know of
something like that?
To make it concrete, here are some example timings for our
locally-defined benchmark, which consists of a large neural-network
simulation program:
Compiler Compiler Run times on machine pale
version host
EGCS 1.1.2 ale 365.740u 0.140s 6:06.06 99.9%
GCC 2.7.2.3 hardhat 370.150u 0.140s 6:10.81 99.8%
GCC 2.95.2 pale 606.070u 0.140s 10:06.46 99.9%
GCC 2.95.1 hardhat 613.880u 0.110s 10:14.14 99.9%
As you can see, GCC versions before 2.95 result in substantially lower
runtimes (~370sec) than later versions (~600sec). (In actual use this
program runs nonstop for several hours or days, so the difference here
is highly significant.) I've also briefly tried to compile with the
CVS version from a couple of weeks ago see if the problem has already
been fixed, but that version appears quite broken and the executables
it compiles generate wildly incorrect results.
Ordinarily I'd try to narrow down a simple test case that demonstrates
the problem, but a) we deliberately use a large, realistic test case
for all performance benchmarks, and b) I'm hoping that someone will
know the solution already, such as some new compiler switch I need to
enable :-). Even so, anyone is welcome to try the benchmark out for
themselves, such as it is. It's archived here temporarily (filesize
206KB; covered by GPL):
http://www.cs.utexas.edu/users/jbednar/lissom2.1_benchmark.tar.gz
Just unpack it, cd to "lissom/production", and run "make clean
shorttest". This will run a one-minute test instead of the above
10-minute testes; this should be more convenient and almost as
precise. Sample output is attached below from running the short test
case with GCC 2.95 and GCC 2.7.2.3 on the same machine.
Thanks,
Jim
-------------------------------------------------------------------------------
bruichladdich:/u/nn/models/lissom/production> cat /proc/version
Linux version 2.2.14 (root@bushmills.cs.utexas.edu) (gcc version 2.95.1 19990816 (release)) #1 SMP Thu Mar 2 16:41:32 CST 2000
bruichladdich:/u/nn/models/lissom/production> cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 7
model name : Pentium III (Katmai)
stepping : 3
cpu MHz : 598.505452
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
sep_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat pse36 mmx fxsr xmm
bogomips : 596.38
bruichladdich:/u/nn/models/lissom/production> rm -f *.o ; env CC=/lusr/bin/gcc make -e shorttest
env CC="/lusr/bin/gcc" DEBUGGING="-DNDEBUG" NPES="1" NMAX="64" MAX_INH_RADIUS="24" MAX_EXC_RADIUS="15" MAX_NUM_EYES="2" SRC_OPTS="" WARNINGS="-Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith" OPTIMS="-DNUM_EYES_IS_CONSTANT -O2 -funroll-loops" DEP=../src/ INCLUDES="-I../src/" LIBS="-lm" LDR_OPTS="-Xlinker -defsym -Xlinker MAIN__=main" make -e -f ../src/Makefile lissom
make[1]: Entering directory `/v/hank/v102/nn/models/lissom/production'
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ind_types.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ipc.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/cmdparam.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/lissom.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/globals.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/kernel.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/inputs.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/binarysave.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/file_io.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ppm_draw.c
../src/ppm_draw.c: In function `ppm_draw_afferent_weights':
../src/ppm_draw.c:1096: warning: suggest explicit braces to avoid ambiguous `else'
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/analyze.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/fit_gauss.c
/lusr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/tilt.c
rm -f lissom
/lusr/bin/gcc -Xlinker -defsym -Xlinker MAIN__=main -DNDEBUG -o lissom ind_types.o ipc.o cmdparam.o lissom.o globals.o kernel.o inputs.o binarysave.o file_io.o ppm_draw.o analyze.o fit_gauss.o tilt.o -lm
make[1]: Leaving directory `/v/hank/v102/nn/models/lissom/production'
/lusr/bin/gcc -v
Reading specs from /lusr/gnu/lib/gcc-lib/i686-pc-linux-gnu/2.95.1/specs
gcc version 2.95.1 19990816 (release)
time ./lissom --command "training 1" --command "training 10" --command "training 100"
Execution of ./lissom started Sun Apr 2 00:33:08 2000 on bruichladdich.cs.utexas.edu.
58MB (max) required for weight storage.
Initializing 15155200-connection (58MB) network (see lissom.init.param for parameters).
Command 2 (init_network) took 5 seconds, completing Sun Apr 2 00:33:13 2000.
Training until iteration 1.
Command 1 (training) took 1 seconds, completing Sun Apr 2 00:33:14 2000.
Training until iteration 10.
Command 4 (training) took 5 seconds, completing Sun Apr 2 00:33:19 2000.
Training until iteration 100.
Command 5 (training) took 56 seconds, completing Sun Apr 2 00:34:15 2000.
Exiting -- Exited normally. There were 0 error(s) and 0 warning(s).
real 1m6.566s
user 1m6.070s
sys 0m0.380s
86.220u 0.810s 1:28.38 98.4% 0+0k 0+0io 14699pf+0w
bruichladdich:/u/nn/models/lissom/production> rm -f *.o ; env CC=/usr/bin/gcc make -e shorttest
env CC="/usr/bin/gcc" DEBUGGING="-DNDEBUG" NPES="1" NMAX="64" MAX_INH_RADIUS="24" MAX_EXC_RADIUS="15" MAX_NUM_EYES="2" SRC_OPTS="" WARNINGS="-Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith" OPTIMS="-DNUM_EYES_IS_CONSTANT -O2 -funroll-loops" DEP=../src/ INCLUDES="-I../src/" LIBS="-lm" LDR_OPTS="-Xlinker -defsym -Xlinker MAIN__=main" make -e -f ../src/Makefile lissom
make[1]: Entering directory `/v/hank/v102/nn/models/lissom/production'
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ind_types.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ipc.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/cmdparam.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/lissom.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/globals.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/kernel.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/inputs.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/binarysave.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/file_io.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/ppm_draw.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/analyze.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/fit_gauss.c
/usr/bin/gcc -c -DNPES=1 -DNMAX="64" -DMAX_EXC_RADIUS="15" -DMAX_INH_RADIUS="24" -DMAX_NUM_EYES="2" -DNDEBUG -Wall -W -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Winline -Wwrite-strings -Wshadow -Wcast-align -Wcast-qual -Wpointer-arith -DNUM_EYES_IS_CONSTANT -O2 -funroll-loops -I../src/ ../src/tilt.c
rm -f lissom
/usr/bin/gcc -Xlinker -defsym -Xlinker MAIN__=main -DNDEBUG -o lissom ind_types.o ipc.o cmdparam.o lissom.o globals.o kernel.o inputs.o binarysave.o file_io.o ppm_draw.o analyze.o fit_gauss.o tilt.o -lm
make[1]: Leaving directory `/v/hank/v102/nn/models/lissom/production'
/usr/bin/gcc -v
Reading specs from /usr/lib/gcc-lib/i486-linux/2.7.2.3/specs
gcc version 2.7.2.3
time ./lissom --command "training 1" --command "training 10" --command "training 100"
Execution of ./lissom started Sun Apr 2 00:36:26 2000 on bruichladdich.cs.utexas.edu.
58MB (max) required for weight storage.
Initializing 15155200-connection (58MB) network (see lissom.init.param for parameters).
Command 3 (testing) took 1 seconds, completing Sun Apr 2 00:36:33 2000.
Command 2 (init_network) took 7 seconds, completing Sun Apr 2 00:36:33 2000.
Training until iteration 1.
Training until iteration 10.
Command 4 (training) took 3 seconds, completing Sun Apr 2 00:36:36 2000.
Training until iteration 100.
Command 5 (training) took 34 seconds, completing Sun Apr 2 00:37:10 2000.
Exiting -- Exited normally. There were 0 error(s) and 0 warning(s).
real 0m44.119s
user 0m43.670s
sys 0m0.370s
54.360u 0.770s 0:56.35 97.8% 0+0k 0+0io 13087pf+0w
bruichladdich:/u/nn/models/lissom/production>
More information about the Gcc-bugs
mailing list