3.2.2 & 3.3 branch & trunk compilation speed comparison (MICO)

[Karel Gardas <kgardas at objectsecurity dot com>]

Hello,

to just not criticize, but to post some usefull data, I've made some tests
with using -time  with compiling core of MICO (http://www.mico.org).

I've compiled only orb directory - there are about 78671 SLOC there.
All compilation was configured the same way and takes:

gcc3.2.2
real    12m52.481s
user    12m31.620s
sys     0m12.590s

gcc3.3
real    16m11.003s
user    15m45.250s
sys     0m13.740s

gcc3.4
real    14m25.133s
user    13m53.470s
sys     0m19.300s

Since I've used -time option (Great that Zack Weinberg mentioned it
yesterday) I've been able to compare times for every file.

Table looks:

File		3.2.2 	3.3 	3.2/3d	3.4	3.2/4d	3.3/4d
--------------------------------------------------------------
os-unix.cc	5.67	7.19	26.81	6.59	16.23	-8.34
dii.cc		17.84	22.14	24.1	19.4	8.74	-12.38
typecode.cc	11.63	14.12	21.41	13.77	18.4	-2.48
any.cc		9.02	10.76	19.29	10.53	16.74	-2.14
codec.cc	7.48	9.57	27.94	9.3	24.33	-2.82
buffer.cc	4.54	5.66	24.67	5.52	21.59	-2.47
context.cc	4.84	6.16	27.27	6.02	24.38	-2.27
except.cc	6.06	7.48	23.43	7.07	16.67	-5.48
dispatch.cc	5.98	7.51	25.59	7.02	17.39	-6.52
string.cc	4.65	6.06	30.32	5.54	19.14	-8.58
object.cc	6.37	8.32	30.61	7.69	20.72	-7.57
address.cc	6.87	8.36	21.69	8.11	18.05	-2.99
ior.cc		16.72	21.59	29.13	18.81	12.5	-12.88
orb.cc		22.93	28.08	22.46	24.99	8.98	-11
boa.cc		11.8	14.68	24.41	13.01	10.25	-11.38
dsi.cc		14.94	19.3	29.18	16.22	8.57	-15.96
transport.cc	5.69	7.18	26.19	6.75	18.63	-5.99
trans../tcp.cc	5.71	7.1	24.34	6.7	17.34	-5.63
trans../udp.cc	5.71	7.14	25.04	6.76	18.39	-5.32
trans../unix.cc	5.61	7.04	25.49	6.53	16.4	-7.24
iop.cc		21.32	26.49	24.25	23.7	11.16	-10.53
util.cc		7.62	9.6	25.98	9.09	19.29	-5.31
basic_seq.cc	5	6.29	25.8	6.14	22.8	-2.38
fast_array.cc	5.44	6.86	26.1	6.38	17.28	-7
ssl.cc		13.87	17.68	27.47	14.68	5.84	-16.97
fixed.cc	5.04	6.33	25.6	5.96	18.25	-5.85
intercept.cc	14.7	19.23	30.82	16.27	10.68	-15.39
codeset.cc	7.64	9.55	25	8.85	15.84	-7.33
queue.cc	6.07	7.51	23.72	7.14	17.63	-4.93
static.cc	25.46	32.66	28.28	29.71	16.69	-9.03
current.cc	13.35	17.38	30.19	13.72	2.77	-21.06
policy_impl.cc	18.19	22.82	25.45	19.17	5.39	-15.99
service_info.cc	13.27	17.32	30.52	13.73	3.47	-20.73
ioptypes.cc	15.66	19.77	26.25	16.82	7.41	-14.92
ssliop.cc	13.57	17.46	28.67	13.88	2.28	-20.5
value.cc	15.91	20.66	29.86	17.03	7.04	-17.57
valuetype.cc	14.84	18.98	27.9	15.49	4.38	-18.39
..etype_impl.cc	17.7	22.63	27.85	19.14	8.14	-15.42
dynany_impl.cc	13.75	15.79	14.84	15.51	12.8	-1.77
policy2.cc	13.6	17.51	28.75	13.83	1.69	-21.02
tckind.cc	13.38	17.26	29	13.64	1.94	-20.97
orb_excepts.cc	13.48	17.38	28.93	13.94	3.41	-19.79
policy.cc	13.36	17.51	31.06	13.77	3.07	-21.36
poa.cc		17.87	22.7	27.03	19.69	10.18	-13.26
poa_base.cc	15.56	19.34	24.29	15.84	1.8	-18.1
poa_impl.cc	23.06	28.39	23.11	25.82	11.97	-9.05
dynany.cc	15.33	19.51	27.27	16.52	7.76	-15.33
uni_base64.cc	0.1	0.1	0	0.13	30	30
uni_unicode.cc	0.16	0.19	18.75	0.2	25	5.26
uni_fromuni.cc	0.33	0.38	15.15	0.41	24.24	7.89
uni_touni.cc	0.35	0.42	20	0.45	28.57	7.14
except2.cc	8.4	10.58	25.95	10.68	27.14	0.95
pi.cc		16	20.79	29.94	17.21	7.56	-17.22
pi_impl.cc	23.26	29.87	28.42	26.83	15.35	-10.18
typecode_seq.cc	13.9	17.78	27.91	14.76	6.19	-16.99
timebase.cc	13.39	17.41	30.02	13.64	1.87	-21.65
ir.cc		59.41	73.92	24.42	70.75	19.09	-4.29
ir_base.cc	16.13	21	30.19	17.48	8.37	-16.76
imr.cc		20.48	25.27	23.39	21.84	6.64	-13.57
mtdebug.cc	5.46	6.54	19.78	6.28	15.02	-3.98

Average		12.19	15.41	25.45	13.54	13.32	-9.38
Max				31.06		30	30
Min				0		1.69	-21.65

Sum		731.47	924.3		812.45


.. means file name shorted to fit into table
3.2/3d - means delta between 3.2.2 and 3.3 compilation time in percents -
likewise for 3.2/4d  and 3.3/4d

3.2.2, 3.3, 3.4 are times in seconds reported on "# cc1plus" line - bigger
times so I assume they are times consumed by user level instead of spend
in sys calls. Right?

Machine is laptop 1GHz PIII/256kb cache/512mb ram/30gb disc.
Compiler command-line looks:

c++ -time  -I../include  -O0 -Wall -fpermissive   -DPIC -fPIC  -c <file
name>.cc -o <file name>.pic.o


Results: as you can see gcc3.4 is slower than gcc3.3 on subsecond
compilations, otherwise it's faster and gcc3.2.2 is clear winner here.


Now I would like to find the best methodology how to make testing of gcc
branches for compile-time regressions.

1) daily or weekly?
2) on whole tree or choose some/one file(s)?
3) would you like to see -ftime-report -fmem-report for some interesting
   files?
4) anyway which files might be 'interesting'?
5) which branch(es) to test?
6) which optimization levels - i.e. I tested only what's the most
   important for development when I try to make compilation time as short
   as possible, by turn off all optimizations...?
7) anything other which should I test for you?

Thanks a lot for your time,

Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com