Re: Doubling of 3.1's Running Time for i686-pc-linux-gnu

[Jeffrey Oldham <oldham at codesourcery dot com>]

A harder look at the data for i686-pc-linux-gnu gcc 3.1:

date	finish	lines of test
	time	summary
Jul  1 03:59      102
Jul  2 04:08      100
Jul  3 03:46      102
Jul  4 03:49      104
Jul  5 03:41      101
Jul  6 03:43      167
Jul  7 03:30      168
Jul  8 03:42      167
Jul  9 02:47      167
Jul 10 01:03        0	build failure
Jul 11 01:42        0	build failure
Jul 12 01:26        0	build failure
Jul 13 03:55      171
Jul 14 03:57      104
Jul 15 04:01      105
Jul 16 04:04      105
Jul 17 02:58      114
Jul 18 04:23      110
Jul 19 04:22       94
Jul 20 09:36       92	build failure so build started by hand later
		        IGNORE this line
Jul 21 05:43       97
Jul 22 01:29        0	build failure
Jul 23 06:03       90
Jul 24 09:28      269	numerous C++ execution errors (see attached)
Jul 25 09:34      271	numerous C++ execution errors

All builds except for 20Jul started at 1:01am, which should be
subtracted from the finish time to yield the total time to bootstrap
and run the regression tests on an i686.  The rightmost column lists
the number of lines in the resulting test summaries.

The extremely high test regressions correlate with numerous C++ and
libjava test regressions.  Jason Merrill resolved this problem late
yesterday, resulting in a significantly shorter running time.  (The
26Jul build died for NFS reasons so I do not have a time.)

To summarize, Jason Merrill seems to have fixed the problem.  Also,
gcc 3.1 compiler running times seem to be sensitive to small code
changes.

Thanks,
Jeffrey D. Oldham
oldham@codesourcery.com

Linux gcc 3.0.1 bootstrap and test regression running times are not
nearly as variable.

i686-pc-linux-gnu gcc 3.0.1 data:

These builds start at 12:01am, not 1:01am as for above.

Jul  1 02:32
Jul  2 02:35
Jul  3 02:30
Jul  4 02:30
Jul  5 02:28
Jul  6 02:29
Jul  7 02:22
Jul  8 02:28
Jul  9 00:11	build failure
Jul 10 01:29
Jul 11 02:01
Jul 12 01:51
Jul 13 02:29
Jul 14 02:29
Jul 15 02:31
Jul 16 02:31
Jul 18 02:33
Jul 19 02:32
Jul 20 01:32
Jul 21 02:35
Jul 22 01:52
Jul 23 02:52
Jul 24 02:35
Jul 25 02:36
Jul 26 02:22
cat <<'EOF' |
Native configuration is i686-pc-linux-gnu

		=== g++ tests ===


Running target unix
FAIL: g++.abi/cxa_vec.C  Execution test
FAIL: g++.brendan/eh1.C  Execution test
FAIL: g++.eh/badalloc1.C  Execution test
FAIL: g++.eh/catch11.C  Execution test
FAIL: g++.eh/catch12.C  Execution test
FAIL: g++.eh/catch3.C  Execution test
FAIL: g++.eh/catch3p.C  Execution test
FAIL: g++.eh/catch4.C  Execution test
FAIL: g++.eh/catch4p.C  Execution test
FAIL: g++.eh/catch5.C  Execution test
FAIL: g++.eh/catch5p.C  Execution test
FAIL: g++.eh/catch6.C  Execution test
FAIL: g++.eh/catch6p.C  Execution test
FAIL: g++.eh/catch7.C  Execution test
FAIL: g++.eh/catch7p.C  Execution test
FAIL: g++.eh/catch8.C  Execution test
FAIL: g++.eh/catch8p.C  Execution test
FAIL: g++.eh/catch9.C  Execution test
FAIL: g++.eh/catch9p.C  Execution test
FAIL: g++.eh/catchptr1.C  Execution test
FAIL: g++.eh/cleanup1.C  Execution test
FAIL: g++.eh/cleanup2.C  Execution test
FAIL: g++.eh/flow1.C  Execution test
FAIL: g++.eh/fntry1.C  Execution test
FAIL: g++.eh/inline2.C  Execution test
FAIL: g++.eh/new1.C  Execution test
FAIL: g++.eh/new2.C  Execution test
FAIL: g++.eh/pdel1.C  Execution test
FAIL: g++.eh/pdel2.C  Execution test
FAIL: g++.eh/ptr1.C  Execution test
FAIL: g++.eh/ptrmem1.C  Execution test
FAIL: g++.eh/rethrow1.C  Execution test
FAIL: g++.eh/rethrow2.C  Execution test
FAIL: g++.eh/rethrow3.C  Execution test
FAIL: g++.eh/rethrow4.C  Execution test
FAIL: g++.eh/rethrow5.C  Execution test
FAIL: g++.eh/rethrow6.C  Execution test
FAIL: g++.eh/spec1.C  Execution test
FAIL: g++.eh/spec2.C  Execution test
FAIL: g++.eh/spec3.C  Execution test
FAIL: g++.eh/spec4.C  Execution test
FAIL: g++.eh/tmpl1.C  Execution test
FAIL: g++.eh/unwind1.C  Execution test
FAIL: g++.eh/vbase1.C  Execution test
FAIL: g++.eh/vbase2.C  Execution test
FAIL: g++.eh/vbase4.C  Execution test
FAIL: g++.ext/instantiate1.C not instantiated (test for errors, line 20)
FAIL: g++.martin/new1.C  Execution test
FAIL: g++.mike/dyncast1.C  Execution test
FAIL: g++.mike/dyncast2.C  Execution test
FAIL: g++.mike/eh10.C  Execution test
FAIL: g++.mike/eh12.C  Execution test
FAIL: g++.mike/eh14.C  Execution test
FAIL: g++.mike/eh16.C  Execution test
FAIL: g++.mike/eh17.C  Execution test
FAIL: g++.mike/eh18.C  Execution test
FAIL: g++.mike/eh2.C  Execution test
FAIL: g++.mike/eh21.C  Execution test
FAIL: g++.mike/eh23.C  Execution test
FAIL: g++.mike/eh24.C  Execution test
FAIL: g++.mike/eh25.C  Execution test
FAIL: g++.mike/eh26.C  Execution test
FAIL: g++.mike/eh27.C  Execution test
FAIL: g++.mike/eh28.C  Execution test
FAIL: g++.mike/eh29.C  Execution test
FAIL: g++.mike/eh3.C  Execution test
FAIL: g++.mike/eh31.C  Execution test
FAIL: g++.mike/eh33.C  Execution test
FAIL: g++.mike/eh34.C  Execution test
FAIL: g++.mike/eh35.C  Execution test
FAIL: g++.mike/eh36.C  Execution test
FAIL: g++.mike/eh37.C  Execution test
FAIL: g++.mike/eh38.C  Execution test
FAIL: g++.mike/eh39.C  Execution test
FAIL: g++.mike/eh40.C  Execution test
FAIL: g++.mike/eh41.C  Execution test
FAIL: g++.mike/eh42.C  Execution test
FAIL: g++.mike/eh44.C  Execution test
FAIL: g++.mike/eh47.C  Execution test
FAIL: g++.mike/eh48.C  Execution test
FAIL: g++.mike/eh49.C  Execution test
FAIL: g++.mike/eh5.C  Execution test
FAIL: g++.mike/eh50.C  Execution test
FAIL: g++.mike/eh51.C  Execution test
FAIL: g++.mike/eh55.C  Execution test
FAIL: g++.mike/eh6.C  Execution test
FAIL: g++.mike/eh8.C  Execution test
FAIL: g++.mike/p7912.C  Execution test
FAIL: g++.mike/p9706.C  Execution test
FAIL: g++.other/array1.C  Execution test
FAIL: g++.other/eh3.C  Execution test
FAIL: g++.other/enum5.C (test for excess errors)
FAIL: g++.other/init7.C  Execution test
FAIL: g++.other/new7.C  Execution test
FAIL: g++.other/singleton.C  Execution test
FAIL: g++.other/vbase2.C  Execution test
FAIL: g++.pt/fntry1.C  Execution test
XPASS: g++.robertl/eb17.C  Execution test
FAIL: g++.robertl/eb31.C  Execution test
FAIL: g++.robertl/eb50.C  Execution test
FAIL: g++.robertl/eb66.C  Execution test
FAIL: g++.robertl/eb88.C  Execution test
FAIL: g++.robertl/eh990323-1.C  Execution test
FAIL: g++.robertl/eh990323-2.C  Execution test
FAIL: g++.robertl/eh990323-3.C  Execution test
FAIL: g++.robertl/eh990323-4.C  Execution test
FAIL: g++.robertl/eh990323-5.C  Execution test
FAIL: g++.robertl/ice990323-2.C  Execution test

		=== g++ Summary ===

# of expected passes		6447
# of unexpected failures	107
# of unexpected successes	1
# of expected failures		85
# of untested testcases		9
/home/oldham/gcc/gcc2/objdir/gcc/testsuite/../g++ version 3.1 20010724 (experimental)

		=== g77 tests ===


Running target unix

		=== g77 Summary ===

# of expected passes		1057
		=== gcc tests ===


Running target unix

		=== gcc Summary ===

# of expected passes		15669
# of expected failures		72
# of unsupported tests		23
/home/oldham/gcc/gcc2/objdir/gcc/xgcc version 3.1 20010724 (experimental)

		=== objc tests ===


Running target unix
FAIL: objc/execute/formal_protocol-6.m execution,  -O 

		=== objc Summary ===

# of expected passes		185
# of unexpected failures	1
/home/oldham/gcc/gcc2/objdir/gcc/xgcc version 3.1 20010724 (experimental)

		=== libjava tests ===


Running target unix
FAIL: G19990210_1 compilation from source
FAIL: G19990210_1 -O compilation from source
FAIL: G19990217_01 compilation from source
FAIL: G19990217_01 -O compilation from source
FAIL: noclass run
FAIL: register run
FAIL: throwit run
FAIL: Divide_1 execution from source compiled test
FAIL: Divide_1 execution from bytecode->native test
FAIL: Divide_1 -O execution from source compiled test
FAIL: Divide_1 -O execution from bytecode->native test
FAIL: Final execution from source compiled test
FAIL: Final execution from bytecode->native test
FAIL: Final -O execution from source compiled test
FAIL: Final -O execution from bytecode->native test
FAIL: G19990302_02 execution from source compiled test
FAIL: G19990302_02 execution from bytecode->native test
FAIL: G19990302_02 -O execution from source compiled test
FAIL: G19990302_02 -O execution from bytecode->native test
FAIL: G19990303_01 execution from source compiled test
FAIL: G19990303_01 execution from bytecode->native test
FAIL: G19990303_01 -O execution from source compiled test
FAIL: G19990303_01 -O execution from bytecode->native test
FAIL: G19990303_02 execution from source compiled test
FAIL: G19990303_02 execution from bytecode->native test
FAIL: G19990303_02 -O execution from source compiled test
FAIL: G19990303_02 -O execution from bytecode->native test
FAIL: G19990304_01 execution from source compiled test
FAIL: G19990304_01 execution from bytecode->native test
FAIL: G19990304_01 -O execution from source compiled test
FAIL: G19990304_01 -O execution from bytecode->native test
FAIL: InterfaceDispatch execution from source compiled test
FAIL: InterfaceDispatch execution from bytecode->native test
FAIL: InterfaceDispatch -O execution from source compiled test
FAIL: InterfaceDispatch -O execution from bytecode->native test
FAIL: Invoke_1 execution from source compiled test
FAIL: Invoke_1 execution from bytecode->native test
FAIL: Invoke_1 -O execution from source compiled test
FAIL: Invoke_1 -O execution from bytecode->native test
FAIL: PR141 execution from source compiled test
FAIL: PR141 execution from bytecode->native test
FAIL: PR141 -O execution from source compiled test
FAIL: PR141 -O execution from bytecode->native test
FAIL: PR218 execution from source compiled test
FAIL: PR218 execution from bytecode->native test
FAIL: PR218 -O execution from source compiled test
FAIL: PR218 -O execution from bytecode->native test
FAIL: Shazam compilation from source
FAIL: StringBuffer_1 execution from source compiled test
FAIL: StringBuffer_1 execution from bytecode->native test
FAIL: StringBuffer_1 -O execution from source compiled test
FAIL: StringBuffer_1 -O execution from bytecode->native test
FAIL: Throw_1 execution from source compiled test
FAIL: Throw_1 execution from bytecode->native test
FAIL: Throw_1 -O execution from source compiled test
FAIL: Throw_1 -O execution from bytecode->native test
FAIL: instinit2 execution from source compiled test
FAIL: instinit2 execution from bytecode->native test
FAIL: instinit2 -O execution from source compiled test
FAIL: instinit2 -O execution from bytecode->native test
FAIL: pr184 execution from source compiled test
FAIL: pr184 execution from bytecode->native test
FAIL: pr184 -O execution from source compiled test
FAIL: pr184 -O execution from bytecode->native test
FAIL: pr83 compilation from source
FAIL: pr83 execution from bytecode->native test
FAIL: pr83 -O execution from source compiled test
FAIL: pr83 -O execution from bytecode->native test
FAIL: TLtest execution from source compiled test
FAIL: TLtest execution from bytecode->native test
FAIL: TLtest -O execution from source compiled test
FAIL: TLtest -O execution from bytecode->native test
XPASS: stringconst output from bytecode->native test
FAIL: stringconst -O compilation from bytecode
FAIL: invokethrow execution from source compiled test
FAIL: invokethrow execution from bytecode->native test
FAIL: invokethrow -O execution from source compiled test
FAIL: invokethrow -O execution from bytecode->native test

		=== libjava Summary ===

# of expected passes		1613
# of unexpected failures	77
# of unexpected successes	1
# of expected failures		16
# of untested testcases		156
		=== libstdc++-v3 tests ===


Running target unix
FAIL: 19_diagnostics/stdexceptions.cc execution test
FAIL: 21_strings/append.cc execution test
FAIL: 21_strings/ctor_copy_dtor.cc execution test
FAIL: 21_strings/element_access.cc execution test
FAIL: 21_strings/insert.cc execution test
FAIL: 21_strings/substr.cc execution test
FAIL: 23_containers/bitset_ctor.cc execution test
FAIL: 23_containers/bitset_members.cc execution test
FAIL: 23_containers/vector_element_access.cc execution test
FAIL: 27_io/ios_members.cc execution test

		=== libstdc++-v3 Summary ===

# of expected passes		271
# of unexpected failures	10
# of expected failures		10

Compiler version: 3.1 20010724 (experimental)