Testsuite ad RedHat 7

[Jakub Jelinek]

On Tue, Oct 10, 2000 at 02:50:52AM +0200, jfm2@club-internet.fr wrote:
> 
> 
> Has anyone run a testsuite on the compiler in RedHat 7 and compared
> with a run using gcc-2.95.  This could settle the question of what
> compiler is the less buggier once and for all.  It is not that
> difficult to make gcc 2.95.2 core dump if you are agressive with
> optimization flags so it cannot deemed perfect.

Every RH7 gcc snapshot rpm I have made publicly available was regression
tested against the previous rpms.
I've run the current testsuite on a 2.95.2 bootstrap, interpretting the
comparison is a little bit harder unless one hacks the testsuite a little
bit for 2.95 (because e.g. some options are missing in 2.95 which the
testsuite uses) which I was lazy to do.
Anyway, e.g. in c-torture/i386 gcc 2.95 gave (g++ and f77 tests have similar figures):
                === gcc Summary ===

# of expected passes            10703
# of unexpected failures        586
# of unexpected successes       3
# of expected failures          49
# of unresolved testcases       482
# of unsupported tests          1069
/usr/src/gcc-2.95.2/obj2/gcc/xgcc version 2.95.2 19991024 (release)
while RHL7 gcc:
                === gcc Summary ===

# of expected passes            12819
# of unexpected failures        13
# of expected failures          56
# of unsupported tests          14
/usr/src/redhat/BUILD/gcc-2.96-20000731/obj-i386-redhat-linux/gcc/xgcc version 2.96 20000731 (Red Hat Linux 7.0)

I've hand edited 2.95.2 results so that failures because of missing -fssa
are accounted for unsupported tests, not FAILs. Most of the 2.95.2
failures are gcc.dg, so I think most of them will be ISO C99 tests or
warning tests. The number of failed tests (not using -fssa) in compile and
execute is actually 137. Without taking time and really go test by test and
check whether it is 2.95.2's fault or it is innocent and just the test is
useful with 2.96+ only I think that's about all that can be said on the
2.95.2 side, so the only think which I've checked were the actual
regressions in the RHL7 compiler against 2.95.2:

c-torture:
- 1 ICE with -gdwarf (ie. dwarf 1, as gcc uses by default stabs on ia32 and
  the other common debugging format is dwarf 2, this is not too high
  priority for me)
- 1 dg test is IMHO broken and gcc is right to refuse it with -ansi
  -pedantic-errors
- 3 improper warning test regressions
- 1 test regression because gcc -MM -MG puts an unexpected newline at the
  end of output
No regressions in C++ tests.
In F77 tests, regressions are:
- 4 similar tests failed with -Os to compile
- 1 test failed to execute with -Os

This comparison (even if I took time and went into details, looked up the
failed tests etc.) would not speak too much about stability of RHL7 compiler
though, because typically when a testsuite entry is submitted, usually it is
either submitted with a fix or somebody writes a fix soon after.
Another testing was rebuilding the whole RHL7 distribution, and that test
passed.
As to whether you can compile 2.2.x kernels with it, you can if you fix some
things in the kernel (like that with the old new K&R preprocessor there are
no GNU macro varargs extensions, so the kernel either cannot use them in
-traditional assembly (like checksum.S) or must avoid using -traditional) -
see H.J.Lu's patches for it. The kernel should work just fine then, it is
just considered as unsupported.
The 2.4.x kernels compile just fine with that compiler.

	Jakub