This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Unexpected failures testing Gcc-2.95.3.test4 with Dejagnu
- To: David Gressett <gressett at iglobal dot net>
- Subject: Re: Unexpected failures testing Gcc-2.95.3.test4 with Dejagnu
- From: Neil Booth <neil at daikokuya dot demon dot co dot uk>
- Date: Fri, 9 Mar 2001 20:53:47 +0000
- Cc: gcc at gcc dot gnu dot org
- References: <5.0.0.25.2.20010309143334.01acc518@mail.iglobal.net>
David Gressett wrote:-
> How are "unexpected" and "expected" defined? In an earlier message, Bernd
> Schmidt said that gcc2.95.3 was looking good: "Basically, all tests have
> come back with good results."
>
> My test summary for gcc-2.95.3.test4 looks like this for the gcc summary
> section for SCO OpenServer 5.0.4
>
> # of expected passes 12046
> # of unexpected failures 972
> # of unexpected successes 2
> # of expected failures 78
> # of unresolved testcases 46
> # of unsupported tests 24
>
> The Linux results were very similar, but not identical.
>
> 972 failures is about 7% of the total number of tests. Are these in some
> sense "expected unexpected failures"?
Dejagnu allows you to flag tests as expected to fail, meaning it's a bug
that still isn't fixed in the compiler.
As for the huge failure count, you're using the GCC 3.0 testsuite, and so
it is reasonable to expect a bunch of failures for bugs fixed by 3.0.
Also, and probably more relevant, is that dejagnu often tests for a bug
by matching error message text. Many error messages have been added /
removed / changed, and 2.95.2 would of course fail those too.
Neil.