This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]

Re: Unexpected failures testing Gcc-2.95.3.test4 with Dejagnu


David Gressett wrote:-

> How are "unexpected" and "expected" defined?  In an earlier message, Bernd 
> Schmidt said that gcc2.95.3 was looking good: "Basically, all tests have 
> come back with good results."
> 
> My test summary for gcc-2.95.3.test4 looks like this for the gcc summary 
> section for SCO OpenServer 5.0.4
> 
> # of expected passes           12046
> # of unexpected failures        972
> # of unexpected successes   2
> # of expected failures            78
> # of unresolved testcases      46
> # of unsupported tests           24
> 
> The Linux results were very similar, but not identical.
> 
> 972 failures is about 7% of the total number of  tests. Are these in some 
> sense "expected unexpected failures"?

Dejagnu allows you to flag tests as expected to fail, meaning it's a bug
that still isn't fixed in the compiler.

As for the huge failure count, you're using the GCC 3.0 testsuite, and so
it is reasonable to expect a bunch of failures for bugs fixed by 3.0.

Also, and probably more relevant, is that dejagnu often tests for a bug
by matching error message text.  Many error messages have been added /
removed / changed, and 2.95.2 would of course fail those too.

Neil.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]