results for egcs-2.91.53 586-pc-linux-gnulibc1

[Joe Buck]

Peter Schmid writes:

> These are the testsuite results for egcs-2.91.53 on the
> i586-pc-linux-gnulibc1 system. There is an internal compiler error in
> g++.pt/explicit69.C which is not caught by the test since it is an XFAIL. 
> Is it possible to fix this?

What a coincidence -- I was composing a message about just this problem
when Peter's message arrived.

It appears that our testing methodology has a severe problem: we can
report PASS for a test, even though the compiler crashes on the test!
Tests for excess errors can pass even though the compiler crashes.
(Obviously we can also get an XFAIL even though a crash was not expected).

So what should be done?  It would be nice to have a separate status, say
FATAL, for internal compiler errors, and to report them even if the
expected result is XFAIL.  (We could then have XFATAL for tests known to
explode the tool that we don't intend to fix, but I believe that the vast
majority of tests marked XFAIL would not be XFATAL).  All tests
corresponding to a crashing tool would report FATAL, even if otherwise any
printed results are correct.

Currently it seems that we are ignoring vital information, since it
appears only in gcc.log or g++.log, e.g.

compiler exited with status 1
output is:
/u/jbuck/gnu/src/egcs-snap/gcc/testsuite/g++.old-deja/g++.jason/destruct3.C:36: Internal compiler error 369.
/u/jbuck/gnu/src/egcs-snap/gcc/testsuite/g++.old-deja/g++.jason/destruct3.C:36: Please submit a full bug report to `egcs-bugs@cygnus.com'.

Perhaps until this can be fixed, Alexandre's test_summary script could
add something like

egrep 'Internal compiler error' g++.log

to the output that is mailed into the list.  (we could do the others
as well, but g++ still crashes the most).