This is the mail archive of the
gcc-bugs@gcc.gnu.org
mailing list for the GCC project.
Re: testsuites classification
- To: <gnuteam at kpit dot com>
- Subject: Re: testsuites classification
- From: Daniel Berlin <dan at cgsoftware dot com>
- Date: Wed, 01 Aug 2001 02:51:21 -0400
- Cc: <gcc-bugs at gcc dot gnu dot org>, <crossgcc at sourceware dot cygnus dot com>
- References: <000101c11a4d$2bd41d90$81100aac@stpp.soft.net>
"GNU Team" <gnuteam@kpit.com> writes:
> I have a question regarding gcc testsuites.
> How do I categorize them ?
> After running the gcc 3.0 test suite on gcc 2.95.3 (sh-coff-gcc)
> I got following results.
>
> GCC PASS 10077
> FAIL 4927
> XPASS 2
> XFAIL 78
> UNRESOLVED 132
> UNSUPPORTED 59
> WARNINGS 23
> ERRORS 3
>
>
> Now, is there any document which classifies these testsuites ?
> ie. which tests must PASS and which may not pass ?
> Or does it mean that compiler has 4500 bugs which must be fixed ?
It means the compiler is failing 4900 tests.
However, this is not an indication of the number of bugs.
For the c-torture tests, for instance, unless the test specifies
otherwise, each test is compiled 8 times (with different flags each
time).
Other than that, all tests that aren't XFAIL, should be passing (in an
ideal world).
>
>
> Arati.
--
"I'm so hyper... (Said with a very dull voice.)
"-Steven Wright