This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Testsuites
- To: Mark Mitchell <mark at codesourcery dot com>
- Subject: Re: Testsuites
- From: Andreas Jaeger <aj at suse dot de>
- Date: 04 Jun 2000 12:30:02 +0200
- Cc: gcc at gcc dot gnu dot org
- References: <20000604015652K.mitchell@codesourcery.com>
>>>>> Mark Mitchell writes:
> Folks --
> In the past, we've used our testsuites as a combination of testsuite
> and bug database.
> The testsuites in GCC are regression test suites. We now have a
> nice GNATS bug-tracking database.
> Let's check in new failing tests as XFAILs. The ideal would be that
> a build of checked-in sources never causes any FAILs -- only XFAILs.
> Then, checking that you haven't introduced new bugs is as simple as
> noticing that there's no test output. Otherwise, people have to
> remember the "magic" list of "normal" failures. That's hard.
> For instance, I just updated my tree, in preparation for checking in
> a change. Rerunning the tests, I suddenly saw failures. It required
> analysis on my part to figure out that the new failures stemmed from
> new tests, not from bugs in my code.
> Bug reports should go in GNATS. The maintainer for that part of the
> code should mark it high priority, so that we know we have to fix it
> before the next release. But the test-case should still be XFAIL --
> it is, after all, expected to fail.
What should we do with existing FAILs? Shouldn't we mark all existing
FAILs as XFAILs now? This has already happened for c++ but not for
the other languages.
Andreas
--
Andreas Jaeger
SuSE Labs aj@suse.de
private aj@arthur.inka.de