This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Using bugzilla to track XFAILs
- From: Tom Tromey <tromey at redhat dot com>
- To: "Giovanni Bajo" <giovannibajo at libero dot it>
- Cc: "Mark Mitchell" <mark at codesourcery dot com>, "Gabriel Dos Reis" <gdr at integrable-solutions dot net>, "Zack Weinberg" <zack at codesourcery dot com>, GCC Hackers <gcc at gcc dot gnu dot org>
- Date: 13 Feb 2004 11:17:00 -0700
- Subject: Re: Using bugzilla to track XFAILs
- References: <061501c3f23d$a464c460$aa422597@bagio>
- Reply-to: tromey at redhat dot com
>>>>> "Giovanni" == Giovanni Bajo <giovannibajo@libero.it> writes:
Giovanni> I propose the introduce a new policy: for each regression
Giovanni> which has been xfailed in the testsuite, there ought be a
Giovanni> bugzilla bug referencing it. We already have a keyword
Giovanni> called 'xfail', but it is currently used only by the
Giovanni> tree-ssa branch.
This sounds great to me.
For libjava our policy has been "0 FAIL". Test cases that provoke
known bugs are marked XFAIL and in theory always have a corresponding
PR.
The idea is that it should always be simple to see how well libjava is
doing without needing some baseline to compare against.
Unfortunately, in practice this doesn't work perfectly both due to
differences between platforms and because no one regularly pays the
required amount of attention to the test results (a problem more
severe for libjava as some of our test cases are separate projects in
their own right and change on their own schedules -- i.e., we can't
XFAIL all new failing tests at the time at which they are added).
Tom