This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: A testcase library
- From: Daniel Jacobowitz <drow at mvista dot com>
- To: Richard Kenner <kenner at vlsi1 dot ultra dot nyu dot edu>
- Cc: gcc at gcc dot gnu dot org
- Date: Mon, 23 Dec 2002 20:49:32 -0500
- Subject: Re: A testcase library
- References: <10212240145.AA23279@vlsi1.ultra.nyu.edu>
On Mon, Dec 23, 2002 at 08:45:36PM -0500, Richard Kenner wrote:
> Suppose that GDB is debugging a program, and the information it needs
> is simply missing from the debug info. Zilch, nada, absolutely no way
> to recover it. That's an XFAIL.
>
> Suppose that a testcase is for a new bug that no one's figured out how
> to fix yet. That's a KFAIL. All KFAILs are required to have an open PR
> associated with them, at least in GDB-land.
>
> The difference, in theory, is that KFAILs represent real problems that
> have not been fixed; and XFAILs represent "expected" failures, problems
> in the system or tools that can not be fixed in the program-under-test.
>
> I still don't get it. Your first example sounds like something that's not
> a bug at all, in which case why would the "test case" still be in the suite?
It's a test. It can have multiple results:
- A correct result
- An incorrect, unavoidable result as a consequence of a bug in the
system
- An incorrect result as a consequence of a bug in the program being
tested.
This is not hypothetical. The GDB testsuite is filled with examples of
this. Often it's discovered the other way round: someone finds a bug
and fixes it, but the test case doesn't pass on some other system and
there's nothing we can do.
--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer