A testcase library

[Daniel Jacobowitz]

On Mon, Dec 23, 2002 at 08:45:36PM -0500, Richard Kenner wrote:
>     Suppose that GDB is debugging a program, and the information it needs
>     is simply missing from the debug info.  Zilch, nada, absolutely no way
>     to recover it.  That's an XFAIL.
> 
>     Suppose that a testcase is for a new bug that no one's figured out how
>     to fix yet.  That's a KFAIL.  All KFAILs are required to have an open PR
>     associated with them, at least in GDB-land.
> 
>     The difference, in theory, is that KFAILs represent real problems that
>     have not been fixed; and XFAILs represent "expected" failures, problems
>     in the system or tools that can not be fixed in the program-under-test. 
> 
> I still don't get it.  Your first example sounds like something that's not
> a bug at all, in which case why would the "test case" still be in the suite?

It's a test.  It can have multiple results:
 - A correct result
 - An incorrect, unavoidable result as a consequence of a bug in the
system
 - An incorrect result as a consequence of a bug in the program being
tested.

This is not hypothetical.  The GDB testsuite is filled with examples of
this.  Often it's discovered the other way round: someone finds a bug
and fixes it, but the test case doesn't pass on some other system and
there's nothing we can do.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer