This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: A testcase library
On Mon, Dec 23, 2002 at 01:25:37PM -0800, Zack Weinberg wrote:
> Ben Elliston <bje@redhat.com> writes:
>
> > I think that KFAIL clarifies the situation (although I, too, have my
> > reservations about having too many test states). It seems few people
> > properly understood the purpose of XFAIL, but most instances of XFAIL
> > are used to express known failures (take the G++ testsuite for
> > example). The number of times XFAIL should be used should be
> > comparatively few.
>
> Can you expand on the difference between KFAIL and XFAIL, please? I
> don't have this new dejagnu version and from over here it sounds like
> a distinction without a difference.
Implementation-wise they are essentially the same. They recognize two
different categories of problems however. The best way I can explain
it is with an example.
Suppose that GDB is debugging a program, and the information it needs
is simply missing from the debug info. Zilch, nada, absolutely no way
to recover it. That's an XFAIL.
Suppose that a testcase is for a new bug that no one's figured out how
to fix yet. That's a KFAIL. All KFAILs are required to have an open PR
associated with them, at least in GDB-land.
The difference, in theory, is that KFAILs represent real problems that
have not been fixed; and XFAILs represent "expected" failures, problems
in the system or tools that can not be fixed in the program-under-test.
They are primarily a device to have no unexpected failures without
abusing XFAIL; this is useful because it makes new regressions very
obvious.
--
Daniel Jacobowitz
MontaVista Software Debian GNU/Linux Developer