PATCH: enable Ada test suite
Mike Stump
mrs@apple.com
Tue Oct 28 18:20:00 GMT 2003
On Tuesday, October 28, 2003, at 06:23 AM, Arnaud Charlet wrote:
> Note that this Web page is not a documentation page: it does not give
> any
> hint about what XFAIL or XPASS mean, etc. It's just a bunch of examples
> and hints.
Imagine that the test suite knew what to expect from every testcase on
every platform. You can think of these expectations as the `we're
ready to ship' indicator. If you've shipped a broken compiler (wrt a
testcase named lookup-15.C) in the past and customers didn't complain,
then shipping it again with the exact same breakage isn't all that bad.
Hit 0 unexpected failures and 0 unexpected passed, and presto, ship
it. Perfection would mean 0 failures of all types (expected and
unexpected). If you can get Ada there, great, but in the real world,
we have infinite testing and infinite bugs, hence, we'll always have
some expected failures.
XFAIL means expected failure. XPASS means unexpected pass.
gcc should swicth to 0 unexpected failure on one platform for releases
as a release requirement, as an existence proof that it can be done.
It doesn't entail too much work, as we can always mark tests that do
fail as expected, if we can't get them fixed.
The expectedness is a good way for a secondary distributor of the tools
to know if they are close to matching the reliability expectation that
we'd like to see the hit for releases. We improve the quality of those
releases and hence gcc's reputation, by teaching all that hitting 0 for
every release it attainable and worthwhile. It is a shame that we
still don't do that.
Maybe Ada can set a good example for all of us to follow...
More information about the Gcc-patches
mailing list