Re: PATCH: enable Ada test suite

[Mike Stump <mrs at apple dot com>]

On Tuesday, October 28, 2003, at 06:23  AM, Arnaud Charlet wrote:
> Note that this Web page is not a documentation page: it does not give 
> any
> hint about what XFAIL or XPASS mean, etc. It's just a bunch of examples
> and hints.

Imagine that the test suite knew what to expect from every testcase on 
every platform.  You can think of these expectations as the `we're 
ready to ship' indicator.  If you've shipped a broken compiler (wrt a 
testcase named lookup-15.C) in the past and customers didn't complain, 
then shipping it again with the exact same breakage isn't all that bad. 
 Hit 0 unexpected failures and 0 unexpected passed, and presto, ship 
it.  Perfection would mean 0 failures of all types (expected and 
unexpected).  If you can get Ada there, great, but in the real world, 
we have infinite testing and infinite bugs, hence, we'll always have 
some expected failures.

XFAIL means expected failure.  XPASS means unexpected pass.

gcc should swicth to 0 unexpected failure on one platform for releases 
as a release requirement, as an existence proof that it can be done.  
It doesn't entail too much work, as we can always mark tests that do 
fail as expected, if we can't get them fixed.

The expectedness is a good way for a secondary distributor of the tools 
to know if they are close to matching the reliability expectation that 
we'd like to see the hit for releases.  We improve the quality of those 
releases and hence gcc's reputation, by teaching all that hitting 0 for 
every release it attainable and worthwhile.  It is a shame that we 
still don't do that.

Maybe Ada can set a good example for all of us to follow...