Re: Small update to reversed_comparison_code

[Mike Stump <mrs at windriver dot com>]

> Date: Tue, 13 Mar 01 21:00:14 EST
> From: kenner@vlsi1.ultra.nyu.edu (Richard Kenner)
> To: mrs@windriver.com

> You miss my point.  Testing would have given the wrong answer.  It
> would have said "There's something wrong with this patch".  But
> there isn't: there was some unrelated latent bug.

To me, I view it differently.  Testing gives you an answer, and what
it says, is right.  Understanding what it says, is hard.  When the
testing system says this patch broke the build, it means just that,
and that is true.  What it doesn't say, is why, or if the problem was
just an unexposed latent bug exposing itself.  Testing cannot give
such subtle answers, nor can we expect such subtle answers from
testing, therefore we cannot judge the answers that testing gives, to
be wrong, they are not, rather, ones interpretation of those answers
can be wrong.

A most unfortunate example, is when a testing system is testing the
RAM on a machine, and giving you the testing results of such tests.
The test results are accurate, it is just that what they test, isn't
the compiler, but RAM, in this case.

So, in your case above, I would argue that your statement of what the
testing system is saying is wrong, it doesn't say that there is
something wrong with this patch.  Rather, it says, there is a
regression; it now fails when your patch is in the tree, and that much
is true.  The fix _could_ be, buy a better machine, it could be to
improve the testcase, it could be to fix the testing infrastructure,
it could be to fix the patch.  The testing system cannot tell you
which, nor can one infer an answer from the testing system.

I happen to think that should one add a warning to gcc that causes 100
testcases in the testsuite to fail, they should _not_ check the work
in, without also fixing the testframework or the testcases, at the
same time.  Sure, I know it is a pain, but I think this comes down to
a simple value judgement.  Is testing worth the pain?