[pph] Clean up PPH tests (issue4572042)

Lawrence Crowl crowl@google.com
Wed Jun 8 21:27:00 GMT 2011


On 6/7/11, Diego Novillo <dnovillo@google.com> wrote:
> After getting new failures due to an unrelated fix, I think this will
> be more trouble than it's worth.
>
> First, we can't get rid of the XPASSes, so those will always be noisy.

Remove all the noise with:

sed -e '
        /-fpph-map=pph.map/     ! {
                /^XPASS: .*test for bogus messages/     d
                /^XPASS: .*test for excess errors/      d
        }

        /^#/            p
        /^ERROR: /      p
        /^XFAIL: /      p
        /^XPASS: /      p
        /^FAIL: /       p

        d
'

>
> Second, some XPASSes will need to be unmarked because we just fixed
> the underlying problem.

That is how we know that what progress we have made.  These should
be reflected in the work list.

>
> Third, we are at such an early stage, that fixes to a test case will
> generally expose failures in other already failing tests, but these
> failures will be in a different place.  So more noise.

That is now we expose remaining tasks.  These should be reflected
in the work list.

>
> I really think that for now the easiest way to keep track of this is
> to have a clean build to compare against.

But the clean build isn't a decent comparison.  The existing tools
only compare against whether or not tests fail.  They fail to
report that a test failed for different reasons.  Given that there
are only twelve tests for which there is a concern, I think we are
better off know where we are.

-- 
Lawrence Crowl



More information about the Gcc-patches mailing list