This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: RFC: Improving support for known testsuite failures
On Thu, Sep 8, 2011 at 2:14 PM, Diego Novillo <dnovillo@google.com> wrote:
> On Thu, Sep 8, 2011 at 07:49, Richard Guenther
> <richard.guenther@gmail.com> wrote:
>
>> Well, I'd rather _fix_ dejagnu then. ?Any specific example you can't
>> eventually xfail by dg-skipping the testcase?
>
> Several I mentioned upthread:
> - Some .exp files do no support xfail markers.
> - Different directories will have their own syntactic sugar (though
> most are using dg-xfail now).
> - When using dg-xfail-if, you also need to xfail other dg- markers to
> avoid UNRESOLVED.
> - Similarly, a dg-xfail-if will cause UNRESOLVED if the test was dg-do
> run or dg-do link.
> - If you are forced to use dg-skip-if, you will miss state changes in the test.
> - Some tests are simply flaky (e.g. the ones that depend on
> environment like those using gdb).
>
> I am purposely avoiding tangling with DejaGNU, though if I could find
> a way of adding a global known-to-fail marker, that would solve my
> problem too. ?Janis, do you think that's feasible?
>
>>> Perhaps if there was a global marker that one could add, that would
>>> solve my problem too. ?I think I'll start with a post-check filter in
>>> contrib/
>>
>> You seem to have a very specific problem ;)
>
> Apparently :) ?I am thinking about release and devel branches, in
> particular. ?That's why I think I'll just add a script in contrib/
>
>> I suppose some
>> patch autotester? ?Our patch autotester simply bootstraps twice
>> and compares the result.
>
> We have those, but doubling the amount of work done by the testers is
> not acceptable. ?What we need is something that will quickly decide
> whether the testsuite run is OK or not.
Cache the comparison result? If you specify a (minimum) revision
required for testing just test against a cached revision that fulfils
the requirement. Something I never implemented for ours.
Richard.
>
> Diego.
>