This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: GCC Buildbot Update - Definition of regression
- From: Joseph Myers <joseph at codesourcery dot com>
- To: Christophe Lyon <christophe dot lyon at linaro dot org>
- Cc: Paulo Matos <pmatos@linki.tools>, "gcc at gcc dot gnu dot org" <gcc at gcc dot gnu dot org>
- Date: Wed, 11 Oct 2017 13:22:35 +0000
- Subject: Re: GCC Buildbot Update - Definition of regression
- Authentication-results: sourceware.org; auth=none
- References: <3b204c53-7613-ffe2-172b-69ad2a781ceb@linki.tools> <alpine.DEB.2.20.1710102111080.20946@digraph.polyomino.org.uk> <726e85ce-9ded-e9b4-6412-edb1da76c768@linki.tools> <CAKdteOYgCJarP+XAvJyYCt3Ei-8-=g8miFJrw_YSc_LBO4gm6A@mail.gmail.com>
On Wed, 11 Oct 2017, Christophe Lyon wrote:
> * {PASS,UNSUPPORTED,UNTESTED,UNRESOLVED}-> XPASS
I don't think any of these should be considered regressions. It's good if
someone manually checks anything that's *consistently* XPASSing, to see if
the XFAIL should be removed or restricted to narrower conditions, but if
the result of a test has become any kind of pass, it cannot possibly be
considered a regression. (You might have a flaky test XFAILed because it
passes or fails at random, though I think that random variation is more
common for GDB than for GCC.)
--
Joseph S. Myers
joseph@codesourcery.com