This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: GCC Buildbot Update - Definition of regression
- From: Jonathan Wakely <jwakely dot gcc at gmail dot com>
- To: Paulo Matos <pmatos@linki.tools>
- Cc: "gcc at gcc dot gnu dot org" <gcc at gcc dot gnu dot org>
- Date: Wed, 11 Oct 2017 11:18:04 +0100
- Subject: Re: GCC Buildbot Update - Definition of regression
- Authentication-results: sourceware.org; auth=none
- References: <3b204c53-7613-ffe2-172b-69ad2a781ceb@linki.tools> <alpine.DEB.2.20.1710102111080.20946@digraph.polyomino.org.uk> <726e85ce-9ded-e9b4-6412-edb1da76c768@linki.tools>
On 11 October 2017 at 07:34, Paulo Matos wrote:
> When someone adds a new test to the testsuite, isn't it supposed to not
> FAIL?
Yes, but sometimes it FAILs because the test is using a new feature
that only works on some targets, and the new test was missing the
right directives to make it UNSUPPORTED on other targets.
> If is does FAIL, shouldn't this be considered a regression?
No, it's not a regression, because it's not something that used to
work and now fails.
Maybe it should still be flagged as red, but it's not strictly a
regression. I would call it a "new failure" rather than regression.