Re: GCC Buildbot Update - Definition of regression

[Hans-Peter Nilsson <hp at bitrange dot com>]

On Tue, 10 Oct 2017, Paulo Matos wrote:

> This is a suggestion. I am keen to have corrections from people who use
> this on a daily basis and/or have a better understanding of each status.

Not mentioning them (oddly I don't see anyone mentioning them)
makes me think you've not looked there so allow me to point out:
consider re-using Geoff Keating's regression tester scripts.
They're all in your nearest gcc checkout, in contrib/regression.
I suggest using whatever definition those scripts define.
They've worked for my regression testing (though my local
automated tester is not active at the moment).  Just remember to
always use the option --add-passes-despite-regression or else
btest-gcc.sh requires a clean bill before adding new PASSes to
the list of PASSing tests considered for regression.  (A clean
bill happens too rarely for non-primary targets, for long times,
for reasons beyond port maintainer powers.)

Also, you may have to fight release maintainers for the
"regression" definition.  Previous arguments have been along the
line of "it's not a regression if there hasn't been a release
with the test for that functionality passing".

brgds, H-P