Re: GCC Buildbot Update - Definition of regression

[Marc Glisse <marc dot glisse at inria dot fr>]

On Wed, 11 Oct 2017, David Malcolm wrote:

> On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote:
> > On 11/10/17 11:15, Christophe Lyon wrote:
> > > You can have a look at
> > > https://git.linaro.org/toolchain/gcc-compare-results.git/
> > > where compare_tests is a patched version of the contrib/ script,
> > > it calls the main perl script (which is not the prettiest thing :-)
> >
> > Thanks, that's useful. I will take a look.
>
> You may also want to look at this script I wrote:
>
>  https://github.com/davidmalcolm/jamais-vu
>
> (it has Python classes for working with DejaGnu output)

By the way, David, how do you handle comparisons for the jit testsuite? jv 
gives

Tests that went away in build/gcc/testsuite/jit/jit.sum: 81
-----------------------------------------------------------

 PASS:  t
 PASS:  test-
 PASS:  test-arith-overflow.c
 PASS:  test-arith-overflow.c.exe iteration 1 of 5: verify_uint_over
 PASS:  test-arith-overflow.c.exe iteration 2 of 5: verify_uint_o
 PASS:  test-arith-overflow.c.exe iteration 3 of 5: verify
[...]

Tests appeared in build/gcc/testsuite/jit/jit.sum: 78
-----------------------------------------------------

 PASS:  test-arith-overflow.c.exe iteration 1
 PASS:  test-arith-overflow.c.exe iteration 2 of
 PASS:  test-arith-overflow.c.exe iteration 4 of 5: verify_u
 PASS:  test-combination.
 PASS:  test-combination.c.exe it
[...]

The issue is more likely in the testsuite, but I assume you have a 
workflow that allows working around the issue?

--
Marc Glisse