GCC Buildbot Update - Definition of regression

[David Malcolm]

On Wed, 2017-10-11 at 16:17 +0200, Marc Glisse wrote:
> On Wed, 11 Oct 2017, David Malcolm wrote:
> 
> > On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote:
> > > 
> > > On 11/10/17 11:15, Christophe Lyon wrote:
> > > > 
> > > > You can have a look at
> > > > https://git.linaro.org/toolchain/gcc-compare-results.git/
> > > > where compare_tests is a patched version of the contrib/
> > > > script,
> > > > it calls the main perl script (which is not the prettiest thing
> > > > :-)
> > > > 
> > > 
> > > Thanks, that's useful. I will take a look.
> > 
> > You may also want to look at this script I wrote:
> > 
> >  https://github.com/davidmalcolm/jamais-vu
> > 
> > (it has Python classes for working with DejaGnu output)
> 
> By the way, David, how do you handle comparisons for the jit
> testsuite? jv 
> gives
> 
> Tests that went away in build/gcc/testsuite/jit/jit.sum: 81
> -----------------------------------------------------------
> 
>   PASS:  t
>   PASS:  test-
>   PASS:  test-arith-overflow.c
>   PASS:  test-arith-overflow.c.exe iteration 1 of 5: verify_uint_over
>   PASS:  test-arith-overflow.c.exe iteration 2 of 5: verify_uint_o
>   PASS:  test-arith-overflow.c.exe iteration 3 of 5: verify
> [...]
> 
> Tests appeared in build/gcc/testsuite/jit/jit.sum: 78
> -----------------------------------------------------
> 
>   PASS:  test-arith-overflow.c.exe iteration 1
>   PASS:  test-arith-overflow.c.exe iteration 2 of
>   PASS:  test-arith-overflow.c.exe iteration 4 of 5: verify_u
>   PASS:  test-combination.
>   PASS:  test-combination.c.exe it
> [...]
> 
> The issue is more likely in the testsuite, but I assume you have a 
> workflow that allows working around the issue?

I believe the issue here is PR jit/69435 ("Truncated lines in
jit.log").
I suspect that the attachment in comment #2 there ought to fix it
(sorry that this issue stalled; in the meantime I've been simplying
verifying the absence of FAILs and checking the number of PASSes in
jit.sum).

Dave