Re: [PATCH] enhance strlen to understand MEM_REF and partial overlaps (PR 86042, 86043)

[Martin Sebor <msebor at gmail dot com>]

On 07/27/2018 10:17 AM, Eric Botcazou wrote:
> > FWIW, there are 128 failures in the GCC test suite on x86_64.
> > Many of these have been there for weeks (e.g., the lto failures
> > due to PR86004), even years (guality).  My script to compare
> > the results against a baseline uses the following regular
> > expression to extract the names of failing (and other
> > "interesting") tests.
>
> Why not just use "make mail-report.log" ?

My script downloads the latest result posted to gcc-testresults
for the target I'm building for, iterates over the interesting
test results in my build logs, compares them to the baseline,
and prints a condensed report like the one in the attachment.
This helps me decide what failures to look into and which ones
are probably safe to ignore.  The tests marked with '!' are
regressions in my run vs the baseline.  I only look at those,
and only those I don't recognize (which admittedly is error-prone).

The others are either the same failures or improvements.
(The numbers in parentheses show the difference between the number
of times the test was seen to fail in the baseline vs in my run.)

What I was trying to highlight is that rolling my own solution like
this makes missing regressions more likely than having a shared
solution would.  It's a separate issue that some of these failures
continue to persist despite having the appearance of regressions
the LTO failures), and another problem are the failures in
the guality tests.  These also make it more likely that
a regression will be missed.

Martin

!  FAIL: gcc.dg/attr-nonstring-2.c (4: +4)
   FAIL: gcc.dg/guality/pr54200.c (1)
   FAIL: gcc.dg/guality/pr54519-1.c (4: -12)
   FAIL: gcc.dg/guality/pr54519-2.c (2: -2)
   FAIL: gcc.dg/guality/pr54519-3.c (8: -8)
   FAIL: gcc.dg/guality/pr54519-4.c (2: -2)
   FAIL: gcc.dg/guality/pr54519-5.c (2: -2)
   FAIL: gcc.dg/guality/pr54970.c (29: -29)
   FAIL: gcc.dg/guality/pr56154-1.c (1: -1)
   FAIL: gcc.dg/guality/pr59776.c (8: -8)
   FAIL: gcc.dg/guality/pr68860-1.c (24: -24)
   FAIL: gcc.dg/guality/sra-1.c (6: -6)
   FAIL: gcc.dg/guality/vla-1.c (8: -8)
   FAIL: g++.dg/guality/pr55665.C (1: -1)
!  FAIL: g++.dg/lto/20091002-1 (1: +1)
!  FAIL: g++.dg/lto/pr64043 (1: +1)
!  FAIL: g++.dg/lto/pr65193 (1: +1)
!  FAIL: g++.dg/lto/pr65302 (1: +1)
!  FAIL: g++.dg/lto/pr65316 (1: +1)
!  FAIL: g++.dg/lto/pr65549 (2: +2)
!  FAIL: g++.dg/lto/pr66180 (1: +1)
!  FAIL: g++.dg/lto/pr66705 (1: +1)
!  FAIL: g++.dg/lto/pr68057 (4: +4)
!  FAIL: g++.dg/lto/pr69077 (1: +1)
!  FAIL: g++.dg/lto/pr69133 (1: +1)
!  FAIL: g++.dg/lto/pr69137 (1: +1)
!  FAIL: g++.dg/lto/pr79000 (1: +1)
!  FAIL: g++.dg/lto/pr81940 (1: +1)
!  FAIL: g++.dg/lto/pr85176 (1: +1)
   FAIL: g++.dg/pr80481.C (3)
   FAIL: g++.dg/pr83239.C (2)
!  FAIL: gfortran.dg/derived_constructor_comps_5.f90 (6: +6)
!  FAIL: gfortran.dg/lto/pr79108 (1: +1)
!  FAIL: ./index0-out.go (1: +1)
! UNRESOLVED: gcc.dg/tree-prof/crossmodule-indircall-1a.c (1: +1)
! UNRESOLVED: gcc.dg/tree-prof/pr59003.c (1: +1)
! UNRESOLVED: gcc.dg/tree-prof/wcoverage-mismatch.c (2: +2)
! UNRESOLVED: g++.dg/bprob/g++-bprob-2.C (2: +2)
! UNRESOLVED: gfortran.dg/derived_constructor_comps_5.f90 (3: +3)
  UNRESOLVED: libgomp.oacc-c++/non-scalar-data.C (1: -1)
  XPASS: gcc.dg/guality/example.c (3: -7)
  XPASS: gcc.dg/guality/guality.c (8: -8)
  XPASS: gcc.dg/guality/inline-params.c (5: -5)
  XPASS: gcc.dg/guality/pr41353-1.c (2: -3)
  XPASS: gcc.dg/guality/pr54970.c (5: -5)
warnings found:
 360   Wabi
   7   Wcast-function-type
   4   Wcpp
   6   Wdeprecated-copy
   5   Wformat-truncation=
   8   Wimplicit-fallthrough=
  14   Wincompatible-pointer-types
  49   Wmaybe-uninitialized
  43   Wmisleading-indentation
  12   Wnonnull
   8   Wsign-compare
   4   Wstringop-truncation
  19   Wuninitialized
   2   Wunused-function
  20   Wunused-parameter