Getting spurious FAILS in testsuite?

[Georg-Johann Lay]

On 01.06.2017 16:16, Marek Polacek wrote:
> On Thu, Jun 01, 2017 at 02:59:37PM +0200, Georg-Johann Lay wrote:
>> Hi, when I am running the gcc testsuite in $builddir/gcc then
>>
>> $ make check-gcc RUNTESTFLAGS='ubsan.exp'
>>
>> comes up with spurious fails.
>>
>> Running target unix
>> Using /usr/share/dejagnu/baseboards/unix.exp as board description file for
>> target.
>> Using /usr/share/dejagnu/config/unix.exp as generic interface file for
>> target.
>> Using /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/config/default.exp as
>> tool-and-target-specific interface file.
>> Running
>> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/gcc.dg/ubsan/ubsan.exp ...
>> FAIL: c-c++-common/ubsan/float-cast-overflow-8.c   -O2  output pattern test
>> FAIL: c-c++-common/ubsan/overflow-mul-4.c   -O0  output pattern test
>>
>> ...
>>
>> when I am running the 1st test alone, then it works:
>>
>> $ make check-gcc RUNTESTFLAGS='ubsan.exp=float-cast-overflow-8.c'
>>
>> In an older log file I found for a different test from the same folder:
>>
>> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-7.h:149:1:
>> runtime error: value <unknown> is outside the range of representable values
>> of type 'unsigned int'
>> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-7.h:149:1:
>> runtime error: value <unknown> is outside the range of representable values
>> of type 'long int'
>> /home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-7.h:149:1:
>> runtime error: PASS: c-c++-common/ubsan
>>
>> BANG: "PASS" output from previous test run shreds this one?
>>
>> /float-cast-overflow-10.c   -O2 -flto -fuse-linker-plugin
>> -fno-fat-lto-objects  execution test
>> FAIL: c-c++-common/ubsan/float-cast-overflow-10.c   -O2 -flto
>> -fuse-linker-plugin -fno-fat-lto-objects  output pattern test
>> Output was:
>> c-c++-common/ubsan/float-cast-overflow-7.h:147:1: runtime error: value
>> <unknown> is outside the range of representable values of type 'signed char'
>> ...
>> c-c++-common/ubsan/float-cast-overflow-7.h:149:1: runtime error:
>> Should match:
>>
>> The last output line stops after "runtime error: ", i.e. at the place
>> where the "PASS" appears.
>>
>> Any ideas?
>
> Does this help?
>
> diff --git gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-8.c gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-8.c
> index 4adb22a..746fe20 100644
> --- gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-8.c
> +++ gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-8.c
> @@ -140,4 +140,4 @@ main ()
>  /* { dg-output "\[^\n\r]*value \[0-9.e+-]* is outside the range of representable values of type 'long long int'\[^\n\r]*(\n|\r\n|\r)" } */
>  /* { dg-output "\[^\n\r]*value -1 is outside the range of representable values of type 'long long unsigned int'\[^\n\r]*(\n|\r\n|\r)" } */
>  /* { dg-output "\[^\n\r]*value \[0-9.e+-]* is outside the range of representable values of type '__int128'\[^\n\r]*(\n|\r\n|\r)" { target { int128 } } } */
> -/* { dg-output "\[^\n\r]*value -1 is outside the range of representable values of type '__int128 unsigned'\[^\n\r]*(\n|\r\n|\r)" { target { int128 } } } */
> +/* { dg-output "\[^\n\r]*value -1 is outside the range of representable values of type '__int128 unsigned'" { target { int128 } } } */
>
> 	Marek

No, still switching back and forth (for a different set of options this 
time):

$ make check-gcc RUNTESTFLAGS='-all ubsan.exp=float-cast-overflow-8.c'

...
PASS: c-c++-common/ubsan/float-cast-overflow-8.c   -O2 -flto 
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test

$ make check-gcc RUNTESTFLAGS='-all ubsan.exp=float-cast-overflow-8.c'

...
FAIL: c-c++-common/ubsan/float-cast-overflow-8.c   -O2 -flto 
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test

In the last case, gcc.log reads (notice the "testsuiPASS" and the 
"trunk/gcc/testsui" at the end):



/home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-7.h:95:1: 
runtime error: 
val/home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuite/c-c++-common/ubsan/float-cast-overflow-7.h:95:1: 
runtime error: value -1 is outside the range of representable values of 
type 'short unsigned int'
/home/georg/gnu/gcc.gnu.org/trunk/gcc/testsuiPASS: 
c-c++-common/ubsan/float-cast-overflow-8.c   -O2 -flto 
-fuse-linker-plugin -fno-fat-lto-objects  execution test
FAIL: c-c++-common/ubsan/float-cast-overflow-8.c   -O2 -flto 
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test
Output was:
c-c++-common/ubsan/float-cast-overflow-7.h:93:1: runtime error: value 
-129 is outside the range of representable values of type 'signed char'

...

c-c++-common/ubsan/float-cast-overflow-7.h:95:1: runtime error: value 
-32769 is outside the range of representable values of type 'short int'
c-c++-common/ubsan/float-cast-overflow-7.h:95:1: runtime error: value -1 
is outside the range of representable values of type 'short unsigned int'
trunk/gcc/testsui
Should match: