Different test results for same source, configure, host?

[Georg-Johann Lay]

Since quite some time now I am observing differences in testsuite
results, even in the case when the compiler is build on the same
host from the same sources, and configured the same way.

What I did:

Host is cfarm421: x86_64 GNU/Linux, Debian 6.12.57-1

1) Checked out same GCC sources (usually trunk) to

$HOME/gnu/source/gcc-master
$HOME/gnu/source/gcc-x

2) In $HOME/gnu/build/gcc-host configure + build + test:

$ ../../source/gcc-master/configure --enable-languages=c,c++ --disable-bootstrap
$ make -j44
$ (cd gcc; make -j44 -k check-gcc)

3) In $HOME/gnu/build/gcc-x-host configure + build + test:

$ ../../source/gcc-x/configure --enable-languages=c,c++ --disable-bootstrap
$ make -j44
$ (cd gcc; make -j44 -k check-gcc)

4) Then compare the test outcomes:

$ $HOME/gnu/source/gcc-master/contrib/compare_tests ~/gnu/build/gcc-host/gcc/testsuite/ ~/gnu/build/gcc-x-host/gcc/testsuite/

compare_tests's output reads:

# Comparing directories
## Dir1=/home/gjl/gnu/build/gcc-host/gcc/testsuite/: 1 sum files
## Dir2=/home/gjl/gnu//build/gcc-x-host/gcc/testsuite/: 1 sum files

# Comparing 1 common sum files
## /bin/sh /home/gjl/gnu/source/gcc-master/contrib/compare_tests  /tmp/gxx-sum1.4194012 /tmp/gxx-sum2.4194012
Non-unique test names: (Eeek!)
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-Wbidi-chars.c  -Wc++-compat   scan-sarif-file "text": "unpaired UTF-8 bidirectional control characters detected"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-1.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<80>'"
       6 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<bf>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<c0>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<c1>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<c2>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<e0>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<e0><80><bf>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<e0><9f><80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<e0><bf>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<ec><80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<ed><a0><80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<f0><80><80><80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<f0><8f><bf><bf>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<f4><90><80><80>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<f5>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<fd><bf><bf><bf>'"
       2 PASS: gcc: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c  -Wc++-compat   scan-sarif-file "text": "invalid UTF-8 character '<ff>'"
       3 PASS: gcc: c-c++-common/goacc/combined-reduction.c scan-tree-dump-times gimple "acc loop reduction.+:v1. private.i." 1
       3 PASS: gcc: c-c++-common/gomp/append-args-6.c scan-tree-dump-times gimple "D.[0-9]+ = __builtin_omp_get_mapped_ptr \\(cp2, 4\\);" 1
       2 PASS: gcc: c-c++-common/gomp/requires-5.c scan-tree-dump original "#pragma omp atomic release"
       3 PASS: gcc: c-c++-common/pr57371-1.c  -Wc++-compat   scan-tree-dump original "return (<retval> = )?x != 0"
       2 PASS: gcc: gcc.dg/attr-noipa.c scan-tree-dump optimized "fn20 \\(\\)"
       2 PASS: gcc: gcc.dg/format/diagnostic-ranges.c diagnostic-ranges-html.py::test_annotation
       2 PASS: gcc: gcc.dg/format/diagnostic-ranges.c diagnostic-ranges-html.py::test_messag
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_0.o assemble, -fcommon
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_0.o-c_lto_20111207-1_3.o execute -fcommon
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_0.o-c_lto_20111207-1_3.o link, -fcommon
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_1.o assemble, -fcommon
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_2.o assemble, -fcommon
       3 PASS: gcc: gcc.dg/lto/20111207-1 c_lto_20111207-1_3.o assemble, -fcommon
       2 PASS: gcc: gcc.dg/sarif-output/multiple-outputs.c (test .sarif output for UTF-8-encoded parseable JSON)
       2 UNSUPPORTED: gcc: gcc.dg/sarif-output/multiple-outputs.c verify-sarif-file: check-jsonschema is missing
       2 PASS: gcc: gcc.dg/vect/vect-fmin-1.c -flto -ffat-lto-objects  scan-tree-dump vect "Detected reduction"
       2 PASS: gcc: gcc.dg/vect/vect-fmin-1.c scan-tree-dump vect "Detected reduction"
       2 PASS: gcc: gcc.dg/vect/vect-reduc-dot-s16a.c -flto -ffat-lto-objects  scan-tree-dump-times vect "vectorized 1 loops" 1
       2 PASS: gcc: gcc.dg/vect/vect-reduc-dot-s16a.c scan-tree-dump-times vect "vectorized 1 loops" 1

## Differences found
# 1 differences in 1 common sum files found


I also see similar diffs for other targets.

Can someone give a hint what I am doing wrong and how to fix that?

Johann