[Bug testsuite/31912] New: Using included gettext changes number of "dump-noaddr" tests being ran with "make -i check".

["rob1weld at aol dot com" <gcc-bugzilla at gcc dot gnu dot org>]

I compiled the target i686-pc-cygwin, two times, with a long list of options. 

One compilation used "--with-included-gettext" the other used
"--without-included-gettext".


The "--with-included-gettext" version, "make -i check" reports:

                === gcc Summary ===
# of expected passes            44340     <---
# of unexpected failures        79
# of unexpected successes       1
# of expected failures          114
# of unresolved testcases       2
# of untested testcases         35
# of unsupported tests          357


The "--without-included-gettext" version, "make -i check" reports:

                === gcc Summary ===
# of expected passes            43732     <---
# of unexpected failures        79
# of unexpected successes       1
# of expected failures          114
# of unresolved testcases       2
# of untested testcases         35
# of unsupported tests          357



Notice that the "with" version has _many_ more "# of expected passes" results,
but that the other values are the same. Essentially this means that both had an
_equal_ number of _failures_, but a _different_ number of passes. 

I see this as a _different_ number of failures (some unreported) and (agree)
that there are a _different_ number of passes.

How are we supposed to know the _maximum_ "# of expected passes" ?

If we do not know what the greatest possible number of possible passes _could_
(might) be then how do we know if something is getting missed ?

I set out to investigate the cause of this difference and found that a
different number of "gcc.c-torture/unsorted/dump-noaddr.c" tests was the main
cause of the problem!


When I type "info gcc" it documents the "dump-noaddr" functions as follows:

`-fdump-noaddr'
     When doing debugging dumps (see `-d' option above), suppress
     address output.  This makes it more feasible to use diff on
     debugging dumps for compiler invocations with different compiler
     binaries and/or different text / bss / data / heap / stack / dso
     start locations.


What does that have to do with "gettext" - Nothing !?

It seems like the test framework is doing additional "dump-noaddr" tests if
"--with-included-gettext" is used as a ./configure option. The other way of
looking at it is that "make -i check" is missing "dump-noaddr" tests if
"--without-included-gettext" is used as a ./configure option. This seems to be
incorrect (to me). The _same_ number of "dump-noaddr" tests should be ran
whether or not we use "included-gettext".


I did a "diff -u -d" of the "make -i check" gcc.log files and found these
differences:


$ diff -u gcc--with-included-gettext.log gcc--without-included-gettext.log
--- gcc--with-included-gettext.log  2007-05-11 20:02:18.281250000 -0700
+++ gcc--without-included-gettext.log  2007-05-11 20:14:07.656250000 -0700
@@ -55398,634 +55398,6 @@
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer  
-DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=1 -x
c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column
-c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer  
-DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer  
-DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=2 -x
c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer  
-DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3
-fomit-frame-pointer  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-loops   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all
-fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-loops -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all
-fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer
-funroll-loops   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all
-fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-loops   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
-fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-loops -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column
-c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer
-funroll-loops   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3
-fomit-frame-pointer -funroll-loops  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions -DMASK=1 -x c --param ggc-min-heapsize=1
-da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions   -DMASK=2 -x c -da -fdump-tree-all
-fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions -DMASK=2 -x c -da -fdump-tree-all
-fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions   -DMASK=2 -x c -da -fdump-tree-all
-fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3
-fomit-frame-pointer -funroll-all-loops -finline-functions  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -g   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -g   -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -O3 -g  comparison
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3 -g  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -Os   -DMASK=1 -x c --param
ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -Os   -DMASK=2 -x c -da
-fdump-tree-all -fdump-noaddr
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -Os  comparison
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -Os  comparison
 Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O0   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
 spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O0 -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
 PASS: gcc.c-torture/unsorted/e.c,  -O0  
@@ -59086,151 +58458,151 @@
 Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/xzz.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/xzz.c
   (timeout = 300)
 spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/xzz.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/xzz.c 
... (many lines deleted)
@@ -133562,7 +132934,7 @@

                === gcc Summary ===

-# of expected passes           44340
+# of expected passes           43732
 # of unexpected failures       79
 # of unexpected successes      1
 # of expected failures         114
@@ -133573,9 +132945,9 @@



For some it might be easier to view a snippet (in non-diff format) starting at
line 284:


File: gcc--with-included-gettext.log

PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer  
-DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=1 -x
c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column
-c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer  
-DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer  
-DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c
   (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=2 -x
c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer  
-DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -O3 -fomit-frame-pointer 
comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.003t.original,  -O3
-fomit-frame-pointer  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.004t.gimple,  -O3
-fomit-frame-pointer  comparison


File: gcc--without-included-gettext.log

PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O0   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O0 -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O0  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O1   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O1 -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O1  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O2   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O2 -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O2  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer  
-fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-loops   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-loops -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer -funroll-loops  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer
-funroll-all-loops -finline-functions -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer -funroll-all-loops
-finline-functions  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -fno-show-column -c  -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
  (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc
-B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -fno-show-column -c -o
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o
/cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -g  



Notice that the gcc.log ("File: gcc--with-included-gettext.log")    does "-O2 
comparison" tests and then does "-O3 -fomit-frame-pointer  comparison" tests.


Notice that the gcc.log ("File: gcc--without-included-gettext.log") does "-O2 
comparison" tests and then does "gcc.c-torture/unsorted/e.c" tests. It _misses_
the "-O3 -fomit-frame-pointer  comparison" tests (and many other "dump-noaddr"
tests also).



Point 1): It would be better if each section's (not just gcc) "summary" showed:

"# of tests available"    (meaning the total number of every possible test
(supported OR not))
"# of expected passes"    (meaning the maximum [supported] that _could_ be
attempted)
"# of unexpected misses"  (meaning that something didn't add up and the test
framework is wrong)


Worse yet, the "summary" sections are not consistent for all tests. Here are a
couple of examples:

The "acats Summary" shows only:
# of expected passes            1940
# of unexpected failures        19
# of unsupported tests          356

The "g++ Summary" shows a bit more:
# of expected passes            13571
# of unexpected failures        28
# of unexpected successes       3
# of expected failures          66
# of unsupported tests          168

The "gcc Summary" shows more categories:
# of expected passes            44340
# of unexpected failures        79
# of unexpected successes       1
# of expected failures          114
# of unresolved testcases       2
# of untested testcases         35
# of unsupported tests          357

Etc. .... ....


The "Installing GCC: Testing" webpage from gcc.gnu.org uses these names for the
tests:

PASS: the test passed as expected 
FAIL: the test unexpectedly failed 
XPASS: the test unexpectedly passed 
XFAIL: the test failed as expected 
ERROR: the testsuite detected an error 
WARNING: the testsuite detected a possible problem 
UNSUPPORTED: the test is not supported on this platform 


It is unclear if the above "unresolved" or "untested" refer to "ERROR" or
"WARNING" in each section of the "summary". Not all compiler errors are getting
grep'd and one must manually inspect the .log files. What is clear is that not
every section is reporting all it's results and none is listing the "TOTAL"
available.

That makes it hard to catch the bug that is the subject of this report (unless
you compile gcc many times with many different options AND study the .log files
- a lot of work).



Point 2): The same number of tests should be performed for "dump-noaddr"
regardless of the "--?--included-gettext" option since "gettext" and
"dump-noaddr" do not have a reason to interact.



Point 3): If you examine the end of the attached .log files you will notice
_many_ errors: "undefined reference to `_libiconv_open'". 

The above "gcc summary" says: "# of unexpected failures 79". If I type this (on
a full length .log file):

$grep 'undefined reference to'
/cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/gcc.log | wc -l
57

$grep _libiconv_open /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/gcc.log
| wc -l
26

(PS: typing those grep commands on the i686-pc-linux-gnu target gives the
number "0").



Note:

"--with-included-gettext"    configure flags: --verbose --with-tune=athlon-xp
--prefix=/usr --enable-objc-gc --enable-concept-checks --disable-multilib
--exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --libexecdir=/usr/lib
--mandir=/usr/share/man --infodir=/usr/share/info
--with-gxx-include-dir=/usr/include/c++/4.2 --enable-libstdcxx-debug
--enable-static --enable-shared --enable-initfini-array --enable-__cxa_atexit
--enable-threads=posix --enable-version-specific-runtime-libs --enable-libssp
--enable-libmudflap --enable-libgomp --disable-werror --enable-nls
--with-included-gettext --enable-decimal-float --with-long-double-128
--enable-debug --enable-java-gc=boehm --with-x --x-includes=/usr/X11R6/include
--x-libraries=/usr/X11R6/lib --enable-java-awt=gtk,xlib --enable-gtk-cairo
--enable-qt-peer --enable-xmlj --enable-gconf-peer --enable-tool-wrappers
--enable-portable-native-sync --enable-libgcj-multifile --enable-libgcj
--with-stabs --enable-hash-synchronization --enable-gc-debug
--enable-interpreter --with-system-zlib --enable-libada --with-tls
--enable-win32-registry --with-cpu=athlon-xp --with-arch=athlon-xp
--disable-checking --enable-languages=c,ada,c++,fortran,java,objc,obj-c++


"--without-included-gettext" configure flags: --verbose --with-tune=athlon-xp
--prefix=/usr --enable-objc-gc --enable-concept-checks --disable-multilib
--exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --libexecdir=/usr/lib
--mandir=/usr/share/man --infodir=/usr/share/info
--with-gxx-include-dir=/usr/include/c++/4.2 --enable-libstdcxx-debug
--enable-static --enable-shared --enable-initfini-array --enable-__cxa_atexit
--enable-threads=posix --enable-version-specific-runtime-libs --enable-libssp
--enable-libmudflap --enable-libgomp --disable-werror --enable-nls
--without-included-gettext --enable-decimal-float --with-long-double-128
--enable-debug --enable-java-gc=boehm --with-x --x-includes=/usr/X11R6/include
--x-libraries=/usr/X11R6/lib --enable-java-awt=gtk,xlib --enable-gtk-cairo
--enable-qt-peer --enable-xmlj --enable-gconf-peer --enable-tool-wrappers
--enable-portable-native-sync --enable-libgcj-multifile --enable-libgcj
--with-stabs --enable-hash-synchronization --enable-gc-debug
--enable-interpreter --with-system-zlib --enable-libada --with-tls
--enable-win32-registry --with-cpu=athlon-xp --with-arch=athlon-xp
--disable-checking --enable-languages=c,ada,c++,fortran,java,objc,obj-c++

The "enter_bug.cgi" page doesn't let you make attachments (on your first post)
so I need to wait until the "Create a New Attachment (proposed patch, testcase,
etc.)" text appears and make a _second_ post to add the .log files.


-- 
           Summary: Using included gettext changes number of "dump-noaddr"
                    tests being ran with "make -i check".
           Product: gcc
           Version: 4.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: rob1weld at aol dot com
 GCC build triplet: i686-pc-cygwin
  GCC host triplet: i686-pc-cygwin
GCC target triplet: i686-pc-cygwin


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31912