Bug 31912 - Using included gettext changes number of "dump-noaddr" tests being ran with "make -i check".
Summary: Using included gettext changes number of "dump-noaddr" tests being ran with "...
Status: UNCONFIRMED
Alias: None
Product: gcc
Classification: Unclassified
Component: testsuite (show other bugs)
Version: 4.2.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-05-13 14:23 UTC by Rob
Modified: 2007-05-13 14:29 UTC (History)
1 user (show)

See Also:
Host: i686-pc-cygwin
Target: i686-pc-cygwin
Build: i686-pc-cygwin
Known to work:
Known to fail:
Last reconfirmed:


Attachments
gcc.log (configured using "without") (4.12 KB, text/plain)
2007-05-13 14:27 UTC, Rob
Details
gcc.log (configured using "with") (7.30 KB, text/plain)
2007-05-13 14:29 UTC, Rob
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rob 2007-05-13 14:23:48 UTC
I compiled the target i686-pc-cygwin, two times, with a long list of options. 

One compilation used "--with-included-gettext" the other used "--without-included-gettext".


The "--with-included-gettext" version, "make -i check" reports:

		=== gcc Summary ===
# of expected passes		44340     <---
# of unexpected failures	79
# of unexpected successes	1
# of expected failures		114
# of unresolved testcases	2
# of untested testcases		35
# of unsupported tests		357


The "--without-included-gettext" version, "make -i check" reports:

		=== gcc Summary ===
# of expected passes		43732     <---
# of unexpected failures	79
# of unexpected successes	1
# of expected failures		114
# of unresolved testcases	2
# of untested testcases		35
# of unsupported tests		357



Notice that the "with" version has _many_ more "# of expected passes" results, but that the other values are the same. Essentially this means that both had an _equal_ number of _failures_, but a _different_ number of passes. 

I see this as a _different_ number of failures (some unreported) and (agree) that there are a _different_ number of passes.

How are we supposed to know the _maximum_ "# of expected passes" ?

If we do not know what the greatest possible number of possible passes _could_ (might) be then how do we know if something is getting missed ?

I set out to investigate the cause of this difference and found that a different number of "gcc.c-torture/unsorted/dump-noaddr.c" tests was the main cause of the problem!


When I type "info gcc" it documents the "dump-noaddr" functions as follows:

`-fdump-noaddr'
     When doing debugging dumps (see `-d' option above), suppress
     address output.  This makes it more feasible to use diff on
     debugging dumps for compiler invocations with different compiler
     binaries and/or different text / bss / data / heap / stack / dso
     start locations.


What does that have to do with "gettext" - Nothing !?

It seems like the test framework is doing additional "dump-noaddr" tests if "--with-included-gettext" is used as a ./configure option. The other way of looking at it is that "make -i check" is missing "dump-noaddr" tests if "--without-included-gettext" is used as a ./configure option. This seems to be incorrect (to me). The _same_ number of "dump-noaddr" tests should be ran whether or not we use "included-gettext".


I did a "diff -u -d" of the "make -i check" gcc.log files and found these differences:


$ diff -u gcc--with-included-gettext.log gcc--without-included-gettext.log
--- gcc--with-included-gettext.log  2007-05-11 20:02:18.281250000 -0700
+++ gcc--without-included-gettext.log  2007-05-11 20:14:07.656250000 -0700
@@ -55398,634 +55398,6 @@
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
 PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3 -fomit-frame-pointer  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-loops   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-loops -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer -funroll-loops   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-loops   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-loops -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer -funroll-loops   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3 -fomit-frame-pointer -funroll-loops  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -g   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -g   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -O3 -g  comparison
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O3 -g  comparison
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -Os   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
-Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
-spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
-PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -Os   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -Os  comparison
... (many lines deleted)
-PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -Os  comparison
 Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O0   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
 spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O0 -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
 PASS: gcc.c-torture/unsorted/e.c,  -O0  
@@ -59086,151 +58458,151 @@
 Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -Os   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/xzz.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/xzz.c    (timeout = 300)
 spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -Os -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/xzz.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/xzz.c 
... (many lines deleted)
@@ -133562,7 +132934,7 @@
 
                === gcc Summary ===
 
-# of expected passes           44340
+# of expected passes           43732
 # of unexpected failures       79
 # of unexpected successes      1
 # of expected failures         114
@@ -133573,9 +132945,9 @@



For some it might be easier to view a snippet (in non-diff format) starting at line 284:


File: gcc--with-included-gettext.log

PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer   -DMASK=1 -x c --param ggc-min-heapsize=1 -da -fdump-tree-all -fdump-noaddr
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/dump-noaddr.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/dump-noaddr.c 
PASS: gcc.c-torture/unsorted/dump-noaddr.c,  -O3 -fomit-frame-pointer   -DMASK=2 -x c -da -fdump-tree-all -fdump-noaddr
PASS: gcc.c-torture/unsorted/dump-noaddr.c.001t.tu,  -O3 -fomit-frame-pointer  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.003t.original,  -O3 -fomit-frame-pointer  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.004t.gimple,  -O3 -fomit-frame-pointer  comparison


File: gcc--without-included-gettext.log

PASS: gcc.c-torture/unsorted/dump-noaddr.c.156r.barriers,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.159r.eh-ranges,  -O2  comparison
PASS: gcc.c-torture/unsorted/dump-noaddr.c.160r.shorten,  -O2  comparison
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O0   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O0 -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O0  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O1   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O1 -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O1  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O2   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O2 -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O2  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-loops   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-loops -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer -funroll-loops  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions  
Executing on host: /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/  -w  -O3 -g   -fno-show-column -c  -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c    (timeout = 300)
spawn /cygdrive/c/gcc-4_2-branch-build/gcc/xgcc -B/cygdrive/c/gcc-4_2-branch-build/gcc/ -w -O3 -g -fno-show-column -c -o /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/e.o /cygdrive/C/makecygwin/gcc-4_2-branch/gcc/testsuite/gcc.c-torture/unsorted/e.c 
PASS: gcc.c-torture/unsorted/e.c,  -O3 -g  



Notice that the gcc.log ("File: gcc--with-included-gettext.log")    does "-O2  comparison" tests and then does "-O3 -fomit-frame-pointer  comparison" tests.


Notice that the gcc.log ("File: gcc--without-included-gettext.log") does "-O2  comparison" tests and then does "gcc.c-torture/unsorted/e.c" tests. It _misses_ the "-O3 -fomit-frame-pointer  comparison" tests (and many other "dump-noaddr" tests also).



Point 1): It would be better if each section's (not just gcc) "summary" showed:

"# of tests available"    (meaning the total number of every possible test (supported OR not))
"# of expected passes"    (meaning the maximum [supported] that _could_ be attempted)
"# of unexpected misses"  (meaning that something didn't add up and the test framework is wrong)


Worse yet, the "summary" sections are not consistent for all tests. Here are a couple of examples:

The "acats Summary" shows only:
# of expected passes		1940
# of unexpected failures	19
# of unsupported tests		356

The "g++ Summary" shows a bit more:
# of expected passes		13571
# of unexpected failures	28
# of unexpected successes	3
# of expected failures		66
# of unsupported tests		168

The "gcc Summary" shows more categories:
# of expected passes		44340
# of unexpected failures	79
# of unexpected successes	1
# of expected failures		114
# of unresolved testcases	2
# of untested testcases		35
# of unsupported tests		357

Etc. .... ....


The "Installing GCC: Testing" webpage from gcc.gnu.org uses these names for the tests:

PASS: the test passed as expected 
FAIL: the test unexpectedly failed 
XPASS: the test unexpectedly passed 
XFAIL: the test failed as expected 
ERROR: the testsuite detected an error 
WARNING: the testsuite detected a possible problem 
UNSUPPORTED: the test is not supported on this platform 


It is unclear if the above "unresolved" or "untested" refer to "ERROR" or "WARNING" in each section of the "summary". Not all compiler errors are getting grep'd and one must manually inspect the .log files. What is clear is that not every section is reporting all it's results and none is listing the "TOTAL" available.

That makes it hard to catch the bug that is the subject of this report (unless you compile gcc many times with many different options AND study the .log files - a lot of work).



Point 2): The same number of tests should be performed for "dump-noaddr" regardless of the "--?--included-gettext" option since "gettext" and "dump-noaddr" do not have a reason to interact.



Point 3): If you examine the end of the attached .log files you will notice _many_ errors: "undefined reference to `_libiconv_open'". 

The above "gcc summary" says: "# of unexpected failures 79". If I type this (on a full length .log file):

$grep 'undefined reference to' /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/gcc.log | wc -l
57

$grep _libiconv_open /cygdrive/c/gcc-4_2-branch-build/gcc/testsuite/gcc/gcc.log | wc -l
26

(PS: typing those grep commands on the i686-pc-linux-gnu target gives the number "0").



Note:

"--with-included-gettext"    configure flags: --verbose --with-tune=athlon-xp --prefix=/usr --enable-objc-gc --enable-concept-checks --disable-multilib --exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-gxx-include-dir=/usr/include/c++/4.2 --enable-libstdcxx-debug --enable-static --enable-shared --enable-initfini-array --enable-__cxa_atexit --enable-threads=posix --enable-version-specific-runtime-libs --enable-libssp --enable-libmudflap --enable-libgomp --disable-werror --enable-nls --with-included-gettext --enable-decimal-float --with-long-double-128 --enable-debug --enable-java-gc=boehm --with-x --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --enable-java-awt=gtk,xlib --enable-gtk-cairo --enable-qt-peer --enable-xmlj --enable-gconf-peer --enable-tool-wrappers --enable-portable-native-sync --enable-libgcj-multifile --enable-libgcj --with-stabs --enable-hash-synchronization --enable-gc-debug --enable-interpreter --with-system-zlib --enable-libada --with-tls --enable-win32-registry --with-cpu=athlon-xp --with-arch=athlon-xp --disable-checking --enable-languages=c,ada,c++,fortran,java,objc,obj-c++


"--without-included-gettext" configure flags: --verbose --with-tune=athlon-xp --prefix=/usr --enable-objc-gc --enable-concept-checks --disable-multilib --exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-gxx-include-dir=/usr/include/c++/4.2 --enable-libstdcxx-debug --enable-static --enable-shared --enable-initfini-array --enable-__cxa_atexit --enable-threads=posix --enable-version-specific-runtime-libs --enable-libssp --enable-libmudflap --enable-libgomp --disable-werror --enable-nls --without-included-gettext --enable-decimal-float --with-long-double-128 --enable-debug --enable-java-gc=boehm --with-x --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --enable-java-awt=gtk,xlib --enable-gtk-cairo --enable-qt-peer --enable-xmlj --enable-gconf-peer --enable-tool-wrappers --enable-portable-native-sync --enable-libgcj-multifile --enable-libgcj --with-stabs --enable-hash-synchronization --enable-gc-debug --enable-interpreter --with-system-zlib --enable-libada --with-tls --enable-win32-registry --with-cpu=athlon-xp --with-arch=athlon-xp --disable-checking --enable-languages=c,ada,c++,fortran,java,objc,obj-c++

The "enter_bug.cgi" page doesn't let you make attachments (on your first post) so I need to wait until the "Create a New Attachment (proposed patch, testcase, etc.)" text appears and make a _second_ post to add the .log files.
Comment 1 Rob 2007-05-13 14:27:20 UTC
Created attachment 13547 [details]
gcc.log (configured using "without")
Comment 2 Rob 2007-05-13 14:29:18 UTC
Created attachment 13548 [details]
gcc.log (configured using "with")