[Bug other/32999] New: Test results script "contrib/test_summary" broken ?

[rob1weld at aol dot com]

For a long time I've filed many test reports and they have a format similar to
this:
                === gcc tests ===

Running target unix
FAIL: gcc.c-torture/compile/20000804-1.c   -O0  (test for excess errors)
FAIL: gcc.c-torture/execute/20050316-2.c  execution,  -O0 
FAIL: gcc.c-torture/execute/20050316-2.c  execution,  -O1 
FAIL: gcc.c-torture/execute/20050316-2.c  execution,  -O2 
...
FAIL:  gcc.dg/dfp/usual-arith-conv.c (test for excess errors)
FAIL:  gcc.dg/builtin-apply4.c execution test
FAIL: gcc.dg/tree-ssa/gen-vect-11.c  scan-tree-dump-times vectorized 1 loops 1
FAIL:  gcc.dg/tree-ssa/gen-vect-11a.c scan-tree-dump-times vectorized 1 loops 1

  === gcc Summary ===

# of expected passes  42751
# of unexpected  failures 122
# of expected failures  116
# of unresolved  testcases 86
# of untested testcases  28
# of unsupported  tests  288
/opt/gcc-4_2-build/gcc/xgcc  version 4.2.2 20070804  (prerelease)


My most recent results are here:
http://gcc.gnu.org/ml/gcc-testresults/2007-08/msg00154.html


I understand that reading of these results and the making of graphs to show the
bug totals, or sending bug reports, is automated (and probably relies on the
reports mailed to gcc-testresults@gcc.gnu.org having a specific format).


In my most recent test I noticed that GCC "C" had 122 fails (mostly DFP) so I
decided to look at other peoples test reports to see if they also had this
problem. See page: http://gcc.gnu.org/ml/gcc-testresults/2007-08/



When I looked at a few other reports for the last few days I saw this sort of
thing (varies from day to day - of course):

                === gcc tests ===

Running target unix
UNRESOLVED: gcc.c-torture/execute/mayalias-2.c execution,  -O3 -g 

                === gcc Summary ===

# of expected passes            42158
# of expected failures          116
# of unresolved testcases       1
# of untested testcases         28
# of unsupported tests          294


Notice that it shows the "UNRESOLVED" line only and not the "FAIL" (like mine
does).

Here are three recent reports (on the same platform as mine):
http://gcc.gnu.org/ml/gcc-testresults/2007-08/msg00084.html
http://gcc.gnu.org/ml/gcc-testresults/2007-08/msg00097.html
http://gcc.gnu.org/ml/gcc-testresults/2007-08/msg00147.html


There are more. Some e-mail addresses are from automated testers and might not
be for incoming e-mails, just an account on the test machine to send e-mail out
- so no point mailing there ...

If it were just ONE person I would e-mail them, but it seems there are many. 

Has the format been changed to NOT reporting the results of the tests. :( 

I don't know why my script works and thiers is broken (or am I wrong).


-- 
           Summary: Test results script "contrib/test_summary" broken ?
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: major
          Priority: P3
         Component: other
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: rob1weld at aol dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32999