[Bug testsuite/30119] New: libjava testsuite output is erratic and unhelpful

[amylaar at gcc dot gnu dot org]

I have tested the patch for PR rtl-optimization/29858 in revision 119261
on gcc01 (i686-pc-linux-gnu)
Compared to a pristine build of revision 119055, these are the
additional failures:

> FAIL: gcc.dg/visibility-11.c scan-assembler memcpy@PLT
14a16,17
> FAIL: gcc.dg/vect/vect-pow-1.c scan-tree-dump pattern recognized
> FAIL: gcc.dg/vect/vect-pow-2.c scan-tree-dump pattern recognized
107a111,112
> FAIL: PR18699 execution - gij test
> FAIL: PR18699 execution - gij test

compared to a pristine build of 119261 (the base version), this are the
aditional failures:

110a111,115
> FAIL: PR18699 execution - gij test
> FAIL: PR18699 execution - gij test
> FAIL: SyncTest execution - gij test
> FAIL: SyncTest execution - gij test
> FAIL: SyncTest execution - gij test

With failures in the gcc core or c++ / libstdc++ testsuite, reproducing
failures is very straight forward : you cut & paste the appropriate line(s)
from the log file in order, and can thus ovserve the failure interactively,
and then use gdb and/or debugging dumps to further investigate.

The log file shows this about the PR18699 failure:
PASS: PR18699 -O3 output - source compiled test
byte compile: /home/amylaar/bld/2006-11-27-29858/i686/gcc/gcj
-B/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/
-B/home/amylaar/bld/2006-11-27-29858/i686/gcc/ --encoding=UTF-8 -C
-I/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar
-g
/home/amylaar/bld/2006-11-27-29858/srcw/libjava/testsuite/libjava.lang/PR18699.java
-d /home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libjava/testsuite
2>@ stdout
PASS: PR18699 byte compilation
PR18699PR18699 set_ld_library_path_env_vars:
ld_library_path=.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc
Setting LD_LIBRARY_PATH to
.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:.:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27-29858/i686/gcc:/home/amylaar/bld/2006-11-27-29858/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27-29858/i686/./gcc:/home/amylaar/bld/2006-11-27-29858/i686/./prev-gcc
FAIL: PR18699 execution - gij test

When I cut & past the line above 'PASS: PR18699 byte compilation', a file named
'@' is created, which contains:

gcj: stdout: No such file or directory

The only README file in the entire libjava testsuite is
testsuite/libjava.verify/README.verify .

The web documentation on testing http://gcc.gnu.org/install/test.html only
has the basic meaning of PASS/ FAIL etc for the benefit of a person who
installs the library without modifying any pieces of the GNU compiler
collection.

I should not be required to reverse-engineer the libjava testsuite before
I can interpret/debug test results for my patches to the gcc core.
There should be easy-to-follow documentation how I can get from
the debugging log to reproducing the failure.

Moreover, considering that there is no regession against the baseline for any
other part of the GNU compiler collection, and the recent track record of
libjava testing, it seems highly likely that the regressions are actually
testsuite failures.


-- 
           Summary: libjava testsuite output is erratic and unhelpful
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: testsuite
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: amylaar at gcc dot gnu dot org
OtherBugsDependingO 29842,29858
             nThis:


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119