This is the mail archive of the
gcc-bugs@gcc.gnu.org
mailing list for the GCC project.
[Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
- From: "amylaar at gcc dot gnu dot org" <gcc-bugzilla at gcc dot gnu dot org>
- To: gcc-bugs at gcc dot gnu dot org
- Date: 19 Dec 2006 12:57:19 -0000
- Subject: [Bug testsuite/30119] libjava testsuite output is erratic and unhelpful
- References: <bug-30119-5394@http.gcc.gnu.org/bugzilla/>
- Reply-to: gcc-bugzilla at gcc dot gnu dot org
------- Comment #10 from amylaar at gcc dot gnu dot org 2006-12-19 12:57 -------
(In reply to comment #9)
> OK, so it looks like this is not a testsuite problem: the problem really is
> with the program being tested.
It is also a testsuite problem when the tests are randonly passing or failing;
such tests are not suitable as such in an automated testsuite.
I suppose you could write a wrapper to run them several times to get a
reasonable high confidence that the test generally passes - if the test does
not take too long - and maybe the probabilities can also be enhanced by
forking specifically designed other programs to perturb the load, but
unless/ until such infrastructure is put in place, these randomly failing
tests make it effectively impossible to pass a regression test.
I've redone to tests on my baseline where the test passed during the automated
test, and the results are again random:
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$
/home/amylaar/bld/2006-11-27/i686/gcc/gcj
-B/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/
-B/home/amylaar/bld/2006-11-27/i686/gcc/ --encoding=UTF-8 -C
-I/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite/../libgcj-4.3.0.jar
-g
/home/amylaar/bld/2006-11-27/srcw/libjava/testsuite/libjava.lang/PR18699.java
-d /home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export
ld_library_path=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ export
LD_LIBRARY_PATH=.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:.:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/./libjava/.libs:/home/amylaar/bld/2006-11-27/i686/gcc:/home/amylaar/bld/2006-11-27/i686/./bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-bfd/.libs:/home/amylaar/bld/2006-11-27/i686/./opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/./prev-opcodes/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libstdc++-v3/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libmudflap/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libssp/.libs:/home/amylaar/bld/2006-11-27/i686/i686-pc-linux-gnu/libgomp/.libs:/home/amylaar/bld/2006-11-27/i686/./gcc:/home/amylaar/bld/2006-11-27/i686/./prev-gcc
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-2" java.lang.NullPointerException
at PR18699.update(PR18699.java:34)
at java.util.Observable.notifyObservers(Observable.java:171)
at PR18699.run(PR18699.java:26)
at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-1" ^[[Ajava.lang.NullPointerException
at PR18699.update(PR18699.java:34)
at java.util.Observable.notifyObservers(Observable.java:171)
at PR18699.run(PR18699.java:26)
at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
Exception in thread "Thread-2" ^[[A
java.lang.NullPointerException
at PR18699.update(PR18699.java:34)
at java.util.Observable.notifyObservers(Observable.java:171)
at PR18699.run(PR18699.java:26)
at java.lang.Thread.run(Thread.java:682)
1
amylaar@gcc01:~/bld/2006-11-27/i686/i686-pc-linux-gnu/libjava/testsuite$ ../gij
-classpath .:../libgcj-4.3.0.jar PR18699;echo $?
0
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30119