Re: Testsuites and Test Results

[Mark Wielaard <mark at klomp dot org>]

Hi,

On Sat, 2003-06-07 at 21:44, Anthony Green wrote:
> I got this from today's HEAD.  It contains mauve, but not jikes.
> 
>                 === libjava Summary ===
>  
> # of expected passes            11618
> # of unexpected failures        16
> # of expected failures          132
> # of untested testcases         12
> 
> These are the unexpected failures.  I'm testing a patch to fix a couple
> of them..

I was just looking at these results. And it seems there are some
regressions from 3.3. I have an autobuilder on powerpc that generates
results for 3.3 and 3.4 each day, but I have not gone back through the
gcc-testresults archive to see when the regressions started.

This is what I know about these failures:

> FAIL: gnu.testlet.java.awt.color.ColorSpace.isCS_sRGB (number 1)

Is a new test. Our code is (and was) clearly wrong/not implemented.

> FAIL: gnu.testlet.java.io.ObjectInputOutput.InputTest:
> gnu.testlet.java.io.ObjectInputOutput.Test$GetPutField (number 1)
> FAIL: gnu.testlet.java.io.ObjectInputOutput.OutputTest: Serializable:
> gnu.testlet.java.io.ObjectInputOutput.Test$Extern () (number 2)
> FAIL: gnu.testlet.java.io.ObjectInputOutput.OutputTest: Serializable:
> test(str=null, x=0) (number 2)

Haven't looked at these yet.

> FAIL: gnu.testlet.java.io.OutputStreamWriter.jdk11:
> OutputStreamWriter(writer, encoding) (number 1)
> FAIL: gnu.testlet.java.io.OutputStreamWriter.jdk11:
> OutputStreamWriter(writer, encoding) // alias (number 1)

These come from the strange deefinition of encoding names returned for
java.io classes. Normally you have to return the canonical (IANA) name
of an encoding. But for java.io classes the spec says to return the
historical (Sun specified) name.

> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error:
> test_shortbyteValue failed - 5 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_shortbyteValue
> failed - 5 (number 1)
> FAIL: gnu.testlet.java.lang.Math.MathTest: Error : test_pow failed - 9
> (number 1)
> FAIL: gnu.testlet.java.lang.Math.sin (number 1)

Haven't looked at these yet.

> FAIL: gnu.testlet.java.lang.String.surrogate: uncaught exception at 
> number 1: java.lang.ArrayIndexOutOfBoundsException: 2

Just committed a patch to solve this issue.

> FAIL: gnu.testlet.java.math.BigDecimal.setScale: quinn (number 2)

New test. I believe there is a patch for this one either java-patches or
the classpath mailinglist.

> FAIL: gnu.testlet.java.net.DatagramSocket.DatagramSocketTest2: Expected
> IOException (number 1)
> FAIL: gnu.testlet.java.net.DatagramSocket.DatagramSocketTest2: uncaught
> exception at "invalid_send_data" number 4:
> java.lang.IllegalArgumentException: Invalid length: -1

This test is disabled on the 3.3 branch.

> FAIL: gnu.testlet.java.net.Socket.SocketTest: Error : test_params failed
> - 1 get port did not return proper values (number 1)
> FAIL: gnu.testlet.java.net.Socket.jdk13: unexpected error: Success
> (number 1)

Don't know about these.

For x86 on the 3.3 branch (with my two recent patches) I now get
(including mauve, but not jacks) just:

                === libjava Summary ===
 
# of expected passes            11476
# of unexpected failures        3
# of unexpected successes       1
# of expected failures          158
# of untested testcases         14

WARNING: program timed out.
FAIL: SyncTest execution - gij test
WARNING: program timed out.
FAIL: SyncTest execution - gij test
FAIL: gnu.testlet.java.awt.color.ColorSpace.isCS_sRGB (number 1)
XPASS: gnu.testlet.java.io.File.jdk11: getCanonicalPath () (number 1)

Cheers,

Mark