Re: Analysis of Mauve failures - Part 1

[Tom Tromey <tromey at redhat dot com>]

>>>>> "Mark" == Mark Wielaard <mark@klomp.org> writes:

Mark> Here are some notes on the Mauve failures I am currently seeing.

Thanks a lot for doing this!  It is a big help for this release.

>> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 28)

Mark> All these failures seem to be because the test tries to
Mark> initialize a java.awt class which fails since java.awt.Toolkit
Mark> cannot be initialized.  Some tests succeed for Object, BitSet
Mark> and Applet. So it seems we must implement more of java.awt first
Mark> before seeing progress on this test.

Meanwhile let's disable these tests in mauve-libgcj.  Feel free to
simply check in such a patch.

Mark> Interaction between a BufferedByteOutputStream wrapped around an
Mark> OutputStream that throws an InterruptedIOException after a
Mark> couple of bytes. Would be nice to get right but using an
Mark> OutputStream that is interrupted seems dangerous anyway.

I think I wrote this test.  It is pretty optimistic.  I seem to recall
general agreement that InterruptedIOException couldn't really be
relied on, since even Sun didn't fully implement it on all platforms.

Mark> Skipping analysis of Character failures (seem to be about 80)
Mark> since Eric has recently rewritten Character support for Unicode
Mark> 3.0 and the tests seem to be written for an older Unicode
Mark> version. Probably needs a rewrite of the Mauve tests.

The Character tests in Mauve have always been a bit flaky, too.  In
the earlier days, when I wrote this code, the specifications were not
as tight as they are now.  So these tests are a bit subjective.

>> FAIL: gnu.testlet.java.lang.Class.ClassTest: Error: test_getClassLoader failed - 1 (number 1)
Mark> It seems to expect a core class like java.lang.String to have a 'null'
Mark> ClassLoader which seems reasonable.

There's a few long threads on this subject.  Basically doing this is
hard, and arguably the current way is correct anyway.  (As I
understand it.)

>> FAIL: gnu.testlet.java.lang.Math.MathTest: Error : test_atan2 failed - 5 (number 1)
>> FAIL: gnu.testlet.java.lang.Math.max (number 2)
>> FAIL: gnu.testlet.java.lang.Math.max (number 18)
Mark> Heisenbugs!

These sure are curious.

Mark> FAIL: gnu.testlet.java.lang.Math.min: Float NaNs (number 1)
Mark> got 0.0 but expected NaN
Mark> FAIL: gnu.testlet.java.lang.Math.min: Float NaNs (number 2)
Mark> got 0.0 but expected NaN
Mark> 2 of 41 tests failed

Are you building with -O?  Math.min might be inlined; maybe the
inlining fails on your platform?  (What platform is it?)

>> FAIL: gnu.testlet.java.lang.StringBuffer.plus (number 1)
Mark> Seems to be a compiler error. method_returning_null() + ""
Mark> should result in the String "null" not null.

I think we have a PR for this already.  I seem to remember running
across this at some point.  I don't have the PR number handy though.

>> FAIL: gnu.testlet.java.lang.reflect.Method.invoke: takes_int (number 7)
Mark> Test expects a NullPointerException not an IllegalArgumentException when
Mark> it supplies a args[0] = null. Quick reading of API makes me think that
Mark> the test is wrong.

I agree.  In cases such as these running the test against the JDK can
also be helpful.

>> FAIL: gnu.testlet.java.lang.reflect.Modifier.toString12 (number 1)
>> FAIL: gnu.testlet.java.lang.reflect.Modifier.toString12 (number 2)
Mark> Test expects Modifier.STRICT to be represented by the String "strict",
Mark> we return "strictfp" which seems correct to me.

This is my fault :-(.  I made this same mistake in libgcj at one point.
The test is definitely in error.

Tom