Re: Mauve Bytecode Verifier Testsuite

[Ranjit Mathew <rmathew at hotmail dot com>]

Hi,

    Curiously, even JDK 1.4.2_03 on my Linux box fails
some verifier test cases:
------------------------- 8< -------------------------
FAIL: /usr/java/j2sdk1.4.2_03/bin/java exception.fail.depth
FAIL: /usr/java/j2sdk1.4.2_03/bin/java exception.fail.uninit
FAIL: /usr/java/j2sdk1.4.2_03/bin/java simple.fail.verify4
FAIL: /usr/java/j2sdk1.4.2_03/bin/java subr.pass.mergeok
FAIL: /usr/java/j2sdk1.4.2_03/bin/java subr.pass.noretend
FAIL: regress
------------------------- 8< -------------------------

Are these expected results or are the testcases wrong?

Or are we interpreting the JVM spec differently than
Sun? Or am I doing something wrong? (See my previous
mail below.)

BTW, even in this limited testsuite the GCJ compiler
seems to fare far worse than the interpreter.

-- gij ------------------ 8< -------------------------
FAIL: /home/ranmath/gcc/bin/gij simple.fail.dupinterface
FAIL: /home/ranmath/gcc/bin/gij subr.fail.jsr10
FAIL: /home/ranmath/gcc/bin/gij subr.fail.jsr8
FAIL: /home/ranmath/gcc/bin/gij subr.fail.merge
FAIL: /home/ranmath/gcc/bin/gij subr.fail.recurse1
FAIL: /home/ranmath/gcc/bin/gij subr.fail.recurse2
FAIL: regress
------------------------- 8< -------------------------

v/s

-- gcj ------------------ 8< -------------------------
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only abstract/fail/abstract1.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only abstract/fail/abstract2.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only abstract/fail/abstract3.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only abstract/fail/abstract5.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only cast/fail/cast.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only exception/fail/uninit.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only new/fail/caught.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only new/fail/cast.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only new/fail/notrun.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only new/fail/new1.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only newarray/fail/multianewarray1.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only null/fail/init.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only null/pass/getfield.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/fail/dupfield.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/fail/dupinterface.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/fail/dupmethod.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/fail/verify2.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/fail/verify4.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only simple/pass/allbytecodes.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only subr/fail/jsr10.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only subr/pass/mergeok.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only subr/pass/mergeok2.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only subr/pass/nonret.class
FAIL: /home/ranmath/gcc/bin/gcj --syntax-only subr/pass/simple.class
FAIL: regress
------------------------- 8< -------------------------

Results are with the current mainline.

Thanks,
Ranjit.


Ranjit Mathew wrote:
> Hi,
> 
>     How *exactly* does one integrate the Mauve
> bytecode verifier testsuite with the standard libjava
> testsuite?
> 
> I tried placing the "verify" folder's contents under
> "libjava/testsuite/libjava.verify" and even the folder
> itself, but the regular testsuite run didn't pick it
> up.
> 
> I then tried to configure and run the tests separately
> myself and I *think* I succeeded, but I just want to
> make sure. Here's what I did:
> 
> 1. Downloaded and installed the jasmin assembler.
> 
>    export JASMIN=/wherever/is/jasmin/binary
> 
> 2. Got the current Mauve "verify" module and configured
>    it.
> 
>    /path/to/verify/configure --with-gij=/wherever/is/current/gij
> 
> 3. "make -k check" on the configured folder.
> 
> I get 65 PASS-es and 7 FAIL-ures:
> 
>     FAIL: /home/ranmath/gcc/bin/gij simple.fail.dupinterface
>     FAIL: /home/ranmath/gcc/bin/gij subr.fail.jsr10
>     FAIL: /home/ranmath/gcc/bin/gij subr.fail.jsr8
>     FAIL: /home/ranmath/gcc/bin/gij subr.fail.merge
>     FAIL: /home/ranmath/gcc/bin/gij subr.fail.recurse1
>     FAIL: /home/ranmath/gcc/bin/gij subr.fail.recurse2
>     FAIL: regress
> 
> (These are with the current mainline gij on
> i686-pc-linux-gnu.)
> 
> Are these the expected results?
> 
> Some observations:
> 
> 1. Some of the assembler files have "^M" characters strewn
>    all over the place. Uggh!
> 
> 2. The "...we've chose to precompile all the .j files to .class
>    files and then import the result." line in
>    "libjava/testsuite/libjava.verify/README.verify" is wrong
>    as I can't see any class files and also because "we've chose"
>    is incorrect.
> 
> 3. What's with the "fail"/"pass" folders? Do they indicate
>    invalid/valid bytecode testcases respectively?
> 
> Thanks in advance for answering!
> 
> Great work, BTW, even if it's just a start - I couldn't
> find any other publically available bytecode verifier
> testsuite.
> 
> Ranjit.
> 


-- 
Ranjit Mathew          Email: rmathew AT hotmail DOT com

Bangalore, INDIA.      Web: http://ranjitmathew.tripod.com/