Re: Testsuites and Test Results

[Ranjit Mathew <rmathew at hotmail dot com>]

> Ranjit> Apart from the humungous GCC testresults mailing
> Ranjit> list archive, is there a place where I can see "canonical"
> Ranjit> testresults for a i686-pc-linux-gnu target for GCJ so
> Ranjit> that I can compare how my builds (and modifications 
> Ranjit> thereof) fare?
> 
> In theory we should have 0 FAIL.  In practice there are always a few
> things we aren't XFAILing.  It would be nice to keep on top of this a
> bit better.

I meant "canonical" results for libjava testsuite, Mauve
and Jacks - maybe we should have weekly test runs against
Mauve and Jacks and have them posted to the GCJ site
so that we can see our progress (or otherwise). I know
that these testsuites keep getting added to, but still...

Now I have a couple of (possibly dumb) questions and
issues (everything is on i686-pc-linux-gnu with a
clean bootstrap of 3.3):

1. What exactly does XFAIL mean? Does it mean that "the
   testcase code is wrong and the compiler flagged an
   error as expected" or "we know that this is a bug
   in the compiler and it will fail until the bug is
   fixed"?

2. I download only "core", "g++" and "java" for any
   GCC release or snapshot. So the normal "make check"
   doesn't really work for me. It's only after seeing
   this page:

   http://gcc.gnu.org/onlinedocs/gccint/libgcj-Tests.html

   that I knew how to go about testing libgcj with
   just the packages I have. IMHO, the first paragraph
   in the page above should also be included in this
   page:

   http://gcc.gnu.org/install/test.html

3. For some weird reason that is not immediately
   obvious to me by looking at the testsuite/Makefile,
   my DejaGNU "runtest" is not picked up by make.
   "runtest" is in "/usr/local/bin" and it definitely
   is in my PATH. I have to explicitly say

   make RUNTEST="/usr/local/bin/runtest" check

4. Even after this, only about 85 tests are actually
   run (all PASS-es).

5. Tcl keeps complaining about INTERPRETER not being
   defined and definining it to "yes" on the make
   command line does not help.

6. I don't know why, but the executables are
   produced with a ".exe" extension. For example,
   "TestEarlyGC.exe", etc. Is this a bug or
   a feature?

Many thanks to anyone who helps me with these
issues.

Ranjit.

-- 
Ranjit Mathew          Email: rmathew AT hotmail DOT com

Bangalore, INDIA.      Web: http://ranjitmathew.tripod.com/