This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
PATCH: install/test.html
- To: gcc-patches at gcc dot gnu dot org
- Subject: PATCH: install/test.html
- From: Gerald Pfeifer <pfeifer at dbai dot tuwien dot ac dot at>
- Date: Wed, 30 Aug 2000 10:43:24 +0200 (CEST)
Revive the testsuite instructions and update them for GCC.
(There is a disclaimer concerning GCC 2.95.x, but I thought it's better
to provide the information for anyone who wants to run the testsuite,
especially in the light of the GCC 3.0 release process.)
I'll install that, once CVS doesn't hang...
Gerald
Index: test.html
===================================================================
RCS file: /cvs/gcc/wwwdocs/htdocs/install/test.html,v
retrieving revision 1.19
diff -c -3 -p -r1.19 test.html
*** test.html 2000/05/15 18:45:19 1.19
--- test.html 2000/08/30 08:38:00
***************
*** 1,20 ****
<html>
<head>
<title>Installation Instructions: Testing GCC</title>
</head>
<body>
<h1 align="center">Installation Instructions: Testing GCC</h1>
! <p>The testsuite has been temporarily removed from the gcc distributions
! until the copyright status of some tests can be determined. It is hoped
! that the testsuite will be included in the gcc releases again soon.
! <p>Until the testsuite is included it is impossible to run the testsuite.
! Sorry.
! <hr>
! <p><a href="./index.html">Return to the GCC Installation page</a></p>
</body>
</html>
--- 1,110 ----
<html>
+
<head>
<title>Installation Instructions: Testing GCC</title>
</head>
<body>
<h1 align="center">Installation Instructions: Testing GCC</h1>
+
+ <p><strong>Please note that this is only applicable to current development
+ versions of GCC and GCC 3.0 or latter. GCC 2.95.x does not come with a
+ testsuite.</strong></p>
+
+ <p>Before you install GCC, you might wish to run the testsuite. This
+ step is optional and may require you to download additional software.</p>
+
+ <p>First, you must have downloaded the testsuites; the full distribution
+ contains testsuites. If you downloaded the "core" compiler plus any front
+ ends, then you do not have the testsuites. You can download the testsuites
+ from the same site where you downloaded the core distribution and language
+ front ends.</p>
+
+ <p>Second, you must have a new version of dejagnu on your system; dejagnu-1.3
+ will not work. We have made dejagnu snapshots available in
+ <a href="ftp://gcc.gnu.org/pub/gcc/infrastructure">
+ gcc.gnu.org:/pub/gcc/infrastructure</a> until a new version of
+ dejagnu can be released. If you find multiple snapshots of dejagnu
+ there, you should probably get the most recent one.
+ (Note, dejagnu will not build/run on NEXTSTEP systems.)</p>
+
+
+ <p>Assuming you've got the testsuites unpacked and have installed an
+ appropriate dejagnu, you can run the testsuite with `<code>cd
+ <i>objdir</i>; make -k check</code>`.
+ This may take a long time. Go get some lunch.</p>
+
+ <p>The testing process will try to test as many components in the GCC
+ distribution as possible, including the C, C++ and Fortran compiler as
+ well as the C++ runtime libraries.</p>
+
+
+ <h2>How can I run the test suite on selected tests?</h2>
+
+ <p>As a first possibility to cut down the number of tests that are run it is
+ possible to use `<code>make check-gcc</code>` or `<code>make check-g++</code>`
+ in the gcc subdirectory of the object directory. To further cut down the
+ tests the following is possible:
+ <pre>
+ make check-gcc RUNTESTFLAGS="execute.exp <other options>"
+ </pre>
+ This will run all gcc execute tests in the testsuite.
+ <pre>
+ make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <other options>"
+ </pre>
+ This will run the g++ "old-deja" tests in the testsuite where the filename mat
+ ches 9805*.
+
+ <p>The *.exp files are located in the testsuite directories of the GCC
+ source, the most important ones being compile.exp, execute.exp, dg.exp
+ and old-deja.exp. To get a list of the possible *.exp files, pipe the
+ output of `<code>make check</code>` into a file and look at the
+ "<code>Running ... .exp</code>" lines.
+
+
+
+ <h2>How to interpret test results</h2>
+
+ <p>After the testsuite has run you'll find various *.sum and *.log
+ files in the testsuite subdirectories. The *.log files contain a
+ detailed log of the compiler invocations and the corresponding
+ results, the *.sum files summarize the results. This summaries list
+ all the tests that have been run with a corrsponding status code:</p>
+
+ <ul>
+ <li>PASS: the test passed as expected
+ <li>FAIL: the test unexpectedly failed
+ <li>XFAIL: the test failed as expected
+ <li>UNSUPPORTED: the test is not supported on this platform
+ <li>ERROR: the testsuite detected an error
+ <li>WARNING: the testsuite detected a possible problem
+ </ul>
+
+ <p> It is normal for some tests to report
+ unexpected failures. At the current time our testing harness does not allow
+ fine grained control over whether or not a test is expected to fail. We expect
+ to fix this problem in future releases.</p>
+
! <h2>Submitting test results</h2>
! <p>If you want to report the results to the GCC project, use the
! <code>contrib/test_summary</code> shell script. Start it in the objdir with
! <pre>
! .../gcc-src/contrib/test_summary -p your_commentary.txt -m gcc-testresults@gcc.gnu.org |sh
! </pre>
! <p>This script uses the <code>Mail</code> program to send the results, so
! make sure it is in your PATH. The file your_commentary_.txt is
! prepended to the testsuite summary and should contain any special
! remarks you have on your results or your build environment. Please
! don't edit the testsuite result block or the subject line, as these
! messages are automatically parsed and presented at the <a
! href="http://gcc.gnu.org/testresults/">GCC testresults</a> web
! page. Here you can also gather information on how specific tests
! behave on different platforms and compare them with your results. A
! few failing testcases are possible even on released versions and you
! should look here first if you think your results are unreasonable.</p>
</body>
</html>