This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Meta-expectations when testing
On Wed, Feb 26, 2003 at 09:59:40AM +0000, Hugh Sasse Staff Elec Eng wrote:
> On Tue, 25 Feb 2003, Janis Johnson wrote:
[snip]
> >
> > Follow the links for test results. Many people report successful builds
> > without even running the test suites, let alone submitting the test
>
> This was the case here. Thanks for pointing that out.
>
> I'd like to propose the following patch to the test.html:
> Its purposes: to point that out and to put a reference in there to
> the list of results, which doesn't seem to be on the test page.
>
> --- test.html.orig 2003-02-25 18:49:09.360007000 +0000
> +++ test.html 2003-02-25 18:55:30.920007000 +0000
> @@ -178,6 +178,12 @@
> over whether or not a test is expected to fail. We expect to fix this
> problem in future releases.
>
> + <p>You may find it useful to compare your results with those from
> +similar builds listed at <a
> +href="http://gcc.gnu.org/buildstat.html";>http://gcc.gnu.org/buildstat.html</a>,
> +but be aware that sometimes a build listed as successful has no test
> +results associated with it.
> +
> <h3 class="section"><a name="TOC3"></a>Submitting test results</h3>
>
> <p>If you want to report the results to the GCC project, use the
The first sentence in test.html (generated from gcc/doc/install.texi) is:
Before you install GCC, we encourage you to run the testsuites and to
compare your results with results from a similar configuration that have
been submitted to the @uref{http://gcc.gnu.org/ml/gcc-testresults/,,
gcc-testresults mailing list}.
Would appending the following to that be sufficient?
Some of these archived results are linked from the build status lists
at @uref{http://gcc.gnu.org/buildstat.html}, although not everyone who
reports a successful build runs the testsuites and submits the results.
(Also, the maintainer of the build status lists sometimes doesn't have
time to go through the gcc-testresults archives after a release to
include new reports in the build status list as she ought to, and
welcomes mail pointing out the URLs of such reports, or patches to the
status list.)
To me, it seems clear enough already that a successful build doesn't
imply that all of the tests passed.
>
> I just tried to submit my results and it failed to work:
> I had no comments to make as such but to keep the command the same I
> did
> touch no_comment
> ../gcc-3.2.1/contrib/test_summary -p no_comment -m gcc-testresults at gcc dot gnu dot org |sh
> tail: cannot open input
>
> [...]
The script moves the *.sum and *.log files to new names, so you can only
run it once unless you move the files back to their original names. You
shouldn't need to supply any options, since the default is to mail to
the address you specified. If you direct the output to a file then you
can look it over before running it, and that way it's available to
modify and run again if there were problems the first time.
Janis