This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: RFC: Improving support for known testsuite failures
On Thu, 8 Sep 2011, Richard Guenther wrote:
> I think it would be more useful to have a script parse gcc-testresults@
> postings from the various autotesters and produce a nice webpage
> with revisions and known FAIL/XPASSes for the target triplets that
> are tested.
Better than parsing gcc-testresults might be a system for uploading full
.sum files (or indeed .logs as well) to a database. gcc-testresults is
useful, but if a test isn't mentioned in a message you don't know if it
passed or wasn't run at all, for example. The database would be big by
gcc.gnu.org standards (maybe multiple GB a day if all the gcc-testresults
posters start uploading full .log files), but not by the standards of many
modern web databases. You'd want a contrib/ script for uploading files
given metadata about the test run (some identifier for the tester, details
of configuration and multilibs involved in the various files) that could
be used for both build tree and installed testing.
(You might want to parse gcc-testresults *as well* for the additional logs
found that way, but a system giving full logs and reliably identifying
successive builds from the same tester could do more things, such as
identifying regressions seen by any individual tester as opposed to a test
that passes for one person on a given target and fails for another person
on that target.)
--
Joseph S. Myers
joseph@codesourcery.com