improving testsuite runtime

Janis Johnson
Fri Sep 19 05:38:00 GMT 2008

On Thu, 2008-09-18 at 21:37 +1000, Ben Elliston wrote:
> While waiting on testsuites this week, I finally snapped and spent some
> time looking at how to speed up the testsuite.

> So, I guess my question is: what now?  What do people feel would be
> required to make this usable?  I assume that the most pressing thing
> would be to have the build system fold the various .log and .sum files
> together so that they look like they were run as a whole.
> Another problem is that this is a bit of guesswork.  Maybe it would be
> better to drive this from make -j, so that make takes care of the
> scheduling (and we can push the longer tests to the front of the
> dependency list, as we do to improve parallel builds now).
> Comments?

I've been thinking about this recently, too, and think that we should
try to get as much parallelism as possible out of the testsuites and use
"make -j" to control it.  A script invoked from make, as part of the
top-level "make check" target for each language, could put together
result files from various pieces so it wouldn't be any more difficult
than it is now to compare test results from multiple test runs.

runtest has an option --directory that could be used to run groups of
tests for each language in parallel, each in its own temporary directory
under gcc/testsuite, as is done now for each language.  The check target
for each language could in turn run targets for each of its directories.
I've played with that a bit but my make skills aren't really up to it.

It's also possible to break the targets down further into directory and
multilib variants like the existing option lang_checks_parallel.

Part of running directories in parallel would be moving more test
directories to the top-level, assuming the history wouldn't be lost, and
ordering the make targets with the longest-running ones first.

Another piece could be moving the tests that are resource hogs into
their own directories to be run sequentially.

Support for more parallelism in the tests should be available by
running "make check" from the top level, which includes running the
library tests.


More information about the Gcc mailing list