This is the mail archive of the
mailing list for the GCC project.
Re: [build, testsuite, v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism
- From: Jakub Jelinek <jakub at redhat dot com>
- To: Rainer Orth <ro at CeBiTec dot Uni-Bielefeld dot DE>
- Cc: gcc-patches at gcc dot gnu dot org, libstdc++ at gcc dot gnu dot org, Janis Johnson <janis dot marie dot johnson at gmail dot com>, Paolo Bonzini <bonzini at gnu dot org>
- Date: Wed, 21 Jul 2010 12:43:52 +0200
- Subject: Re: [build, testsuite, v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism
- References: <yddhbjt40ty.fsf@manam.CeBiTec.Uni-Bielefeld.DE>
- Reply-to: Jakub Jelinek <jakub at redhat dot com>
On Wed, Jul 21, 2010 at 11:54:17AM +0200, Rainer Orth wrote:
> [Please keep me on the Cc:, I'm not subscribed to libstdc++. Thanks.]
I don't think we want to backport this to 4.4 or 4.5.
There are reasons why I've tried not to split the testing into too many
pieces, there is some constant cost needed to start runtest and e.g. compile
testcases that are used in the various dejagnu guards.
+ execute.exp=execute/2000* \
+ execute.exp=execute/2001* \
+ execute.exp=execute/2002* \
+ execute.exp=execute/2003* \
+ execute.exp=execute/2004* \
+ execute.exp=execute/2005* \
+ execute.exp=execute/2006* \
+ execute.exp=execute/2007* \
+ execute.exp=execute/2008* \
+ execute.exp=execute/2009* \
+ execute.exp=execute/2010* \
+ execute.exp=execute/201\[1-9\]* \
+ execute.exp=execute/20\[2-9\]* \
+ execute.exp=execute/2\[1-9\]* \
+ execute.exp=execute/\[a-eA-E\]* \
+ execute.exp=execute/\[f-mF-M\]* \
+ execute.exp=execute/\[n-pN-P\]* \
+ execute.exp=execute/\[q-zQ-Z\]* \
is IMHO way too small division. The test counts for 20* are:
so, while it might be appropriate to run 2000, 2001, 2002, 2003 and 2004
tests in separate jobs, you might as well run say 2005-2009 in another one
(or two). On the other side, you don't mention guality.exp at all, and that
one takes several minutes to complete even on fast hw.
Much better than counting number of tests in some cases is to look at
...exp completed in ... seconds
lines (preferrably from a fast box than very slow one).
Anyway, even if you just count tests, start with counting the tests done
by that job and divide, decide into how many subjobs to split (the current
tuning was tuned for make -j24 check to make -j48 check, though the
testsuite changed a little since it was split, by adding new tests and by
adding -flto/-fwhopr testing) and then try to keep jobs roughly the same if
easily possible. I don't think subdividing current jobs to more than say 5
or so parts is a good idea. So say for execute.exp (or compile.exp) try
to have roughly the same sized chunks. execute.exp has currently 1122 tests
and is ATM run as two parallel jobs, one with 2*.c (419 tests) and the rest.
So, you should AIM at roughly 100 tests in each group. Perhaps
200\[069\]*, 200\[14\]*, 200\[27\]*, 200\[358\]*, 20\[1-9\]*,
>From the other former subjob, there are currently 286 9*.c tests, 188 pr*.c
tests and 229 other tests. So you want to split the 9* group into say 3,
pr*.c into 2 and the other tests into 2 pieces.