This is the mail archive of the
mailing list for the GCC project.
Re: [PATCH] RE: gcc parallel make check
- From: Jakub Jelinek <jakub at redhat dot com>
- To: VandeVondele Joost <joost dot vandevondele at mat dot ethz dot ch>
- Cc: Yury Gribov <y dot gribov at samsung dot com>, "gcc at gcc dot gnu dot org" <gcc at gcc dot gnu dot org>, "fortran at gcc dot gnu dot org" <fortran at gcc dot gnu dot org>, "gcc-patches at gcc dot gnu dot org" <gcc-patches at gcc dot gnu dot org>
- Date: Tue, 9 Sep 2014 15:24:14 +0200
- Subject: Re: [PATCH] RE: gcc parallel make check
- Authentication-results: sourceware.org; auth=none
- References: <908103EDB4893A42920B21D3568BFD93150F4103 at MBX23 dot d dot ethz dot ch> <20140905143740 dot GL17454 at tucnak dot redhat dot com> <908103EDB4893A42920B21D3568BFD93150F414C at MBX23 dot d dot ethz dot ch> <20140905145304 dot GM17454 at tucnak dot redhat dot com> <908103EDB4893A42920B21D3568BFD93150F7F45 at MBX23 dot d dot ethz dot ch> <540ED02A dot 9080002 at samsung dot com> <20140909101043 dot GQ17454 at tucnak dot redhat dot com> <908103EDB4893A42920B21D3568BFD93150F8014 at MBX23 dot d dot ethz dot ch>
- Reply-to: Jakub Jelinek <jakub at redhat dot com>
On Tue, Sep 09, 2014 at 10:57:09AM +0000, VandeVondele Joost wrote:
> > No. As I wrote earlier, splitting on filenames and test counts only is only
> > very rough split, all the splits really need to be backed out by real timing
> > data from popular targets.
> Furthermore, for parallel performance, it is not
> so important that times are distributed evenly (it is anyway unlikely the
> number of goals is exactly divided by N of -jN), but rather that the goals
> are ordered (executed) from slow to fast (similar to omp schedule guided).
> Most of the real bottlenecks are single letter patterns (e.g. p* since
> prxxxx is such a common filename), and this is ultimately limiting.
I disagree. If e.g. in gcc.dg/ more than a third of testcases are pr*.c,
then running dg.exp=p* in one job and dg.exp=a* in another one etc. is
simply a bad idea, the pr*.c should be split more and some other letters
just be done together.
Even that can be done semi-automatically.
If you get whitespace right, one can provide multiple different wildcards to
a single *.exp file, e.g.
make check-gcc RUNTESTFLAGS="dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*'" should
cover all tests starting with p other than pr[0-8]*.c (where you could split
say pr[0-2]* into another job, pr[3-5]* into another and pr[6-8]* into
The fact that some check-gcc or check-gfortran test job is early in the list
doesn't mean it will be started early, you need to consider also all other
potentially long jobs like check-g++, check-target-libgomp,