[PATCH] RE: gcc parallel make check

[VandeVondele Joost]

> No.  As I wrote earlier, splitting on filenames and test counts only is only
> very rough split, all the splits really need to be backed out by real timing
> data from popular targets.  

I'm actually doing quite some testing trying to get a reasonable balance, checking 'completed in' in all *.log.sep files. However, it is important that the procedure is semi-automatic, otherwise few people will be interested in doing so. Furthermore, for parallel performance, it is not so important that times are distributed evenly (it is anyway unlikely the number of goals is exactly divided by N of -jN), but rather that the goals are ordered (executed) from slow to fast (similar to omp schedule guided). Most of the real bottlenecks are single letter patterns (e.g. p* since prxxxx is such a common filename), and this is ultimately limiting.

In the project (CP2K) I'm working on, we also parallelize testing over directories, but we keep a list of approximate runtimes per directory, and keep that (global) list sorted. Testing follows that list. As a result, we have near perfect parallel speedup, despite (or because) timings per directory ranging from a few 100s to 1s. 

> Also, I'm afraid of some tests being left out
> unintentionally (e.g. the wildcards created at some point, then a new test
> is added with a weird starting character that hasn't been used before and
> suddenly it will not be tested with make -j?).

I agree this is an issue, partially addressed by not having to write patterns by hand anymore (i.e. a script does this), and by having the script check its input. There are something like 10 testnames that do not fall in [0-9A-Za-z], as mentioned in a previous email.