Re: If you had a month to improve gcc build parallelization, where would you begin?

[Diego Novillo <dnovillo at google dot com>]

On 2013-04-04 04:49 , Richard Biener wrote:
> But yes, testing is the most time-consuming part, and the testsuite
> harness overhead is big.  A combined bootstrap&test target could
> indeed interleave target library build and testing.  General improvements
> to parallel testing support (hack upstream dejagnu?) to be less
> statically divided would also help.

Yes.  Fortunately, the testsuite is the easiest component to 
parallelize.  Internally, I've implemented a setup over our build system 
to split all the files in all the testsuite directories in N independent 
sets.  This brings testing times from 1.5hrs down to 10min.  There are 
some issues, however. Some directories have implicit dependencies: 
http://gcc.gnu.org/ml/fortran/2013-02/msg00106.html.


> Not sure if improving things for people with distributed compile setup 
> will help many people. It isn't my testing setup at least. At least 
> make sure you don't regress things for people without distributed 
> compile ;)

Agreed.  make -j should be first priority.  Multicores are easier to 
come by than clusters.  And improvements in single-box builds, will also 
help distributed setups (better dependencies).


Diego.