This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: If you had a month to improve gcc build parallelization, where would you begin?
- From: Geert Bosch <bosch at adacore dot com>
- To: Joern Rennecke <joern dot rennecke at embecosm dot com>
- Cc: Jeff Law <law at redhat dot com>, David Fang <fang at csl dot cornell dot edu>, Simon Baldwin <simonb at google dot com>, gcc at gcc dot gnu dot org
- Date: Thu, 4 Apr 2013 00:00:15 -0400
- Subject: Re: If you had a month to improve gcc build parallelization, where would you begin?
- References: <CAPTY64o0UBQBwnq_GMNOBRmdBV4QTc+En3Q7pLn6iR1aKXKQTA at mail dot gmail dot com> <43ABCE7D-03A4-4534-9A3C-79360A7AEC75 at adacore dot com> <Pine dot LNX dot 4 dot 64 dot 1304031751290 dot 19270 at hal-00 dot csl dot cornell dot edu> <20130403195359 dot dxngsssuo8cgsww4-nzlynne at webmail dot spamcop dot net> <515CDC53 dot 5000009 at redhat dot com> <20130403234402 dot or5qa8khmsgs8k40-nzlynne at webmail dot spamcop dot net>
On Apr 3, 2013, at 23:44, Joern Rennecke <joern.rennecke@embecosm.com> wrote:
> How does that work?
> The binaries have to get the all the machines of the clusters somewhere.
> Does this assume you are using NFS or similar for your build directory?
> Won't the overhead of using that instead of local disk kill most of the
> parallelization benefit of a cluster over a single SMP machine?
This will be true regardless of communication method. There is so little
opportunity for parallelism that anything more than 4-8 local cores is
pretty much wasted. On a 4-core machine, more than 50% of the wall time
is spent on things that will not use more than those 4 cores regardless.
If the other 40-50% or so can be cut by a factor 4 compared to 4-core
execution, we still are talking about at most a 30% improvement on the
total wall time. Even a small serial overhead for communicating sources
and binaries will still reduce this 30%.
We need to improve the Makefiles before it makes sense to use more
parallelism. Otherwise we'll just keep running into Amdahl's law.
-Geert