GCC Multi-Threading Ideas

Allan Sandfeld Jensen linux@carewolf.com
Fri Jan 24 19:32:00 GMT 2020


On Freitag, 24. Januar 2020 17:29:06 CET Nicholas Krause wrote:
> On 1/24/20 3:18 AM, Allan Sandfeld Jensen wrote:
> > On Freitag, 24. Januar 2020 04:38:48 CET Nicholas Krause wrote:
> >> On 1/23/20 12:19 PM, Nicholas Krause wrote:
> >>> On 1/23/20 3:39 AM, Allan Sandfeld Jensen wrote:
> >>>> On Montag, 20. Januar 2020 20:26:46 CET Nicholas Krause wrote:
> >>>>> Greetings All,
> >>>>> 
> >>>>> Unfortunately due to me being rather busy with school and other
> >>>>> things I
> >>>>> will not be able to post my article to the wiki for awhile. However
> >>>>> there is a  rough draft here:
> >>>>> https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIp
> >>>>> C9
> >>>>> gUMj
> >>>>> 
> >>>>> Oxk/edit that may change a little for people to read in the meantime.
> >>>> 
> >>>> This comment might not be suited for your project, but now that I
> >>>> think about
> >>>> it: If we want to improve gcc toolchain buildspeed with better
> >>>> multithreading.
> >>>> I think the most sensible would be fixing up gold multithreading and
> >>>> enabling
> >>>> it by default. We already get most of the benefits of multicore
> >>>> architectures
> >>>> by running multiple compile jobs in parallel (yes, I know you are
> >>>> focusing on
> >>>> cases where that for some reason doesn't work, but it is still the
> >>>> case in
> >>>> most situations). The main bottleneck is linking. The code is even
> >>>> already
> >>>> there in gold and have been for years, it just haven't been deemed
> >>>> ready for
> >>>> being enabled by default.
> >>>> 
> >>>> Is anyone even working on that?
> >>>> 
> >>>> Best regards
> >>>> Allan
> >>> 
> >>> Allan,
> >>> You would need both depending on the project, some are more compiler
> >>> bottle necked and others linker. I mentioned that issue at Cauldron as
> >>> the other side would be the linker.
> >>> 
> >>> Nick
> >> 
> >> Sorry for the second message Allan but make -j does not scale well
> >> beyond 4 or
> >> 8 threads and that's considering a 4 core or 8 machine.
> > 
> > It doesn't? I generally build with -j100, but then also use icecream to
> > distribute builds to multiple machines in the office. That probably also
> > makes linking times more significant to my case.
> > 
> > 'Allan
> 
> Allan,
> 
> I ran a gcc build on a machine with make -j32 and -j64 that had 64 cores.
> There was literally only a 4 minute increase in build speed. Good question
> through.
> 
Right. I guess it entirely depends on what you are building. If you are 
building gcc, it is probably bound by multiple configure runs, and separate 
iterations. What I usually build is Qt and Chromium, where thousands of files 
can be compiled from a single configure run (more than 20000 in the case of 
Chromium), plus those configure runs are much faster. For Chromium there is 
almost a linear speed up with the number of parallel jobs you run up to around 
100. With -j100 I can build Chromium in 10 minutes, with 2 minutes being 
linking time (5 minutes linking if using bfd linker). With -j8 it takes 2 
hours.

But I guess that means multithreading the compiler can make sense to your 
case, even if it doesn't to mine.

Regards
'Allan




More information about the Gcc mailing list