This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: GCC Multi-Threading Ideas
On Freitag, 24. Januar 2020 17:29:06 CET Nicholas Krause wrote:
> On 1/24/20 3:18 AM, Allan Sandfeld Jensen wrote:
> > On Freitag, 24. Januar 2020 04:38:48 CET Nicholas Krause wrote:
> >> On 1/23/20 12:19 PM, Nicholas Krause wrote:
> >>> On 1/23/20 3:39 AM, Allan Sandfeld Jensen wrote:
> >>>> On Montag, 20. Januar 2020 20:26:46 CET Nicholas Krause wrote:
> >>>>> Greetings All,
> >>>>>
> >>>>> Unfortunately due to me being rather busy with school and other
> >>>>> things I
> >>>>> will not be able to post my article to the wiki for awhile. However
> >>>>> there is a rough draft here:
> >>>>> https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIp
> >>>>> C9
> >>>>> gUMj
> >>>>>
> >>>>> Oxk/edit that may change a little for people to read in the meantime.
> >>>>
> >>>> This comment might not be suited for your project, but now that I
> >>>> think about
> >>>> it: If we want to improve gcc toolchain buildspeed with better
> >>>> multithreading.
> >>>> I think the most sensible would be fixing up gold multithreading and
> >>>> enabling
> >>>> it by default. We already get most of the benefits of multicore
> >>>> architectures
> >>>> by running multiple compile jobs in parallel (yes, I know you are
> >>>> focusing on
> >>>> cases where that for some reason doesn't work, but it is still the
> >>>> case in
> >>>> most situations). The main bottleneck is linking. The code is even
> >>>> already
> >>>> there in gold and have been for years, it just haven't been deemed
> >>>> ready for
> >>>> being enabled by default.
> >>>>
> >>>> Is anyone even working on that?
> >>>>
> >>>> Best regards
> >>>> Allan
> >>>
> >>> Allan,
> >>> You would need both depending on the project, some are more compiler
> >>> bottle necked and others linker. I mentioned that issue at Cauldron as
> >>> the other side would be the linker.
> >>>
> >>> Nick
> >>
> >> Sorry for the second message Allan but make -j does not scale well
> >> beyond 4 or
> >> 8 threads and that's considering a 4 core or 8 machine.
> >
> > It doesn't? I generally build with -j100, but then also use icecream to
> > distribute builds to multiple machines in the office. That probably also
> > makes linking times more significant to my case.
> >
> > 'Allan
>
> Allan,
>
> I ran a gcc build on a machine with make -j32 and -j64 that had 64 cores.
> There was literally only a 4 minute increase in build speed. Good question
> through.
>
Right. I guess it entirely depends on what you are building. If you are
building gcc, it is probably bound by multiple configure runs, and separate
iterations. What I usually build is Qt and Chromium, where thousands of files
can be compiled from a single configure run (more than 20000 in the case of
Chromium), plus those configure runs are much faster. For Chromium there is
almost a linear speed up with the number of parallel jobs you run up to around
100. With -j100 I can build Chromium in 10 minutes, with 2 minutes being
linking time (5 minutes linking if using bfd linker). With -j8 it takes 2
hours.
But I guess that means multithreading the compiler can make sense to your
case, even if it doesn't to mine.
Regards
'Allan