This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [RFC] Old school parallelization of WPA streaming
- From: Andi Kleen <ak at linux dot intel dot com>
- To: Jan Hubicka <hubicka at ucw dot cz>
- Cc: Richard Biener <rguenther at suse dot de>, Michael Matz <matz at suse dot de>, gcc-patches at gcc dot gnu dot org, dnovillo at google dot com, dmalcolm at redhat dot com, jakub at redhat dot com
- Date: Thu, 29 Aug 2013 19:36:49 -0700
- Subject: Re: [RFC] Old school parallelization of WPA streaming
- Authentication-results: sourceware.org; auth=none
- References: <20130821141747 dot GD24782 at kam dot mff dot cuni dot cz> <20130821145853 dot GB2139 at tassilo dot jf dot intel dot com> <aa686933-7ae6-461b-9403-d5d47ad11289 at email dot android dot com> <alpine dot LNX dot 2 dot 00 dot 1308281728110 dot 9949 at wotan dot suse dot de> <alpine dot LNX dot 2 dot 00 dot 1308290931540 dot 20077 at zhemvz dot fhfr dot qr> <20130829125103 dot GB20627 at kam dot mff dot cuni dot cz> <alpine dot LNX dot 2 dot 00 dot 1308291536280 dot 20077 at zhemvz dot fhfr dot qr> <20130829135845 dot GC20627 at kam dot mff dot cuni dot cz>
On Thu, Aug 29, 2013 at 03:58:45PM +0200, Jan Hubicka wrote:
> > > Said that, I now have the fork() patch in all my trees and enjoy 50% faster
> > > WPA times. I changed my mind about claim that stremaing should be disk bound -
> > > it is hard to hope for disk boundness for something that should fit in cache.
> >
> > It should at least limit its fork rate according to -flto=N or jobserver.
> It limits forks to -flto=N.
> If the patch seems resonable, I will look into posiblity of adding my jobserver client
> based on GNU make code.
>
> I also think with -flto we want wrapper to figure out number of threads and suppy
> default =N (i.e. nonparallel lto would be -flto=0). Most people don't want to worry
> about =n/=jobserv parameters and those few projects that don't want to start too many
> processes to not explode in memory use can get their build machinery right.
>
Job server should do that already. You get whatever the user specifies with -j
on the top level make. That's imho the right area to control this.
The only problem is we need to work around the jobserver pipe bug first
I suspect this may need a change in make :-/
-Andi