This is the mail archive of the
mailing list for the GCC project.
Re: [RFC] Old school parallelization of WPA streaming
- From: Jan Hubicka <hubicka at ucw dot cz>
- To: Richard Biener <rguenther at suse dot de>
- Cc: Andi Kleen <ak at linux dot intel dot com>, Jan Hubicka <hubicka at ucw dot cz>, gcc-patches at gcc dot gnu dot org, dnovillo at google dot com, dmalcolm at redhat dot com
- Date: Wed, 21 Aug 2013 23:53:37 +0200
- Subject: Re: [RFC] Old school parallelization of WPA streaming
- References: <20130821141747 dot GD24782 at kam dot mff dot cuni dot cz> <20130821145853 dot GB2139 at tassilo dot jf dot intel dot com> <aa686933-7ae6-461b-9403-d5d47ad11289 at email dot android dot com>
> >One risk is if someone streams to a spinning disk it may add more seeks
> >the parallel IO. But I think it's a reasonable tradeoffs.
> It'll also wreck all WPA dump files.
We do not dump anything during the main streaming. If we now stream 2GB for firefox,
I think we can hope to mostly fit in cache with the whole machinery.
We will need to flush cgraph file prior forking and close it in forked process.
It is only one that remains cross fork boundary IMO.
> >We should also use a faster compressor
> And we should avoid uncompressing the function sections...
Yep, we also need to avoid carring whole tree stream of the original source
unit whenever we stream out function from it. I think function sections should
have two parts - the references to global trees that is uncompressed and
transleted during WPA streaming plus compressed binary blob with the body that
is copied over.
> That said, the patch is enough of a hack that I don't ever want to debug a bug in it....
> I also fail to see why threads should not work here. Maybe simply annotate gcc with openmp?
It means pushing global state of lto-streamer into a context variable + moving
it out of GGC or making GGC thread safe. I would hope that David Malcolm would
be interested in doing this, but it is bit more I have time for right now during
the labs conference.
To be honest I fail to see how bug in openmp annotated program would be easier
to debug than the fork variant.