Re: [whopr] Design/implementation alternatives for the driver and WPA

[Kenneth Zadeck <zadeck at naturalbridge dot com>]

Ollie Wild wrote:
> On Wed, Jun 4, 2008 at 9:14 AM, Chris Lattner <clattner@apple.com 
> <mailto:clattner@apple.com>> wrote:
>
>
>     1) start with all code in memory and see how far you can get.  It
>     seems that on reasonable developer machines (e.g. 2GB memory) that
>     we can handle C programs on the order of a million lines of code,
>     or C++ code on the order of 400K lines of code without a problem
>     with LLVM.
>
>
> This is essentially what the lto branch does today, and I don't see 
> any reason to disable this feature.  In the language of the WHOPR 
> design, the lto branch supports LGEN + LTRANS, with WPA bypassed 
> completely.  For implementing WPA, my intention is to add a new flag 
> (-fpartition or whatever else people think is suitable) to instruct 
> the lto1 front end to perform partitioning (aka repackaging) of .o 
> files, execute summary IPA analysese, and kick off a separate LTRANS 
> phase.
This is what lto does today because this was the easiest thing to do to 
be able to continue to develop and test the other parts of the 
system.    it is stupidly implemented - it required only five lines of 
code (two of them being curly braces according to the gcc coding 
standards) so it allowed us to work on other things.

However this was not the point of my mail. The point of my mail was 
whopr's design that seems to basically sacrifice almost all 
interprocedural analysis and transformation except for inlining in order 
to scale so as to be able to compile programs of such size that most of 
the gcc community (including the distros) will never see.   I realize 
that there is handwaving that sure, there is this or that could possibly 
be implemented by someone else for programs of modest scale, but that is 
not what whopr is all about.  

I do not want to imply that google's needs are not real and that they 
should not use gcc to fulfill them.   I only want to raise the point 
that whopr is at one end of a spectrum in which REAL tradeoffs are being 
made in the quality of compilation vs size of program handled and there 
there is a real possibility that being able to handle an entire program 
with these tradeoffs is going to yield the fastest program or a 
reasonable compilation time.

kenny