Re: is LTO aimed for large programs?

[Richard Guenther <richard dot guenther at gmail dot com>]

On Mon, Nov 9, 2009 at 10:44 AM, Tobias Burnus <burnus@net-b.de> wrote:
> On 11/09/2009 12:03 AM, Basile STARYNKEVITCH wrote:
>> is gcc-trunk -flto -O2 aimed for medium sized programs (something like
>> bash), or for bigger ones (something like the linux kernel, the Xorg
>> server, the Qt or GTK graphical toolkit libraries, or bootstrapping GCC
>> itself.
>
> My understanding is that LTO aims at both, but that one needs to use
> -fwhopr for really large systems as the otherwise e.g. the memory usage
> may exceed the available memory. I don't know whether one can really
> estimated how much memory compilation needs. It is surely not a simple
> function on the number of code lines.
>
> I tried -flto successfully for our 100 kLoC Fortran code and there lto1
> needs <1/2 GB of RAM (370 MB if I recall correctly). (Thanks to
> especially Richard; initially more than 4 GB were needed and lto1
> crashed thus). Toon also used LTO [1] for their HIRLAM weather
> forecasting program, which has according to [2] 1.2 MLoC in Fortran and
> O(10 kLoC) in C.
>
> If I recall correctly, bootstrapping GCC also works in principle, except
> for problems when comparing stage2 with stage3.
>
>
>> Perhaps the question is when not to use -flto and use -fwhopr instead?
>
> My rule of thumb is: Try -flto first, if it does not work (running out
> of memory), try -fwhopr. I think the advantage of -flto is also that it
> is better tested, while -fwhopr has known issues.

Indeed for 4.5 I wouldn't recommend on using -fwhopr at all.

The compile-time penalty is dependent on the complexity of
the IPA passes we run - and I have no idea if they are currently
worse than linear in the number of cgraph nodes or edges.

The main issue with LTO is memory usage as you have to
keep the whole program in memory.

Another issue is that with excessive inlining (which we avoid
by default, but of course you can tweak params) you can
more easily hit non-linear time- and memory complexity
in function local passes.

Richard.