Re: Resolving LTO symbols for static library boundary

[Allan Sandfeld Jensen <linux at carewolf dot com>]

On Dienstag, 6. Februar 2018 01:01:29 CET Jan Hubicka wrote:
> Dne 2018-02-05 18:44, Richard Biener napsal:
> > On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen
> > 
> > <linux@carewolf.com> wrote:
> >> Hello GCC
> >> 
> >> In trying to make it possible to use LTO for distro-builds of Qt, I
> >> have again
> >> hit the problem of static libraries. In Qt in general we for LTO rely
> >> on a
> >> library boundary, where LTO gets resolved when generating the library
> >> but no
> >> LTO-symbols are exported in the shared library. This ensure the
> >> library
> >> has a
> >> well defined binary compatible interface and gets LTO optimizations
> >> within
> >> each library. For some private libraries we use static libraries
> >> however,
> >> because we don't need binary compatibility, but though we don't need
> >> BC
> >> 
> >> between Qt versions, the libraries should still be linkable with
> >> different gcc
> >> versions (and with different compilers). However when LTO is enabled
> >> the
> >> static libraries will contain definitions that depend on a single gcc
> >> version
> >> making it unsuitable for distribution.
> >> 
> >> One solution is to enable fat-lto object files for static libraries
> >> but
> >> that
> >> is both a waste of space and compile time, and disables any LTO
> >> optimization
> >> within the library. Ideally I would like to have the static library do
> >> LTO
> >> optimizations internally just like a shared library, but then exported
> >> as
> >> static library.
> >> 
> >> I suspect this is more of gcc task than ar/ld task, as it basically
> >> boils down
> >> to gcc doing for a static library what it does for shared library, but
> >> maybe
> >> export the result as a single combined .o file, that can then be ar'ed
> >> into a
> >> compatible static library.
> >> 
> >> Is this possible?
> > 
> > Hmm. I think you could partially link the static archive contents into
> > a single relocatable object. Or we could add a mode where you do a
> > 1to1 LTO link of the objects and stop at the ltrans object files. You
> > could stuff those into an archive again.
> > 
> > I'm not sure how far Honza got partial LTO linking to work?
> 
> Parital linking of lto .o files into single non-lto .o file should work
> and it will get you cross-module optimization done. The problem is that
> without resolution info compiler needs to assume that all symbols
> exported by object files are possibly referneced by the later
> incremental link and thus the code quality will definitly not be
> comparable with what you get for LTO on final binary or DSO. Still
> should be better than non-lto build.
> I would be curious if it is useful for you in practice.
> 
How would I do that partial link, and what are the requirements?

Best regards
'Allan