Re: Fwd: LLVM collaboration?

[Rafael EspÃndola <rafael dot espindola at gmail dot com>]

On 11 February 2014 12:28, Renato Golin <renato.golin@linaro.org> wrote:
> Now copying Rafael, which can give us some more insight on the LLVM LTO side.

Thanks.

> On 11 February 2014 09:55, Renato Golin <renato.golin@linaro.org> wrote:
>> Hi Jan,
>>
>> I think this is a very good example where we could all collaborate
>> (including binutils).

It is. Both LTO models (LLVM and GCC) were considered form the start
of the API design and I think we got a better plugin model as a
result.

>> If I got it right, LTO today:
>>
>> - needs the drivers to explicitly declare the plugin
>> - needs the library available somewhere

True.

>> - may have to change the library loading semantics (via LD_PRELOAD)

That depends on the library being loaded. RPATH works just fine too.

>> Since both toolchains do the magic, binutils has no incentive to
>> create any automatic detection of objects.

It is mostly a historical decision. At the time the design was for the
plugin to be matched to the compiler, and so the compiler could pass
that information down to the linker.

> The trouble however is that one needs to pass explicit --plugin argument
> specifying the particular plugin to load and so GCC ships with its own wrappers
> (gcc-nm/gcc-ld/gcc-ar and the gcc driver itself) while LLVM does similar thing.

These wrappers should not be necessary. While the linker currently
requires a command line option, bfd has support for searching for a
plugin. It will search <inst>/lib/bfd-plugin. See for example the
instructions at http://llvm.org/docs/GoldPlugin.html.

This was done because ar and nm are not normally bound to any
compiler. Had we realized this issue earlier we would probably have
supported searching for plugins in the linker too.

So it seems that what you want could be done by

* having bfd-ld and gold search bfd-plugins (maybe rename the directory?)
* support loading multiple plugins, and asking each to see if it
supports a given file. That ways we could LTO when having a part GCC
and part LLVM build.
* maybe be smart about version and load new ones first? (libLLVM-3.4
before libLLVM-3.3 for example). Probably the first one should always
be the one given in the command line.

For OS X the situation is a bit different. There instead of a plugin
the linker loads a library: libLTO.dylib. When doing LTO with a newer
llvm, one needs to set DYLD_LIBRARY_PATH. I think I proposed setting
that from clang some time ago, but I don't remember the outcome.

In theory GCC could implement a libLTO.dylib and set
DYLD_LIBRARY_PATH. The gold/bfd plugin that LLVM uses is basically a
API mapping the other way, so the job would be inverting it. The LTO
model ld64 is a bit more strict about knowing all symbol definitions
and uses (including inline asm), so there would be work to be done to
cover that, but the simple cases shouldn't be too hard.

Cheers,
Rafael