Re: LTO symtab sections vs. missing symbols (libcalls maybe?) and lto-plugin vs. COFF

[Richard Guenther <richard dot guenther at gmail dot com>]

On Thu, Oct 14, 2010 at 4:59 PM, Dave Korn <dave.korn.cygwin@gmail.com> wrote:
>
> ? ?Hello list,
>
> ?When I compile this source with -flto:
>
>> extern int retval;
>> int func (void)
>> {
>> ? return retval;
>> }
>
> ... the LTO symbol table contains both symbols:
>
>> /gnu/binutils/git.repo/obj/ld/test/func.o: ? ? file format pe-i386
>>
>> Contents of section .gnu.lto_.symtab.227b80e3:
>> ?0000 66756e63 00000000 00000000 00000000 ?func............
>> ?0010 4b000000 72657476 616c0000 02000000 ?K...retval......
>> ?0020 00000000 00005100 0000 ? ? ? ? ? ? ? ......Q...
>
> ?But when I compile this:
>
>> extern int printf (const char *fmt, ...);
>>
>> extern const char *text;
>> extern int func (void);
>>
>> int retval = 0;
>>
>> int main (int argc, const char **argv)
>> {
>> ? printf ("%s\n", text);
>> ? return func ();
>> }
>
> ... there is no sign of printf:
>
>> /gnu/binutils/git.repo/obj/ld/test/main.o: ? ? file format pe-i386
>>
>> Contents of section .gnu.lto_.symtab.6e8eaf64:
>> ?0000 6d61696e 00000000 00000000 00000000 ?main............
>> ?0010 4b000000 66756e63 00000200 00000000 ?K...func........
>> ?0020 00000000 5b000000 74657874 00000200 ?....[...text....
>> ?0030 00000000 00000000 5f000000 72657476 ?........_...retv
>> ?0040 616c0000 00000000 00000000 00006100 ?al............a.
>> ?0050 0000 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ..
>
> ?Nor indeed is there any sign of puts, which is what the generated ltrans0.s
> file ends up optimising it to (as indeed does the native code in the original
> .o file). ?I'm assuming that this is by design and is for some reason along
> the lines that we don't even know whether or which function calls are actually
> going to be emitted until link time?

I think this is because builtin functions are handled specially (their decls
are not streamed, likewise I think we don't stream their cgraph nodes).
As you noted we may eventually fold printf to puts (we may also generate
a memcpy call out of an aggregate assignment), so it might not make
your life perfect if we emit symbols for those calls (as indeed we don't know
which ones we will emit at link time).

> ?It makes life complicated in the linker though, because it means there are
> symbols present in the object files that the plugin adds via the
> add_input_files callback that weren't in the original symbols the linker
> presented via add_symbols when it initially claimed the object file containing
> the IR.
>
> ?The consequence of this is that either there are going to be undefined
> symbols in the final executable, or the linker has to perform another round of
> library scanning. ?It occurred to me that the semantics of this might even not
> have been decided yet, since ELF platforms are perfectly happy with undefined
> symbols at final link time. ?The only documentation I know of that specifies
> the linker plugin API is the GCC wiki whopr/driver page(*), and it doesn't say
> anything explicit about this; maybe I've failed to infer something that I
> should have.

Yeah, I think you have to deal with undefined references to "standard"
functions (mostly from libc, libm but maybe also from libpthread or so).

> ?So, is there a defined way in which this is supposed to work? ?And if the
> linker is supposed to rescan the libs after the plugin adds files, is it
> supposed to offer any archive members it finds to the plugin for claiming?
> Should there be multiple iterations of claiming files and calling
> all_symbols_read? ?And if not, what about if the archive members we find on
> the second pass contain LTO IR?
>
> ?It occurs to me that maybe this is what the add_input_library hook is for:
> perhaps a simple fix would be for collect2 to pass a list of all the stdlibs
> to the plugin as a plugin option, and it could add_input_library them after
> it's finished adding object files. ?Would that be a reasonable approach?
>
> ?(Right now I have a "working" COFF lto-plugin, but the link fails with
> unresolved symbols unless I manually add (e.g.) "-Wl,-u,_puts (... etc.)" to
> the command-line to make sure all the required libc archive members get pulled
> in during the first pass over libs!)

I have no idea about the linker-plugin side, but we could of course
avoid generating any calls that were not there before (by for example
streaming builtin decls and only if they are used).  But that's as much
a workaround as fixing things up in the linker afterwards ...

Richard.

> ? ?cheers,
> ? ? ?DaveK
> --
> (*) - http://gcc.gnu.org/wiki/whopr/driver
>
>