Bug 44992 - ld -r breaks LTO
Summary: ld -r breaks LTO
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 4.5.3
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords: lto
: 43576 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-07-19 15:50 UTC by Andi Kleen
Modified: 2012-05-07 11:50 UTC (History)
4 users (show)

See Also:
Host: x86_64-linux
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2010-07-19 16:20:08


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andi Kleen 2010-07-19 15:50:43 UTC
When object files containing LTO sections are linked together with ld -r
the linker merges all sections with the same name. The LTO reader cannot
deal with this. Currently it will just ignore excessive data behind
the first entry for most sections, which leads to all kinds of errors.

I have been working on a patch kit to solve this, this is just a tracking
bug for it.
Comment 1 Richard Biener 2010-07-19 16:20:08 UTC
And partial linking support will break mixed LTO / non-LTO objects.  Unless
we drop all non-LTO sections from LTO objects and thus the .text sections
of partially linked mixed LTO / non-LTO objects will be still used in the
final link.
Comment 2 Andi Kleen 2010-07-19 16:31:51 UTC
Not sure I understand the comment.

The case I've been looking at is ld -r (without a LTO code generation stage)
to combine existing object and then using gold for the final linking/LTO code generation based on the  combined objects.
For me it seems like gold handles this correctly (with my patches)

Your scenario sounds like ld -r with code generation? That's probably harder.


Comment 3 Jakub Jelinek 2010-07-19 19:35:54 UTC
I must say I don't like your solution.  IMHO much better is instead add a header to LTO sections, which says the length of the LTO chunk (similarly e.g. to how .debug_info section chunks have length in the header), perhaps version, followed by that length of bytes of compressed stream, and change LTO reader to iterate over all chunks in the LTO section.
This will also have advantage that it doesn't require non-ELF object formats to have section length not a multiple of 4, etc.
Comment 4 Andi Kleen 2010-07-19 19:46:42 UTC
This is actually what I tried first, but it turned out to be quite complicated,
had to change a lot of code and my patch was growing and growing and it didn't
fit clearly with the different readers etc.

That is why I ended up with the post fixes instead which is much simpler
and isolated in a single part of the reader.

I don't understand the comment about section lengths? There's no dependency
on section lengths being a multiple of 4
Comment 5 Richard Biener 2010-07-20 09:00:09 UTC
I was refering to a situation like

 gcc -c -flto t1.c
 gcc -c t2.c
 gcc -o t.o -r -nostdlib t1.o t2.o [-flto]
 gcc -o t t.o -flto

which would break with your solution (it's broken right now as well, of course).

We could make it work by not emitting code but only LTO sections for t1.o.
Then the partial link would cause us to have both LTO sections and regular
sections, so at link-time we'd pick up the LTO parts for re-optimization
and link the code parts without re-optimization.  We'd still need a way
to merge LTO sections (or mangle them like you do), of course.
Comment 6 ak 2010-07-23 05:34:12 UTC
Subject: Bug 44992

Author: ak
Date: Fri Jul 23 05:33:51 2010
New Revision: 162443

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=162443
Log:
gcc:

2010-07-10   Andi Kleen <ak@linux.intel.com>

        PR lto/44992
        * lto-opts.c (lto_write_options): Add NULL file_data argument to 
	lto_get_section_name.
	* lto-section-out.c (lto_destroy_simple_output_block): Likewise.
	* lto-streamer-out.c (produce_asm): Likewise.
	(copy_function): Likewise.
	(produce_symtab): Likewise.
	(produce_asm_for_decls): Likewise.
	* lto-streamer.c (lto_get_section_name): Add file_data argument.	
	Rewrite to add random postfix to LTO sections.
	* lto-streamer.h (lto_file_decl_data): Add next, id, resolutions.
	(lto_get_section_name): Add file_data argument to prototype.
	

lto:

2010-07-10   Andi Kleen <ak@linux.intel.com>

        PR lto/44992
        * lto.c: Include splay-tree.h
	(lto_resolution_read): Change to walk file_ids tree and parse
	extra file_id in resolution file.
	(lto_section_with_id): Add.
	(create_subid_section_table): Add.
	(lwstate): Add.
	(lto_create_files_from_ids): Add.
	(lto_file_read): Change to handle sub file ids and create list
	of file_datas. Add output argument for count.
	(get_section_data): Pass file_data to lto_get_section_name.
	(lto_flatten_file): Add.
	(read_cgraph_and_symbols): Handle linked lists of file_datas.
	
lto-plugin:

2010-07-10   Andi Kleen <ak@linux.intel.com>

        PR lto/44992
        * lto-plugin.c (sym_aux): Add.
	(plugin_symtab): Remove slots. Add aux and id.
	(parse_table_entry): Change to use aux instead of slots.
	(LTO_SECTION_PREFIX): Add.
	(translate): Improve buffer allocation. Change to append
	symbols to existing out buffer.
	(get_section): Remove.
	(process_symtab): Add. 
	(free_2): Free symtab->aux.
	(write_resolution): Handle aux instead of slots.
	Print sub id to resolution file.
	(claim_file_handler): Clear lto_file. Replace get_symtab/translate
	calls with call to process_symtab.  

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-opts.c
    trunk/gcc/lto-section-out.c
    trunk/gcc/lto-streamer-out.c
    trunk/gcc/lto-streamer.c
    trunk/gcc/lto-streamer.h
    trunk/gcc/lto/ChangeLog
    trunk/gcc/lto/lto.c
    trunk/lto-plugin/ChangeLog
    trunk/lto-plugin/lto-plugin.c

Comment 7 Richard Biener 2010-08-31 09:13:18 UTC
Fixed.
Comment 8 Andi Kleen 2010-08-31 09:32:23 UTC
Sorry this is not fixed yet, only partially. Still working on the last bits,
in particular passthrough of non LTOed code like assembler functions.
Comment 9 Jakub Jelinek 2011-03-25 19:53:20 UTC
GCC 4.6.0 is being released, adjusting target milestone.
Comment 10 Andi Kleen 2011-10-07 05:42:50 UTC
I consider this fixed now because everything I need works together
with HJ's binutils.

Mainstream binutils will hopefull catch up eventually.
Comment 11 Richard Biener 2012-05-07 11:50:07 UTC
*** Bug 43576 has been marked as a duplicate of this bug. ***