Bug 51916 - FAIL: gcc.dg/lto/trans-mem-3 c_lto_trans-mem-3_0.o-c_lto_trans-mem-3_1.o link, -flto (internal compiler error)
Summary: FAIL: gcc.dg/lto/trans-mem-3 c_lto_trans-mem-3_0.o-c_lto_trans-mem-3_1.o link...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 4.7.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-01-20 13:01 UTC by Dominique d'Humieres
Modified: 2012-01-23 14:08 UTC (History)
3 users (show)

See Also:
Host: *-apple-darwin*
Target: *-apple-darwin*
Build: *-apple-darwin*
Known to work:
Known to fail:
Last reconfirmed: 2012-01-21 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dominique d'Humieres 2012-01-20 13:01:19 UTC
The test for gcc.dg/lto/trans-mem-3* fails on *-apple-darwin*  as

[macbook] f90/bug% gcc47 /opt/gcc/work/gcc/testsuite/gcc.dg/lto/trans-mem-3_0.c -c -o obj_0.o -flto
[macbook] f90/bug% gcc47 /opt/gcc/work/gcc/testsuite/gcc.dg/lto/trans-mem-3_1.c -c -o obj_1.o -flto -fgnu-tm
[macbook] f90/bug% gcc47 obj_0.o obj_1.o -flto
lto1: internal compiler error: in streamer_get_builtin_tree, at tree-streamer-in.c:1077
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: gcc47 returned 1 exit status
collect2: error: lto-wrapper returned 1 exit status

The internal compiler error disappears if I add -fgnu-tm:

[macbook] f90/bug% gcc47 obj_0.o obj_1.o -flto -fgnu-tm
[macbook] f90/bug%
Comment 1 Aldy Hernandez 2012-01-20 15:29:45 UTC
I don't have a Darwin machine on which to test the link sequence.  Do you mind finding out which is the fcode in question?  It is a matter of setting a gdb breakpoint on the assert and typing "print fcode".
Comment 2 Dominique d'Humieres 2012-01-20 16:09:11 UTC
> I don't have a Darwin machine on which to test the link sequence.  Do you mind
> finding out which is the fcode in question?  It is a matter of setting a gdb
> breakpoint on the assert and typing "print fcode".

No luck!-(I get 'No symbol "fcode" in current context.' and if not '<value optimized out>').
The only things I have been to print stepping through 
streamer_get_builtin_tree (ib=<value optimized out>, data_in=<value optimized out>)
are

(gdb) s
streamer_read_uhwi (ib=0x7fff5fbfd820) at ../../work/gcc/data-streamer-in.c:93
93	{
(gdb) p *ib
$4 = {data = 0x142090d84 "\037", p = 1432, len = 1621}
Comment 3 Aldy Hernandez 2012-01-20 16:25:19 UTC
> No luck!-(I get 'No symbol "fcode" in current context.' and if not '<value
> optimized out>').
> The only things I have been to print stepping through
> streamer_get_builtin_tree (ib=<value optimized out>, data_in=<value optimized
> out>)

Try building the compiler without optimization (-O0).
Comment 4 Dominique d'Humieres 2012-01-20 19:15:08 UTC
> Try building the compiler without optimization (-O0).

I have never done that!-(I guess I have to pass something like CFLAGS and CXXFLAGS in configure or make). What is the official way to do it?
Comment 5 Aldy Hernandez 2012-01-20 20:03:09 UTC
> I have never done that!-(I guess I have to pass something like CFLAGS and
> CXXFLAGS in configure or make). What is the official way to do it?

 From the Debugging GCC wiki (http://gcc.gnu.org/wiki/DebuggingGCC):

To build a debuggable compiler, configure the compiler normally and then

make STAGE1_CFLAGS="-g -O0" all-stage1
Comment 6 Dominique d'Humieres 2012-01-20 22:46:47 UTC
>  From the Debugging GCC wiki (http://gcc.gnu.org/wiki/DebuggingGCC):
>
> To build a debuggable compiler, configure the compiler normally and then
>
> make STAGE1_CFLAGS="-g -O0" all-stage1

With that I get

Breakpoint 1, streamer_get_builtin_tree (ib=find_location_expression: Corrupted DWARF expression.
) at ../../_clean/gcc/tree-streamer-in.c:1067

and 'Corrupted DWARF expression' for every attempt to print something. I am lost!-(
Comment 7 Patrick Marlier 2012-01-20 23:57:09 UTC
Some more info:

(gdb) bt
#0  0x00007fff884ec283 in exit ()
#1  0x0000000100e238fa in diagnostic_action_after_output (context=0x141397020, diagnostic=0x7fff5fbfee80) at ../../trunk/gcc/diagnostic.c:243
#2  0x0000000100e2477d in diagnostic_report_diagnostic (context=0x141397020, diagnostic=0x7fff5fbfee80) at ../../trunk/gcc/diagnostic.c:552
#3  0x0000000100e2589b in internal_error (gmsgid=0x100f076ef "in %s, at %s:%d") at ../../trunk/gcc/diagnostic.c:845
#4  0x0000000100e25a35 in fancy_abort (file=Could not find the frame base for "fancy_abort".
) at ../../trunk/gcc/diagnostic.c:899
#5  0x0000000100c72b41 in streamer_get_builtin_tree (ib=0x7fff5fbff070, data_in=0x1419126a0) at ../../trunk/gcc/tree-streamer-in.c:1077
#6  0x0000000100853d12 in lto_input_tree (ib=0x7fff5fbff070, data_in=0x1419126a0) at ../../trunk/gcc/lto-streamer-in.c:1175
#7  0x00000001000368f3 in lto_read_decls (decl_data=0x141f45000, data=0x1420ac410, resolutions=0x0) at ../../trunk/gcc/lto/lto.c:925
#8  0x000000010003740c in lto_file_finalize (file_data=0x141f45000, file=0x141903a10) at ../../trunk/gcc/lto/lto.c:1167
#9  0x000000010003744d in lto_create_files_from_ids (file=0x141903a10, file_data=0x141f45000, count=0x7fff5fbff204) at ../../trunk/gcc/lto/lto.c:1177
#10 0x0000000100037553 in lto_file_read (file=0x141903a10, resolution_file=0x0, count=0x7fff5fbff204) at ../../trunk/gcc/lto/lto.c:1217
#11 0x000000010003f5ec in read_cgraph_and_symbols (nfiles=2, fnames=0x14190f300) at ../../trunk/gcc/lto/lto.c:2618
#12 0x0000000100040008 in lto_main () at ../../trunk/gcc/lto/lto.c:2932
#13 0x00000001009e7f29 in compile_file () at ../../trunk/gcc/toplev.c:557
#14 0x00000001009ead90 in do_compile () at ../../trunk/gcc/toplev.c:1938
#15 0x00000001009eaf15 in toplev_main (argc=17, argv=0x1419003c0) at ../../trunk/gcc/toplev.c:2014
#16 0x000000010004324e in main (argc=16, argv=0x7fff5fbff2b8) at ../../trunk/gcc/main.c:36

Fails as before the fcode corresponding to the TM_COMMIT.
The problem is that the lto-wrapper does not collect parameters from input files.

The tracing brings me into libiberty, we see that simple_object_start_read returns NULL with obj_0 or obj_1. 

Breakpoint 1, run_gcc (argc=4, argv=0x7fff5fbff598) at ../../trunk/gcc/lto-wrapper.c:482
482	      sobj = simple_object_start_read (fd, file_offset, NULL, &errmsg, &err);
(gdb) p argv[i]
$2 = 0x7fff5fbff751 "obj_0.o"
(gdb) n
483	      if (!sobj)
(gdb) p sobj
$3 = (simple_object_read *) 0x0


Tracing a bit more, I see that simple_object_mach_o_match() ends with this:
*errmsg = "Mach-O file found but no segment name specified";
Indeed, lto-wrapper always gives NULL for the segment name.
sobj = simple_object_start_read (fd, file_offset, NULL, &errmsg, &err);

So, since there is no documentation/specification of the simple_object_start_read function I cannot say if it is a gcc/lto bug or a libiberty bug.

Finally, there is a problem with lto-wrapper.c
      if (!simple_object_find_section (sobj, LTO_SECTION_NAME_PREFIX "." "opts",
                       &offset, &length, &errmsg, &err))
In MacOSX, I don't know the section name for opts (I have __wrapper_index, __wrapper_sects, and __wrapper_names) but if I add the segment name "__GNU_LTO" , the section name is ".gnu.lto_.opts" (not existing in my files).

To sum up, LTO files with MacOS does not supported opts or cannot be retrieve in lto-wrapper. Need inputs from MacOS (and maybe LTO) maintainers...

Thanks!
Patrick Marlier.
Comment 8 Iain Sandoe 2012-01-21 10:04:58 UTC
(In reply to comment #7)

Thanks for the debugging Patrick.

> Tracing a bit more, I see that simple_object_mach_o_match() ends with this:
> *errmsg = "Mach-O file found but no segment name specified";
> Indeed, lto-wrapper always gives NULL for the segment name.
> sobj = simple_object_start_read (fd, file_offset, NULL, &errmsg, &err);
> 
> So, since there is no documentation/specification of the
> simple_object_start_read function I cannot say if it is a gcc/lto bug or a
> libiberty bug.

include/simple_object.h:

/* Create an simple_object_read given DESCRIPTOR, an open file
   descriptor, and OFFSET, an offset within the file.  The offset is
   for use with archives, and should be 0 for an ordinary object file.
   The descriptor must remain open until done with the returned
   simple_object_read.  SEGMENT_NAME is used on Mach-O and is required
   on that platform: it means to only look at sections within the
   segment with that name.  It is ignored for other object file
   formats.  On error, this function returns NULL, and sets *ERRMSG to
   an error string and sets *ERR to an errno value or 0 if there is no
   relevant errno.  */

extern simple_object_read *
simple_object_start_read (int descriptor, off_t offset,
                          const char *segment_name, const char **errmsg,
                          int *err);

> Finally, there is a problem with lto-wrapper.c
>       if (!simple_object_find_section (sobj, LTO_SECTION_NAME_PREFIX "."
> "opts",
>                        &offset, &length, &errmsg, &err))
> In MacOSX, I don't know the section name for opts (I have __wrapper_index,
> __wrapper_sects, and __wrapper_names) but if I add the segment name "__GNU_LTO"
> , the section name is ".gnu.lto_.opts" (not existing in my files).

the sections are there (inside the wrapper - if you objdump or hexdump -C the objects you'll see them) and if I apply this hack the link works (the objects are correctly formed as of now):

Index: gcc/lto-wrapper.c
===================================================================
--- gcc/lto-wrapper.c   (revision 183359)
+++ gcc/lto-wrapper.c   (working copy)
@@ -479,7 +479,7 @@ run_gcc (unsigned argc, char *argv[])
       fd = open (argv[i], O_RDONLY);
       if (fd == -1)
        continue;
-      sobj = simple_object_start_read (fd, file_offset, NULL, &errmsg, &err);
+      sobj = simple_object_start_read (fd, file_offset, "__GNU_LTO", &errmsg, &err);
       if (!sobj)
        {
          close (fd);


... so this is a build/config issue - or, alternatively, the segment name can be specified as above since it is ignored for non-mach-o platforms.

thanks
Iain
Comment 9 Iain Sandoe 2012-01-21 10:34:59 UTC
(In reply to comment #8)
> (In reply to comment #7)


> ... so this is a build/config issue - or, alternatively, the segment name can
> be specified as above since it is ignored for non-mach-o platforms.

note:
gcc/lto/lto-object.c 
hardwires this without any ill effect on other platforms.  So, since the test and the section are lto-specific, I'd suspect that this is the right solution rather than some tricky config machinery.
Comment 10 Richard Biener 2012-01-23 10:11:48 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #7)
> 
> 
> > ... so this is a build/config issue - or, alternatively, the segment name can
> > be specified as above since it is ignored for non-mach-o platforms.
> 
> note:
> gcc/lto/lto-object.c 
> hardwires this without any ill effect on other platforms.  So, since the test
> and the section are lto-specific, I'd suspect that this is the right solution
> rather than some tricky config machinery.

Yes.  Can you please post it to gcc-patches@ and commit it?  It's preapproved
as obviously correct.  Thx.
Comment 11 Dominique d'Humieres 2012-01-23 10:21:38 UTC
> Yes.  Can you please post it to gcc-patches@ and commit it?  It's preapproved
> as obviously correct.  Thx.

A patch has already been submitted by Patrick Marlier at http://gcc.gnu.org/ml/gcc-patches/2012-01/msg01063.html . I have successfully regstrapped with it on x86_64-apple-darwin10 and powerpc-apple-darwin9.
Comment 12 Aldy Hernandez 2012-01-23 14:07:49 UTC
Author: aldyh
Date: Mon Jan 23 14:07:41 2012
New Revision: 183433

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=183433
Log:
       PR lto/51916
        * lto-wrapper.c (run_gcc): Pass the LTO section name to
        simple_object_start_read.


Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-wrapper.c
Comment 13 Aldy Hernandez 2012-01-23 14:08:42 UTC
fixed on trunk