Bug 69188 - ICE when linking objects at different optimization levels with LTO and profile generation enabled. (Works with 4.9.3.)
Summary: ICE when linking objects at different optimization levels with LTO and profil...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 5.1.0
: P2 normal
Target Milestone: 5.5
Assignee: Martin Liška
URL:
Keywords: ice-on-valid-code, link-failure, lto
Depends on:
Blocks:
 
Reported: 2016-01-08 02:14 UTC by anthonyfk
Modified: 2017-01-24 21:44 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work: 4.9.4
Known to fail: 5.4.0, 6.2.0
Last reconfirmed: 2016-12-01 00:00:00


Attachments
Log of link step with -v. (1.42 KB, text/plain)
2016-01-08 02:14 UTC, anthonyfk
Details
Script used to reproduce. (194 bytes, application/x-shellscript)
2016-01-08 02:15 UTC, anthonyfk
Details
Source file 1. (146 bytes, text/plain)
2016-01-08 02:15 UTC, anthonyfk
Details
Source file 2. (187 bytes, text/plain)
2016-01-08 02:16 UTC, anthonyfk
Details
Source file 3. (80 bytes, text/plain)
2016-01-08 02:16 UTC, anthonyfk
Details
4.9.3 lsame.s (6.50 KB, text/plain)
2016-01-09 04:15 UTC, anthonyfk
Details
4.9.3 slamch.s (7.14 KB, text/plain)
2016-01-09 04:16 UTC, anthonyfk
Details
4.9.3 slamchtst.s (9.12 KB, text/plain)
2016-01-09 04:16 UTC, anthonyfk
Details
5.1.0 lsame.s (6.78 KB, text/plain)
2016-01-09 04:17 UTC, anthonyfk
Details
5.1.0 slamch.s (7.38 KB, text/plain)
2016-01-09 04:17 UTC, anthonyfk
Details
5.1.0 slamchtst.s (9.30 KB, text/plain)
2016-01-09 04:18 UTC, anthonyfk
Details

Note You need to log in before you can comment on or make changes to this bug.
Description anthonyfk 2016-01-08 02:14:36 UTC
Created attachment 37257 [details]
Log of link step with -v.

Running the given "compile" script on these reduced sources gives the following:

$ ./compile 
lto1: internal compiler error: in lto_varpool_replace_node, at lto/lto-symtab.c:143
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: /usr/local/solido/tools/20160101/bin/gfortran returned 1 exit status
compilation terminated.
/usr/local/solido/tools/20160101/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status

This is with a custom built gcc-5.3.0. I've also tried with the gcc-5-20160105 snapshot and the same thing occurs. I'm using binutils 2.25.1.

Found while compiling lapack-3.4.2 for our build environment; tried reducing as much as possible. "-save-temps" just resulted in .s files; I'm guessing the .f's will be more useful.
Comment 1 anthonyfk 2016-01-08 02:15:11 UTC
Created attachment 37258 [details]
Script used to reproduce.
Comment 2 anthonyfk 2016-01-08 02:15:38 UTC
Created attachment 37259 [details]
Source file 1.
Comment 3 anthonyfk 2016-01-08 02:16:01 UTC
Created attachment 37260 [details]
Source file 2.
Comment 4 anthonyfk 2016-01-08 02:16:20 UTC
Created attachment 37261 [details]
Source file 3.
Comment 5 anthonyfk 2016-01-08 02:18:22 UTC
I forgot to mention, modifying the compile script in any of the following ways prevents the ICE from occurring:

1. Compile all sources at same optimization levels.
2. Remove -fprofile-generate.
3. Remove -flto.
Comment 6 anthonyfk 2016-01-08 03:07:40 UTC
With the gcc-6-20160103 snapshot the line in lto-symtab.c changes from 143 to 119:

lto1: internal compiler error: in lto_varpool_replace_node, at lto/lto-symtab.c:119
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: /usr/local/solido/tools/20160101/bin/gfortran returned 1 exit status
compilation terminated.
/usr/local/solido/tools/20160101/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Comment 7 anthonyfk 2016-01-09 04:15:45 UTC
Created attachment 37287 [details]
4.9.3 lsame.s
Comment 8 anthonyfk 2016-01-09 04:16:18 UTC
Created attachment 37288 [details]
4.9.3 slamch.s
Comment 9 anthonyfk 2016-01-09 04:16:44 UTC
Created attachment 37289 [details]
4.9.3 slamchtst.s
Comment 10 anthonyfk 2016-01-09 04:17:12 UTC
Created attachment 37290 [details]
5.1.0 lsame.s
Comment 11 anthonyfk 2016-01-09 04:17:37 UTC
Created attachment 37291 [details]
5.1.0 slamch.s
Comment 12 anthonyfk 2016-01-09 04:18:02 UTC
Created attachment 37292 [details]
5.1.0 slamchtst.s
Comment 13 anthonyfk 2016-01-09 04:19:38 UTC
I've discovered that the error occurred between 4.9.3 and 5.1.0. I've attached the .s assembly files for the objects from both of those versions.
Comment 14 Matt Godbolt 2016-06-30 09:15:37 UTC
We confirm seeing the same issue (a large proprietary system, when building optimized tests at -O3 but linking against test libraries compiled with -O2).
Comment 15 Matt Godbolt 2016-06-30 09:58:25 UTC
I just tried on GCC 6.1; same issue (lto-symtab.c:119 similarly to anthony's snapshot build).
Comment 16 anthonyfk 2016-07-29 18:53:47 UTC
Updating a few fields and marking as a regression to hopefully catch some attention. I'm glad someone else was able to confirm the bug. I'm happy to provide any more help or test patches.
Comment 17 anthonyfk 2016-07-29 18:54:41 UTC
Fixing version field since the bug was found in 5.1.0.
Comment 18 Richard Biener 2016-12-01 14:21:26 UTC
Confirmed.

lto1: internal compiler error: in lto_varpool_replace_node, at lto/lto-symtab.c:122
0x62e519 lto_varpool_replace_node
        /space/rguenther/src/gcc-git/gcc/lto/lto-symtab.c:122
0x62f67a lto_symtab_merge_symbols_1
        /space/rguenther/src/gcc-git/gcc/lto/lto-symtab.c:882
0x62f67a lto_symtab_merge_symbols()
        /space/rguenther/src/gcc-git/gcc/lto/lto-symtab.c:939
0x62352e read_cgraph_and_symbols
        /space/rguenther/src/gcc-git/gcc/lto/lto.c:2930
0x62352e lto_main()
        /space/rguenther/src/gcc-git/gcc/lto/lto.c:3304
Please submit a full bug report,

The test is just using -O -fprofile-generate -flto -fPIC and letting option
auto-deduce work at link time (no options given). One file is built with -O0.

118     static void
119     lto_varpool_replace_node (varpool_node *vnode,
120                               varpool_node *prevailing_node)
121     {
122       gcc_assert (!vnode->definition || prevailing_node->definition);
123       gcc_assert (!vnode->analyzed || prevailing_node->analyzed);
124
125       prevailing_node->clone_referring (vnode);
126       if (vnode->force_output)
(gdb) p vnode->definition
$1 = 1
(gdb) p prevailing_node->definition
$2 = 0
(gdb) p vnode->analyzed
$3 = 1
(gdb) p prevailing_node->analyzed
$4 = 0
Comment 19 Martin Liška 2017-01-13 12:15:07 UTC
I've got patch for that, currently testing.
Comment 20 Martin Liška 2017-01-20 09:45:58 UTC
Author: marxin
Date: Fri Jan 20 09:45:04 2017
New Revision: 244692

URL: https://gcc.gnu.org/viewcvs?rev=244692&root=gcc&view=rev
Log:
Do not declare artificial variables in tree-profile.c to have a definition (PR lto/69188).

2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* gcc.dg/lto/pr69188_0.c: New test.
	* gcc.dg/lto/pr69188_1.c: New test.
2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* tree-profile.c (init_ic_make_global_vars): Do not call
	finalize_decl.
	(gimple_init_gcov_profiler): Likewise.

Added:
    trunk/gcc/testsuite/gcc.dg/lto/pr69188_0.c
    trunk/gcc/testsuite/gcc.dg/lto/pr69188_1.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/tree-profile.c
Comment 21 Martin Liška 2017-01-20 09:46:40 UTC
Fixed on trunk.
Comment 22 ktkachov 2017-01-23 11:23:26 UTC
Author: ktkachov
Date: Mon Jan 23 11:22:54 2017
New Revision: 244779

URL: https://gcc.gnu.org/viewcvs?rev=244779&root=gcc&view=rev
Log:
[testsuite] Fix FAIL: gcc.dg/lto/pr69188 on bare-metal targets

	PR lto/69188
	* gcc.dg/lto/pr69188_0.c: Require profiling support for testcase.


Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gcc.dg/lto/pr69188_0.c
Comment 23 Martin Liška 2017-01-23 12:25:26 UTC
Author: marxin
Date: Mon Jan 23 12:24:54 2017
New Revision: 244797

URL: https://gcc.gnu.org/viewcvs?rev=244797&root=gcc&view=rev
Log:
Do not declare artificial variables in tree-profile.c to have a definition (PR lto/69188).

2017-01-23  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* tree-profile.c (init_ic_make_global_vars): Do not call
	finalize_decl.
	(gimple_init_gcov_profiler): Likewise.
2017-01-23  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* gcc.dg/lto/pr69188_0.c: New test.
	* gcc.dg/lto/pr69188_1.c: New test.

Added:
    branches/gcc-5-branch/gcc/testsuite/gcc.dg/lto/pr69188_0.c
    branches/gcc-5-branch/gcc/testsuite/gcc.dg/lto/pr69188_1.c
Modified:
    branches/gcc-5-branch/gcc/ChangeLog
    branches/gcc-5-branch/gcc/testsuite/ChangeLog
    branches/gcc-5-branch/gcc/tree-profile.c
Comment 24 Martin Liška 2017-01-23 12:26:37 UTC
Author: marxin
Date: Mon Jan 23 12:26:05 2017
New Revision: 244799

URL: https://gcc.gnu.org/viewcvs?rev=244799&root=gcc&view=rev
Log:
Do not declare artificial variables in tree-profile.c to have a definition (PR lto/69188).

2017-01-23  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* tree-profile.c (init_ic_make_global_vars): Do not call
	finalize_decl.
	(gimple_init_gcov_profiler): Likewise.
2017-01-23  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2017-01-20  Martin Liska  <mliska@suse.cz>

	PR lto/69188
	* gcc.dg/lto/pr69188_0.c: New test.
	* gcc.dg/lto/pr69188_1.c: New test.

Added:
    branches/gcc-6-branch/gcc/testsuite/gcc.dg/lto/pr69188_0.c
    branches/gcc-6-branch/gcc/testsuite/gcc.dg/lto/pr69188_1.c
Modified:
    branches/gcc-6-branch/gcc/ChangeLog
    branches/gcc-6-branch/gcc/testsuite/ChangeLog
    branches/gcc-6-branch/gcc/tree-profile.c
Comment 25 Martin Liška 2017-01-23 12:42:27 UTC
Fixed on all active branches.
Comment 26 anthonyfk 2017-01-23 14:57:02 UTC
Thank you very much for the fix! I've tried the patch as applied to gcc-6-branch with an otherwise fresh tarball of gcc 6.3.0. When compiling lapack-3.6.1 I no longer get the original bug, but I do get:

lto1: internal compiler error: resolution sub id 0x7485cad21ad62861 not in object file
0x54e61b lto_resolution_read
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:1929
0x54e61b lto_file_read
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:2092
0x54e61b read_cgraph_and_symbols
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:2806
0xc9045a lto_main()
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:3310
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: /usr/local/solido/tools/20170120/bin/gfortran returned 1 exit status
compilation terminated.
/usr/local/solido/tools/20170120/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:129: ../xeigtsts] Error 1

I tried searching the bug tracker to see if this was fixed already but didn't have any luck. I'll try to put together a reduced test case in the next week, but it's unfortunately a busy time of year for me so that may take longer. At the very least, I can try with the latest tarball from the 6 branch easily.
Comment 27 Martin Liška 2017-01-23 15:03:32 UTC
(In reply to anthonyfk from comment #26)
> Thank you very much for the fix! I've tried the patch as applied to
> gcc-6-branch with an otherwise fresh tarball of gcc 6.3.0. When compiling
> lapack-3.6.1 I no longer get the original bug, but I do get:
> 
> lto1: internal compiler error: resolution sub id 0x7485cad21ad62861 not in
> object file
> 0x54e61b lto_resolution_read
> 	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:1929
> 0x54e61b lto_file_read
> 	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:2092
> 0x54e61b read_cgraph_and_symbols
> 	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:2806
> 0xc9045a lto_main()
> 	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:3310
> Please submit a full bug report,
> with preprocessed source if appropriate.
> Please include the complete backtrace with any bug report.
> See <http://gcc.gnu.org/bugs.html> for instructions.
> lto-wrapper: fatal error: /usr/local/solido/tools/20170120/bin/gfortran
> returned 1 exit status
> compilation terminated.
> /usr/local/solido/tools/20170120/bin/ld: error: lto-wrapper failed
> collect2: error: ld returned 1 exit status
> make[2]: *** [Makefile:129: ../xeigtsts] Error 1
> 
> I tried searching the bug tracker to see if this was fixed already but
> didn't have any luck. I'll try to put together a reduced test case in the
> next week, but it's unfortunately a busy time of year for me so that may
> take longer. At the very least, I can try with the latest tarball from the 6
> branch easily.

Reduce test-case would be handy. Can you please re-test that also on trunk?

Thanks
Comment 28 anthonyfk 2017-01-24 21:44:04 UTC
The plot thickens.

Before trying a newer gcc, I ran a quick sanity check to make sure that the bug in comment 26 was repeatable. It compiled fine! So I set the compilation of lapack-3.6.1 up in a loop which would break on failure. After several iterations I got a different failure:

lto1: internal compiler error: in read_cgraph_and_symbols, at lto/lto.c:2761
0x54dcb9 read_cgraph_and_symbols
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:2761
0xc9045a lto_main()
	/tmp/solido-build-20170120/gcc-6.3.0/gcc/lto/lto.c:3310
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: /usr/local/solido/tools/20170120/bin/gfortran returned 1 exit status
compilation terminated.
/usr/local/solido/tools/20170120/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:134: ../xeigtstc] Error 1

My best guess is that this is some race condition (I have 12 cores going at once) but I can't imagine how. Regardless, I'm convinced that this is a different issue that may have more to do with the environment than gcc. Sorry for the noise! If I have time, I'll try to make a reduced, reproducible case as a new bug.

Thanks again!