Bug 65380 - [5 Regression][ICF] LTO: ICE in add_symbol_to_partition_1, at lto/lto-partition.c:158
Summary: [5 Regression][ICF] LTO: ICE in add_symbol_to_partition_1, at lto/lto-partiti...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 5.0
: P1 normal
Target Milestone: 5.0
Assignee: Jan Hubicka
URL:
Keywords:
Depends on:
Blocks: 65303
  Show dependency treegraph
 
Reported: 2015-03-10 16:42 UTC by Tobias Burnus
Modified: 2015-12-02 11:13 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2015-03-10 00:00:00


Attachments
one.ii (1.29 KB, text/plain)
2015-03-10 16:42 UTC, Tobias Burnus
Details
two.ii (1.86 KB, text/plain)
2015-03-10 16:43 UTC, Tobias Burnus
Details
one.o.000i.cgraph: -fdump-ipa-cgraph for the LTO step for the unpatched GCC (r221482) (11.37 KB, text/plain)
2015-03-18 08:22 UTC, Tobias Burnus
Details
Shell script, which compiles ChibiOS and indeed triggers gcc failure (154 bytes, application/x-shellscript)
2015-12-01 22:33 UTC, milan.plzik
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Burnus 2015-03-10 16:42:54 UTC
Created attachment 35003 [details]
one.ii

Follow up to PR65276, PR65302, PR65316, PR65361 - still for the same program.

Somewhat reduced test case (NOTE: The order one.ii/two.ii matters; if one reverses it, it compiles):

 g++ -c -w -O2 -flto -std=c++11 one.ii two.ii
 gcc -r -nostdlib one.o two.o

fails with:

lto1: internal compiler error: in add_symbol_to_partition_1, at lto/lto-partition.c:158
0x637bd0 add_symbol_to_partition_1
        ../../gcc/lto/lto-partition.c:157
0x637a1a add_symbol_to_partition_1
        ../../gcc/lto/lto-partition.c:202
0x63942c lto_balanced_map(int)
        ../../gcc/lto/lto-partition.c:640
0x632089 do_whole_program_analysis
        ../../gcc/lto/lto.c:3304
0x632089 lto_main()
        ../../gcc/lto/lto.c:3465
Comment 1 Tobias Burnus 2015-03-10 16:43:34 UTC
Created attachment 35004 [details]
two.ii
Comment 2 Jan Hubicka 2015-03-10 19:55:20 UTC
Will take a look.  Is it an open source program? I may include it into my testing :)
Comment 3 Tobias Burnus 2015-03-11 10:29:34 UTC
(In reply to Jan Hubicka from comment #2)
> Will take a look.  Is it an open source program?

Thanks. Unfortunately, it isn't open source; it's mostly used in house and there were talks about open-sourcing it, but it has not (yet) happened.
Comment 4 Jan Hubicka 2015-03-16 00:36:18 UTC
 Hmm, this one compiles just fine for me with today mainline.  Does the problem still reproduce for you?  Can you possibly dump out node->debug() and c?
Comment 5 Tobias Burnus 2015-03-16 07:52:00 UTC
(In reply to Jan Hubicka from comment #4)
> Hmm, this one compiles just fine for me with today mainline.  Does the
> problem still reproduce for you?  Can you possibly dump out node->debug()
> and c?

Unfortunately, it still fails for me with today's r221445 (for the very first invocation of line lto-partition.c:158). For

157   gcc_assert (c != SYMBOL_EXTERNAL
158               && (c == SYMBOL_DUPLICATE || !symbol_partitioned_p (node)));


Running

lto1 -quiet -dumpbase one.o -mtune=generic -march=x86-64 -mtune=generic -march=x86-64 -auxbase one -O2 -version -fexceptions -fmath-errno -fsigned-zeros -ftrapping-math -fno-trapv -fno-openmp -fno-openacc -fltrans-output-list=/tmp/ccX5yPa0.ltrans.out -fwpa -fresolution=two.res @/tmp/ccYbGLe0

in the debugger gives:


(gdb) p c
$1 = SYMBOL_EXTERNAL

(gdb) p node->debug()
_ZTCSt14basic_ofstreamIcSt11char_traitsIcEE0_So/188 (_ZTCSt14basic_ofstreamIcSt11char_traitsIcEE0_So) @0x7ffff1d38880
  Type: variable definition analyzed alias
  Visibility: externally_visible external public visibility_specified visibility:hidden virtual artificial
  References: _ZTCN7Utility2IO12GUnzipStreamE0_So/7 (alias)
  Referring: _ZTTSt14basic_ofstreamIcSt11char_traitsIcEE/178 (addr)_ZTTSt14basic_ofstreamIcSt11char_traitsIcEE/178 (addr)
  Availability: available
  Varpool flags: read-only const-value-known
$2 = void
Comment 6 Tobias Burnus 2015-03-16 07:59:26 UTC
Note that it compiles if I add "-fno-ipa-icf".
Comment 7 Jan Hubicka 2015-03-17 00:51:13 UTC
> Note that it compiles if I add "-fno-ipa-icf".

Yeah, but it is partitioning bug; it should be able to deal with whatever aliases ICF creates.
I will take a look tonight or tomorrow.

Honza
Comment 8 Jan Hubicka 2015-03-18 02:13:28 UTC
I am with terrible internet connection and it still does not reproduce for me (I suppose difference between GNU LD and gold).
It is however clear what happens, we try to add symbol's alias that is external and we do not expect external symbols to be in partitions.

Does the following (untested) help?
Index: lto-partition.c
===================================================================
--- lto-partition.c     (revision 221399)
+++ lto-partition.c     (working copy)
@@ -198,7 +198,7 @@
   /* Add all aliases associated with the symbol.  */
 
   FOR_EACH_ALIAS (node, ref)
-    if (!node->weakref)
+    if (!node->weakref && !DECL_ExTERNAL (node->decl))
       add_symbol_to_partition_1 (part, ref->referring);
 
   /* Ensure that SAME_COMDAT_GROUP lists all allways added in a group.  */
Comment 9 Jan Hubicka 2015-03-18 02:14:34 UTC
please also attachg WPA dump of -fdump-ipa-cgraph. I would be interested what visibility _ZTCN7Utility2IO12GUnzipStreamE0_So/7 has.
Comment 10 Jan Hubicka 2015-03-18 02:32:13 UTC
Actually perhaps we manage to produce external alias of non-external symbol that also should not happen.
Index: ipa-icf.c
===================================================================
--- ipa-icf.c   (revision 221461)
+++ ipa-icf.c   (working copy)
@@ -809,6 +809,15 @@ sem_function::merge (sem_item *alias_ite
   bool original_address_matters = original->address_matters_p ();
   bool alias_address_matters = alias->address_matters_p ();
 
+  if (DECL_EXTERNAL (alias->decl))
+    {
+      if (dump_file)
+       fprintf (dump_file,
+                "Not unifying; "
+                "alias is external.\n\n");
+      return false;
+    }
+
   if (DECL_NO_INLINE_WARNING_P (original->decl)
       != DECL_NO_INLINE_WARNING_P (alias->decl))
     {
@@ -1767,6 +1870,14 @@ sem_variable::merge (sem_item *alias_ite
                 "Symbol aliases are not supported by target\n\n");
       return false;
     }
+  if (DECL_EXTERNAL (alias->decl))
+    {
+      if (dump_file)
+       fprintf (dump_file,
+                "Not unifying; "
+                "alias is external.\n\n");
+      return false;
+    }
 
   sem_variable *alias_var = static_cast<sem_variable *> (alias_item);
 

May help.
Comment 11 Tobias Burnus 2015-03-18 08:22:17 UTC
Created attachment 35050 [details]
one.o.000i.cgraph: -fdump-ipa-cgraph for the LTO step for the unpatched GCC (r221482)

(In reply to Jan Hubicka from comment #8, #9 and #10)

First, I can still reproduce the issue with the unpatched GCC. In addition, the big program also compiles successfully with -fno-ipa-icf.

Regarding "ld": As /usr/bin/ld is very old, that's with the current "ld" from the binutils-gdb GIT (via GCC's --with-plugin-ld=) - it fails identical when I replace the ld.bfd by ld.gold.

Regarding -fdump-ipa-cgraph: If you prefer the dump of a patched version, please tell me.

Regarding the patch in comment 8 (with "ExTERNAL" typo fixed): Unfortunately, it doesn't solve the problem - I still get the same ICE.


Regarding the patch in comment 9: I had to change "alias" to "alias_item" due to

../../gcc/ipa-icf.c:1779:22: error: ‘alias’ was not declared in this scope
   if (DECL_EXTERNAL (alias->decl))

but afterwards: It no longer has an ICE!
Comment 12 Martin Liška 2015-03-18 13:19:27 UTC
(In reply to Tobias Burnus from comment #11)
> Created attachment 35050 [details]
> one.o.000i.cgraph: -fdump-ipa-cgraph for the LTO step for the unpatched GCC
> (r221482)
> 
> (In reply to Jan Hubicka from comment #8, #9 and #10)
> 
> First, I can still reproduce the issue with the unpatched GCC. In addition,
> the big program also compiles successfully with -fno-ipa-icf.
> 
> Regarding "ld": As /usr/bin/ld is very old, that's with the current "ld"
> from the binutils-gdb GIT (via GCC's --with-plugin-ld=) - it fails identical
> when I replace the ld.bfd by ld.gold.
> 
> Regarding -fdump-ipa-cgraph: If you prefer the dump of a patched version,
> please tell me.
> 
> Regarding the patch in comment 8 (with "ExTERNAL" typo fixed):
> Unfortunately, it doesn't solve the problem - I still get the same ICE.
> 
> 
> Regarding the patch in comment 9: I had to change "alias" to "alias_item"
> due to
> 
> ../../gcc/ipa-icf.c:1779:22: error: ‘alias’ was not declared in this scope
>    if (DECL_EXTERNAL (alias->decl))
> 
> but afterwards: It no longer has an ICE!

Hello.

What's your target where you have the PR? I'm also unable to reproduce the issue. Even with BFD.

Thanks,
Martin
Comment 13 Tobias Burnus 2015-03-18 15:08:37 UTC
(In reply to Martin Liška from comment #12)
> What's your target where you have the PR? I'm also unable to reproduce the
> issue. Even with BFD.

Build/host/target is a CentOS 6.6 x86_64-unknown-linux-gnu system, with the GIT version of GCC installed and in the $PATH, the GIT version of binutils not in the path (neither during build nor during run). GCC is configured with:
  --with-plugin-ld=<path to GIT version>/bin/ld --enable-checking=yes
  --enable-languages=c,c++,fortran,lto
Thus, for LTO the plugin-ld is invoked via the (implicit) -fuse-linker-plugin.

The GIT version of binutils is build with /usr/bin/gcc (= 4.4) and with configure options --with-python (= nonsystem Python 2.7.8) --enable-plugins --enable-ld --enable-gold.

The CentOS6 ships with glibc glibc-2.12-1.149.el6_6.5 and binutils-2.20.51.0.2-5.42.el6.

Just to make sure, I have just re-bootstrapped GCC (r221491) and re-downloaded the attached files - and it still ICEs for me without the patch in comment 10.
Comment 14 Jan Hubicka 2015-03-18 18:28:22 UTC
Ah, thanks! The patch fixed as in coment #10 is OK for mainline.  We obviously should not try to do any merging of external symbols; it is pointless. We only want to calculate equivalences on these to merge non-externals referencing them.

Patch is OK with the fixes and testcase.
Comment 15 Martin Liška 2015-03-19 17:37:46 UTC
Author: marxin
Date: Thu Mar 19 17:37:15 2015
New Revision: 221519

URL: https://gcc.gnu.org/viewcvs?rev=221519&root=gcc&view=rev
Log:
Fix PR ipa/65380.

	PR ipa/65380
	* ipa-icf.c (sem_function::merge): Do not merge DECL_EXTERNAL symbols.
	(sem_variable::merge): Likewise.


Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/ipa-icf.c
Comment 16 Martin Liška 2015-03-19 17:38:03 UTC
Fixed in 5.0.0.
Comment 17 milan.plzik 2015-11-29 17:36:48 UTC
with gcc 5.2.0, I still get this error when e.g. compiling ChibiOS example:
$ make
Linking build/ch.elf
lto1: internal compiler error: in add_symbol_to_partition_1, at lto/lto-partition.c:158
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.archlinux.org/> for instructions.
lto-wrapper: fatal error: arm-none-eabi-gcc returned 1 exit status
compilation terminated.
/usr/lib/gcc/arm-none-eabi/5.2.0/../../../../arm-none-eabi/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
../../../os/common/ports/ARMCMx/compilers/GCC/rules.mk:238: recipe for target 'build/ch.elf' failed
make: *** [build/ch.elf] Error 1

$ arm-none-eabi-gcc -v
Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-none-eabi/5.2.0/lto-wrapper
Target: arm-none-eabi
Configured with: /build/arm-none-eabi-gcc/src/gcc-5.2.0/configure --target=arm-none-eabi --prefix=/usr --with-sysroot=/usr/arm-none-eabi --with-native-system-header-dir=/include --libexecdir=/usr/lib --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-system-zlib --with-newlib --with-headers=/usr/arm-none-eabi/include --with-python-dir=share/gcc-arm-none-eabi --with-gmp --with-mpfr --with-mpc --with-isl --with-libelf --enable-gnu-indirect-function --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='Arch Repository' --with-bugurl=https://bugs.archlinux.org/ --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r
Thread model: single
gcc version 5.2.0 (Arch Repository)
Comment 18 Martin Liška 2015-11-30 08:09:26 UTC
(In reply to milan.plzik from comment #17)
> with gcc 5.2.0, I still get this error when e.g. compiling ChibiOS example:
> $ make
> Linking build/ch.elf
> lto1: internal compiler error: in add_symbol_to_partition_1, at
> lto/lto-partition.c:158
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See <https://bugs.archlinux.org/> for instructions.
> lto-wrapper: fatal error: arm-none-eabi-gcc returned 1 exit status
> compilation terminated.
> /usr/lib/gcc/arm-none-eabi/5.2.0/../../../../arm-none-eabi/bin/ld:
> lto-wrapper failed
> collect2: error: ld returned 1 exit status
> ../../../os/common/ports/ARMCMx/compilers/GCC/rules.mk:238: recipe for
> target 'build/ch.elf' failed
> make: *** [build/ch.elf] Error 1
> 
> $ arm-none-eabi-gcc -v
> Using built-in specs.
> COLLECT_GCC=arm-none-eabi-gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-none-eabi/5.2.0/lto-wrapper
> Target: arm-none-eabi
> Configured with: /build/arm-none-eabi-gcc/src/gcc-5.2.0/configure
> --target=arm-none-eabi --prefix=/usr --with-sysroot=/usr/arm-none-eabi
> --with-native-system-header-dir=/include --libexecdir=/usr/lib
> --enable-languages=c,c++ --enable-plugins --disable-decimal-float
> --disable-libffi --disable-libgomp --disable-libmudflap
> --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls
> --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld
> --with-system-zlib --with-newlib --with-headers=/usr/arm-none-eabi/include
> --with-python-dir=share/gcc-arm-none-eabi --with-gmp --with-mpfr --with-mpc
> --with-isl --with-libelf --enable-gnu-indirect-function
> --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm'
> --with-pkgversion='Arch Repository'
> --with-bugurl=https://bugs.archlinux.org/
> --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r
> Thread model: single
> gcc version 5.2.0 (Arch Repository)

Hi.

Can you please provide a test-case which I can reproduce?

Thanks,
Martin
Comment 19 milan.plzik 2015-12-01 22:33:11 UTC
Created attachment 36886 [details]
Shell script, which compiles ChibiOS and indeed triggers gcc failure

Attached script should trigger the problem when compiling ChibiOS sources.
Comment 20 Martin Liška 2015-12-02 10:16:46 UTC
(In reply to milan.plzik from comment #19)
> Created attachment 36886 [details]
> Shell script, which compiles ChibiOS and indeed triggers gcc failure
> 
> Attached script should trigger the problem when compiling ChibiOS sources.

Hi.

Well the script is fine, however it requires a base metal compiler.
Would it be possible to run the test-case on one of there cross compilers:

cross-armv6hl-gcc5
cross-armv7hl-gcc5

Or I can test it on an aarch64 machine, does it expose the failure?

Problem with the issue is that it requires multiple LTO object files
that eventually trigger the failure.

Thanks,
Martin
Comment 21 milan.plzik 2015-12-02 11:13:31 UTC
Unfortunately, I currently don't have dev setup for any non-MCU ARM platform. Also, ChibiOS non-MCU platforms, so I guess I would need to identify different test case, assuming it is possible to trigger this error on non-bare-metal platforms. 

Anyway, if this is wrong place to report problems with bare-metal gcc targets, please let me know where should this be filed :).