Bug 97295 - ICE on firefox built with lto+pgo: dist/include/mozilla/Casting.h:64:1: internal compiler error: in to_frequency, at profile-count.c:273
Summary: ICE on firefox built with lto+pgo: dist/include/mozilla/Casting.h:64:1: inter...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: ipa (show other bugs)
Version: 11.0
: P3 normal
Target Milestone: ---
Assignee: Martin Liška
URL:
Keywords: lto
Depends on:
Blocks: mozillametabug
  Show dependency treegraph
 
Reported: 2020-10-05 18:37 UTC by Sergei Trofimovich
Modified: 2020-10-19 16:55 UTC (History)
6 users (show)

See Also:
Host:
Target:
Build:
Known to work: 10.2.1, 11.0
Known to fail: 9.3.0
Last reconfirmed: 2020-10-06 00:00:00


Attachments
ICE-testcase-gcc-10.2.0.tar.gz (364.00 KB, application/gzip)
2020-10-05 18:40 UTC, Sergei Trofimovich
Details
b.tar.gz (792 bytes, application/gzip)
2020-10-09 08:05 UTC, Sergei Trofimovich
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sergei Trofimovich 2020-10-05 18:37:49 UTC
It's an upstream report of downstream bug https://bugs.gentoo.org/746578 where firefox-81.0 built with gcc-9.3.0 or gcc-10.2.0 ICEs on LTO+PGO build:

The crash happens somewhere in register allocator:

"""
$ LANG=C g++-10.2.0 -o TestFloatingPoint TestFloatingPoint.o  Unified_cpp_mfbt0.o -shared
during RTL pass: ira
/var/tmp/portage/www-client/firefox-81.0.1/work/firefox_build/dist/include/mozilla/Casting.h: In function 'BitwiseCast':
/var/tmp/portage/www-client/firefox-81.0.1/work/firefox_build/dist/include/mozilla/Casting.h:64:1: internal compiler error: in to_frequency, at profile-count.c:273
0x5b0db1 profile_count::to_frequency(function*) const
        /usr/src/debug/sys-devel/gcc-10.2.0-r2/gcc-10.2.0/gcc/profile-count.c:273
0x9edea9 regstat_bb_compute_ri
        /usr/src/debug/sys-devel/gcc-10.2.0-r2/gcc-10.2.0/gcc/regstat.c:200
0x9edea9 regstat_compute_ri()
        /usr/src/debug/sys-devel/gcc-10.2.0-r2/gcc-10.2.0/gcc/regstat.c:253
0x8a5958 ira
        /usr/src/debug/sys-devel/gcc-10.2.0-r2/gcc-10.2.0/gcc/ira.c:5294
0x8a5958 execute
        /usr/src/debug/sys-devel/gcc-10.2.0-r2/gcc-10.2.0/gcc/ira.c:5666
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://bugs.gentoo.org/> for instructions.
lto-wrapper: fatal error: /usr/bin/g++-10.2.0 returned 1 exit status
compilation terminated.
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
"""

$ LANG=C g++-10.2.0 -v
Using built-in specs.
COLLECT_GCC=/usr/bin/g++-10.2.0
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/10.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-10.2.0-r2/work/gcc-10.2.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/10.2.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/g++-v10 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/10.2.0/python --enable-languages=c,c++,go,jit,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 10.2.0-r2 p3' --disable-esp --enable-libstdcxx-time --enable-host-shared --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --enable-vtable-verify --without-zstd --enable-lto --with-isl --disable-isl-version-check --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (Gentoo 10.2.0-r2 p3)
Comment 1 Sergei Trofimovich 2020-10-05 18:40:00 UTC
Created attachment 49309 [details]
ICE-testcase-gcc-10.2.0.tar.gz

ICE-testcase-gcc-10.2.0.tar.gz contains two object files that seems to be enough to feed into gcc-10.2.0 to get IRA crash. Unfortunately it's not a source-based example. Is it useful to get the idea where ira goes wrong?

I hope it's a simple(ish) bug in not traversing some subset of CFG.
Comment 2 Sergei Trofimovich 2020-10-05 18:52:47 UTC
Also fails on unpatched releases/gcc-10.2.0 built as:

"""
$ ${HOME}/dev/git/gcc-10-build/gcc/xg++ -B${HOME}/dev/git/gcc-10-build/gcc -v

Reading specs from /home/slyfox/dev/git/gcc-10-build/gcc/specs
COLLECT_GCC=/home/slyfox/dev/git/gcc-10-build/gcc/xg++
COLLECT_LTO_WRAPPER=/home/slyfox/dev/git/gcc-10-build/gcc/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../gcc-10/configure --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-languages=c,c++ --disable-bootstrap --with-multilib-list=m64 --prefix=/home/slyfox/dev/git/gcc-10-build/../gcc-native-quick-installed --disable-nls --without-isl --disable-libsanitizer --disable-libvtv --disable-libgomp --disable-libstdcxx-pch --disable-libunwind-exceptions CFLAGS='-O0 -ggdb3 ' CXXFLAGS='-O0 -ggdb3 ' --with-sysroot=/usr/x86_64-HEAD-linux-gnu --enable-valgrind-annotations --without-zstd
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (GCC)
"""
Comment 3 Sergei Trofimovich 2020-10-05 22:34:51 UTC
Poking at the crash to get clues:

"""
(gdb) bt
#0  internal_error (gmsgid=0x285ac9f "in %s, at %s:%d") at ../../gcc-10/gcc/diagnostic.c:1706
#1  0x0000000001f7c34a in fancy_abort (file=0x20c3e90 "../../gcc-10/gcc/profile-count.c", line=273, function=0x20c3e81 "to_frequency") at ../../gcc-10/gcc/diagnostic.c:1778
#2  0x0000000000f0563a in profile_count::to_frequency (this=0x7ffff759b190, fun=0x7ffff7585000) at ../../gcc-10/gcc/profile-count.c:273
#3  0x0000000000f3b5a4 in regstat_bb_compute_ri (bb=0x7ffff759b138, live=0x2ee0c20) at ../../gcc-10/gcc/regstat.c:200
#4  0x0000000000f3b9df in regstat_compute_ri () at ../../gcc-10/gcc/regstat.c:253
#5  0x0000000000d23399 in ira (f=0x0) at ../../gcc-10/gcc/ira.c:5294
#6  0x0000000000d23d95 in (anonymous namespace)::pass_ira::execute (this=0x2ddb750) at ../../gcc-10/gcc/ira.c:5666
#7  0x0000000000ec631d in execute_one_pass (pass=0x2ddb750) at ../../gcc-10/gcc/passes.c:2502
#8  0x0000000000ec6655 in execute_pass_list_1 (pass=0x2ddb750) at ../../gcc-10/gcc/passes.c:2590
#9  0x0000000000ec6686 in execute_pass_list_1 (pass=0x2dda4d0) at ../../gcc-10/gcc/passes.c:2591
#10 0x0000000000ec66df in execute_pass_list (fn=0x7ffff7585000, pass=0x2dd6790) at ../../gcc-10/gcc/passes.c:2601
#11 0x00000000009c5de3 in cgraph_node::expand (this=0x7ffff758e2d0) at ../../gcc-10/gcc/cgraphunit.c:2300
#12 0x00000000009c682d in output_in_order () at ../../gcc-10/gcc/cgraphunit.c:2578
#13 0x00000000009c6e6d in symbol_table::compile (this=0x7ffff773e100) at ../../gcc-10/gcc/cgraphunit.c:2819
#14 0x00000000008c41e7 in lto_main () at ../../gcc-10/gcc/lto/lto.c:653
#15 0x00000000010222ab in compile_file () at ../../gcc-10/gcc/toplev.c:458
#16 0x00000000010254cd in do_compile () at ../../gcc-10/gcc/toplev.c:2278
#17 0x00000000010257d8 in toplev::main (this=0x7fffffffd526, argc=21, argv=0x2db0700) at ../../gcc-10/gcc/toplev.c:2417
#18 0x0000000001f4b529 in main (argc=21, argv=0x7fffffffd638) at ../../gcc-10/gcc/main.c:39
"""

"""
(gdb) fr 2
#2  0x0000000000f0563a in profile_count::to_frequency (this=0x7ffff759b190, fun=0x7ffff7585000) at ../../gcc-10/gcc/profile-count.c:273
273       gcc_assert (REG_BR_PROB_BASE == BB_FREQ_MAX

(gdb) call print_generic_decl(stderr,  fun->decl, 0)
  static long unsigned int BitwiseCast (double);

(gdb) call debug_gimple_stmt(fun->gimple_body)
# .MEM_6 = VDEF <.MEM_5(D)>
BitwiseCast (aFrom_2(D), &temp);
"""


"""
$ lto-dump -dump-body=BitwiseCast Unified_cpp_mfbt0.o
Gimple Body of Function: BitwiseCast
BitwiseCast (const double aFrom)
{
  <bb 2> [count: 1509]:
  _3 = VIEW_CONVERT_EXPR<long unsigned int>(aFrom_2(D));
  return _3;

}

$ lto-dump -dump-body=BitwiseCast TestFloatingPoint.o
...
Gimple Body of Function: BitwiseCast
BitwiseCast (const double aFrom)
{
  long unsigned int temp;
  long unsigned int D.4528;

  <bb 2> :
  BitwiseCast (aFrom_2(D), &temp);
  _4 = temp;
  temp ={v} {CLOBBER};

  <bb 3> :
<L0>:
  return _4;

}

Gimple Body of Function: BitwiseCast
BitwiseCast (const double aFrom, long unsigned int * aResult)
{
  long unsigned int D.4534;

  <bb 2> :
  _2 = MEM <long unsigned int> [(char * {ref-all})&aFrom];
  MEM <long unsigned int> [(char * {ref-all})aResult_3(D)] = _2;
  return;

}
...
"""

Source definition of the template:

"""
// from firefox-81.0.1/mfbt/Casting.h

template <typename To, typename From>
inline void BitwiseCast(const From aFrom, To* aResult) {
  static_assert(sizeof(From) == sizeof(To),
                "To and From must have the same size");

  // We could maybe downgrade these to std::is_trivially_copyable, but the
  // various STLs we use don't all provide it.
  static_assert(std::is_trivial<From>::value,
                "shouldn't bitwise-copy a type having non-trivial "
                "initialization");
  static_assert(std::is_trivial<To>::value,
                "shouldn't bitwise-copy a type having non-trivial "
                "initialization");

  std::memcpy(static_cast<void*>(aResult), static_cast<const void*>(&aFrom),
              sizeof(From));
}
"""

My interpretation of the above: under some circumstances (different profile data?) gcc generates two variants of specialised form of BitwiseCast<double,unsigned long>:
1. VIEW_CONVERT_EXPR form
2. and 'call BitwiseCast (const double aFrom, long unsigned int * aResult)' form

Once LTO merges both definitions somehow one of them becomes unreachable for initial part of register allocator (initial analysis?) but not later phase (actual register allocation?). And assertion fails.
Comment 4 Martin Liška 2020-10-06 11:15:20 UTC
Thank you for the report. I can reproduce it but it would be handy to provide a pre-processed source files (-E option) for the 2 objects.
Comment 5 Martin Liška 2020-10-07 10:23:20 UTC
So it's pretty clear what happens, it's about mixing of different flags:

TestFloatingPoint.o:

'-fno-openmp' '-fno-openacc' '-mtune=generic' '-march=x86-64' '-O3' '-O0' '-Wno-error=maybe-uninitialized' '-Wno-error=deprecated-declarations' '-Wno-error=array-bounds' '-Wno-error=coverage-mismatch' '-Wno-error=free-nonheap-object' '-Wno-error=class-memaccess' '-Wno-error=deprecated-copy' '-Wno-error=shadow' '-flto' '-flifetime-dse=1' '-fstack-protector-strong' '-fPIC' '-fno-strict-aliasing' '-ffunction-sections' '-fdata-sections' '-fno-exceptions' '-fno-math-errno' '-fomit-frame-pointer' '-funwind-tables'

while Unified_cpp_mfbt0.o:

'-fno-openmp' '-fno-openacc' '-mtune=generic' '-march=x86-64' '-O3' '-Wno-error=maybe-uninitialized' '-Wno-error=deprecated-declarations' '-Wno-error=array-bounds' '-Wno-error=coverage-mismatch' '-Wno-error=free-nonheap-object' '-Wno-error=class-memaccess' '-Wno-error=deprecated-copy' '-flto' '-flifetime-dse=1' '-fstack-protector-strong' '-fPIC' '-fno-strict-aliasing' '-ffunction-sections' '-fdata-sections' '-fno-exceptions' '-fno-math-errno' '-fomit-frame-pointer' '-funwind-tables' '-fprofile-use' '-fprofile-correction'

the profiles are merged in ipa_merge_profiles where match=false due to a different CFG and we forget to drop back dst->count. I'll prepare a patch for it.
Comment 6 Sergei Trofimovich 2020-10-09 08:05:03 UTC
Created attachment 49336 [details]
b.tar.gz

Managed to craft self-contained example which is able to crash both gcc-10 and gcc-11. Attached as b.tar.gz.

The session looks like:

$ LANG=C bash -x ./mk2.bash
+ rm TestFloatingPoint.o Unified_cpp_mfbt0.o main.o Unified_cpp_mfbt0.gcda main.gcda main
+ cxx=g++-11.0.0
+ cxxflags=(-flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile "$@")
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o Unified_cpp_mfbt0.o -c -O1 -fprofile-generate -fprofile-correction Unified_cpp_mfbt0.cpp
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o main.o -c -O1 -fprofile-generate -fprofile-correction main.cc
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o main -O1 -fprofile-generate main.o Unified_cpp_mfbt0.o
+ ./main
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o Unified_cpp_mfbt0.o -c -O1 -fprofile-use -fprofile-correction Unified_cpp_mfbt0.cpp
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o TestFloatingPoint.o -c -O0 TestFloatingPoint.cpp
+ g++-11.0.0 -flto -fPIC -O1 -Wno-error=coverage-mismatch -Wno-coverage-mismatch -Wno-missing-profile -o TestFloatingPoint -O1 -shared TestFloatingPoint.o Unified_cpp_mfbt0.o
during RTL pass: ira
TestFloatingPoint.cpp: In function 'BitwiseCast':
TestFloatingPoint.cpp:11:1: internal compiler error: in to_frequency, at profile-count.c:273
   11 | }
      | ^
0x5c6aaf profile_count::to_frequency(function*) const
        /usr/src/debug/sys-devel/gcc-11.0.0_pre9999/gcc-11.0.0_pre9999/gcc/profile-count.c:273
0x9f2511 regstat_bb_compute_ri
        /usr/src/debug/sys-devel/gcc-11.0.0_pre9999/gcc-11.0.0_pre9999/gcc/regstat.c:200
0x9f2511 regstat_compute_ri()
        /usr/src/debug/sys-devel/gcc-11.0.0_pre9999/gcc-11.0.0_pre9999/gcc/regstat.c:253
0x8b5680 ira
        /usr/src/debug/sys-devel/gcc-11.0.0_pre9999/gcc-11.0.0_pre9999/gcc/ira.c:5300
0x8b5680 execute
        /usr/src/debug/sys-devel/gcc-11.0.0_pre9999/gcc-11.0.0_pre9999/gcc/ira.c:5672
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://bugs.gentoo.org/> for instructions.
lto-wrapper: fatal error: /usr/bin/g++-11.0.0 returned 1 exit status
compilation terminated.
/usr/lib/gcc/x86_64-pc-linux-gnu/11.0.0/../../../../x86_64-pc-linux-gnu/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
Comment 7 Sergei Trofimovich 2020-10-09 08:09:00 UTC
(In reply to Sergei Trofimovich from comment #6)
> Created attachment 49336 [details]
> b.tar.gz
> 
> Managed to craft self-contained example which is able to crash both gcc-10
> and gcc-11. Attached as b.tar.gz.

On this test I observe failures on: 8.4.0, 9.3.0, 10.2.0, 11.0.0. Last working version is 7.5.0.
Comment 8 Martin Liška 2020-10-09 11:04:43 UTC
Thank you Sergei for the nice test-case.
It started with r8-4715-g35cd23ebb6d05e13.
Comment 9 GCC Commits 2020-10-15 07:57:07 UTC
The master branch has been updated by Martin Liska <marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:508e2d88a4c512e8b8685cf5ba201ad48e6bb58d

commit r11-3908-g508e2d88a4c512e8b8685cf5ba201ad48e6bb58d
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Oct 13 16:44:47 2020 +0200

    IPA: fix profile handling in IRA
    
    gcc/ChangeLog:
    
            PR ipa/97295
            * profile-count.c (profile_count::to_frequency): Move part of
            gcc_assert to STATIC_ASSERT.
            * regs.h (REG_FREQ_FROM_BB): Do not use count.to_frequency for
            a function that does not have count_max initialized.
Comment 10 Martin Liška 2020-10-15 07:59:20 UTC
Fixed on master so far.
Comment 11 GCC Commits 2020-10-15 08:43:45 UTC
The releases/gcc-10 branch has been updated by Martin Liska <marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:be8b62c99cf8baea6ee8163af8e85aa0e8634222

commit r10-8891-gbe8b62c99cf8baea6ee8163af8e85aa0e8634222
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Oct 13 16:44:47 2020 +0200

    IPA: fix profile handling in IRA
    
    gcc/ChangeLog:
    
            PR ipa/97295
            * profile-count.c (profile_count::to_frequency): Move part of
            gcc_assert to STATIC_ASSERT.
            * regs.h (REG_FREQ_FROM_BB): Do not use count.to_frequency for
            a function that does not have count_max initialized.
    
    (cherry picked from commit 508e2d88a4c512e8b8685cf5ba201ad48e6bb58d)
Comment 12 GCC Commits 2020-10-15 09:07:37 UTC
The releases/gcc-9 branch has been updated by Martin Liska <marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:12c9413228d2955126ff5c45194f8aacf1aa81f6

commit r9-8996-g12c9413228d2955126ff5c45194f8aacf1aa81f6
Author: Martin Liska <mliska@suse.cz>
Date:   Tue Oct 13 16:44:47 2020 +0200

    IPA: fix profile handling in IRA
    
    gcc/ChangeLog:
    
            PR ipa/97295
            * profile-count.c (profile_count::to_frequency): Move part of
            gcc_assert to STATIC_ASSERT.
            * regs.h (REG_FREQ_FROM_BB): Do not use count.to_frequency for
            a function that does not have count_max initialized.
    
    (cherry picked from commit 508e2d88a4c512e8b8685cf5ba201ad48e6bb58d)
Comment 13 Martin Liška 2020-10-15 09:08:33 UTC
Fixed now on all problematic code streams.