% wget trippelsdorf.de/testcase.tar.bz2 % tar xvjf testcase.tar.bz2 % g++ -w -march=amdfam10 -flto=4 -fprofile-use -fprofile-correction -ffunction-sections -fdata-sections -fno-rtti -fno-exceptions -fno-math-errno -std=gnu++0x -pipe -O3 Unified_cpp_js_src1.ii Unified_cpp_js_src2.ii Unified_cpp_js_src3.ii lto1: internal compiler error: in ipa_merge_profiles, at ipa-utils.c:540 0x7f7d55 ipa_merge_profiles(cgraph_node*, cgraph_node*) ../../gcc/gcc/ipa-utils.c:540 0xf57883 ipa_icf::sem_function::merge(ipa_icf::sem_item*) ../../gcc/gcc/ipa-icf.c:625 0xf5cc86 ipa_icf::sem_item_optimizer::merge_classes(unsigned int) ../../gcc/gcc/ipa-icf.c:2241 0xf60bf9 ipa_icf::sem_item_optimizer::execute() ../../gcc/gcc/ipa-icf.c:1602 0xf61671 ipa_icf_driver ../../gcc/gcc/ipa-icf.c:2319 0xf61671 ipa_icf::pass_ipa_icf::execute(function*) ../../gcc/gcc/ipa-icf.c:2366 Please submit a full bug report, with preprocessed source if appropriate. Please include the complete backtrace with any bug report. See <http://gcc.gnu.org/bugs.html> for instructions -fno-ipa-icf "fixes" the issue.
Do you have Honza an idea how to handle correctly situation, where ipa_profile is called before IPA ICF and we mark speculative an edge in: #0 cgraph_edge::make_speculative (this=this@entry=0x7ffff69e8c98, n2=0x7ffff6933468, direct_count=1776, direct_frequency=<optimized out>) at ../../gcc/cgraph.c:1027 #1 0x0000000000844d6f in ipa_profile () at ../../gcc/ipa-profile.c:641 #2 (anonymous namespace)::pass_ipa_profile::execute (this=<optimized out>) at ../../gcc/ipa-profile.c:742 #3 0x000000000091bee2 in execute_one_pass (pass=pass@entry=0x1bc76b0) at ../../gcc/passes.c:2155 #4 0x000000000091cbc2 in execute_ipa_pass_list (pass=0x1bc76b0) at ../../gcc/passes.c:2545 #5 0x00000000005e222b in do_whole_program_analysis () at ../../gcc/lto/lto.c:3253 #6 0x00000000005e25e6 in lto_main () at ../../gcc/lto/lto.c:3431 #7 0x00000000009d48f2 in compile_file () at ../../gcc/toplev.c:555 #8 0x00000000009d6bdf in do_compile () at ../../gcc/toplev.c:1977 #9 toplev_main (argc=28, argv=0x1ba4970) at ../../gcc/toplev.c:2053 #10 0x00000000012140dc in main (argc=26, argv=0x7fffffffdd38) at ../../gcc/main.c:36 Thank you, Martin
speculative edges come in pairs (direct, indirect edge) that is obtained by speculative_call_info. THen you need to sum counts (instead of taking ones from BB) and turn them back to frequencies (because it is profile only counts should be non-0)
(In reply to Jan Hubicka from comment #2) > speculative edges come in pairs (direct, indirect edge) that is obtained by > speculative_call_info. THen you need to sum counts (instead of taking ones > from BB) and turn them back to frequencies (because it is profile only > counts should be non-0) Suggested patch https://gcc.gnu.org/ml/gcc-patches/2014-10/msg02764.html Jan, so you mean that counts from BBs and counts from speculative edges should be sumed?
Happend again today. Honza can you please take a look at Ilya's patch? https://gcc.gnu.org/ml/gcc-patches/2014-10/msg03089.html
I don't seem to be able to make sense of Ilya's logic in the probability calculation (incrementing the count with bb->count seem wrong). Will try to update his patch today.
mine.
Created attachment 34490 [details] Proposed fix I am going to try build firefox with PDO myself with the attached patch. It is bit tricky to merge the speculation right becuase both variant may disagree on speculative target.
Markus, I don't seem to be able to train firefox with current tree because of unrelated issues. If you would have chance to test the patch on your setup, it would be cool.
I will give your patch a try. In case it might help here's the script I use for LTO/PGO (as you can see it starts the instrumented browser. I then just visit a couple of webpages to train it): markus@x4 mozilla-central % cat profile_build #!/bin/zsh mv .mozconfig .mozconfig_tmp && cp .mozconfig_profile_gen .mozconfig && nice -n 19 make -f client.mk && killall firefox /var/tmp/moz-build-dir/dist/bin/firefox && rm /var/tmp/moz-build-dir/**/Makefile && rm /var/tmp/moz-build-dir/**/*.o && rm /var/tmp/moz-build-dir/**/config.status && rm /var/tmp/moz-build-dir/**/configure.pkl && cp .mozconfig_profile_use .mozconfig && nice -n 19 make -f client.mk && make DESTDIR=/var/tmp/firefox-destdir -C /var/tmp/moz-build-dir install && rm -fr /var/tmp/moz-build-dir mv .mozconfig_tmp .mozconfig Where .mozconfig_profile_gen contains: export CFLAGS="-march=native -fno-semantic-interposition -ffunction-sections -fdata-sections" export CXXFLAGS="-march=native -fno-semantic-interposition -fprofile-generate -ffunction-sections -fdata-sections" and .mozconfig_profile_use contains: export CFLAGS="-march=native -fno-semantic-interposition -ffunction-sections -fdata-sections" export CXXFLAGS="-march=native -fno-semantic-interposition -flto=4 -fdevirtualize-at-ltrans -fprofile-use -fprofile-correction -ffunction-sections -fdata-sections"
Unfortunately your patch dosen't work: lto1: internal compiler error: Segmentation fault 0xa12f6f crash_signal ../../gcc/gcc/toplev.c:381 0x88b190 ipa_merge_profiles(cgraph_node*, cgraph_node*) ../../gcc/gcc/ipa-utils.c:637 0x603722 lto_cgraph_replace_node ../../gcc/gcc/lto/lto-symtab.c:124 0x604cf3 lto_symtab_merge_symbols_1 ../../gcc/gcc/lto/lto-symtab.c:619 0x604cf3 lto_symtab_merge_symbols() ../../gcc/gcc/lto/lto-symtab.c:647 0x5fa52e read_cgraph_and_symbols ../../gcc/gcc/lto/lto.c:3109 0x5fa52e lto_main() ../../gcc/gcc/lto/lto.c:3436 Please submit a full bug report, with preprocessed source if appropriate. Please include the complete backtrace with any bug report. See <http://gcc.gnu.org/bugs.html> for instruction
And if I try the obvious fix: 637 else if (e2 && e2->speculative) I hit the assert in ipa-inline.c later on: lto1: internal compiler error: in edge_badness, at ipa-inline.c:912 0x100bf73 edge_badness ../../gcc/gcc/ipa-inline.c:912 0x100f005 update_edge_key ../../gcc/gcc/ipa-inline.c:1038 0x100f005 inline_small_functions ../../gcc/gcc/ipa-inline.c:1608 0x10117ba ipa_inline ../../gcc/gcc/ipa-inline.c:2179 0x10117ba execute ../../gcc/gcc/ipa-inline.c:2551 Please submit a full bug report, with preprocessed source if appropriate.
Martin pointed out that the assertion gets also hit in a pure LTO Firefox build. See PR64686. When I comment out that assertion, Firefox LTO/PGO now builds fine with your patch and the trivial fix from comment 11.
Author: hubicka Date: Tue Jan 20 20:03:23 2015 New Revision: 219910 URL: https://gcc.gnu.org/viewcvs?rev=219910&root=gcc&view=rev Log: PR ipa/63576 * ipa-utils.c (ipa_merge_profiles): Merge speculative edges. Modified: trunk/gcc/ChangeLog trunk/gcc/ipa-utils.c
Fixed. Thanks.