Bug 53321 - [4.8 Regression] LTO bootstrap failed with bootstrap-profiled
Summary: [4.8 Regression] LTO bootstrap failed with bootstrap-profiled
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: middle-end (show other bugs)
Version: 4.8.0
: P3 normal
Target Milestone: 4.8.0
Assignee: Uroš Bizjak
URL: http://gcc.gnu.org/ml/gcc-patches/201...
Keywords:
Depends on:
Blocks: 53433
  Show dependency treegraph
 
Reported: 2012-05-11 11:18 UTC by H.J. Lu
Modified: 2012-08-02 18:06 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2012-05-25 00:00:00


Attachments
Preprocessed source (67.83 KB, application/x-gzip)
2012-05-25 20:03 UTC, Uroš Bizjak
Details
Patch that cures various "may be used uninitialized" errors during LTO bootstrap (613 bytes, patch)
2012-07-03 18:17 UTC, Uroš Bizjak
Details | Diff
A different patch (569 bytes, patch)
2012-07-06 20:22 UTC, H.J. Lu
Details | Diff
The updated patch (1.73 KB, patch)
2012-07-06 23:17 UTC, H.J. Lu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description H.J. Lu 2012-05-11 11:18:46 UTC
On Linux/x86-64, revision 187375 failed to profiledbootstrap:

../../src-trunk/gcc/gcov.c:2348:1: internal compiler error: vector VEC(inline_edge_summary_t,base) index domain error, in inline_edge_summary at ipa-inline.h:200
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[6]: *** [gcov.o] Error 1

when configured with

--prefix=/usr/local --enable-clocale=gnu --with-system-zlib --enable-sh
ared --with-demangler-in-ld --with-build-config=bootstrap-lto --with-fpmath=sse 
--enable-languages=c,c++,fortran,java,lto,objc

Revision 187371 is OK.
Comment 1 H.J. Lu 2012-05-11 11:20:37 UTC
It may be caused by revision 187375:

http://gcc.gnu.org/ml/gcc-cvs/2012-05/msg00371.html
Comment 2 Jan Hubicka 2012-05-11 15:48:32 UTC
I fixed identically looking ICE seen on Mozilla build yesterday. Hopefully the boostrap is fixed now, too.

Honza
Comment 3 H.J. Lu 2012-05-11 15:52:38 UTC
(In reply to comment #2)
> I fixed identically looking ICE seen on Mozilla build yesterday. Hopefully the
> boostrap is fixed now, too.
> 

It still fails with the same error as of revision 187408.
Does revision 187408 have your fix for Mozilla?
Comment 4 Jan Hubicka 2012-05-11 18:25:06 UTC
was comitted as revision 187382.  I am trying profiledbootstrap now.

Honza
Comment 5 Jan Hubicka 2012-05-12 11:03:01 UTC
I need to add disable-werror otherwise we fail on extra warnings, but with that my bootstrap works. Is it still failing for you?  The unreferenced nodes removal is very basic thing so when it breaks it should show up quite consistently.

Honza
Comment 6 H.J. Lu 2012-05-19 18:14:17 UTC
(In reply to comment #5)
> I need to add disable-werror otherwise we fail on extra warnings, but with that
> my bootstrap works. Is it still failing for you?  The unreferenced nodes
> removal is very basic thing so when it breaks it should show up quite
> consistently.

It still fails for me.  Did you configure GCC with --with-build-config=bootstrap-lto?
Comment 7 Uroš Bizjak 2012-05-25 20:03:51 UTC
Created attachment 27500 [details]
Preprocessed source

This preprocessed source from the bootstrap fails with -g -O2 -fprofile-generate:

~/gcc-build/gcc/cc1plus -O2 -fprofile-generate -quiet gcov.ii 
../../gcc-svn/trunk/gcc/gcov.c:2348:1: internal compiler error: vector VEC(inline_edge_summary_t,base) index domain error, in inline_edge_summary at ipa-inline.h:200
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
Comment 8 Uroš Bizjak 2012-05-25 20:05:57 UTC
Confirmed on x86_64-pc-linux-gnu with:

Target: x86_64-unknown-linux-gnu
gcc version 4.8.0 20120525 (experimental) [trunk revision 187887] (GCC)
Comment 9 Uroš Bizjak 2012-05-25 20:10:54 UTC
(gdb) bt
#0  internal_error (gmsgid=0x1214dd8 "vector %s %s domain error, in %s at %s:%u") at ../../gcc-svn/trunk/gcc/diagnostic.c:951
#1  0x0000000000ef31a9 in vec_assert_fail (op=0x11d3093 "index", struct_name=0xfa61d0 "VEC(inline_edge_summary_t,base)", file=<optimized out>, line=200, 
    function=0xfa6fd0 "inline_edge_summary") at ../../gcc-svn/trunk/gcc/vec.c:528
#2  0x000000000079114e in VEC_inline_edge_summary_t_base_index (ix_=<optimized out>, vec_=<optimized out>, file_=<optimized out>, line_=<optimized out>, function_=<optimized out>)
    at ../../gcc-svn/trunk/gcc/ipa-inline.h:145
#3  inline_edge_summary (edge=<optimized out>) at ../../gcc-svn/trunk/gcc/ipa-inline.h:199
#4  cgraph_propagate_frequency_1 (node=0x7ffff109b138, data=0x7fffffffdb20) at ../../gcc-svn/trunk/gcc/cgraph.c:1805
#5  0x0000000000794810 in cgraph_for_node_and_aliases (node=0x7ffff109b138, callback=0x790f10 <cgraph_propagate_frequency_1(cgraph_node*, void*)>, data=0x7fffffffdb20, 
    include_overwritable=true) at ../../gcc-svn/trunk/gcc/cgraph.c:1633
#6  0x0000000000794a92 in cgraph_propagate_frequency (node=0x7ffff109b138) at ../../gcc-svn/trunk/gcc/cgraph.c:1839
#7  0x0000000000930487 in symtab_remove_unreachable_nodes (before_inlining_p=<optimized out>, file=0x0) at ../../gcc-svn/trunk/gcc/ipa.c:455
#8  0x000000000079b6a0 in ipa_passes () at ../../gcc-svn/trunk/gcc/cgraphunit.c:1853
Comment 10 Uroš Bizjak 2012-05-26 10:09:35 UTC
Reduced testcase:

--cut here--
typedef long unsigned int size_t;

extern "C"
{
  extern void *memcpy (void *__dest, __const void *__src, size_t __n);
}

extern char *src, *sources;
extern int n_sources;

static void
find_source (const char *file_name)
{
  memcpy (src, sources, n_sources * sizeof (*sources));
}

extern const char *gcov_read_string (void);

static void read_graph_file (void)
{
  find_source (gcov_read_string ());
}

static void process_file (void)
{
  read_graph_file ();
}

int main ()
{
  process_file ();
}
--cut here--

~/gcc-build/gcc/cc1plus -O2 -fprofile-generate -quiet t.C
t.C:32:1: internal compiler error: vector VEC(inline_edge_summary_t,base) index domain error, in inline_edge_summary at ipa-inline.h:200
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
Comment 11 Steven Bosscher 2012-05-26 19:39:28 UTC
At the point of the ICE:
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/stevenb/devel/build-test2/gcc/cc1plus -quiet -O2 -fprofile-generate t.cc

Breakpoint 2, internal_error (gmsgid=0x18d10b8 "vector %s %s domain error, in %s at %s:%u") at ../../trunk/gcc/diagnostic.c:955
955       va_start (ap, gmsgid);
(gdb) up 4
#4  0x00000000009bc54c in cgraph_propagate_frequency_1 (node=0x7ffff70d5888, data=0x7fffffffde40) at ../../trunk/gcc/cgraph.c:1805
1805              if (inline_edge_summary (edge)->loop_depth)
(gdb) p debug_cgraph()
callgraph:

__gcov_indirect_call_profiler/25 (<built-in>) @0x7ffff72603a8
  Type: function
  Visibility: external public artificial
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: main/3 (1.00 per call)
  Calls:
__gcov_ior_profiler/24 (<built-in>) @0x7ffff7260138
  Type: function
  Visibility: external public artificial
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: _ZL11find_sourcePKc.isra.0/9 (1.00 per call)
  Calls:
__gcov_average_profiler/23 (<built-in>) @0x7ffff7260000
  Type: function
  Visibility: external public artificial
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: _ZL11find_sourcePKc.isra.0/9 (1.00 per call)
  Calls:
__gcov_one_value_profiler/22 (<built-in>) @0x7ffff70d59c0
  Type: function
  Visibility: external public artificial
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: _ZL11find_sourcePKc.isra.0/9 (1.00 per call)
  Calls:
_ZL11find_sourcePKc.isra.0/9 (void _ZL11find_sourcePKc.isra.0()) @0x7ffff70d5ea0
  Type: function
  Visibility: artificial
  References: long int __gcov0._ZL11find_sourcePKc.isra.0 [1]/12 (read)long int __gcov0._ZL11find_sourcePKc.isra.0 [1]/12 (write)int n_sources/5 (read)char* sources/6 (read)char* src/7 (read)long int __gcov3._ZL11find_sourcePKc.isra.0 [3]/13 (addr)long int __gcov6._ZL11find_sourcePKc.isra.0 [2]/14 (addr)long int __gcov7._ZL11find_sourcePKc.isra.0 [1]/15 (addr)
  Referring:
  Availability: local
  Function flags: analyzed body local finalized
  Called by: _ZL15read_graph_filev/1 (1.00 per call)
  Calls: void* memcpy(void*, const void*, size_t)/8 (1.00 per call) __gcov_ior_profiler/24 (1.00 per call) __gcov_average_profiler/23 (1.00 per call) __gcov_one_value_profiler/22 (1.00 per call)
void* memcpy(void*, const void*, size_t)/8 (void* memcpy(void*, const void*, size_t)) @0x7ffff70d5d68
  Type: function
  Visibility: external public visibility_specified
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: _ZL11find_sourcePKc.isra.0/9 (1.00 per call)
  Calls:
const char* gcov_read_string()/4 (const char* gcov_read_string()) @0x7ffff70d5c30
  Type: function
  Visibility: external public
  References:
  Referring:
  Availability: not_available
  Function flags:
  Called by: _ZL15read_graph_filev/1 (1.00 per call) (can throw external)
  Calls:
main/3 (int main()) @0x7ffff70d5af8
  Type: function
  Visibility: public
  Address is taken.
  References: long int* __gcov_indirect_call_counters/11 (read)void* __gcov_indirect_call_callee/10 (read)main/3 (addr)void* __gcov_indirect_call_callee/10 (write)long int __gcov0.main [2]/16 (read)long int __gcov0.main [2]/16 (write)long int __gcov0.main [2]/16 (read)long int __gcov0.main [2]/16 (write)
  Referring: main/3 (addr)
  Availability: available
  Function flags: analyzed body finalized only_called_at_startup
  Called by:
  Calls: _ZL15read_graph_filev/1 (1.00 per call) (can throw external) __gcov_indirect_call_profiler/25 (1.00 per call)
_ZL15read_graph_filev/1 (void read_graph_file()) @0x7ffff70d5888
  Type: function
  Visibility: prevailing_def_ironly
  References: long int __gcov0._ZL15read_graph_filev [3]/17 (read)long int __gcov0._ZL15read_graph_filev [3]/17 (write)long int __gcov0._ZL15read_graph_filev [3]/17 (read)long int __gcov0._ZL15read_graph_filev [3]/17 (write)long int __gcov0._ZL15read_graph_filev [3]/17 (read)long int __gcov0._ZL15read_graph_filev [3]/17 (write)
  Referring:
  Availability: local
  Function flags: analyzed body local finalized
  Called by: main/3 (1.00 per call) (can throw external)
  Calls: _ZL11find_sourcePKc.isra.0/9 (1.00 per call) const char* gcov_read_string()/4 (1.00 per call) (can throw external)
$41 = void

Note the isra function. The ICE goes away with -fno-ipa-sra, and can be triggered with "-O1 -fipa-sra -fprofile-generate".
Comment 12 Steven Bosscher 2012-05-26 19:42:26 UTC
Note, btw, that verify_cgraph() doesn't catch this. Honza, you loved checkers so much a few years ago -- maybe this checker (also yours??) should be enhanced :-)
Comment 13 Jan Hubicka 2012-05-28 08:26:48 UTC
Hehe, i still love them ;)
But inline summaries are separate of cgraph, so it is not cgraph checker's work to verify it ;))

I think the problem is in new profile updating logic in unreachable node removal. I will look into it.
Comment 14 H.J. Lu 2012-06-23 14:51:07 UTC
(In reply to comment #13)
> Hehe, i still love them ;)
> But inline summaries are separate of cgraph, so it is not cgraph checker's work
> to verify it ;))
> 
> I think the problem is in new profile updating logic in unreachable node
> removal. I will look into it.

Any updates?
Comment 15 Uroš Bizjak 2012-07-02 09:48:57 UTC
Jan, can you please fix this bug, profiledbootstrap LTO autotesters are broken for more than a month due to this problem. The reduced testcase is in Comment #10.
Comment 16 Uroš Bizjak 2012-07-03 15:57:34 UTC
Following patch fixes testcase failure for me, and also enables lto-bootstrap:

--cut here--
Index: ipa.c
===================================================================
--- ipa.c       (revision 189217)
+++ ipa.c       (working copy)
@@ -449,11 +449,6 @@ symtab_remove_unreachable_nodes (bool before_inlin
   verify_symtab ();
 #endif
 
-  /* If we removed something, perhaps profile could be improved.  */
-  if (changed && optimize && inline_edge_summary_vec)
-    FOR_EACH_DEFINED_FUNCTION (node)
-      cgraph_propagate_frequency (node);
-
   return changed;
 }
 
--cut here--
Comment 17 Uroš Bizjak 2012-07-03 18:17:09 UTC
Created attachment 27736 [details]
Patch that cures various "may be used uninitialized" errors during LTO bootstrap

This patch is needed to cure various "may be used uninitialized" errors during LTO bootstrap. This patch brings the bootstrap up to PR 53433.

Adding the patch from PR 53433 successfully finish LTO profiledbootstrap.
Comment 18 uros 2012-07-04 13:49:24 UTC
Author: uros
Date: Wed Jul  4 13:49:19 2012
New Revision: 189261

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=189261
Log:
	PR middle-end/53321
	* ipa.c (symtab_remove_unreachable_nodes): Partially revert r187375
	to not call cgraph_propagate_frequency if something was changed.

testsuite/ChangLog:

	PR middle-end/53321
	* g++.dg/torture/pr53321.C: New test.


Added:
    trunk/gcc/testsuite/g++.dg/torture/pr53321.C
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/ipa.c
    trunk/gcc/testsuite/ChangeLog
Comment 19 Uroš Bizjak 2012-07-04 13:56:15 UTC
Fixed.
Comment 20 Uroš Bizjak 2012-07-04 13:57:53 UTC
.
Comment 21 H.J. Lu 2012-07-06 18:29:50 UTC
The fix caused PR 53865.  The even smaller testcase:

[hjl@gnu-mic-2 pr53321]$ cat x.i
extern char *src;
extern const char *sources;
extern unsigned int n_sources;

static void
find_source (const char *file_name)
{
 __builtin_memcpy (src, sources, n_sources);
}

int main ()
{
  find_source (sources);
}
[hjl@gnu-mic-2 pr53321]$ /export/build/gnu/gcc/build-x86_64-linux/gcc/xgcc -B/export/build/gnu/gcc/build-x86_64-linux/gcc/ -O2 -fprofile-generate -S x.i
x.i:14:1: internal compiler error: vector VEC(inline_edge_summary_t,base) index domain error, in inline_edge_summary at ipa-inline.h:200
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
[hjl@gnu-mic-2 pr53321]$
Comment 22 H.J. Lu 2012-07-06 18:46:09 UTC
After inlining, IPA calls tree_profiling which rebuilds edges:

#2  0x0000000000683ccd in rebuild_cgraph_edges ()
    at /export/gnu/import/git/gcc/gcc/cgraphbuild.c:433
#3  0x0000000000c02de6 in tree_profiling ()
    at /export/gnu/import/git/gcc/gcc/tree-profile.c:564
#4  0x0000000000a1be4f in execute_one_pass (pass=0x1908520)
    at /export/gnu/import/git/gcc/gcc/passes.c:2165
#5  0x0000000000a1cb0c in execute_ipa_pass_list (pass=0x1908520)
    at /export/gnu/import/git/gcc/gcc/passes.c:2532
#6  0x000000000068b874 in ipa_passes ()
    at /export/gnu/import/git/gcc/gcc/cgraphunit.c:1844

  basic_block bb; 
  struct cgraph_node *node = cgraph_get_node (current_function_decl);
  gimple_stmt_iterator gsi;

  cgraph_node_remove_callees (node);
  ipa_remove_all_references (&node->symbol.ref_list);

  node->count = ENTRY_BLOCK_PTR->count;

and creates new edges.  After that, all things went downhill

#0  internal_error (
    gmsgid=0x158c110 "vector %s %s domain error, in %s at %s:%u")
    at /export/gnu/import/git/gcc/gcc/diagnostic.c:955
#1  0x000000000124f707 in vec_assert_fail (op=0x12fc800 "index", 
    struct_name=0x12fc7e0 "VEC(inline_edge_summary_t,base)", 
    file=0x12fc808 "/export/gnu/import/git/gcc/gcc/ipa-inline.h", line=200, 
    function=0x12fd380 "inline_edge_summary")
    at /export/gnu/import/git/gcc/gcc/vec.c:527
#2  0x000000000067c685 in VEC_inline_edge_summary_t_base_index (
    vec_=0x1aea360, ix_=6, 
    file_=0x12fc808 "/export/gnu/import/git/gcc/gcc/ipa-inline.h", line_=200, 
    function_=0x12fd380 "inline_edge_summary")
    at /export/gnu/import/git/gcc/gcc/ipa-inline.h:145
#3  0x000000000067c6cd in inline_edge_summary (edge=0x7ffff1ab67b8)
    at /export/gnu/import/git/gcc/gcc/ipa-inline.h:199

since inline_edge_summary_vec was never updated.
Comment 23 H.J. Lu 2012-07-06 20:22:55 UTC
Created attachment 27753 [details]
A different patch

This patch calls inline_free_summary to clear
stale inline summary.  It fixed PR 53865.  I am
testing LTO profile bootstrap now.
Comment 24 H.J. Lu 2012-07-06 23:17:18 UTC
Created attachment 27755 [details]
The updated patch
Comment 25 hjl@gcc.gnu.org 2012-08-02 16:58:41 UTC
Author: hjl
Date: Thu Aug  2 16:58:33 2012
New Revision: 190090

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=190090
Log:
Add free inline summary pass after pass_early_local_passes 

	PR middle-end/53321
	PR middle-end/53865
	* ipa-inline-analysis.c (inline_free_summary): Return if
	inline_edge_summary_vec is NULL.

	* ipa-split.c (execute_split_functions): Check if a function
	is inlinable only if inline_edge_summary_vec != NULL.

	* ipa.c (symtab_remove_unreachable_nodes): Restore
	cgraph_propagate_frequency call when something was changed.
	(free_inline_summary): New function.
	(pass_ipa_free_inline_summary): New pass.

	* passes.c (init_optimization_passes): Add
	pass_ipa_free_inline_summary before pass_ipa_tree_profile.

	* timevar.def (TV_IPA_FREE_INLINE_SUMMARY): New.

	* tree-pass.h (pass_ipa_free_inline_summary): New.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/ipa-inline-analysis.c
    trunk/gcc/ipa-split.c
    trunk/gcc/ipa.c
    trunk/gcc/passes.c
    trunk/gcc/timevar.def
    trunk/gcc/tree-pass.h
Comment 26 H.J. Lu 2012-08-02 18:06:00 UTC
Fixed.