This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH] Clean up early inliner
On Mon, 12 Apr 2010, Jan Hubicka wrote:
> >
> > Does it? Not for me (neither with nor without the patch).
> > It of course sorrry()s out.
>
> Yes, it sorrys out. I was just concerned about memory time before it does
> so (as sorrying happens only after all decisios are fixed and function is
> being output so if we recursively inline too deep, we would consume too
> much of RAM)
> >
> > Anyway, I'll shortcut this as well to save some compile-time.
> >
> > Note that for the testcase above it doesn't make sense to
> > inline bomb() into t at all, still we do it. To improve
> > the situation here we'd indeed need to compute cgraph SCCs
> > and simply use the heuristic to not increase SCC size by
> > inlining (thus, not inline A into B if they are in different
> > non-singleton SCCs).
>
> Well, this is questionable for recursive inlining in general.
> Our general strategy to handle these is to make recursive functions inlined as
> small functions and handled by same way.
>
> In my experience we have three cases we are shooting for:
>
> 1) functions that are written in recursive way, but the recursion really
> is of fixed depth of 1 or 2 and inlining eliminate them. We have plenty
> of such examples in GCC.
> 2) normal recursive functions making many invocations of self with small
> depth (such as recursive walking of RTL).
>
> In this case inlining into caller tends to reduce amount of calls
> noticeably and helps. Recursive inlining can hurt as the call overhead
> of function gets higher (i.e. we have more registers to save in prologue)
> and consequently the leaves of recursion that are in majority can be
> more expensive. Sometimes it helps when the optimizations hits or we
> suceed reducing number of leaves a lot.
>
> Since in many cases recursion depth is just 1 or 2, we are relatively often
> on the losing side.
>
> 3) functions that do basically iteration via recursion and tends to do very
> deep recursion chain. In this case recursive inlining helps because it
> reduces return stack mispredict to some constant fraction.
>
> For 1/2 inlining recursive functions into their callers make sense, for 3 it
> does not. But this is bit difficult to decide statically. I made no
> experiments with tunning this too much. Not even making profile driven inliner
> too much smarter on this except by convincing recursive inliner to give up
> on functions with small average recursion depth.
Yeah, but then recursive inlining is basically an interprocedural
tracer - and we shouldn't need to fully inline recursive calls but
only the interesting parts. Maybe something that can be considered
when implementing partial inlining?
Anyway, the following is what I now successfully bootstrapped and
tested on x86_64-unknown-linux-gnu. Ok for trunk?
Thanks,
Richard.
2010-04-12 Richard Guenther <rguenther@suse.de>
* ipa.c (cgraph_postorder): Adjust postorder to guarantee
single-iteration always-inline inlining.
* ipa-inline.c (cgraph_mark_inline): Do not return anything.
(cgraph_decide_inlining): Do not handle always-inline
specially.
(try_inline): Remove always-inline cycle detection special case.
Do not recurse on always-inlines.
(cgraph_early_inlining): Do not iterate if not optimizing.
(cgraph_gate_early_inlining): remove.
(pass_early_inline): Run unconditionally.
(gate_cgraph_decide_inlining): New function.
(pass_ipa_inline): Use it. Do not run the IPA inliner if
not inlining or optimizing.
(cgraph_decide_inlining_of_small_functions): Also consider
always-inline functions.
(cgraph_default_inline_p): Return true for nodes which should
disregard inline limits.
(estimate_function_body_sizes): Assume zero size and time for
nodes which are marked as disregarding inline limits.
(cgraph_decide_recursive_inlining): Do not perform recursive
inlining on always-inline nodes.
* gcc.dg/torture/inline-2.c: New testcase.
Index: gcc/ipa-inline.c
===================================================================
*** gcc/ipa-inline.c.orig 2010-04-12 11:48:18.000000000 +0200
--- gcc/ipa-inline.c 2010-04-12 13:37:44.000000000 +0200
*************** enum inlining_mode {
*** 160,168 ****
INLINE_SIZE,
INLINE_ALL
};
static bool
! cgraph_decide_inlining_incrementally (struct cgraph_node *, enum inlining_mode,
! int);
/* Statistics we collect about inlining algorithm. */
--- 160,169 ----
INLINE_SIZE,
INLINE_ALL
};
+
static bool
! cgraph_decide_inlining_incrementally (struct cgraph_node *, enum inlining_mode);
! static void cgraph_flatten (struct cgraph_node *node);
/* Statistics we collect about inlining algorithm. */
*************** cgraph_mark_inline_edge (struct cgraph_e
*** 346,356 ****
return false;
}
! /* Mark all calls of EDGE->CALLEE inlined into EDGE->CALLER.
! Return following unredirected edge in the list of callers
! of EDGE->CALLEE */
! static struct cgraph_edge *
cgraph_mark_inline (struct cgraph_edge *edge)
{
struct cgraph_node *to = edge->caller;
--- 347,355 ----
return false;
}
! /* Mark all calls of EDGE->CALLEE inlined into EDGE->CALLER. */
! static void
cgraph_mark_inline (struct cgraph_edge *edge)
{
struct cgraph_node *to = edge->caller;
*************** cgraph_mark_inline (struct cgraph_edge *
*** 370,377 ****
edge = next;
}
}
-
- return edge;
}
/* Estimate the growth caused by inlining NODE into all callees. */
--- 369,374 ----
*************** cgraph_default_inline_p (struct cgraph_n
*** 479,484 ****
--- 476,484 ----
{
tree decl = n->decl;
+ if (n->local.disregard_inline_limits)
+ return true;
+
if (!flag_inline_small_functions && !DECL_DECLARED_INLINE_P (decl))
{
if (reason)
*************** cgraph_decide_recursive_inlining (struct
*** 727,732 ****
--- 727,738 ----
int depth = 0;
int n = 0;
+ /* It does not make sense to recursively inline always-inline functions
+ as we are going to sorry() on the remaining calls anyway. */
+ if (node->local.disregard_inline_limits
+ && lookup_attribute ("always_inline", DECL_ATTRIBUTES (node->decl)))
+ return false;
+
if (optimize_function_for_size_p (DECL_STRUCT_FUNCTION (node->decl))
|| (!flag_inline_functions && !DECL_DECLARED_INLINE_P (node->decl)))
return false;
*************** cgraph_decide_inlining_of_small_function
*** 916,923 ****
for (node = cgraph_nodes; node; node = node->next)
{
! if (!node->local.inlinable || !node->callers
! || node->local.disregard_inline_limits)
continue;
if (dump_file)
fprintf (dump_file, "Considering inline candidate %s.\n", cgraph_node_name (node));
--- 922,928 ----
for (node = cgraph_nodes; node; node = node->next)
{
! if (!node->local.inlinable || !node->callers)
continue;
if (dump_file)
fprintf (dump_file, "Considering inline candidate %s.\n", cgraph_node_name (node));
*************** cgraph_decide_inlining_of_small_function
*** 1128,1133 ****
--- 1133,1218 ----
BITMAP_FREE (updated_nodes);
}
+ /* Flatten NODE from the IPA inliner. */
+
+ static void
+ cgraph_flatten (struct cgraph_node *node)
+ {
+ struct cgraph_edge *e;
+
+ /* We shouldn't be called recursively when we are being processed. */
+ gcc_assert (node->aux == NULL);
+
+ node->aux = (void *)(size_t) INLINE_ALL;
+
+ for (e = node->callees; e; e = e->next_callee)
+ {
+ struct cgraph_node *orig_callee;
+
+ if (e->call_stmt_cannot_inline_p)
+ continue;
+
+ if (!e->callee->analyzed)
+ {
+ if (dump_file)
+ fprintf (dump_file,
+ "Not inlining: Function body not available.\n");
+ continue;
+ }
+
+ /* We've hit cycle? It is time to give up. */
+ if (e->callee->aux)
+ {
+ if (dump_file)
+ fprintf (dump_file,
+ "Not inlining %s into %s to avoid cycle.\n",
+ cgraph_node_name (e->callee),
+ cgraph_node_name (e->caller));
+ e->inline_failed = CIF_RECURSIVE_INLINING;
+ continue;
+ }
+
+ /* When the edge is already inlined, we just need to recurse into
+ it in order to fully flatten the leaves. */
+ if (!e->inline_failed)
+ {
+ cgraph_flatten (e->callee);
+ continue;
+ }
+
+ if (cgraph_recursive_inlining_p (node, e->callee, &e->inline_failed))
+ {
+ if (dump_file)
+ fprintf (dump_file, "Not inlining: recursive call.\n");
+ continue;
+ }
+
+ if (!tree_can_inline_p (e))
+ {
+ if (dump_file)
+ fprintf (dump_file, "Not inlining: %s",
+ cgraph_inline_failed_string (e->inline_failed));
+ continue;
+ }
+
+ /* Inline the edge and flatten the inline clone. Avoid
+ recursing through the original node if the node was cloned. */
+ if (dump_file)
+ fprintf (dump_file, " Inlining %s into %s.\n",
+ cgraph_node_name (e->callee),
+ cgraph_node_name (e->caller));
+ orig_callee = e->callee;
+ cgraph_mark_inline_edge (e, true, NULL);
+ if (e->callee != orig_callee)
+ orig_callee->aux = (void *)(size_t) INLINE_ALL;
+ cgraph_flatten (e->callee);
+ if (e->callee != orig_callee)
+ orig_callee->aux = NULL;
+ }
+
+ node->aux = NULL;
+ }
+
/* Decide on the inlining. We do so in the topological order to avoid
expenses on updating data structures. */
*************** cgraph_decide_inlining (void)
*** 1140,1146 ****
XCNEWVEC (struct cgraph_node *, cgraph_n_nodes);
int old_size = 0;
int i;
- bool redo_always_inline = true;
int initial_size = 0;
cgraph_remove_function_insertion_hook (function_insertion_hook_holder);
--- 1225,1230 ----
*************** cgraph_decide_inlining (void)
*** 1178,1242 ****
node->aux = 0;
if (dump_file)
! fprintf (dump_file, "\nInlining always_inline functions:\n");
! /* In the first pass mark all always_inline edges. Do this with a priority
! so none of our later choices will make this impossible. */
! while (redo_always_inline)
! {
! redo_always_inline = false;
! for (i = nnodes - 1; i >= 0; i--)
{
- struct cgraph_edge *e, *next;
-
- node = order[i];
-
- /* Handle nodes to be flattened, but don't update overall unit
- size. */
- if (lookup_attribute ("flatten",
- DECL_ATTRIBUTES (node->decl)) != NULL)
- {
- if (dump_file)
- fprintf (dump_file,
- "Flattening %s\n", cgraph_node_name (node));
- cgraph_decide_inlining_incrementally (node, INLINE_ALL, 0);
- }
-
- if (!node->local.disregard_inline_limits)
- continue;
if (dump_file)
fprintf (dump_file,
! "\nConsidering %s size:%i (always inline)\n",
! cgraph_node_name (node), node->global.size);
! old_size = overall_size;
! for (e = node->callers; e; e = next)
! {
! next = e->next_caller;
! if (!e->inline_failed || e->call_stmt_cannot_inline_p)
! continue;
! if (cgraph_recursive_inlining_p (e->caller, e->callee,
! &e->inline_failed))
! continue;
! if (!tree_can_inline_p (e))
! continue;
! if (cgraph_mark_inline_edge (e, true, NULL))
! redo_always_inline = true;
! if (dump_file)
! fprintf (dump_file,
! " Inlined into %s which now has size %i.\n",
! cgraph_node_name (e->caller),
! e->caller->global.size);
! }
! /* Inlining self recursive function might introduce new calls to
! themselves we didn't see in the loop above. Fill in the proper
! reason why inline failed. */
! for (e = node->callers; e; e = e->next_caller)
! if (e->inline_failed)
! e->inline_failed = CIF_RECURSIVE_INLINING;
! if (dump_file)
! fprintf (dump_file,
! " Inlined for a net change of %+i size.\n",
! overall_size - old_size);
}
}
--- 1262,1290 ----
node->aux = 0;
if (dump_file)
! fprintf (dump_file, "\nFlattening functions:\n");
! /* In the first pass handle functions to be flattened. Do this with
! a priority so none of our later choices will make this impossible. */
! for (i = nnodes - 1; i >= 0; i--)
! {
! node = order[i];
!
! /* Handle nodes to be flattened, but don't update overall unit
! size. Calling the incremental inliner here is lame,
! a simple worklist should be enough. What should be left
! here from the early inliner (if it runs) is cyclic cases.
! Ideally when processing callees we stop inlining at the
! entry of cycles, possibly cloning that entry point and
! try to flatten itself turning it into a self-recursive
! function. */
! if (lookup_attribute ("flatten",
! DECL_ATTRIBUTES (node->decl)) != NULL)
{
if (dump_file)
fprintf (dump_file,
! "Flattening %s\n", cgraph_node_name (node));
! cgraph_flatten (node);
}
}
*************** cgraph_decide_inlining (void)
*** 1313,1398 ****
return 0;
}
- /* Try to inline edge E from incremental inliner. MODE specifies mode
- of inliner.
-
- We are detecting cycles by storing mode of inliner into cgraph_node last
- time we visited it in the recursion. In general when mode is set, we have
- recursive inlining, but as an special case, we want to try harder inline
- ALWAYS_INLINE functions: consider callgraph a->b->c->b, with a being
- flatten, b being always inline. Flattening 'a' will collapse
- a->b->c before hitting cycle. To accommodate always inline, we however
- need to inline a->b->c->b.
-
- So after hitting cycle first time, we switch into ALWAYS_INLINE mode and
- stop inlining only after hitting ALWAYS_INLINE in ALWAY_INLINE mode. */
- static bool
- try_inline (struct cgraph_edge *e, enum inlining_mode mode, int depth)
- {
- struct cgraph_node *callee = e->callee;
- enum inlining_mode callee_mode = (enum inlining_mode) (size_t) callee->aux;
- bool always_inline = e->callee->local.disregard_inline_limits;
- bool inlined = false;
-
- /* We've hit cycle? */
- if (callee_mode)
- {
- /* It is first time we see it and we are not in ALWAY_INLINE only
- mode yet. and the function in question is always_inline. */
- if (always_inline && mode != INLINE_ALWAYS_INLINE)
- {
- if (dump_file)
- {
- indent_to (dump_file, depth);
- fprintf (dump_file,
- "Hit cycle in %s, switching to always inline only.\n",
- cgraph_node_name (callee));
- }
- mode = INLINE_ALWAYS_INLINE;
- }
- /* Otherwise it is time to give up. */
- else
- {
- if (dump_file)
- {
- indent_to (dump_file, depth);
- fprintf (dump_file,
- "Not inlining %s into %s to avoid cycle.\n",
- cgraph_node_name (callee),
- cgraph_node_name (e->caller));
- }
- e->inline_failed = (e->callee->local.disregard_inline_limits
- ? CIF_RECURSIVE_INLINING : CIF_UNSPECIFIED);
- return false;
- }
- }
-
- callee->aux = (void *)(size_t) mode;
- if (dump_file)
- {
- indent_to (dump_file, depth);
- fprintf (dump_file, " Inlining %s into %s.\n",
- cgraph_node_name (e->callee),
- cgraph_node_name (e->caller));
- }
- if (e->inline_failed)
- {
- cgraph_mark_inline (e);
-
- /* In order to fully inline always_inline functions, we need to
- recurse here, since the inlined functions might not be processed by
- incremental inlining at all yet.
-
- Also flattening needs to be done recursively. */
-
- if (mode == INLINE_ALL || always_inline)
- cgraph_decide_inlining_incrementally (e->callee, mode, depth + 1);
- inlined = true;
- }
- callee->aux = (void *)(size_t) callee_mode;
- return inlined;
- }
-
/* Return true when N is leaf function. Accept cheap (pure&const) builtins
in leaf functions. */
static bool
--- 1361,1366 ----
*************** leaf_node_p (struct cgraph_node *n)
*** 1408,1445 ****
}
/* Decide on the inlining. We do so in the topological order to avoid
! expenses on updating data structures.
! DEPTH is depth of recursion, used only for debug output. */
static bool
cgraph_decide_inlining_incrementally (struct cgraph_node *node,
! enum inlining_mode mode,
! int depth)
{
struct cgraph_edge *e;
bool inlined = false;
cgraph_inline_failed_t failed_reason;
- enum inlining_mode old_mode;
#ifdef ENABLE_CHECKING
verify_cgraph_node (node);
#endif
- old_mode = (enum inlining_mode) (size_t)node->aux;
-
if (mode != INLINE_ALWAYS_INLINE && mode != INLINE_SIZE_NORECURSIVE
&& lookup_attribute ("flatten", DECL_ATTRIBUTES (node->decl)) != NULL)
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file, "Flattening %s\n", cgraph_node_name (node));
! }
mode = INLINE_ALL;
}
- node->aux = (void *)(size_t) mode;
-
/* First of all look for always inline functions. */
if (mode != INLINE_SIZE_NORECURSIVE)
for (e = node->callees; e; e = e->next_callee)
--- 1376,1404 ----
}
/* Decide on the inlining. We do so in the topological order to avoid
! expenses on updating data structures. */
static bool
cgraph_decide_inlining_incrementally (struct cgraph_node *node,
! enum inlining_mode mode)
{
struct cgraph_edge *e;
bool inlined = false;
cgraph_inline_failed_t failed_reason;
#ifdef ENABLE_CHECKING
verify_cgraph_node (node);
#endif
if (mode != INLINE_ALWAYS_INLINE && mode != INLINE_SIZE_NORECURSIVE
&& lookup_attribute ("flatten", DECL_ATTRIBUTES (node->decl)) != NULL)
{
if (dump_file)
! fprintf (dump_file, "Incrementally flattening %s\n",
! cgraph_node_name (node));
mode = INLINE_ALL;
}
/* First of all look for always inline functions. */
if (mode != INLINE_SIZE_NORECURSIVE)
for (e = node->callees; e; e = e->next_callee)
*************** cgraph_decide_inlining_incrementally (st
*** 1449,1509 ****
continue;
if (e->call_stmt_cannot_inline_p)
continue;
- /* When the edge is already inlined, we just need to recurse into
- it in order to fully flatten the leaves. */
- if (!e->inline_failed && mode == INLINE_ALL)
- {
- inlined |= try_inline (e, mode, depth);
- continue;
- }
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Considering to always inline inline candidate %s.\n",
! cgraph_node_name (e->callee));
! }
if (cgraph_recursive_inlining_p (node, e->callee, &e->inline_failed))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file, "Not inlining: recursive call.\n");
! }
continue;
}
if (!tree_can_inline_p (e))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: %s",
! cgraph_inline_failed_string (e->inline_failed));
! }
continue;
}
if (gimple_in_ssa_p (DECL_STRUCT_FUNCTION (node->decl))
!= gimple_in_ssa_p (DECL_STRUCT_FUNCTION (e->callee->decl)))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file, "Not inlining: SSA form does not match.\n");
! }
continue;
}
if (!e->callee->analyzed)
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: Function body no longer available.\n");
! }
continue;
}
! inlined |= try_inline (e, mode, depth);
}
/* Now do the automatic inlining. */
--- 1408,1452 ----
continue;
if (e->call_stmt_cannot_inline_p)
continue;
if (dump_file)
! fprintf (dump_file,
! "Considering to always inline inline candidate %s.\n",
! cgraph_node_name (e->callee));
if (cgraph_recursive_inlining_p (node, e->callee, &e->inline_failed))
{
if (dump_file)
! fprintf (dump_file, "Not inlining: recursive call.\n");
continue;
}
if (!tree_can_inline_p (e))
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: %s",
! cgraph_inline_failed_string (e->inline_failed));
continue;
}
if (gimple_in_ssa_p (DECL_STRUCT_FUNCTION (node->decl))
!= gimple_in_ssa_p (DECL_STRUCT_FUNCTION (e->callee->decl)))
{
if (dump_file)
! fprintf (dump_file, "Not inlining: SSA form does not match.\n");
continue;
}
if (!e->callee->analyzed)
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: Function body no longer available.\n");
continue;
}
!
! if (dump_file)
! fprintf (dump_file, " Inlining %s into %s.\n",
! cgraph_node_name (e->callee),
! cgraph_node_name (e->caller));
! cgraph_mark_inline (e);
! inlined = true;
}
/* Now do the automatic inlining. */
*************** cgraph_decide_inlining_incrementally (st
*** 1530,1550 ****
if (cgraph_recursive_inlining_p (node, e->callee, &e->inline_failed))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file, "Not inlining: recursive call.\n");
! }
continue;
}
if (gimple_in_ssa_p (DECL_STRUCT_FUNCTION (node->decl))
!= gimple_in_ssa_p (DECL_STRUCT_FUNCTION (e->callee->decl)))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: SSA form does not match.\n");
! }
continue;
}
--- 1473,1487 ----
if (cgraph_recursive_inlining_p (node, e->callee, &e->inline_failed))
{
if (dump_file)
! fprintf (dump_file, "Not inlining: recursive call.\n");
continue;
}
if (gimple_in_ssa_p (DECL_STRUCT_FUNCTION (node->decl))
!= gimple_in_ssa_p (DECL_STRUCT_FUNCTION (e->callee->decl)))
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: SSA form does not match.\n");
continue;
}
*************** cgraph_decide_inlining_incrementally (st
*** 1563,1576 ****
&& cgraph_estimate_growth (e->callee) > allowed_growth)
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: code size would grow by %i.\n",
! cgraph_estimate_size_after_inlining (1, e->caller,
! e->callee)
! - e->caller->global.size);
! }
continue;
}
if (!cgraph_check_inline_limits (node, e->callee, &e->inline_failed,
--- 1500,1510 ----
&& cgraph_estimate_growth (e->callee) > allowed_growth)
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: code size would grow by %i.\n",
! cgraph_estimate_size_after_inlining (1, e->caller,
! e->callee)
! - e->caller->global.size);
continue;
}
if (!cgraph_check_inline_limits (node, e->callee, &e->inline_failed,
*************** cgraph_decide_inlining_incrementally (st
*** 1578,1617 ****
|| e->call_stmt_cannot_inline_p)
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file, "Not inlining: %s.\n",
! cgraph_inline_failed_string (e->inline_failed));
! }
continue;
}
if (!e->callee->analyzed)
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: Function body no longer available.\n");
! }
continue;
}
if (!tree_can_inline_p (e))
{
if (dump_file)
! {
! indent_to (dump_file, depth);
! fprintf (dump_file,
! "Not inlining: %s.",
! cgraph_inline_failed_string (e->inline_failed));
! }
continue;
}
if (cgraph_default_inline_p (e->callee, &failed_reason))
! inlined |= try_inline (e, mode, depth);
}
BITMAP_FREE (visited);
}
- node->aux = (void *)(size_t) old_mode;
return inlined;
}
--- 1512,1548 ----
|| e->call_stmt_cannot_inline_p)
{
if (dump_file)
! fprintf (dump_file, "Not inlining: %s.\n",
! cgraph_inline_failed_string (e->inline_failed));
continue;
}
if (!e->callee->analyzed)
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: Function body no longer available.\n");
continue;
}
if (!tree_can_inline_p (e))
{
if (dump_file)
! fprintf (dump_file,
! "Not inlining: %s.",
! cgraph_inline_failed_string (e->inline_failed));
continue;
}
if (cgraph_default_inline_p (e->callee, &failed_reason))
! {
! if (dump_file)
! fprintf (dump_file, " Inlining %s into %s.\n",
! cgraph_node_name (e->callee),
! cgraph_node_name (e->caller));
! cgraph_mark_inline (e);
! inlined = true;
! }
}
BITMAP_FREE (visited);
}
return inlined;
}
*************** cgraph_early_inlining (void)
*** 1633,1659 ****
if (sorrycount || errorcount)
return 0;
! while (iterations < PARAM_VALUE (PARAM_EARLY_INLINER_MAX_ITERATIONS)
! && cgraph_decide_inlining_incrementally (node,
! iterations
! ? INLINE_SIZE_NORECURSIVE : INLINE_SIZE, 0))
! {
timevar_push (TV_INTEGRATION);
todo |= optimize_inline_calls (current_function_decl);
- iterations++;
timevar_pop (TV_INTEGRATION);
}
! if (dump_file)
! fprintf (dump_file, "Iterations: %i\n", iterations);
cfun->always_inline_functions_inlined = true;
- return todo;
- }
! /* When inlining shall be performed. */
! static bool
! cgraph_gate_early_inlining (void)
! {
! return flag_early_inlining;
}
struct gimple_opt_pass pass_early_inline =
--- 1564,1603 ----
if (sorrycount || errorcount)
return 0;
!
! if (!optimize
! || flag_no_inline
! || !flag_early_inlining)
! {
! /* When not optimizing or not inlining inline only always-inline
! functions. */
! cgraph_decide_inlining_incrementally (node, INLINE_ALWAYS_INLINE);
timevar_push (TV_INTEGRATION);
todo |= optimize_inline_calls (current_function_decl);
timevar_pop (TV_INTEGRATION);
}
! else
! {
! /* We iterate incremental inlining to get trivial cases of indirect
! inlining. */
! while (iterations < PARAM_VALUE (PARAM_EARLY_INLINER_MAX_ITERATIONS)
! && cgraph_decide_inlining_incrementally (node,
! iterations
! ? INLINE_SIZE_NORECURSIVE
! : INLINE_SIZE))
! {
! timevar_push (TV_INTEGRATION);
! todo |= optimize_inline_calls (current_function_decl);
! iterations++;
! timevar_pop (TV_INTEGRATION);
! }
! if (dump_file)
! fprintf (dump_file, "Iterations: %i\n", iterations);
! }
!
cfun->always_inline_functions_inlined = true;
! return todo;
}
struct gimple_opt_pass pass_early_inline =
*************** struct gimple_opt_pass pass_early_inline
*** 1661,1667 ****
{
GIMPLE_PASS,
"einline", /* name */
! cgraph_gate_early_inlining, /* gate */
cgraph_early_inlining, /* execute */
NULL, /* sub */
NULL, /* next */
--- 1605,1611 ----
{
GIMPLE_PASS,
"einline", /* name */
! NULL, /* gate */
cgraph_early_inlining, /* execute */
NULL, /* sub */
NULL, /* next */
*************** estimate_function_body_sizes (struct cgr
*** 1786,1791 ****
--- 1730,1743 ----
int freq;
tree funtype = TREE_TYPE (node->decl);
+ if (node->local.disregard_inline_limits)
+ {
+ inline_summary (node)->self_time = 0;
+ inline_summary (node)->self_size = 0;
+ inline_summary (node)->time_inlining_benefit = 0;
+ inline_summary (node)->size_inlining_benefit = 0;
+ }
+
if (dump_file)
fprintf (dump_file, "Analyzing function body size: %s\n",
cgraph_node_name (node));
*************** inline_write_summary (cgraph_node_set se
*** 2045,2056 ****
ipa_prop_write_jump_functions (set);
}
struct ipa_opt_pass_d pass_ipa_inline =
{
{
IPA_PASS,
"inline", /* name */
! NULL, /* gate */
cgraph_decide_inlining, /* execute */
NULL, /* sub */
NULL, /* next */
--- 1997,2026 ----
ipa_prop_write_jump_functions (set);
}
+ /* When to run IPA inlining. Inlining of always-inline functions
+ happens during early inlining. */
+
+ static bool
+ gate_cgraph_decide_inlining (void)
+ {
+ /* We'd like to skip this if not optimizing or not inlining as
+ all always-inline functions have been processed by early
+ inlining already. But this breaks EH with C++ somehow with
+
+ g++.dg/torture/pr31863.C: In destructor 'Serializer<unsigned int, Loki::Typelist<ClassSpec<unsigned int, A040, 40u>, Loki::NullType> >::~Serializer()':
+ g++.dg/torture/pr31863.C:231:7: error: statement marked for throw, but doesn't
+ Serializer<unsigned int, ClassSpec<unsigned int, A040, 40u> >::~Serializer (this.18352_8, D.118411_7);
+
+ so leave it on unconditionally for now. */
+ return 1;
+ }
+
struct ipa_opt_pass_d pass_ipa_inline =
{
{
IPA_PASS,
"inline", /* name */
! gate_cgraph_decide_inlining, /* gate */
cgraph_decide_inlining, /* execute */
NULL, /* sub */
NULL, /* next */
Index: gcc/ipa.c
===================================================================
*** gcc/ipa.c.orig 2010-04-12 11:48:18.000000000 +0200
--- gcc/ipa.c 2010-04-12 12:12:01.000000000 +0200
*************** cgraph_postorder (struct cgraph_node **o
*** 70,75 ****
--- 70,81 ----
node2->aux = edge->next_caller;
else
node2->aux = &last;
+ /* Break possible cycles involving always-inline
+ functions by ignoring edges from always-inline
+ functions to non-always-inline functions. */
+ if (edge->caller->local.disregard_inline_limits
+ && !edge->callee->local.disregard_inline_limits)
+ continue;
if (!edge->caller->aux)
{
if (!edge->caller->callers)
Index: gcc/testsuite/gcc.dg/torture/inline-2.c
===================================================================
*** /dev/null 1970-01-01 00:00:00.000000000 +0000
--- gcc/testsuite/gcc.dg/torture/inline-2.c 2010-04-12 12:12:01.000000000 +0200
***************
*** 0 ****
--- 1,35 ----
+ /* { dg-do link } */
+
+ extern inline void foo2 (void) __attribute__((always_inline,gnu_inline));
+ extern inline void foo1 (void) __attribute__((always_inline,gnu_inline));
+ void bar1 (void);
+ void bar2 (void);
+
+ extern inline void __attribute__((always_inline,gnu_inline))
+ foo2 (void)
+ {
+ bar2 ();
+ }
+
+ void
+ bar1 (void)
+ {
+ foo2 ();
+ }
+
+ void
+ bar2 (void)
+ {
+ foo1 ();
+ }
+
+ extern inline void __attribute__((always_inline,gnu_inline))
+ foo1 (void)
+ {
+ bar1 ();
+ }
+
+ int main()
+ {
+ return 0;
+ }