This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH] Indirect call graph edges
Hi,
On Mon, Apr 26, 2010 at 03:29:06PM +0200, Jan Hubicka wrote:
> > > +/* Structure containing additional information about an indirect call. */
> > > +
> > > +struct GTY(()) cgraph_indirect_call_info
> > > +{
> > > + /* Index of the parameter that is called. */
> > > + int param_index;
>
> I guess for ipa-pta we will want to have parameter of SSA name being
> called so we can build may edges, but this is for later.
I need this integer anyway so I can update it as inlining proceeds (it
always identifies the relevant parameter in the function the noe is
inlined to).
This field is initialised to -1 meaning this is not a call to
parameter (though if you leave it like that it will hit an assert now
so that we have some extra testing but otherwise it is easy to ignore
such edges).
> > > +
> > > + /* Whether this edge has already been looked at by indirect inlining. */
> > > + unsigned int inlining_processed : 1;
>
> Hmm, does this have to sit in the global datastructure? I know that
> inliner is pretty bad on this having all its local stuff in the
> cgraph_node instead of storing it on-side like others does, but
> perhaps this can just be bitmap somewhere in inliner?
No, it doesn't, it just made the edge duplication hook a bit more
straightforward. I have replaced it with a bitmap
(iinlining_processed_edges).
> > > @@ -1726,6 +1845,12 @@ dump_cgraph_node (FILE *f, struct cgraph
> > > }
> > > fprintf (f, "\n");
> > >
> > > + for (edge = node->indirect_calls; edge; edge = edge->next_callee)
> > > + indirect_calls_count++;
> > > + if (indirect_calls_count)
> > > + fprintf (f, " has %i outgoing edges for indirect calls.\n",
> > > + indirect_calls_count);
>
> I guess we should have better dumping here later, but this can be
> done incrementally.
Well, I just wanted to see that something was indeed there. But sure,
I can add whatever dumping you would like.
> > > -ipa_write_param_call_note (struct output_block *ob,
> > > - struct ipa_param_call_note *note)
> > > +ipa_write_indirect_edge_info (struct output_block *ob,
> > > + struct cgraph_edge *cs)
> > > {
> > > - gcc_assert (!note->processed);
> > > - lto_output_uleb128_stream (ob->main_stream, gimple_uid (note->stmt));
> > > - lto_output_sleb128_stream (ob->main_stream, note->formal_id);
> > > - lto_output_sleb128_stream (ob->main_stream, note->count);
> > > - lto_output_sleb128_stream (ob->main_stream, note->frequency);
> > > - lto_output_sleb128_stream (ob->main_stream, note->loop_nest);
> > > + struct cgraph_indirect_call_info *ii = cs->indirect_info;
> > > + struct bitpack_d *bp;
> > > +
> > > + lto_output_sleb128_stream (ob->main_stream, ii->param_index);
> > > + bp = bitpack_create ();
> > > + bp_pack_value (bp, ii->inlining_processed, 1);
>
> Hmm, what is the point of streaming this?
No. Actually a subsequnt patch of mine turned this into an assert.
>
> Otherwise the patch looks OK.
OK, so I assume the patch below is approved and will commit it
tomorrow if there are no objections.
Thanks,
Martin
2010-04-27 Martin Jambor <mjambor@suse.cz>
* cgraph.h (struct cgraph_node): New field indirect_calls.
(struct cgraph_indirect_call_info): New type.
(struct cgraph_edge): Removed field indirect_call. New fields
indirect_info, indirect_inlining_edge and indirect_unknown_callee.
(cgraph_create_indirect_edge): Declare.
(cgraph_make_edge_direct): Likewise.
(enum LTO_cgraph_tags): New item LTO_cgraph_indirect_edge.
* ipa-prop.h (struct ipa_param_call_note): Removed.
(struct ipa_node_params): Removed field param_calls.
(ipa_create_all_structures_for_iinln): Declare.
* cgraph.c: Described indirect edges and uids in initial comment.
(cgraph_add_edge_to_call_site_hash): New function.
(cgraph_edge): Search also among the indirect edges, use
cgraph_add_edge_to_call_site_hash to add edges to the call site hash.
(cgraph_set_call_stmt): Possibly turn an indirect edge into a direct
one, use cgraph_add_edge_to_call_site_hash to add edges to the call
site hash.
(initialize_inline_failed): Assign a reason to indirect edges.
(cgraph_create_edge_1): New function.
(cgraph_create_edge): Moved some functionality to
cgraph_create_edge_1.
(cgraph_create_indirect_edge): New function.
(cgraph_edge_remove_callee): Add an assert checking for
non-indirectness.
(cgraph_edge_remove_caller): Special-case indirect edges.
(cgraph_remove_edge): Likewise.
(cgraph_set_edge_callee): New function.
(cgraph_redirect_edge_callee): Use cgraph_set_edge_callee.
(cgraph_make_edge_direct): New function.
(cgraph_update_edges_for_call_stmt_node): Do nothing only when also
the declaration of the call statement matches.
(cgraph_node_remove_callees): Special-case indirect edges.
(cgraph_clone_edge): Likewise.
(cgraph_clone_node): Clone also the indirect edges.
(dump_cgraph_node): Dump indirect_inlining_edge flag instead of
indirect_call, dump count of indirect_calls edges.
* ipa-prop.c (iinlining_processed_edges): New variable.
(ipa_note_param_call): Create indirect edges instead of
creating notes. New parameter node.
(ipa_analyze_call_uses): New parameter node, pass it on to
ipa_note_param_call.
(ipa_analyze_stmt_uses): Likewise.
(ipa_analyze_params_uses): Pass node to ipa_analyze_stmt_uses.
(print_edge_addition_message): Work on edges rather than on notes.
(update_call_notes_after_inlining): Likewise, renamed to
update_indirect_edges_after_inlining.
(ipa_create_all_structures_for_iinln): New function.
(ipa_free_node_params_substructures): Do not free notes.
(ipa_edge_duplication_hook): Propagate bits within
iinlining_processed_edges bitmap.
(ipa_node_duplication_hook): Do not duplicate notes.
(free_all_ipa_structures_after_ipa_cp): Renamed to
ipa_free_all_structures_after_ipa_cp.
(free_all_ipa_structures_after_iinln): Renamed to
ipa_free_all_structures_after_iinln.g
(ipa_write_param_call_note): Removed.
(ipa_read_param_call_note): Removed.
(ipa_write_indirect_edge_info): New function.
(ipa_read_indirect_edge_info): Likewise.
(ipa_write_node_info): Do not stream notes, do stream information
in indirect edges.
(ipa_read_node_info): Likewise.
(lto_ipa_fixup_call_notes): Removed.
* ipa-cp.c (pass_ipa_cp): Set stmt_fixup to NULL.
* ipa-inline.c (pass_ipa_inline): Likewise.
* cgraphunit.c (verify_cgraph_node): Check also indirect edges.
* cif-code.def (INDIRECT_UNKNOWN_CALL): New reason.
* tree-inline.c (copy_bb): Removed an unnecessary double check for
is_gimple_call.
* tree-inline.c (get_indirect_callee_fndecl): Do not consider indirect
edges.
* lto-cgraph.c (output_outgoing_cgraph_edges): New function.
(output_cgraph): Stream also indirect edges.
(lto_output_edge): Added capability to stream indirect edges.
(input_edge): Likewise.
(input_cgraph_1): Likewise.
* testsuite/gcc.dg/lto/20091209-1_0.c: New testcase.
Index: icln/gcc/cgraph.h
===================================================================
--- icln.orig/gcc/cgraph.h
+++ icln/gcc/cgraph.h
@@ -184,6 +184,9 @@ struct GTY((chain_next ("%h.next"), chai
struct cgraph_edge *callers;
struct cgraph_node *next;
struct cgraph_node *previous;
+ /* List of edges representing indirect calls with a yet undetermined
+ callee. */
+ struct cgraph_edge *indirect_calls;
/* For nested functions points to function the node is nested in. */
struct cgraph_node *origin;
/* Points to first nested function, if any. */
@@ -315,6 +318,14 @@ typedef enum {
CIF_N_REASONS
} cgraph_inline_failed_t;
+/* Structure containing additional information about an indirect call. */
+
+struct GTY(()) cgraph_indirect_call_info
+{
+ /* Index of the parameter that is called. */
+ int param_index;
+};
+
struct GTY((chain_next ("%h.next_caller"), chain_prev ("%h.prev_caller"))) cgraph_edge {
/* Expected number of executions: calculated in profile.c. */
gcov_type count;
@@ -325,6 +336,9 @@ struct GTY((chain_next ("%h.next_caller"
struct cgraph_edge *prev_callee;
struct cgraph_edge *next_callee;
gimple call_stmt;
+ /* Additional information about an indirect call. Not cleared when an edge
+ becomes direct. */
+ struct cgraph_indirect_call_info *indirect_info;
PTR GTY ((skip (""))) aux;
/* When equal to CIF_OK, inline this call. Otherwise, points to the
explanation why function was not inlined. */
@@ -340,8 +354,12 @@ struct GTY((chain_next ("%h.next_caller"
int uid;
/* Depth of loop nest, 1 means no loop nest. */
unsigned short int loop_nest;
- /* Whether this edge describes a call that was originally indirect. */
- unsigned int indirect_call : 1;
+ /* Whether this edge was made direct by indirect inlining. */
+ unsigned int indirect_inlining_edge : 1;
+ /* Whether this edge describes an indirect call with an undetermined
+ callee. */
+ unsigned int indirect_unknown_callee : 1;
+ /* Whether this edge is still a dangling */
/* True if the corresponding CALL stmt cannot be inlined. */
unsigned int call_stmt_cannot_inline_p : 1;
/* Can this call throw externally? */
@@ -443,7 +461,8 @@ void cgraph_node_remove_callees (struct
struct cgraph_edge *cgraph_create_edge (struct cgraph_node *,
struct cgraph_node *,
gimple, gcov_type, int, int);
-
+struct cgraph_edge *cgraph_create_indirect_edge (struct cgraph_node *, gimple,
+ gcov_type, int, int);
struct cgraph_node * cgraph_get_node (tree);
struct cgraph_node *cgraph_node (tree);
bool cgraph_same_body_alias (tree, tree);
@@ -469,6 +488,7 @@ struct cgraph_node * cgraph_clone_node (
int, bool, VEC(cgraph_edge_p,heap) *);
void cgraph_redirect_edge_callee (struct cgraph_edge *, struct cgraph_node *);
+void cgraph_make_edge_direct (struct cgraph_edge *, struct cgraph_node *);
struct cgraph_asm_node *cgraph_add_asm_node (tree);
@@ -638,6 +658,7 @@ enum LTO_cgraph_tags
LTO_cgraph_overwritable_node,
LTO_cgraph_unavail_node,
LTO_cgraph_edge,
+ LTO_cgraph_indirect_edge,
LTO_cgraph_last_tag
};
Index: icln/gcc/ipa-prop.h
===================================================================
--- icln.orig/gcc/ipa-prop.h
+++ icln/gcc/ipa-prop.h
@@ -135,32 +135,6 @@ struct ipcp_lattice
tree constant;
};
-/* Each instance of the following structure describes a statement that calls a
- function parameter. Those referring to statements within the same function
- are linked in a list. */
-struct ipa_param_call_note
-{
- /* Expected number of executions: calculated in profile.c. */
- gcov_type count;
- /* Linked list's next */
- struct ipa_param_call_note *next;
- /* Statement that contains the call to the parameter above. */
- gimple stmt;
- /* When in LTO, we the above stmt will be NULL and we need an uid. */
- unsigned int lto_stmt_uid;
- /* Index of the parameter that is called. */
- int formal_id;
- /* Expected frequency of executions within the function. see cgraph_edge in
- cgraph.h for more on this. */
- int frequency;
- /* Depth of loop nest, 1 means no loop nest. */
- unsigned short int loop_nest;
- /* Set when we have already found the target to be a compile time constant
- and turned this into an edge or when the note was found unusable for some
- reason. */
- bool processed;
-};
-
/* Structure describing a single formal parameter. */
struct ipa_param_descriptor
{
@@ -184,8 +158,6 @@ struct ipa_node_params
/* Pointer to an array of structures describing individual formal
parameters. */
struct ipa_param_descriptor *params;
- /* List of structures enumerating calls to a formal parameter. */
- struct ipa_param_call_note *param_calls;
/* Only for versioned nodes this field would not be NULL,
it points to the node that IPA cp cloned from. */
struct cgraph_node *ipcp_orig_node;
@@ -338,8 +310,9 @@ void ipa_free_edge_args_substructures (s
void ipa_free_node_params_substructures (struct ipa_node_params *);
void ipa_free_all_node_params (void);
void ipa_free_all_edge_args (void);
-void free_all_ipa_structures_after_ipa_cp (void);
-void free_all_ipa_structures_after_iinln (void);
+void ipa_create_all_structures_for_iinln (void);
+void ipa_free_all_structures_after_ipa_cp (void);
+void ipa_free_all_structures_after_iinln (void);
void ipa_register_cgraph_hooks (void);
/* This function ensures the array of node param infos is big enough to
Index: icln/gcc/cgraph.c
===================================================================
--- icln.orig/gcc/cgraph.c
+++ icln/gcc/cgraph.c
@@ -34,10 +34,16 @@ The callgraph:
based on DECL_UID. The call-graph nodes are created lazily using
cgraph_node function when called for unknown declaration.
- The callgraph at the moment does not represent indirect calls or calls
- from other compilation unit. Flag NEEDED is set for each node that may
- be accessed in such an invisible way and it shall be considered an
- entry point to the callgraph.
+ The callgraph at the moment does not represent all indirect calls or calls
+ from other compilation units. Flag NEEDED is set for each node that may be
+ accessed in such an invisible way and it shall be considered an entry point
+ to the callgraph.
+
+ On the other hand, the callgraph currently does contain some edges for
+ indirect calls with unknown callees which can be accessed through
+ indirect_calls field of a node. It should be noted however that at the
+ moment only calls which are potential candidates for indirect inlining are
+ added there.
Interprocedural information:
@@ -48,6 +54,9 @@ The callgraph:
rtl_info used by RTL backend to propagate data from already compiled
functions to their callers.
+ Moreover, each node has a uid which can be used to keep information in
+ on-the-side arrays. UIDs are reused and therefore reasonably dense.
+
Inlining plans:
The function inlining information is decided in advance and maintained
@@ -722,6 +731,19 @@ edge_eq (const void *x, const void *y)
return ((const struct cgraph_edge *) x)->call_stmt == y;
}
+/* Add call graph edge E to call site hash of its caller. */
+
+static inline void
+cgraph_add_edge_to_call_site_hash (struct cgraph_edge *e)
+{
+ void **slot;
+ slot = htab_find_slot_with_hash (e->caller->call_site_hash,
+ e->call_stmt,
+ htab_hash_pointer (e->call_stmt),
+ INSERT);
+ gcc_assert (!*slot);
+ *slot = e;
+}
/* Return the callgraph edge representing the GIMPLE_CALL statement
CALL_STMT. */
@@ -742,26 +764,28 @@ cgraph_edge (struct cgraph_node *node, g
solution. It is not good idea to add pointer into CALL_EXPR itself
because we want to make possible having multiple cgraph nodes representing
different clones of the same body before the body is actually cloned. */
- for (e = node->callees; e; e= e->next_callee)
+ for (e = node->callees; e; e = e->next_callee)
{
if (e->call_stmt == call_stmt)
break;
n++;
}
+ if (!e)
+ for (e = node->indirect_calls; e; e = e->next_callee)
+ {
+ if (e->call_stmt == call_stmt)
+ break;
+ n++;
+ }
+
if (n > 100)
{
node->call_site_hash = htab_create_ggc (120, edge_hash, edge_eq, NULL);
for (e2 = node->callees; e2; e2 = e2->next_callee)
- {
- void **slot;
- slot = htab_find_slot_with_hash (node->call_site_hash,
- e2->call_stmt,
- htab_hash_pointer (e2->call_stmt),
- INSERT);
- gcc_assert (!*slot);
- *slot = e2;
- }
+ cgraph_add_edge_to_call_site_hash (e2);
+ for (e2 = node->indirect_calls; e2; e2 = e2->next_callee)
+ cgraph_add_edge_to_call_site_hash (e2);
}
return e;
@@ -773,26 +797,31 @@ cgraph_edge (struct cgraph_node *node, g
void
cgraph_set_call_stmt (struct cgraph_edge *e, gimple new_stmt)
{
+ tree decl;
+
if (e->caller->call_site_hash)
{
htab_remove_elt_with_hash (e->caller->call_site_hash,
e->call_stmt,
htab_hash_pointer (e->call_stmt));
}
+
e->call_stmt = new_stmt;
+ if (e->indirect_unknown_callee
+ && (decl = gimple_call_fndecl (new_stmt)))
+ {
+ /* Constant propagation (and possibly also inlining?) can turn an
+ indirect call into a direct one. */
+ struct cgraph_node *new_callee = cgraph_node (decl);
+
+ cgraph_make_edge_direct (e, new_callee);
+ }
+
push_cfun (DECL_STRUCT_FUNCTION (e->caller->decl));
e->can_throw_external = stmt_can_throw_external (new_stmt);
pop_cfun ();
if (e->caller->call_site_hash)
- {
- void **slot;
- slot = htab_find_slot_with_hash (e->caller->call_site_hash,
- e->call_stmt,
- htab_hash_pointer
- (e->call_stmt), INSERT);
- gcc_assert (!*slot);
- *slot = e;
- }
+ cgraph_add_edge_to_call_site_hash (e);
}
/* Like cgraph_set_call_stmt but walk the clone tree and update all
@@ -894,7 +923,9 @@ initialize_inline_failed (struct cgraph_
{
struct cgraph_node *callee = e->callee;
- if (!callee->analyzed)
+ if (e->indirect_unknown_callee)
+ e->inline_failed = CIF_INDIRECT_UNKNOWN_CALL;
+ else if (!callee->analyzed)
e->inline_failed = CIF_BODY_NOT_AVAILABLE;
else if (callee->local.redefined_extern_inline)
e->inline_failed = CIF_REDEFINED_EXTERN_INLINE;
@@ -906,15 +937,16 @@ initialize_inline_failed (struct cgraph_
e->inline_failed = CIF_FUNCTION_NOT_CONSIDERED;
}
-/* Create edge from CALLER to CALLEE in the cgraph. */
-
-struct cgraph_edge *
-cgraph_create_edge (struct cgraph_node *caller, struct cgraph_node *callee,
- gimple call_stmt, gcov_type count, int freq, int nest)
+/* Allocate a cgraph_edge structure and fill it with data according to the
+ parameters of which only CALLEE can be NULL (when creating an indirect call
+ edge). */
+
+static struct cgraph_edge *
+cgraph_create_edge_1 (struct cgraph_node *caller, struct cgraph_node *callee,
+ gimple call_stmt, gcov_type count, int freq, int nest)
{
struct cgraph_edge *edge;
-
/* LTO does not actually have access to the call_stmt since these
have not been loaded yet. */
if (call_stmt)
@@ -940,47 +972,83 @@ cgraph_create_edge (struct cgraph_node *
}
edge->aux = NULL;
-
edge->caller = caller;
edge->callee = callee;
+ edge->prev_caller = NULL;
+ edge->next_caller = NULL;
+ edge->prev_callee = NULL;
+ edge->next_callee = NULL;
+
+ edge->count = count;
+ gcc_assert (count >= 0);
+ edge->frequency = freq;
+ gcc_assert (freq >= 0);
+ gcc_assert (freq <= CGRAPH_FREQ_MAX);
+ edge->loop_nest = nest;
+
edge->call_stmt = call_stmt;
push_cfun (DECL_STRUCT_FUNCTION (caller->decl));
edge->can_throw_external
= call_stmt ? stmt_can_throw_external (call_stmt) : false;
pop_cfun ();
- edge->prev_caller = NULL;
+ edge->call_stmt_cannot_inline_p =
+ (call_stmt ? gimple_call_cannot_inline_p (call_stmt) : false);
+ if (call_stmt && caller->call_site_hash)
+ cgraph_add_edge_to_call_site_hash (edge);
+
+ edge->indirect_info = NULL;
+ edge->indirect_inlining_edge = 0;
+
+ return edge;
+}
+
+/* Create edge from CALLER to CALLEE in the cgraph. */
+
+struct cgraph_edge *
+cgraph_create_edge (struct cgraph_node *caller, struct cgraph_node *callee,
+ gimple call_stmt, gcov_type count, int freq, int nest)
+{
+ struct cgraph_edge *edge = cgraph_create_edge_1 (caller, callee, call_stmt,
+ count, freq, nest);
+
+ edge->indirect_unknown_callee = 0;
+ initialize_inline_failed (edge);
+
edge->next_caller = callee->callers;
if (callee->callers)
callee->callers->prev_caller = edge;
- edge->prev_callee = NULL;
edge->next_callee = caller->callees;
if (caller->callees)
caller->callees->prev_callee = edge;
caller->callees = edge;
callee->callers = edge;
- edge->count = count;
- gcc_assert (count >= 0);
- edge->frequency = freq;
- gcc_assert (freq >= 0);
- gcc_assert (freq <= CGRAPH_FREQ_MAX);
- edge->loop_nest = nest;
- edge->indirect_call = 0;
- edge->call_stmt_cannot_inline_p =
- (call_stmt ? gimple_call_cannot_inline_p (call_stmt) : false);
- if (call_stmt && caller->call_site_hash)
- {
- void **slot;
- slot = htab_find_slot_with_hash (caller->call_site_hash,
- edge->call_stmt,
- htab_hash_pointer
- (edge->call_stmt),
- INSERT);
- gcc_assert (!*slot);
- *slot = edge;
- }
+ return edge;
+}
+
+
+/* Create an indirect edge with a yet-undetermined callee where the call
+ statement destination is a formal parameter of the caller with index
+ PARAM_INDEX. */
+
+struct cgraph_edge *
+cgraph_create_indirect_edge (struct cgraph_node *caller, gimple call_stmt,
+ gcov_type count, int freq, int nest)
+{
+ struct cgraph_edge *edge = cgraph_create_edge_1 (caller, NULL, call_stmt,
+ count, freq, nest);
+
+ edge->indirect_unknown_callee = 1;
initialize_inline_failed (edge);
+ edge->indirect_info = GGC_NEW (struct cgraph_indirect_call_info);
+ edge->indirect_info->param_index = -1;
+
+ edge->next_callee = caller->indirect_calls;
+ if (caller->indirect_calls)
+ caller->indirect_calls->prev_callee = edge;
+ caller->indirect_calls = edge;
+
return edge;
}
@@ -989,6 +1057,7 @@ cgraph_create_edge (struct cgraph_node *
static inline void
cgraph_edge_remove_callee (struct cgraph_edge *e)
{
+ gcc_assert (!e->indirect_unknown_callee);
if (e->prev_caller)
e->prev_caller->next_caller = e->next_caller;
if (e->next_caller)
@@ -1007,7 +1076,12 @@ cgraph_edge_remove_caller (struct cgraph
if (e->next_callee)
e->next_callee->prev_callee = e->prev_callee;
if (!e->prev_callee)
- e->caller->callees = e->next_callee;
+ {
+ if (e->indirect_unknown_callee)
+ e->caller->indirect_calls = e->next_callee;
+ else
+ e->caller->callees = e->next_callee;
+ }
if (e->caller->call_site_hash)
htab_remove_elt_with_hash (e->caller->call_site_hash,
e->call_stmt,
@@ -1036,8 +1110,9 @@ cgraph_remove_edge (struct cgraph_edge *
/* Call all edge removal hooks. */
cgraph_call_edge_removal_hooks (e);
- /* Remove from callers list of the callee. */
- cgraph_edge_remove_callee (e);
+ if (!e->indirect_unknown_callee)
+ /* Remove from callers list of the callee. */
+ cgraph_edge_remove_callee (e);
/* Remove from callees list of the callers. */
cgraph_edge_remove_caller (e);
@@ -1046,6 +1121,20 @@ cgraph_remove_edge (struct cgraph_edge *
cgraph_free_edge (e);
}
+/* Set callee of call graph edge E and add it to the corresponding set of
+ callers. */
+
+static void
+cgraph_set_edge_callee (struct cgraph_edge *e, struct cgraph_node *n)
+{
+ e->prev_caller = NULL;
+ if (n->callers)
+ n->callers->prev_caller = e;
+ e->next_caller = n->callers;
+ n->callers = e;
+ e->callee = n;
+}
+
/* Redirect callee of E to N. The function does not update underlying
call expression. */
@@ -1056,12 +1145,37 @@ cgraph_redirect_edge_callee (struct cgra
cgraph_edge_remove_callee (e);
/* Insert to callers list of the new callee. */
- e->prev_caller = NULL;
- if (n->callers)
- n->callers->prev_caller = e;
- e->next_caller = n->callers;
- n->callers = e;
- e->callee = n;
+ cgraph_set_edge_callee (e, n);
+}
+
+/* Make an indirect EDGE with an unknown callee an ordinary edge leading to
+ CALLEE. */
+
+void
+cgraph_make_edge_direct (struct cgraph_edge *edge, struct cgraph_node *callee)
+{
+ edge->indirect_unknown_callee = 0;
+
+ /* Get the edge out of the indirect edge list. */
+ if (edge->prev_callee)
+ edge->prev_callee->next_callee = edge->next_callee;
+ if (edge->next_callee)
+ edge->next_callee->prev_callee = edge->prev_callee;
+ if (!edge->prev_callee)
+ edge->caller->indirect_calls = edge->next_callee;
+
+ /* Put it into the normal callee list */
+ edge->prev_callee = NULL;
+ edge->next_callee = edge->caller->callees;
+ if (edge->caller->callees)
+ edge->caller->callees->prev_callee = edge;
+ edge->caller->callees = edge;
+
+ /* Insert to callers list of the new callee. */
+ cgraph_set_edge_callee (edge, callee);
+
+ /* We need to re-determine the inlining status of the edge. */
+ initialize_inline_failed (edge);
}
@@ -1090,9 +1204,10 @@ cgraph_update_edges_for_call_stmt_node (
if (e)
{
- /* See if the call is already there. It might be because of indirect
- inlining already found it. */
- if (new_call && e->callee->decl == new_call)
+ /* See if the edge is already there and has the correct callee. It
+ might be so because of indirect inlining has already updated
+ it. */
+ if (new_call && e->callee && e->callee->decl == new_call)
return;
/* Otherwise remove edge and create new one; we can't simply redirect
@@ -1170,7 +1285,8 @@ cgraph_node_remove_callees (struct cgrap
{
f = e->next_callee;
cgraph_call_edge_removal_hooks (e);
- cgraph_edge_remove_callee (e);
+ if (!e->indirect_unknown_callee)
+ cgraph_edge_remove_callee (e);
cgraph_free_edge (e);
}
node->callees = NULL;
@@ -1626,6 +1742,8 @@ void
dump_cgraph_node (FILE *f, struct cgraph_node *node)
{
struct cgraph_edge *edge;
+ int indirect_calls_count = 0;
+
fprintf (f, "%s/%i(%i)", cgraph_node_name (node), node->uid,
node->pid);
dump_addr (f, " @", (void *)node);
@@ -1707,8 +1825,8 @@ dump_cgraph_node (FILE *f, struct cgraph
edge->frequency / (double)CGRAPH_FREQ_BASE);
if (!edge->inline_failed)
fprintf(f, "(inlined) ");
- if (edge->indirect_call)
- fprintf(f, "(indirect) ");
+ if (edge->indirect_inlining_edge)
+ fprintf(f, "(indirect_inlining) ");
if (edge->can_throw_external)
fprintf(f, "(can throw external) ");
}
@@ -1720,8 +1838,8 @@ dump_cgraph_node (FILE *f, struct cgraph
edge->callee->uid);
if (!edge->inline_failed)
fprintf(f, "(inlined) ");
- if (edge->indirect_call)
- fprintf(f, "(indirect) ");
+ if (edge->indirect_inlining_edge)
+ fprintf(f, "(indirect_inlining) ");
if (edge->count)
fprintf (f, "("HOST_WIDEST_INT_PRINT_DEC"x) ",
(HOST_WIDEST_INT)edge->count);
@@ -1735,6 +1853,12 @@ dump_cgraph_node (FILE *f, struct cgraph
}
fprintf (f, "\n");
+ for (edge = node->indirect_calls; edge; edge = edge->next_callee)
+ indirect_calls_count++;
+ if (indirect_calls_count)
+ fprintf (f, " has %i outgoing edges for indirect calls.\n",
+ indirect_calls_count);
+
if (node->same_body)
{
struct cgraph_node *n;
@@ -1854,11 +1978,30 @@ cgraph_clone_edge (struct cgraph_edge *e
freq = e->frequency * (gcov_type) freq_scale / CGRAPH_FREQ_BASE;
if (freq > CGRAPH_FREQ_MAX)
freq = CGRAPH_FREQ_MAX;
- new_edge = cgraph_create_edge (n, e->callee, call_stmt, count, freq,
- e->loop_nest + loop_nest);
+
+ if (e->indirect_unknown_callee)
+ {
+ tree decl;
+
+ if (call_stmt && (decl = gimple_call_fndecl (call_stmt)))
+ {
+ struct cgraph_node *callee = cgraph_node (decl);
+ new_edge = cgraph_create_edge (n, callee, call_stmt, count, freq,
+ e->loop_nest + loop_nest);
+ }
+ else
+ {
+ new_edge = cgraph_create_indirect_edge (n, call_stmt, count, freq,
+ e->loop_nest + loop_nest);
+ new_edge->indirect_info->param_index = e->indirect_info->param_index;
+ }
+ }
+ else
+ new_edge = cgraph_create_edge (n, e->callee, call_stmt, count, freq,
+ e->loop_nest + loop_nest);
new_edge->inline_failed = e->inline_failed;
- new_edge->indirect_call = e->indirect_call;
+ new_edge->indirect_inlining_edge = e->indirect_inlining_edge;
new_edge->lto_stmt_uid = stmt_uid;
if (update_original)
{
@@ -1929,6 +2072,10 @@ cgraph_clone_node (struct cgraph_node *n
cgraph_clone_edge (e, new_node, e->call_stmt, e->lto_stmt_uid,
count_scale, freq, loop_nest, update_original);
+ for (e = n->indirect_calls; e; e = e->next_callee)
+ cgraph_clone_edge (e, new_node, e->call_stmt, e->lto_stmt_uid,
+ count_scale, freq, loop_nest, update_original);
+
new_node->next_sibling_clone = n->clones;
if (n->clones)
n->clones->prev_sibling_clone = new_node;
Index: icln/gcc/ipa-prop.c
===================================================================
--- icln.orig/gcc/ipa-prop.c
+++ icln/gcc/ipa-prop.c
@@ -41,6 +41,10 @@ VEC (ipa_node_params_t, heap) *ipa_node_
/* Vector where the parameter infos are actually stored. */
VEC (ipa_edge_args_t, gc) *ipa_edge_args_vector;
+/* Bitmap with all UIDs of call graph edges that have been already processed
+ by indirect inlining. */
+static bitmap iinlining_processed_edges;
+
/* Holders of ipa cgraph hooks: */
static struct cgraph_edge_hook_list *edge_removal_hook_holder;
static struct cgraph_node_hook_list *node_removal_hook_holder;
@@ -745,39 +749,31 @@ ipa_is_ssa_with_stmt_def (tree t)
return false;
}
-/* Creates a new note describing a call to a parameter number FORMAL_ID and
- attaches it to the linked list of INFO. It also sets the called flag of the
- parameter. STMT is the corresponding call statement. */
+/* Create a new indirect call graph edge describing a call to a parameter
+ number FORMAL_ID and and set the called flag of the parameter. NODE is the
+ caller. STMT is the corresponding call statement. */
static void
-ipa_note_param_call (struct ipa_node_params *info, int formal_id,
- gimple stmt)
+ipa_note_param_call (struct cgraph_node *node, int formal_id, gimple stmt)
{
- struct ipa_param_call_note *note;
+ struct cgraph_edge *cs;
basic_block bb = gimple_bb (stmt);
+ int freq;
- note = XCNEW (struct ipa_param_call_note);
- note->formal_id = formal_id;
- note->stmt = stmt;
- note->lto_stmt_uid = gimple_uid (stmt);
- note->count = bb->count;
- note->frequency = compute_call_stmt_bb_frequency (current_function_decl, bb);
- note->loop_nest = bb->loop_depth;
-
- note->next = info->param_calls;
- info->param_calls = note;
-
- return;
+ freq = compute_call_stmt_bb_frequency (current_function_decl, bb);
+ cs = cgraph_create_indirect_edge (node, stmt, bb->count, freq,
+ bb->loop_depth);
+ cs->indirect_info->param_index = formal_id;
}
-/* Analyze the CALL and examine uses of formal parameters of the caller
+/* Analyze the CALL and examine uses of formal parameters of the caller NODE
(described by INFO). Currently it checks whether the call calls a pointer
that is a formal parameter and if so, the parameter is marked with the
- called flag and a note describing the call is created. This is very simple
- for ordinary pointers represented in SSA but not-so-nice when it comes to
- member pointers. The ugly part of this function does nothing more than
- tries to match the pattern of such a call. An example of such a pattern is
- the gimple dump below, the call is on the last line:
+ called flag and an indirect call graph edge describing the call is created.
+ This is very simple for ordinary pointers represented in SSA but not-so-nice
+ when it comes to member pointers. The ugly part of this function does
+ nothing more than trying to match the pattern of such a call. An example of
+ such a pattern is the gimple dump below, the call is on the last line:
<bb 2>:
f$__delta_5 = f.__delta;
@@ -817,7 +813,8 @@ ipa_note_param_call (struct ipa_node_par
*/
static void
-ipa_analyze_call_uses (struct ipa_node_params *info, gimple call)
+ipa_analyze_call_uses (struct cgraph_node *node, struct ipa_node_params *info,
+ gimple call)
{
tree target = gimple_call_fn (call);
gimple def;
@@ -838,7 +835,7 @@ ipa_analyze_call_uses (struct ipa_node_p
/* assuming TREE_CODE (var) == PARM_DECL */
index = ipa_get_param_decl_index (info, var);
if (index >= 0)
- ipa_note_param_call (info, index, call);
+ ipa_note_param_call (node, index, call);
return;
}
@@ -935,20 +932,21 @@ ipa_analyze_call_uses (struct ipa_node_p
index = ipa_get_param_decl_index (info, rec);
if (index >= 0 && !ipa_is_param_modified (info, index))
- ipa_note_param_call (info, index, call);
+ ipa_note_param_call (node, index, call);
return;
}
-/* Analyze the statement STMT with respect to formal parameters (described in
- INFO) and their uses. Currently it only checks whether formal parameters
- are called. */
+/* Analyze the call statement STMT with respect to formal parameters (described
+ in INFO) of caller given by NODE. Currently it only checks whether formal
+ parameters are called. */
static void
-ipa_analyze_stmt_uses (struct ipa_node_params *info, gimple stmt)
+ipa_analyze_stmt_uses (struct cgraph_node *node, struct ipa_node_params *info,
+ gimple stmt)
{
if (is_gimple_call (stmt))
- ipa_analyze_call_uses (info, stmt);
+ ipa_analyze_call_uses (node, info, stmt);
}
/* Scan the function body of NODE and inspect the uses of formal parameters.
@@ -973,7 +971,7 @@ ipa_analyze_params_uses (struct cgraph_n
for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
{
gimple stmt = gsi_stmt (gsi);
- ipa_analyze_stmt_uses (info, stmt);
+ ipa_analyze_stmt_uses (node, info, stmt);
}
}
@@ -1029,9 +1027,8 @@ update_jump_functions_after_inlining (st
by JFUNC. NODE is the node where the call is. */
static void
-print_edge_addition_message (FILE *f, struct ipa_param_call_note *nt,
- struct ipa_jump_func *jfunc,
- struct cgraph_node *node)
+print_edge_addition_message (FILE *f, struct cgraph_edge *e,
+ struct ipa_jump_func *jfunc)
{
fprintf (f, "ipa-prop: Discovered an indirect call to a known target (");
if (jfunc->type == IPA_JF_CONST_MEMBER_PTR)
@@ -1042,8 +1039,8 @@ print_edge_addition_message (FILE *f, st
else
print_node_brief(f, "", jfunc->value.constant, 0);
- fprintf (f, ") in %s: ", cgraph_node_name (node));
- print_gimple_stmt (f, nt->stmt, 2, TDF_SLIM);
+ fprintf (f, ") in %s: ", cgraph_node_name (e->caller));
+ print_gimple_stmt (f, e->call_stmt, 2, TDF_SLIM);
}
/* Update the param called notes associated with NODE when CS is being inlined,
@@ -1053,41 +1050,47 @@ print_edge_addition_message (FILE *f, st
unless NEW_EDGES is NULL. Return true iff a new edge(s) were created. */
static bool
-update_call_notes_after_inlining (struct cgraph_edge *cs,
- struct cgraph_node *node,
- VEC (cgraph_edge_p, heap) **new_edges)
+update_indirect_edges_after_inlining (struct cgraph_edge *cs,
+ struct cgraph_node *node,
+ VEC (cgraph_edge_p, heap) **new_edges)
{
- struct ipa_node_params *info = IPA_NODE_REF (node);
struct ipa_edge_args *top = IPA_EDGE_REF (cs);
- struct ipa_param_call_note *nt;
+ struct cgraph_edge *ie, *next_ie;
bool res = false;
- for (nt = info->param_calls; nt; nt = nt->next)
+ ipa_check_create_edge_args ();
+
+ for (ie = node->indirect_calls; ie; ie = next_ie)
{
+ struct cgraph_indirect_call_info *ici = ie->indirect_info;
struct ipa_jump_func *jfunc;
- if (nt->processed)
+ next_ie = ie->next_callee;
+ if (bitmap_bit_p (iinlining_processed_edges, ie->uid))
continue;
+ /* If we ever use indirect edges for anything other than indirect
+ inlining, we will need to skip those with negative param_indices. */
+ gcc_assert (ici->param_index >= 0);
+
/* We must check range due to calls with variable number of arguments: */
- if (nt->formal_id >= ipa_get_cs_argument_count (top))
+ if (ici->param_index >= ipa_get_cs_argument_count (top))
{
- nt->processed = true;
+ bitmap_set_bit (iinlining_processed_edges, ie->uid);
continue;
}
- jfunc = ipa_get_ith_jump_func (top, nt->formal_id);
+ jfunc = ipa_get_ith_jump_func (top, ici->param_index);
if (jfunc->type == IPA_JF_PASS_THROUGH
&& jfunc->value.pass_through.operation == NOP_EXPR)
- nt->formal_id = jfunc->value.pass_through.formal_id;
+ ici->param_index = jfunc->value.pass_through.formal_id;
else if (jfunc->type == IPA_JF_CONST
|| jfunc->type == IPA_JF_CONST_MEMBER_PTR)
{
struct cgraph_node *callee;
- struct cgraph_edge *new_indirect_edge;
tree decl;
- nt->processed = true;
+ bitmap_set_bit (iinlining_processed_edges, ie->uid);
if (jfunc->type == IPA_JF_CONST_MEMBER_PTR)
decl = jfunc->value.member_cst.pfn;
else
@@ -1105,32 +1108,29 @@ update_call_notes_after_inlining (struct
res = true;
if (dump_file)
- print_edge_addition_message (dump_file, nt, jfunc, node);
+ print_edge_addition_message (dump_file, ie, jfunc);
- new_indirect_edge = cgraph_create_edge (node, callee, nt->stmt,
- nt->count, nt->frequency,
- nt->loop_nest);
- new_indirect_edge->lto_stmt_uid = nt->lto_stmt_uid;
- new_indirect_edge->indirect_call = 1;
- ipa_check_create_edge_args ();
+ cgraph_make_edge_direct (ie, callee);
+ ie->indirect_inlining_edge = 1;
if (new_edges)
- VEC_safe_push (cgraph_edge_p, heap, *new_edges, new_indirect_edge);
+ VEC_safe_push (cgraph_edge_p, heap, *new_edges, ie);
top = IPA_EDGE_REF (cs);
}
else
{
- /* Ancestor jum functions and pass theoughs with operations should
+ /* Ancestor jump functions and pass theoughs with operations should
not be used on parameters that then get called. */
gcc_assert (jfunc->type == IPA_JF_UNKNOWN);
- nt->processed = true;
+ bitmap_set_bit (iinlining_processed_edges, ie->uid);
}
}
+
return res;
}
/* Recursively traverse subtree of NODE (including node) made of inlined
cgraph_edges when CS has been inlined and invoke
- update_call_notes_after_inlining on all nodes and
+ update_indirect_edges_after_inlining on all nodes and
update_jump_functions_after_inlining on all non-inlined edges that lead out
of this subtree. Newly discovered indirect edges will be added to
*NEW_EDGES, unless NEW_EDGES is NULL. Return true iff a new edge(s) were
@@ -1144,7 +1144,7 @@ propagate_info_to_inlined_callees (struc
struct cgraph_edge *e;
bool res;
- res = update_call_notes_after_inlining (cs, node, new_edges);
+ res = update_indirect_edges_after_inlining (cs, node, new_edges);
for (e = node->callees; e; e = e->next_callee)
if (!e->inline_failed)
@@ -1216,13 +1216,6 @@ ipa_free_node_params_substructures (stru
if (info->params)
free (info->params);
- while (info->param_calls)
- {
- struct ipa_param_call_note *note = info->param_calls;
- info->param_calls = note->next;
- free (note);
- }
-
memset (info, 0, sizeof (*info));
}
@@ -1317,6 +1310,10 @@ ipa_edge_duplication_hook (struct cgraph
new_args->jump_functions = (struct ipa_jump_func *)
duplicate_ggc_array (old_args->jump_functions,
sizeof (struct ipa_jump_func) * arg_count);
+
+ if (iinlining_processed_edges
+ && bitmap_bit_p (iinlining_processed_edges, src->uid))
+ bitmap_set_bit (iinlining_processed_edges, dst->uid);
}
/* Hook that is called by cgraph.c when a node is duplicated. */
@@ -1326,7 +1323,6 @@ ipa_node_duplication_hook (struct cgraph
__attribute__((unused)) void *data)
{
struct ipa_node_params *old_info, *new_info;
- struct ipa_param_call_note *note;
int param_count;
ipa_check_create_node_params ();
@@ -1340,17 +1336,6 @@ ipa_node_duplication_hook (struct cgraph
sizeof (struct ipa_param_descriptor) * param_count);
new_info->ipcp_orig_node = old_info->ipcp_orig_node;
new_info->count_scale = old_info->count_scale;
-
- for (note = old_info->param_calls; note; note = note->next)
- {
- struct ipa_param_call_note *nn;
-
- nn = (struct ipa_param_call_note *)
- xcalloc (1, sizeof (struct ipa_param_call_note));
- memcpy (nn, note, sizeof (struct ipa_param_call_note));
- nn->next = new_info->param_calls;
- new_info->param_calls = nn;
- }
}
/* Register our cgraph hooks if they are not already there. */
@@ -1387,11 +1372,19 @@ ipa_unregister_cgraph_hooks (void)
node_duplication_hook_holder = NULL;
}
+/* Allocate all necessary data strucutures necessary for indirect inlining. */
+
+void
+ipa_create_all_structures_for_iinln (void)
+{
+ iinlining_processed_edges = BITMAP_ALLOC (NULL);
+}
+
/* Free all ipa_node_params and all ipa_edge_args structures if they are no
longer needed after ipa-cp. */
void
-free_all_ipa_structures_after_ipa_cp (void)
+ipa_free_all_structures_after_ipa_cp (void)
{
if (!flag_indirect_inlining)
{
@@ -1405,8 +1398,10 @@ free_all_ipa_structures_after_ipa_cp (vo
longer needed after indirect inlining. */
void
-free_all_ipa_structures_after_iinln (void)
+ipa_free_all_structures_after_iinln (void)
{
+ BITMAP_FREE (iinlining_processed_edges);
+
ipa_free_all_edge_args ();
ipa_free_all_node_params ();
ipa_unregister_cgraph_hooks ();
@@ -1974,40 +1969,31 @@ ipa_read_jump_function (struct lto_input
}
}
-/* Stream out a parameter call note. */
+/* Stream out parts of cgraph_indirect_call_info corresponding to CS that are
+ relevant to indirect inlining to OB. */
static void
-ipa_write_param_call_note (struct output_block *ob,
- struct ipa_param_call_note *note)
+ipa_write_indirect_edge_info (struct output_block *ob,
+ struct cgraph_edge *cs)
{
- gcc_assert (!note->processed);
- lto_output_uleb128_stream (ob->main_stream, gimple_uid (note->stmt));
- lto_output_sleb128_stream (ob->main_stream, note->formal_id);
- lto_output_sleb128_stream (ob->main_stream, note->count);
- lto_output_sleb128_stream (ob->main_stream, note->frequency);
- lto_output_sleb128_stream (ob->main_stream, note->loop_nest);
+ struct cgraph_indirect_call_info *ii = cs->indirect_info;
+
+ lto_output_sleb128_stream (ob->main_stream, ii->param_index);
}
-/* Read in a parameter call note. */
+/* Read in parts of cgraph_indirect_call_info corresponding to CS that are
+ relevant to indirect inlining from IB. */
static void
-ipa_read_param_call_note (struct lto_input_block *ib,
- struct ipa_node_params *info)
-
+ipa_read_indirect_edge_info (struct lto_input_block *ib,
+ struct data_in *data_in ATTRIBUTE_UNUSED,
+ struct cgraph_edge *cs)
{
- struct ipa_param_call_note *note = XCNEW (struct ipa_param_call_note);
-
- note->lto_stmt_uid = (unsigned int) lto_input_uleb128 (ib);
- note->formal_id = (int) lto_input_sleb128 (ib);
- note->count = (gcov_type) lto_input_sleb128 (ib);
- note->frequency = (int) lto_input_sleb128 (ib);
- note->loop_nest = (int) lto_input_sleb128 (ib);
+ struct cgraph_indirect_call_info *ii = cs->indirect_info;
- note->next = info->param_calls;
- info->param_calls = note;
+ ii->param_index = (int) lto_input_sleb128 (ib);
}
-
/* Stream out NODE info to OB. */
static void
@@ -2019,8 +2005,6 @@ ipa_write_node_info (struct output_block
int j;
struct cgraph_edge *e;
struct bitpack_d *bp;
- int note_count = 0;
- struct ipa_param_call_note *note;
encoder = ob->decl_state->cgraph_node_encoder;
node_ref = lto_cgraph_encoder_encode (encoder, node);
@@ -2046,12 +2030,8 @@ ipa_write_node_info (struct output_block
for (j = 0; j < ipa_get_cs_argument_count (args); j++)
ipa_write_jump_function (ob, ipa_get_ith_jump_func (args, j));
}
-
- for (note = info->param_calls; note; note = note->next)
- note_count++;
- lto_output_uleb128_stream (ob->main_stream, note_count);
- for (note = info->param_calls; note; note = note->next)
- ipa_write_param_call_note (ob, note);
+ for (e = node->indirect_calls; e; e = e->next_callee)
+ ipa_write_indirect_edge_info (ob, e);
}
/* Srtream in NODE info from IB. */
@@ -2064,7 +2044,6 @@ ipa_read_node_info (struct lto_input_blo
int k;
struct cgraph_edge *e;
struct bitpack_d *bp;
- int i, note_count;
ipa_initialize_node_params (node);
@@ -2094,10 +2073,8 @@ ipa_read_node_info (struct lto_input_blo
for (k = 0; k < ipa_get_cs_argument_count (args); k++)
ipa_read_jump_function (ib, ipa_get_ith_jump_func (args, k), data_in);
}
-
- note_count = lto_input_uleb128 (ib);
- for (i = 0; i < note_count; i++)
- ipa_read_param_call_note (ib, info);
+ for (e = node->indirect_calls; e; e = e->next_callee)
+ ipa_read_indirect_edge_info (ib, data_in, e);
}
/* Write jump functions for nodes in SET. */
@@ -2222,29 +2199,3 @@ ipa_update_after_lto_read (void)
ipa_set_called_with_variable_arg (IPA_NODE_REF (cs->callee));
}
}
-
-/* Walk param call notes of NODE and set their call statements given the uid
- stored in each note and STMTS which is an array of statements indexed by the
- uid. */
-
-void
-lto_ipa_fixup_call_notes (struct cgraph_node *node, gimple *stmts)
-{
- struct ipa_node_params *info;
- struct ipa_param_call_note *note;
-
- ipa_check_create_node_params ();
- info = IPA_NODE_REF (node);
- note = info->param_calls;
- /* If there are no notes or they have already been fixed up (the same fixup
- is called for both inlining and ipa-cp), there's nothing to do. */
- if (!note || note->stmt)
- return;
-
- do
- {
- note->stmt = stmts[note->lto_stmt_uid];
- note = note->next;
- }
- while (note);
-}
Index: icln/gcc/ipa-cp.c
===================================================================
--- icln.orig/gcc/ipa-cp.c
+++ icln/gcc/ipa-cp.c
@@ -1282,7 +1282,7 @@ ipcp_driver (void)
ipcp_print_profile_data (dump_file);
}
/* Free all IPCP structures. */
- free_all_ipa_structures_after_ipa_cp ();
+ ipa_free_all_structures_after_ipa_cp ();
if (dump_file)
fprintf (dump_file, "\nIPA constant propagation end\n");
return 0;
@@ -1346,7 +1346,7 @@ struct ipa_opt_pass_d pass_ipa_cp =
ipcp_read_summary, /* read_summary */
NULL, /* write_optimization_summary */
NULL, /* read_optimization_summary */
- lto_ipa_fixup_call_notes, /* stmt_fixup */
+ NULL, /* stmt_fixup */
0, /* TODOs */
NULL, /* function_transform */
NULL, /* variable_transform */
Index: icln/gcc/ipa-inline.c
===================================================================
--- icln.orig/gcc/ipa-inline.c
+++ icln/gcc/ipa-inline.c
@@ -1321,6 +1321,8 @@ cgraph_decide_inlining (void)
cgraph_remove_function_insertion_hook (function_insertion_hook_holder);
if (in_lto_p && flag_indirect_inlining)
ipa_update_after_lto_read ();
+ if (flag_indirect_inlining)
+ ipa_create_all_structures_for_iinln ();
max_count = 0;
max_benefit = 0;
@@ -1441,7 +1443,7 @@ cgraph_decide_inlining (void)
/* Free ipa-prop structures if they are no longer needed. */
if (flag_indirect_inlining)
- free_all_ipa_structures_after_iinln ();
+ ipa_free_all_structures_after_iinln ();
if (dump_file)
fprintf (dump_file,
@@ -2137,7 +2139,7 @@ struct ipa_opt_pass_d pass_ipa_inline =
inline_read_summary, /* read_summary */
NULL, /* write_optimization_summary */
NULL, /* read_optimization_summary */
- lto_ipa_fixup_call_notes, /* stmt_fixup */
+ NULL, /* stmt_fixup */
0, /* TODOs */
inline_transform, /* function_transform */
NULL, /* variable_transform */
Index: icln/gcc/cgraphunit.c
===================================================================
--- icln.orig/gcc/cgraphunit.c
+++ icln/gcc/cgraphunit.c
@@ -607,6 +607,24 @@ verify_cgraph_node (struct cgraph_node *
error ("Inline clone is needed");
error_found = true;
}
+ for (e = node->indirect_calls; e; e = e->next_callee)
+ {
+ if (e->aux)
+ {
+ error ("aux field set for indirect edge from %s",
+ identifier_to_locale (cgraph_node_name (e->caller)));
+ error_found = true;
+ }
+ if (!e->indirect_unknown_callee
+ || !e->indirect_info)
+ {
+ error ("An indirect edge from %s is not marked as indirect or has "
+ "associated indirect_info, the corresponding statement is: ",
+ identifier_to_locale (cgraph_node_name (e->caller)));
+ debug_gimple_stmt (e->call_stmt);
+ error_found = true;
+ }
+ }
for (e = node->callers; e; e = e->next_caller)
{
if (e->count < 0)
@@ -759,10 +777,10 @@ verify_cgraph_node (struct cgraph_node *
gsi_next (&gsi))
{
gimple stmt = gsi_stmt (gsi);
- tree decl;
- if (is_gimple_call (stmt) && (decl = gimple_call_fndecl (stmt)))
+ if (is_gimple_call (stmt))
{
struct cgraph_edge *e = cgraph_edge (node, stmt);
+ tree decl = gimple_call_fndecl (stmt);
if (e)
{
if (e->aux)
@@ -771,25 +789,38 @@ verify_cgraph_node (struct cgraph_node *
debug_gimple_stmt (stmt);
error_found = true;
}
- if (e->callee->same_body_alias)
+ if (!e->indirect_unknown_callee)
{
- error ("edge points to same body alias:");
- debug_tree (e->callee->decl);
- error_found = true;
+ if (e->callee->same_body_alias)
+ {
+ error ("edge points to same body alias:");
+ debug_tree (e->callee->decl);
+ error_found = true;
+ }
+ else if (!node->global.inlined_to
+ && !e->callee->global.inlined_to
+ && decl
+ && !clone_of_p (cgraph_node (decl),
+ e->callee))
+ {
+ error ("edge points to wrong declaration:");
+ debug_tree (e->callee->decl);
+ fprintf (stderr," Instead of:");
+ debug_tree (decl);
+ error_found = true;
+ }
}
- else if (!node->global.inlined_to
- && !e->callee->global.inlined_to
- && !clone_of_p (cgraph_node (decl), e->callee))
+ else if (decl)
{
- error ("edge points to wrong declaration:");
- debug_tree (e->callee->decl);
- fprintf (stderr," Instead of:");
- debug_tree (decl);
+ error ("an indirect edge with unknown callee "
+ "corresponding to a call_stmt with "
+ "a known declaration:");
error_found = true;
+ debug_gimple_stmt (e->call_stmt);
}
e->aux = (void *)1;
}
- else
+ else if (decl)
{
error ("missing callgraph edge for call stmt:");
debug_gimple_stmt (stmt);
@@ -805,7 +836,7 @@ verify_cgraph_node (struct cgraph_node *
for (e = node->callees; e; e = e->next_callee)
{
- if (!e->aux && !e->indirect_call)
+ if (!e->aux)
{
error ("edge %s->%s has no corresponding call_stmt",
identifier_to_locale (cgraph_node_name (e->caller)),
@@ -813,6 +844,17 @@ verify_cgraph_node (struct cgraph_node *
debug_gimple_stmt (e->call_stmt);
error_found = true;
}
+ e->aux = 0;
+ }
+ for (e = node->indirect_calls; e; e = e->next_callee)
+ {
+ if (!e->aux)
+ {
+ error ("an indirect edge from %s has no corresponding call_stmt",
+ identifier_to_locale (cgraph_node_name (e->caller)));
+ debug_gimple_stmt (e->call_stmt);
+ error_found = true;
+ }
e->aux = 0;
}
}
Index: icln/gcc/cif-code.def
===================================================================
--- icln.orig/gcc/cif-code.def
+++ icln/gcc/cif-code.def
@@ -84,3 +84,7 @@ DEFCIFCODE(MISMATCHED_ARGUMENTS, N_("mis
/* Call was originally indirect. */
DEFCIFCODE(ORIGINALLY_INDIRECT_CALL,
N_("originally indirect function call not considered for inlining"))
+
+/* Ths edge represents an indirect edge with a yet-undetermined callee . */
+DEFCIFCODE(INDIRECT_UNKNOWN_CALL,
+ N_("indirect function call with a yet undetermined callee"))
Index: icln/gcc/tree-inline.c
===================================================================
--- icln.orig/gcc/tree-inline.c
+++ icln/gcc/tree-inline.c
@@ -1688,9 +1688,8 @@ copy_bb (copy_body_data *id, basic_block
/* Constant propagation on argument done during inlining
may create new direct call. Produce an edge for it. */
if ((!edge
- || (edge->indirect_call
+ || (edge->indirect_inlining_edge
&& id->transform_call_graph_edges == CB_CGE_MOVE_CLONES))
- && is_gimple_call (stmt)
&& (fn = gimple_call_fndecl (stmt)) != NULL)
{
struct cgraph_node *dest = cgraph_node (fn);
@@ -3473,7 +3472,7 @@ get_indirect_callee_fndecl (struct cgrap
struct cgraph_edge *cs;
cs = cgraph_edge (node, stmt);
- if (cs)
+ if (cs && !cs->indirect_unknown_callee)
return cs->callee->decl;
return NULL_TREE;
@@ -3556,7 +3555,7 @@ expand_call_inline (basic_block bb, gimp
/* If this call was originally indirect, we do not want to emit any
inlining related warnings or sorry messages because there are no
guarantees regarding those. */
- if (cg_edge->indirect_call)
+ if (cg_edge->indirect_inlining_edge)
goto egress;
if (lookup_attribute ("always_inline", DECL_ATTRIBUTES (fn))
Index: icln/gcc/lto-cgraph.c
===================================================================
--- icln.orig/gcc/lto-cgraph.c
+++ icln/gcc/lto-cgraph.c
@@ -139,15 +139,21 @@ lto_output_edge (struct lto_simple_outpu
intptr_t ref;
struct bitpack_d *bp;
- lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_edge);
+ if (edge->indirect_unknown_callee)
+ lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_indirect_edge);
+ else
+ lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_edge);
ref = lto_cgraph_encoder_lookup (encoder, edge->caller);
gcc_assert (ref != LCC_NOT_FOUND);
lto_output_sleb128_stream (ob->main_stream, ref);
- ref = lto_cgraph_encoder_lookup (encoder, edge->callee);
- gcc_assert (ref != LCC_NOT_FOUND);
- lto_output_sleb128_stream (ob->main_stream, ref);
+ if (!edge->indirect_unknown_callee)
+ {
+ ref = lto_cgraph_encoder_lookup (encoder, edge->callee);
+ gcc_assert (ref != LCC_NOT_FOUND);
+ lto_output_sleb128_stream (ob->main_stream, ref);
+ }
lto_output_sleb128_stream (ob->main_stream, edge->count);
@@ -157,7 +163,7 @@ lto_output_edge (struct lto_simple_outpu
bp_pack_value (bp, edge->inline_failed, HOST_BITS_PER_INT);
bp_pack_value (bp, edge->frequency, HOST_BITS_PER_INT);
bp_pack_value (bp, edge->loop_nest, 30);
- bp_pack_value (bp, edge->indirect_call, 1);
+ bp_pack_value (bp, edge->indirect_inlining_edge, 1);
bp_pack_value (bp, edge->call_stmt_cannot_inline_p, 1);
bp_pack_value (bp, edge->can_throw_external, 1);
lto_output_bitpack (ob->main_stream, bp);
@@ -399,6 +405,25 @@ add_node_to (lto_cgraph_encoder_t encode
lto_cgraph_encoder_encode (encoder, node);
}
+/* Output all callees or indirect outgoing edges. EDGE must be the first such
+ edge. */
+
+static void
+output_outgoing_cgraph_edges (struct cgraph_edge *edge,
+ struct lto_simple_output_block *ob,
+ lto_cgraph_encoder_t encoder)
+{
+ if (!edge)
+ return;
+
+ /* Output edges in backward direction, so the reconstructed callgraph match
+ and it is easy to associate call sites in the IPA pass summaries. */
+ while (edge->next_callee)
+ edge = edge->next_callee;
+ for (; edge; edge = edge->prev_callee)
+ lto_output_edge (ob, edge, encoder);
+}
+
/* Output the part of the cgraph in SET. */
void
@@ -467,16 +492,8 @@ output_cgraph (cgraph_node_set set)
for (csi = csi_start (set); !csi_end_p (csi); csi_next (&csi))
{
node = csi_node (csi);
- if (node->callees)
- {
- /* Output edges in backward direction, so the reconstructed callgraph
- match and it is easy to associate call sites in the IPA pass summaries. */
- edge = node->callees;
- while (edge->next_callee)
- edge = edge->next_callee;
- for (; edge; edge = edge->prev_callee)
- lto_output_edge (ob, edge, encoder);
- }
+ output_outgoing_cgraph_edges (node->callees, ob, encoder);
+ output_outgoing_cgraph_edges (node->indirect_calls, ob, encoder);
}
lto_output_uleb128_stream (ob->main_stream, 0);
@@ -496,7 +513,6 @@ output_cgraph (cgraph_node_set set)
lto_destroy_simple_output_block (ob);
}
-
/* Overwrite the information in NODE based on FILE_DATA, TAG, FLAGS,
STACK_SIZE, SELF_TIME and SELF_SIZE. This is called either to initialize
NODE or to replace the values in it, for instance because the first
@@ -666,11 +682,14 @@ input_node (struct lto_file_decl_data *f
}
-/* Read an edge from IB. NODES points to a vector of previously read
- nodes for decoding caller and callee of the edge to be read. */
+/* Read an edge from IB. NODES points to a vector of previously read nodes for
+ decoding caller and callee of the edge to be read. If INDIRECT is true, the
+ edge being read is indirect (in the sense that it has
+ indirect_unknown_callee set). */
static void
-input_edge (struct lto_input_block *ib, VEC(cgraph_node_ptr, heap) *nodes)
+input_edge (struct lto_input_block *ib, VEC(cgraph_node_ptr, heap) *nodes,
+ bool indirect)
{
struct cgraph_node *caller, *callee;
struct cgraph_edge *edge;
@@ -686,9 +705,14 @@ input_edge (struct lto_input_block *ib,
if (caller == NULL || caller->decl == NULL_TREE)
internal_error ("bytecode stream: no caller found while reading edge");
- callee = VEC_index (cgraph_node_ptr, nodes, lto_input_sleb128 (ib));
- if (callee == NULL || callee->decl == NULL_TREE)
- internal_error ("bytecode stream: no callee found while reading edge");
+ if (!indirect)
+ {
+ callee = VEC_index (cgraph_node_ptr, nodes, lto_input_sleb128 (ib));
+ if (callee == NULL || callee->decl == NULL_TREE)
+ internal_error ("bytecode stream: no callee found while reading edge");
+ }
+ else
+ callee = NULL;
count = (gcov_type) lto_input_sleb128 (ib);
@@ -706,10 +730,14 @@ input_edge (struct lto_input_block *ib,
|| caller_resolution == LDPR_PREEMPTED_IR)
return;
- edge = cgraph_create_edge (caller, callee, NULL, count, freq, nest);
+ if (indirect)
+ edge = cgraph_create_indirect_edge (caller, NULL, count, freq, nest);
+ else
+ edge = cgraph_create_edge (caller, callee, NULL, count, freq, nest);
+
+ edge->indirect_inlining_edge = bp_unpack_value (bp, 1);
edge->lto_stmt_uid = stmt_id;
edge->inline_failed = inline_failed;
- edge->indirect_call = bp_unpack_value (bp, 1);
edge->call_stmt_cannot_inline_p = bp_unpack_value (bp, 1);
edge->can_throw_external = bp_unpack_value (bp, 1);
bitpack_delete (bp);
@@ -732,7 +760,9 @@ input_cgraph_1 (struct lto_file_decl_dat
while (tag)
{
if (tag == LTO_cgraph_edge)
- input_edge (ib, nodes);
+ input_edge (ib, nodes, false);
+ else if (tag == LTO_cgraph_indirect_edge)
+ input_edge (ib, nodes, true);
else
{
node = input_node (file_data, ib, tag);
Index: icln/gcc/lto-streamer-in.c
===================================================================
--- icln.orig/gcc/lto-streamer-in.c
+++ icln/gcc/lto-streamer-in.c
@@ -1248,6 +1248,8 @@ fixup_call_stmt_edges_1 (struct cgraph_n
struct cgraph_edge *cedge;
for (cedge = node->callees; cedge; cedge = cedge->next_callee)
cedge->call_stmt = stmts[cedge->lto_stmt_uid];
+ for (cedge = node->indirect_calls; cedge; cedge = cedge->next_callee)
+ cedge->call_stmt = stmts[cedge->lto_stmt_uid];
}
/* Fixup call_stmt pointers in NODE and all clones. */
Index: icln/gcc/testsuite/gcc.dg/lto/20091209-1_0.c
===================================================================
--- /dev/null
+++ icln/gcc/testsuite/gcc.dg/lto/20091209-1_0.c
@@ -0,0 +1,23 @@
+/* Stream an indirect edge in and out. */
+
+/* { dg-lto-do link } */
+/* { dg-lto-options {{ -O3 -fno-early-inlining -flto }} } */
+
+volatile int something;
+
+static void hooray ()
+{
+ something = 1;
+}
+
+static void hiphip (void (*f)())
+{
+ something = 2;
+ f ();
+}
+
+int main (int argc, int *argv[])
+{
+ hiphip (hooray);
+ return 0;
+}