C++ and inlining

[Richard Guenther]

It seems the topic has to be raised again.  The problem is the same as
discussed during the last discussion - basically the used ought to have
more control over what gets inlined (or not).  I want to focus on _hints_
to the compiler, not forcing it, as that may be impossible in some
circumstances.

So, in my case there is CFD using Expression Templates and, frankly, gcc
is lousy in optimizing the math kernel loops due to not doing enough
inlining inside the loops.

I proposed a function attribute, __attribute__((leafify)), to hint gcc
to inline all calls inside a function (see
http://gcc.gnu.org/ml/gcc-patches/2004-05/msg00187.html, pinged some times
already, but still not reviewed).  Analogous to this one may want the
ability to hint inlining at the callsite like other compilers allow
using pragmas like

#pragma inline
foo();

which would hint the compiler at inlining foo at this particular
call-site, or

#pragma inline complete
bar();

which would be a leafified expansion of bar at this particular call-site.

Of course, inline hinting per function is already available using the
C/C++ inline keyword.

I guess, without further discussion about adding more inlining hinting
capabilities to the compiler, the leafify patch won't get ever reviewed
and considered.

The fact is, with the per-call cgraph infrastructure we have now it is
easy to implement the above in a non-intrusive manner.  For reference you
find the cgraph part of the leafify attribute implementation below (the
other parts are of course documentation and registering of the function
attribute).

For performance comparisons you can look at the gnuplot-charts at
http://www.tat.physik.uni-tuebingen.de/~rguenth/gcc/monitor-summary.html
In short (ia64), numbers improve from ~8s to ~4s for mainline, and from
~8s (note - same as mainline, loop optimizers have no effect on
non-leafified loops) to ~2.8s on lno.

Thanks for your attention,
Richard.


Index: cgraphunit.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/cgraphunit.c,v
retrieving revision 1.66
diff -u -p -c -3 -r1.66 cgraphunit.c
*** cgraphunit.c	20 Jun 2004 08:34:44 -0000	1.66
--- cgraphunit.c	24 Jun 2004 14:48:33 -0000
*************** cgraph_decide_inlining_of_small_function
*** 1437,1442 ****
--- 1437,1503 ----
    free (heap_node);
  }

+ /* Find callgraph nodes closing a circle in the graph.  The
+    resulting hashtab can be used to avoid walking the circles.
+    Uses the cgraph nodes ->aux field which needs to be zero
+    before and will be zero after operation.  */
+
+ static void
+ cgraph_find_cycles (struct cgraph_node *node, htab_t cycles)
+ {
+   struct cgraph_edge *e;
+
+   if (node->aux)
+     {
+       void **slot;
+       slot = htab_find_slot (cycles, node, INSERT);
+       if (!*slot)
+ 	{
+ 	  if (cgraph_dump_file)
+ 	    fprintf (cgraph_dump_file, "Cycle contains %s\n", cgraph_node_name (node));
+ 	  *slot = node;
+ 	}
+       return;
+     }
+
+   node->aux = node;
+   for (e = node->callees; e; e = e->next_callee)
+     {
+        cgraph_find_cycles (e->callee, cycles);
+     }
+   node->aux = 0;
+ }
+
+ /* Leafify the cgraph node.  We have to be careful in recursing
+    as to not run endlessly in circles of the callgraph.
+    We do so by using a hashtab of cycle entering nodes as generated
+    by cgraph_find_cycles.  */
+
+ static void
+ cgraph_leafify_node (struct cgraph_node *node, htab_t cycles)
+ {
+   struct cgraph_edge *e;
+
+   for (e = node->callees; e; e = e->next_callee)
+     {
+       /* Inline call, if possible, and recurse.  Be sure we are not
+ 	 entering callgraph circles here.  */
+       if (e->inline_failed
+ 	  && e->callee->local.inlinable
+ 	  && !cgraph_recursive_inlining_p (node, e->callee,
+ 				  	   &e->inline_failed)
+ 	  && !htab_find (cycles, e->callee))
+ 	{
+ 	  if (cgraph_dump_file)
+     	    fprintf (cgraph_dump_file, " inlining %s", cgraph_node_name (e->callee));
+           cgraph_mark_inline_edge (e);
+ 	  cgraph_leafify_node (e->callee, cycles);
+ 	}
+       else if (cgraph_dump_file)
+ 	fprintf (cgraph_dump_file, " !inlining %s", cgraph_node_name (e->callee));
+     }
+ }
+
  /* Decide on the inlining.  We do so in the topological order to avoid
     expenses on updating data structures.  */

*************** cgraph_decide_inlining (void)
*** 1475,1480 ****
--- 1536,1559 ----

        node = order[i];

+       /* Handle nodes to be leafified, but don't update overall unit size.  */
+       if (lookup_attribute ("leafify", DECL_ATTRIBUTES (node->decl)) != NULL)
+         {
+ 	  int old_overall_insns = overall_insns;
+ 	  htab_t cycles;
+   	  if (cgraph_dump_file)
+     	    fprintf (cgraph_dump_file,
+ 	     	     "Leafifying %s\n", cgraph_node_name (node));
+ 	  cycles = htab_create (7, htab_hash_pointer, htab_eq_pointer, NULL);
+ 	  cgraph_find_cycles (node, cycles);
+ 	  cgraph_leafify_node (node, cycles);
+ 	  htab_delete (cycles);
+ 	  overall_insns = old_overall_insns;
+ 	  /* We don't need to consider always_inline functions inside the leafified
+ 	     function anymore.  */
+ 	  continue;
+         }
+
        if (!node->local.disregard_inline_limits)
  	continue;
        if (cgraph_dump_file)