[Bug tree-optimization/56572] GCC generates non-optimal transactional code when inlining nested transaction.

[aldyh at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56572

--- Comment #7 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
Well, we could tweak the inliner cost model, thus causing the early inliner to
inline the nested transactions earlier.  With the attached patch, f() gets
inlined into g() early enough so that pass_ipa_tm sees the nested transaction
before the uninstrumented path has been added.

So with this patch, we could twiddle IPA tm to remove nested transactions
without it handling the unnecessarily complex uninstrumented/instrumented code
paths.  However, it seems to me that the proper IPA inliner may add other
(possibly nested) transactions, which will have us back to square one by tmmark
time (??).

This sounds like a good start, and if we find that the latter IPA inliner
creates other unnecessary nested transactions (in other test cases), we could
bite the bullet and do the whole thing in tmmark, handling both uninstrumented
and instrumented code paths.

Waddaya think?

diff --git a/gcc/tree-inline.c b/gcc/tree-inline.c
index f42ade02..1cb9e58 100644
--- a/gcc/tree-inline.c
+++ b/gcc/tree-inline.c
@@ -3969,7 +3969,7 @@ init_inline_once (void)
   eni_size_weights.target_builtin_call_cost = 1;
   eni_size_weights.div_mod_cost = 1;
   eni_size_weights.omp_cost = 40;
-  eni_size_weights.tm_cost = 10;
+  eni_size_weights.tm_cost = 2;
   eni_size_weights.time_based = false;
   eni_size_weights.return_cost = 1;