[Bug tree-optimization/56572] GCC generates non-optimal transactional code when inlining nested transaction.

[rth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56572

--- Comment #5 from Richard Henderson <rth at gcc dot gnu.org> ---
Yes, removing nested transactions (when possible) is worth doing.

The problem is not just with ipa_inline, but also with early_inline.
Indeed, for an example this small, I would have expected early_inline
to have already nested f into g between lower_tm and ipa_tm.

I think I'd always intended that ipa_tm delete nested transations as
it saw them, since _even if_ f isn't inlined into g, the transational
clone of f (which g should call) also should not have the nested
transaction.  I.e. inside a transactional clone we know we must be
within an outer transaction.

As a sort of pass-within-a-pass, we're already walking the transaction
tree, and the blocks therein, in order to transform calls.  I would
think it would be easy enough to add a similar pass-within-a-pass to
walk the transaction tree and delete nested transactions.  Hopefully
we could do this before actually creating the uninstrumented path.

I think moving the creation of the uninstrumented path after ipa_inline
is probably a mistake.  The inliner makes decisions e.g. based on the
number of calls to a function.  If we double the number of calls with
the creation of the uninst path, that could significantly change the
decisions that ipa_inline would make.


r~