[Bug rtl-optimization/92281] Inconsistent canonicalization of (minus (minus A B) C)

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92281

--- Comment #4 from Segher Boessenkool <segher at gcc dot gnu.org> ---
(In reply to Richard Earnshaw from comment #2)
> Yes, but since 
>   (A - B) - C = A - B - C = A - C - B = (A - C) - B
> we can clearly swap the order of the two RHS operands here.

My intent was to show the two rtx shapes, and that neither is a defined
canonical form.

>  This would be
> a special rule similar to the rules that we have that rewrite 
>   A - (B + C)
> as
>   (A - B) - C.

That isn't a canonical form, either!  Not according to the documentation,
anyway.

> My suggestion would be that we should have a rule here that re-orders things
> so
> that B is the most 'complex' operation and C the simplest, using the normal
> precedence ordering (complex > REG > CONST).

But minus isn't commutative, and reordering with minus introduces negs which
is wrong (it is canonical to *remove* such negs).

> > What targets would it break, and how?
> 
> Hard to tell, until we try it.  Mostly the 'breakage' would be some combine
> patterns might no-longer match if the target only had one and the ordering
> were not canonical (leading to some missed optimizations).  On targets that
> have both orderings, some patterns might become redundant and never match
> unless directly generated by the back-end.

The breakage will be that many targets optimise worse than they did before.
And this is non-obvious to detect, usually.

> > What makes combine come up with something else for these two cases?
> 
> Sorry, I don't understand what you're asking here?  Why does it produce
> these two separate canoncializations in one compilation?  I've no idea,
> hence the bug report.

A lot of what combine does is *not* canonicalisation.  But combine comes up
with only one result for every attempted combination, making that a kind-of
de-facto canonicalisation.

And yes, that is what I asked: in both cases it combined the same insn with
a simple pseudo move, in both cases on the RHS in that insn.  And it came
up with different results.

This may be unavoidable, or combine does something weird, or the RTL that
combine started with was non-canonical or unexpected in some other way, etc.

So I'd like to know where the difference was introduced.  Was it in combine
at all, to start with?  It can be in simplify-rtx as well for example.