[Bug tree-optimization/124560] `min(z - y, x - y)` -> `min(z, x) - y`

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=124560

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Jeevitha from comment #2)
> I am trying to learn GCC tree optimizations and started with this issue. I
> have a patch for it, but currently it only handles unsigned types. Let me
> explain why I excluded signed types.
> 
> The bug report mentions that the transformation is valid for both signed and
> unsigned types, but I believe it is not safe for signed types, even under
> TYPE_OVERFLOW_UNDEFINED. Could you please confirm whether this assumption is
> correct?

No, it is the exact opposite. Going from what was undefined to defined is ok. 
What is problematic is going from being defined to being undefined.

So I think the check in this case should be just `!TYPE_OVERFLOW_SANITIZED
(type)`. As you still want to sanitize this case.

Thanks,
Andrew Pinski

> 
> int x = INT_MAX, y = -1, z = 0;
> 
> /* src */
> x - y = INT_MAX - (-1) = INT_MAX + 1  /* signed overflow - UB */
> z - y = 0 - (-1)       = 1
> min(UB, 1)             = UB
> 
> /* tgt */
> min(INT_MAX, 0)        = 0
> 0 - (-1)               = 1            /* no overflow - defined */
> 
> In this case, the original expression results in undefined behavior, while
> the transformed version produces a defined result. 
> 
> Therefore, I am restricting the transformation to unsigned types only.
> diff --git a/gcc/match.pd b/gcc/match.pd
> index 7b652afb43d..f59eb40bb97 100644
> --- a/gcc/match.pd
> +++ b/gcc/match.pd
> @@ -7054,6 +7054,14 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT)
>            && integer_nonzerop (fold_build2 (GE_EXPR, boolean_type_node, @3,
> @4)))
>        (max @2 @4))))))
> 
> +/* Optimize min(x - y, z - y) -> min(x, z) - y
> +            max(x - y, z - y) -> max(x, z) - y  */
> +(for minmax (min max)
> +  (simplify
> +    (minmax (minus @0 @1) (minus @2 @1))
> +    (if (TYPE_UNSIGNED (type))
> +      (minus (minmax @0 @2) @1))))
> +
>  /* Optimize (((signed)a CMP 0) ? max<a,CST2> : CST3 */
>  (for cmp    (lt  ge)
>       minmax (min max)