[Bug tree-optimization/98513] [10/11 Regression] Wrong code with -O3 since r10-2804-gbf05a3bbb58b3558

[amacleod at redhat dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98513

--- Comment #7 from Andrew Macleod <amacleod at redhat dot com> ---
This is in the legacy intersection code 

we have
[-INF, minus_1_29(D) + 2]   intersect  [  -INF + 1, -INF + 2]

 it falls into this block:

else if ((operand_less_p (vr1min, *vr0max) == 1
            || operand_equal_p (vr1min, *vr0max, 0))
           && operand_less_p (*vr0min, vr1min) == 1)
    {
      /* [  (  ]  ) or [  ](  ) */
      if (*vr0type == VR_ANTI_RANGE
          && vr1type == VR_ANTI_RANGE)
        *vr0max = vr1max;
      else if (*vr0type == VR_RANGE
               && vr1type == VR_RANGE)
        *vr0min = vr1min;
      else if (*vr0type == VR_RANGE
               && vr1type == VR_ANTI_RANGE)
        {
          if (TREE_CODE (vr1min) == INTEGER_CST)
  -->       *vr0max = int_const_binop (MINUS_EXPR, vr1min,
                                       build_int_cst (TREE_TYPE (vr1min), 1));
          else
            *vr0max = vr1min;
        }
(gdb)  p operand_less_p (vr1min, *vr0max)
$19 = 1
(gdb) p operand_less_p (*vr0min, vr1min)
$21 = 1

and ends up setting vr0max to (vr1min - 1), which is -INF

and so returns [-INF, -INF]


It seems like it *should* have entered an earlier hunk here maybe?

else if ((maxeq || operand_less_p (vr1max, *vr0max) == 1)
           && (mineq || operand_less_p (*vr0min, vr1min) == 1))
    {
      /* [ (  ) ] or [(  ) ] or [ (  )] */

this looks like the  [ ( ) ] case?  if I interpret this correctly

it fails to enter this block because:


(gdb) p operand_less_p (vr1max, *vr0max)
$22 = -2
which is operand_less_p (-INF + 2, minus_1_29(D) + 2)

so it claims they cannot be compared at compile time, and thus doesn't drop
into this block. 

Im not sure what should be done here... The easiest thing to do is simply punt
when we get a -2 back anywhere... and leave vr0 as it is. thats conservative
and safe.  Im not even sure how best to add those checks in where needed.

Otherwise we'll have to delve into why we got a -2, and eventually maybe
substitute +INF for vr0max...  but really, I think you'd have to do that sort
of check for each of the operand_less_p() calls to be correct,  and figure out
when you want to substitute a +INF or -INF and recalculate the expression.

Although maybe you have a more concise idea of how to handle this.  Perhaps its
more localized than it appears to me at first glance.