[Bug tree-optimization/84933] [8 Regression] ICE in set_value_range, at tree-vrp.c:288 since r257852

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84933

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
            Version|unknown                     |8.0.1
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
      else if (is_min
               /* As a special exception preserve non-null ranges.  */
               && !(TYPE_UNSIGNED (TREE_TYPE (min))
                    && integer_zerop (max)))
        {
          tree one = build_int_cst (TREE_TYPE (max), 1);
          min = int_const_binop (PLUS_EXPR, max, one);
          max = vrp_val_max (TREE_TYPE (max));
          t = VR_RANGE;
        }

the issue is we end up with ~[0, 2147483647] which we want to canonicalize
to [2147483647 + 1, type-max].  But with -fstrict-enums TYPE_MAX_VALUE
of the enumeral type is 1.

So we have out-of-bound ranges as input if we look at TYPE_MIN/MAX_VALUE
which leads to GIGO.

Not sure what to do here at this point.  I was always uncomfortable with
using TYPE_MIN/MAX_VALUE and given we use wi::min/max_value in a lot of
places already we could do the same in vrp_val_min/max.  Another fix
would be to prune those out-of-bound ranges in both
set_and_canonicalize_value_range and set_value_range.  Or a more
localized fix in set_and_canonicalize_value_range instead of looking
for an exact match of vrp_val_is_min/max check <=/>= here and thus in
this case drop into

      if (is_min && is_max)
        {
          /* We cannot deal with empty ranges, drop to varying.
             ???  This could be VR_UNDEFINED instead.  */
          set_value_range_to_varying (vr);
          return;
        }

we still have the issue of using TYPE_MIN/MAX_VALUE and allowing
out-of-bound ranges at the same time though.

vrp_val_is_min/max has the issue to only work reliably for integral
types as well.  Auditing all users of vrp_val_[is_]{min,max} is
necessary here.

So the simplest fix I can think of is

Index: gcc/tree-vrp.c
===================================================================
--- gcc/tree-vrp.c      (revision 258641)
+++ gcc/tree-vrp.c      (working copy)
@@ -386,8 +386,13 @@ set_and_canonicalize_value_range (value_
   /* Anti-ranges that can be represented as ranges should be so.  */
   if (t == VR_ANTI_RANGE)
     {
-      bool is_min = vrp_val_is_min (min);
-      bool is_max = vrp_val_is_max (max);
+      /* For -fstrict-enums we may receive out-of-range ranges so consider
+         values < -INF and values > INF as -INF/INF as well.  */
+      tree type = TREE_TYPE (min);
+      bool is_min = (INTEGRAL_TYPE_P (type)
+                    && tree_int_cst_compare (min, TYPE_MIN_VALUE (type)) <=
0);
+      bool is_max = (INTEGRAL_TYPE_P (type)
+                    && tree_int_cst_compare (max, TYPE_MAX_VALUE (type)) >=
0);

       if (is_min && is_max)
        {