ragen-op-float: Fix up float_binary_op_range_finish [PR107668]

The following testcase ICEs, because when !HONOR_NANS but
HONOR_SIGNED_ZEROS, if we see
lhs = op1 * op2;
and know that lhs is [-0.0, 0.0] and op2 is [0.0, 0.0], the
division of these two yields UNDEFINED and clear_nan () on it
fails an assert.  With HONOR_NANS it would actually result in
a known NAN, but when NANs aren't honored, we clear the NAN bits.
Now, for the above case we actually don't know anything about
the op1 range (except that it isn't a NAN/INF because of
!HONOR_NANS !HONOR_INFINITIES), so I think the best is just
to return VARYING for the case we get UNDEFINED as well.

If we want, the op[12]_range methods perhaps can handle the
corner cases earlier separately, say for
lhs [0.0, 0.0] and op2 [0.0, 0.0] when HONOR_SIGNED_ZEROS this
would be just [0.0, MAX].

2022-11-16  Jakub Jelinek  <jakub@redhat.com>

	PR tree-optimization/107668
	* range-op-float.cc (float_binary_op_range_finish): Set VARYING
	also when r is UNDEFINED.

	* gcc.dg/ubsan/pr107668.c: New test.

diff --git a/gcc/range-op-float.cc b/gcc/range-op-float.cc
index 53a0928c6aa9193c0bdd08433e11e679f3d40be5..0c4ec8dd1782b091106a0d858ebe75a3b5b5fb03 100644 (file)
--- a/gcc/range-op-float.cc
+++ b/gcc/range-op-float.cc
@@ -1891,8 +1891,9 @@ float_binary_op_range_finish (bool ret, frange &r, tree type,
   // or the reverse operation introduced a known NAN.
   // Say for lhs = op1 * op2 if lhs is [-0, +0] and op2 is too,
   // 0 / 0 is known NAN.  Just punt in that case.
+  // If NANs aren't honored, we get for 0 / 0 UNDEFINED, so punt as well.
   // Or if lhs is a known NAN, we also don't know anything.
-  if (r.known_isnan () || lhs.known_isnan ())
+  if (r.known_isnan () || lhs.known_isnan () || r.undefined_p ())
     {
       r.set_varying (type);
       return true;

diff --git a/gcc/testsuite/gcc.dg/ubsan/pr107668.c b/gcc/testsuite/gcc.dg/ubsan/pr107668.c
new file mode 100644 (file)
index 0000000..e94d6cd
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ubsan/pr107668.c
@@ -0,0 +1,12 @@
+/* PR tree-optimization/107668 */
+/* { dg-do compile } */
+/* { dg-options "-ffast-math -fno-associative-math -fsanitize=float-cast-overflow -fno-guess-branch-probability -fsigned-zeros" } */
+
+_Complex int c;
+int i;
+
+void
+foo (void)
+{
+  c /= (_Complex) i;
+}