tree_binary_nonnegative_warnv_p for RDIV_EXPR does RECURSE (op0) && RECURSE (op1), but that doesn't work so well when the denominator can be 0. I guess it is still ok when finite-math-only (or no-nans and no-signed-zeros maybe?), or when we can prove that the denominator is non-zero.

Applying pattern match.pd:832, gimple-match.c:11245
Match-and-simplified ABS_EXPR <val_5> to val_5

(In reply to Andrew Pinski from comment #2)
> Applying pattern match.pd:832, gimple-match.c:11245
> Match-and-simplified ABS_EXPR <val_5> to val_5

Which is:
(simplify
 (abs tree_expr_nonnegative_p@0)
 @0)

    case MAX_EXPR:
      return RECURSE (op0) || RECURSE (op1);

This is not true if one is a NAN.

(In reply to Andrew Pinski from comment #4)
>     case MAX_EXPR:
>       return RECURSE (op0) || RECURSE (op1);
> 
> This is not true if one is a NAN.

And the reason why it is not true for NAN is simple:
If one of the two arguments is NaN, the value of the other argument is returned

What does tree_expr_nonnegative_p call non-negative? A natural definition would exclude NaN, but for REAL_CST we just return ! REAL_VALUE_NEGATIVE.

(In reply to Marc Glisse from comment #6)
> What does tree_expr_nonnegative_p call non-negative? A natural definition
> would exclude NaN, but for REAL_CST we just return ! REAL_VALUE_NEGATIVE.

Is there sth like -NaN?!

Anyway, the bug is clearly in tree_expr_nonnegative_p and siblings.  Btw, tree_binary_nonzero_warnv_p looks similarly suspicious.

I guess first of all

diff --git a/gcc/fold-const.c b/gcc/fold-const.c
index faa184a2bbd..cb3de26f9f1 100644
--- a/gcc/fold-const.c
+++ b/gcc/fold-const.c
@@ -12949,7 +12949,8 @@ tree_single_nonnegative_warnv_p (tree t, bool *strict_overflow_p, int depth)
       return tree_int_cst_sgn (t) >= 0;
 
     case REAL_CST:
-      return ! REAL_VALUE_NEGATIVE (TREE_REAL_CST (t));
+      return (! REAL_VALUE_ISNAN (TREE_REAL_CST (t))
+             && ! REAL_VALUE_NEGATIVE (TREE_REAL_CST (t)));
 
     case FIXED_CST:
       return ! FIXED_VALUE_NEGATIVE (TREE_FIXED_CST (t));

but then how do we want to deal with MAX_EXPR (and MIN_EXPR which after the
above could see an optimization for min (NaN, x)?).  We'd have to special
case real-type MAX_EXPR (what about real complex/vector?  tree_single_nonnegative_warnv_p doesn't handle complex/vector constants
but at least we should be correct in the uplevel treatment).

> Is there sth like -NaN?!

signbit can tell you.

On Thu, 17 May 2018, schwab@linux-m68k.org wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85811
> 
> --- Comment #8 from Andreas Schwab <schwab@linux-m68k.org> ---
> > Is there sth like -NaN?!
> 
> signbit can tell you.

True.  I guess that also asks for the guarantee on the operand
for tree_single_nonnegative_warnv_p returning true.  Does it
guarantee that OP >= 0 compares as true?  Or merely !(OP < 0)?
So does it make sense for the function to return true for +NaN
or should we err on the side of caution and not do that?

The master branch has been updated by Jeff Law <law@gcc.gnu.org>:

https://gcc.gnu.org/g:1be4878116a2be82552bd59c3c1c9adcac3d106b

commit r11-5152-g1be4878116a2be82552bd59c3c1c9adcac3d106b
Author: Roger Sayle <roger@nextmovesoftware.com>
Date:   Wed Nov 18 15:09:46 2020 -0700

    Fix middle-end/85811: Introduce tree_expr_maybe_non_p et al.
    
    The motivation for this patch is PR middle-end/85811, a wrong-code
    regression entitled "Invalid optimization with fmax, fabs and nan".
    The optimization involves assuming max(x,y) is non-negative if (say)
    y is non-negative, i.e. max(x,2.0).  Unfortunately, this is an invalid
    assumption in the presence of NaNs.  Hence max(x,+qNaN), with IEEE fmax
    semantics will always return x even though the qNaN is non-negative.
    Worse, max(x,2.0) may return a negative value if x is -sNaN.
    
    I'll quote Joseph Myers (many thanks) who describes things clearly as:
    > (a) When both arguments are NaNs, the return value should be a qNaN,
    > but sometimes it is an sNaN if at least one argument is an sNaN.
    > (b) Under TS 18661-1 semantics, if either argument is an sNaN then the
    > result should be a qNaN (whereas if one argument is a qNaN and the
    > other is not a NaN, the result should be the non-NaN argument).
    > Various implementations treat sNaNs like qNaNs here.
    
    Under this logic, the tree_expr_nonnegative_p for IEEE fmax should be:
    
        CASE_CFN_FMAX:
        CASE_CFN_FMAX_FN:
          /* Usually RECURSE (arg0) || RECURSE (arg1) but NaNs complicate
             things.  In the presence of sNaNs, we're only guaranteed to be
             non-negative if both operands are non-negative.  In the presence
             of qNaNs, we're non-negative if either operand is non-negative
             and can't be a qNaN, or if both operands are non-negative.  */
          if (tree_expr_maybe_signaling_nan_p (arg0) ||
              tree_expr_maybe_signaling_nan_p (arg1))
            return RECURSE (arg0) && RECURSE (arg1);
          return RECURSE (arg0) ? (!tree_expr_maybe_nan_p (arg0)
                                  || RECURSE (arg1))
                                : (RECURSE (arg1)
                                  && !tree_expr_maybe_nan_p (arg1));
    
    Which indeed resolves the wrong code in the PR.  The infrastructure that
    makes this possible are the two new functions tree_expr_maybe_nan_p and
    tree_expr_maybe_signaling_nan_p which test whether a value may potentially
    be a NaN or a signaling NaN respectively.  In fact, this patch adds seven
    new predicates to the middle-end:
    
    bool tree_expr_finite_p (const_tree);
    bool tree_expr_infinite_p (const_tree);
    bool tree_expr_maybe_infinite_p (const_tree);
    bool tree_expr_signaling_nan_p (const_tree);
    bool tree_expr_maybe_signaling_nan_p (const_tree);
    bool tree_expr_nan_p (const_tree);
    bool tree_expr_maybe_nan_p (const_tree);
    
    These functions correspond to the "must" and "may" operators in modal logic,
    and allow us to triage expressions in the middle-end; definitely a NaN,
    definitely not a NaN, and unknown at compile-time, etc.  A prime example of
    the utility of these functions is that a IEEE floating point value promoted
    from an integer type can't be a NaN or infinite.  Hence (double)i+0.0 where
    i is an integer can be simplified to (double)i even with -fsignaling-nans.
    Currently in GCC optimizations are enabled/disabled based on whether the
    expression's type supports NaNs or sNaNs; with these new predicates they
    can be controlled by whether the actual operands may or may not be NaNs.
    
    Having added these extremely useful helper functions to the middle-end,
    I couldn't help by use then in a few places in fold-const.c, builtins.c
    and match.pd.  In the near term, these can/should be used in places
    where the tree optimizers test for HONOR_NANS, HONOR_INFINITIES or
    HONOR_SNANS, or explicitly test whether a REAL_CST is a NaN or Inf.
    In the longer term (I'm not volunteering) these predicates could perhaps
    be hooked into the middle-end's SSA chaining and/or VRP machinery,
    allowing finiteness to propagated around the CFG, much like we
    currently propagate value ranges.
    
    This patch has been tested on x86_64-pc-linux-gnu with a "make bootstrap"
    and "make -k check".
    Ok for mainline?
    
    2020-08-15  Roger Sayle  <roger@nextmovesoftware.com>
    
    gcc/ChangeLog
            PR middle-end/85811
            * fold-const.c (tree_expr_finite_p): New function to test whether
            a tree expression must be finite, i.e. not a FP NaN or infinity.
            (tree_expr_infinite_p):  New function to test whether a tree
            expression must be infinite, i.e. a FP infinity.
            (tree_expr_maybe_infinite_p): New function to test whether a tree
            expression may be infinite, i.e. a FP infinity.
            (tree_expr_signaling_nan_p): New function to test whether a tree
            expression must evaluate to a signaling NaN (sNaN).
            (tree_expr_maybe_signaling_nan_p): New function to test whether a
            tree expression may be a signaling NaN (sNaN).
            (tree_expr_nan_p): New function to test whether a tree expression
            must evaluate to a (quiet or signaling) NaN.
            (tree_expr_maybe_nan_p): New function to test whether a tree
            expression me be a (quiet or signaling) NaN.
    
            (tree_binary_nonnegative_warnv_p) [MAX_EXPR]: In the presence
            of NaNs, MAX_EXPR is only guaranteed to be non-negative, if both
            operands are non-negative.
            (tree_call_nonnegative_warnv_p) [CASE_CFN_FMAX,CASE_CFN_FMAX_FN]:
            In the presence of signaling NaNs, fmax is only guaranteed to be
            non-negative if both operands are negative.  In the presence of
            quiet NaNs, fmax is non-negative if either operand is non-negative
            and not a qNaN, or both operands are non-negative.
    
            * fold-const.h (tree_expr_finite_p, tree_expr_infinite_p,
            tree_expr_maybe_infinite_p, tree_expr_signaling_nan_p,
            tree_expr_maybe_signaling_nan_p, tree_expr_nan_p,
            tree_expr_maybe_nan_p): Prototype new functions here.
    
            * builtins.c (fold_builtin_classify) [BUILT_IN_ISINF]: Fold to
            a constant if argument is known to be (or not to be) an Infinity.
            [BUILT_IN_ISFINITE]: Fold to a constant if argument is known to
            be (or not to be) finite.
            [BUILT_IN_ISNAN]: Fold to a constant if argument is known to be
            (or not to be) a NaN.
            (fold_builtin_fpclassify): Check tree_expr_maybe_infinite_p and
            tree_expr_maybe_nan_p instead of HONOR_INFINITIES and HONOR_NANS
            respectively.
            (fold_builtin_unordered_cmp): Fold UNORDERED_EXPR to a constant
            when its arguments are known to be (or not be) NaNs.  Check
            tree_expr_maybe_nan_p instead of HONOR_NANS when choosing between
            unordered and regular forms of comparison operators.
    
            * match.pd (ordered(x,y)->true/false): Constant fold ORDERED_EXPR
            if its operands are known to be (or not to be) NaNs.
            (unordered(x,y)->true/false): Constant fold UNORDERED_EXPR if its
            operands are known to be (or not to be) NaNs.
            (sqrt(x)*sqrt(x)->x): Check tree_expr_maybe_signaling_nan_p instead
            of HONOR_SNANS.
    
    gcc/testsuite/ChangeLog
            PR middle-end/85811
            * gcc.dg/pr85811.c: New test.
            * gcc.dg/fold-isfinite-1.c: New test.
            * gcc.dg/fold-isfinite-2.c: New test.
            * gcc.dg/fold-isinf-1.c: New test.
            * gcc.dg/fold-isinf-2.c: New test.
            * gcc.dg/fold-isnan-1.c: New test.
            * gcc.dg/fold-isnan-2.c: New test.

Are there backports pending?

Possibly could be backported even if not a regression but I guess the wrong-code is really restricted to cases we don't hit in the wild.

That said, not objecting if anybody wants to backport to GCC 10.

Fixed on mainline.  [Sorry I've only just noticed this hadn't been closed].