[Bug tree-optimization/79800] New: wrong snprintf result range with precision in a narrow negative-positive range

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79800

            Bug ID: 79800
           Summary: wrong snprintf result range with precision in a narrow
                    negative-positive range
           Product: gcc
           Version: 7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: msebor at gcc dot gnu.org
  Target Milestone: ---

As mentioned in the patch for bug 79692
(https://gcc.gnu.org/ml/gcc-patches/2017-02/msg01564.html), the return value of
a call to snprintf with a floating directive, a non-constant floating argument,
and with precision specified by the asterisk whose argument's range includes
both negative and non-negative values (and the upper bound is low), is in the
wrong range.

The following test case shows the problem.  A negative precision must be
treated as if no precision were specified.  The result of formatting 1.23456789
using %a is 0x1.3c0ca4283de1bp+0, which is 20 bytes.  However, the
-fprintf-return-value optimization sets the range of the return value to [6,
15], causing the test program to print "20 < 20" when it should print "20 >=
20" instead.

$ cat z.c && gcc -O2 -Wall -Wextra -Wpedantic z.c && ./a.out 
volatile double x = 1.23456789;

void f (int p)
{
  if (p < -1 || 0 < p)
    p = -1;

  char d[30];
  int n = __builtin_sprintf (d, "%.*a", p, x);
  const char *s = n < 20 ? "< 20" : ">= 20";

  __builtin_printf ("%i %s\n%zu: \"%s\"\n", n, s, __builtin_strlen (d), d);
}

volatile int i;

int main ()
{
  volatile int i = -1;
  f (i);
}

20 < 20
20: "0x1.3c0ca4283de1bp+0"