[Bug tree-optimization/78586] [7 Regression] Wrong code caused by printf-return-value

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78586

--- Comment #10 from Martin Sebor <msebor at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #8)
> The far more important thing is that the handling of different argtype and
> dirtype really doesn't work.  Consider:
> 
> __attribute__((noinline, noclone)) int
> foo (int x)
> {
>   if (x < 4096 + 8 || x >= 4096 + 256 + 8)
>     return -1;
>   char buf[5];
>   return __builtin_sprintf (buf, "%hhd", x + 1);
> }
> 
> int
> main ()
> {
>   if (__SCHAR_MAX__ != 127 || __CHAR_BIT__ != 8 || __SIZEOF_INT__ != 4)
>     return 0;
>   if (foo (4095 + 9) != 1
>       || foo (4095 + 32) != 2
>       || foo (4095 + 127) != 3
>       || foo (4095 + 128) != 4
>       || foo (4095 + 240) != 3
>       || foo (4095 + 248) != 2
>       || foo (4095 + 255) != 2
>       || foo (4095 + 256) != 1)
>     __builtin_abort ();
>   return 0;
> }
> 
> We have VR_RANGE for the SSA_NAME:
>   # RANGE [4105, 4360] NONZERO 4607
>   _3 = x_6(D) + 1;
> and because both 4105 and 4360 are positive, we use argmin 4105 and argmax
> 4360 and recurse with those, and the recursive call for INTEGER_CST folds
> the integer to dirtype (signed char here), which yields 9 and 8 and thus we
> assume that it must be exactly a single char.  But that is not true, in this
> case the range of (signed char) _3 is actually [ -128, 127 ], so all values
> of dirtype.  In other cases it could be a subset of the values, but still
> the boundary values wouldn't be the boundary values of the range in the
> larger type.

Yes, this is a bug.  Thanks for the test case!  Let me look into fixing it.

> Another thing is, for the non-VR_RANGE case, it sets res.argmin and
> res.argmax always to the argmin and argmax values.  But those values aren't
> necessarily the boundaries of actual value range, those are just some values
> where we assume it will emit minimum or maximum number of characters.  So, I
> think if typeprec <= argprec, we shouldn't set res.arg{min,max} at all,
> otherwise we should always set it to the minimum and maximum value of the
> argtype.

I'll have to spend some time on this to understand what the problem is so I can
test it.