[Bug tree-optimization/77899] New: incorrect VR_RANGE for a signed char function argument

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77899

            Bug ID: 77899
           Summary: incorrect VR_RANGE for a signed char function argument
           Product: gcc
           Version: 7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: msebor at gcc dot gnu.org
  Target Milestone: ---

Similar to bug 77898 but with different and more wide-ranging symptoms (it
persists past even the last invocation of the VRP pass), in the following
program the range of the variable i is bounded by its type to [-128, 127]. 
Thus the result of the (p += i) expression is guaranteed to be in the range [d
+ 2, d + 257] and the if expression always false.  However, GCC represents the
range of the signed char offset by the VR_ANTI_RANGE ~[128,
18446744073709551487] and fails to optimize the if statement away.

This is another case that popped up during the testing of my patch for bug
77608 where it affects the result of __builtin_object_size.

$ cat x.c && /build/gcc-trunk-svn/gcc/xgcc -B /build/gcc-trunk-svn/gcc -O2 -S
-Wall -Wextra -Wpedantic -fdump-tree-vrp=/dev/stdout x.c
void f (signed char i)
{
  char d [260];

  const char *p = &d[130];

  p += i;

  if (p < d + 2 || d + 257 < p)
    __builtin_abort ();
}

;; Function f (f, funcdef_no=0, decl_uid=1791, cgraph_uid=0, symbol_order=0)

;; 1 loops found
;;
;; Loop 0
;;  header 0, latch 1
;;  depth 0, outer -1
;;  nodes: 0 1 2 3 4 5
;; 2 succs { 4 3 }
;; 3 succs { 4 5 }
;; 4 succs { }
;; 5 succs { 1 }

SSA replacement table
N_i -> { O_1 ... O_j } means that N_i replaces O_1, ..., O_j

p_7 -> { p_3 }
Incremental SSA update started at block: 2
Number of blocks in CFG: 6
Number of blocks to update: 2 ( 33%)



Value ranges after VRP:

_1: ~[128, 18446744073709551487]
i_2(D): VARYING
p_3: VARYING
p_7: VARYING


f (signed char i)
{
  const char * p;
  char d[260];
  sizetype _1;

  <bb 2>:
  _1 = (sizetype) i_2(D);
  p_3 = &d[130] + _1;
  if (&MEM[(void *)&d + 2B] > p_3)
    goto <bb 4>;
  else
    goto <bb 3>;

  <bb 3>:
  if (&MEM[(void *)&d + 257B] < p_3)
    goto <bb 4>;
  else
    goto <bb 5>;

  <bb 4>:
  __builtin_abort ();

  <bb 5>:
  d ={v} {CLOBBER};
  return;

}



;; Function f (f, funcdef_no=0, decl_uid=1791, cgraph_uid=0, symbol_order=0)

;; 1 loops found
;;
;; Loop 0
;;  header 0, latch 1
;;  depth 0, outer -1
;;  nodes: 0 1 2 3 4
;; 2 succs { 3 4 }
;; 3 succs { }
;; 4 succs { 1 }

Value ranges after VRP:

_1: ~[128, 18446744073709551487]
i_2(D): VARYING
p_3: VARYING
_7: [0, +INF]
_8: [0, +INF]
_9: [0, +INF]


f (signed char i)
{
  const char * p;
  char d[260];
  sizetype _1;
  _Bool _7;
  _Bool _8;
  _Bool _9;

  <bb 2>:
  _1 = (sizetype) i_2(D);
  p_3 = &d[130] + _1;
  _7 = &MEM[(void *)&d + 257B] < p_3;
  _8 = &MEM[(void *)&d + 2B] > p_3;
  _9 = _8 | _7;
  if (_9 != 0)
    goto <bb 3>;
  else
    goto <bb 4>;

  <bb 3>:
  __builtin_abort ();

  <bb 4>:
  d ={v} {CLOBBER};
  return;

}