[Bug c/56917] New: -ftrapv detects a overflow wrongly.

[akr at fsij dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56917

             Bug #: 56917
           Summary: -ftrapv detects a overflow wrongly.
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: akr@fsij.org


Created attachment 29855
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29855
steps to reproduce the -ftrapv problem

I found that following program aborts with -ftrapv with gcc-4.8.0 on
Debian GNU/Linux (x86_64).

  % cat tst.c
  unsigned long ul = 0x8000000000000000UL;

  int main(int argc, char *argv[])
  {
    long l;
    l = (-(long)(ul-1))-1;
    return 0;
  }
  % /home/src/gcc/bin/gcc -g -ftrapv -Wall tst.c
  tst.c: In function 'main':
  tst.c:5:8: warning: variable 'l' set but not used [-Wunused-but-set-variable]
     long l;
      ^
  % ./a.out 
  zsh: abort (core dumped)  ./a.out

I think it should not abort on x86_64.

Since ul is LONG_MAX+1 (in indefinite precision), ul-1 is LONG_MAX which is
representable in long.
So the cast, (long)(ul-1), doesn't change the value.
Its negation, -(long)(ul-1), is -LONG_MAX which is also representable in long.
Its predecessor, (-(long)(ul-1))-1, is -LONG_MAX-1 = LONG_MIN which is also
representable in long.

gcc-ftrapv-problem-log.txt contains the log with gcc -v.