[Bug tree-optimization/70484] New: Wrong optimization with aliasing and access via char

["ch3root at openwall dot com" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70484

            Bug ID: 70484
           Summary: Wrong optimization with aliasing and access via char
           Product: gcc
           Version: 6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: ch3root at openwall dot com
  Target Milestone: ---

The function:

int f(int *pi, long *pl)
{
  *pi = 1;             // (1)
  *pl = 0;             // (2)
  return *(char *)pi;  // (3)
}

is optimized (with -O2) to always return 1:

f:
        movl    $1, (%rdi)
        movl    $1, %eax
        movq    $0, (%rsi)
        ret

This is wrong if pi and pl both point to the same allocated block. In this case
the function should return 0.

The first impression could be that it's invalid to call this function with
equal arguments as it violates strict aliasing rules. This is wrong.

Suppose the function is called like this:

  void *p = malloc(sizeof(long));
  int result = f(p, p);

Then (1) is clearly Ok.

(2) is fine too because allocated memory can be repurposed freely. C11, 6.5p6,
reads: "If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue
becomes the effective type of the object for that access and for subsequent
accesses that do not modify the stored value."

(3) is fine according to 6.5p7 because a character type can be used to access
anything.

Full example with a function:

----------------------------------------------------------------------
extern void *malloc (__SIZE_TYPE__);
extern void abort (void);

__attribute__((noinline,noclone))
int f(int *pi, long *pl)
{
  *pi = 1;
  *pl = 0;
  return *(char *)pi;
}

int main()
{
  void *p = malloc(sizeof(long));
  if (f(p, p) != 0)
    abort();
}
----------------------------------------------------------------------

Full example with volatile:

----------------------------------------------------------------------
extern void *malloc (__SIZE_TYPE__);
extern void abort (void);

int main()
{
  void *volatile p = malloc(sizeof(long));
  int *pi = p;
  long *pl = p;
  *pi = 1;
  *pl = 0;
  if (*(char *)pi != 0)
    abort();
}
----------------------------------------------------------------------

Tested on gcc 6.0.0 20160331. According to https://gcc.godbolt.org/ the bug is
present since at least 4.4.7.