[Bug tree-optimization/96007] New: -O2 miscompiles memcmp an object with a string literal containing '\0'

[jerryfromearth at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96007

            Bug ID: 96007
           Summary: -O2 miscompiles memcmp an object with a string literal
                    containing '\0'
           Product: gcc
           Version: 9.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jerryfromearth at gmail dot com
  Target Milestone: ---

Hi,

While comparing an object to a literal string that contains "\0" (e.g.
memcmp((&u128), "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16)), -O2 produces
incorrect result, while -O1 and -O0 produces correct result.

This affects at least gcc 9.2, 9.3 and 10.1.
gcc 9.1 is fine.

```
$ cat test.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct tagMYUINT128
{
  unsigned long u64High;
  unsigned long u64Low;
} MYUINT128, *PMYUINT128;

int main(int argc, char **argv) {
  MYUINT128 u128 = {0};
  char arr[] = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0";
  printf("All results are expected to be non-zero.\n");

  // Test A, u128={0x0, 0x1234}
  printf("Test A. u128={0x0, 0x1234}\n");
  u128.u64High = 0x0;
  u128.u64Low  = 0x1234;

  // Comparing to a literal string is broken.
  printf("memcmp1 result=%d\n", memcmp((&u128),
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16));
  printf("memcmp2 result=%d\n", __builtin_memcmp((&u128),
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16));

  // Comparing to a array pointer works.
  printf("memcmp3 result=%d\n", memcmp((&u128), arr, 16));
  printf("memcmp4 result=%d\n", __builtin_memcmp((&u128), arr, 16));

  return 0;
}

$ gcc -O2 -o test test.c && ./test
All results are expected to be non-zero.
Test A. u128={0x0, 0x1234}
memcmp1 result=0
memcmp2 result=0
memcmp3 result=52
memcmp4 result=52

$ gcc -O1 -o test test.c && ./test
All results are expected to be non-zero.
Test A. u128={0x0, 0x1234}
memcmp1 result=52
memcmp2 result=52
memcmp3 result=52
memcmp4 result=52
```

godbolt link: https://godbolt.org/z/9M3L8h (it contains more tests)

I looked into the list of bugs fixed in 9.2 and found this probably relevant
issue: 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90892

Cheers,
Jiarui Hong