[Bug tree-optimization/63641] New: Invalid shift leads to incorrect code on 32-bit system

[ian at airs dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63641

            Bug ID: 63641
           Summary: Invalid shift leads to incorrect code on 32-bit system
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: ian at airs dot com

Compile and run this program with -m32 -O2 on an x86 system.

#include <stdio.h>

int f (unsigned char) __attribute__ ((noinline));

int
f (unsigned char b)
{
  if (0x0 <= b && b <= 0x8)
    goto L;
  if (b == 0x0b)
    goto L;
  if (0x0e <= b && b <= 0x1a)
    goto L;
  if (0x1c <= b && b <= 0x1f)
    goto L;
  return 0;
 L:
  return 1;
}

int
main ()
{
  printf ("%d\n", f (' '));
}

The program should print 0.  However, when compiled with -m32 -O2 with current
mainline (revision 216611) it prints 1.

The generated code for f is:

00000000 <f>:
   0:   8b 4c 24 04             mov    0x4(%esp),%ecx
   4:   31 c0                   xor    %eax,%eax
   6:   80 f9 20                cmp    $0x20,%cl
   9:   77 0a                   ja     15 <f+0x15>
   b:   b8 ff c9 ff f7          mov    $0xf7ffc9ff,%eax
  10:   d3 e8                   shr    %cl,%eax
  12:   83 e0 01                and    $0x1,%eax
  15:   f3 c3                   repz ret 

The bug is obvious: when the value in %cl is 0x20, the shr does nothing.  The
ja needs to be a jae.