[Bug target/56875] New: vax target miscompiles short negative constants for 64bit values

[martin at netbsd dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56875

             Bug #: 56875
           Summary: vax target miscompiles short negative constants for
                    64bit values
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: martin@netbsd.org


Created attachment 29823
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29823
Use the D format specifie for ashqs second arg

The documentation for VAX operand formatting codes says:

   D    64-bit immediate operand

and:

/* The purpose of D is to get around a quirk or bug in vax assembler
   whereby -1 in a 64-bit immediate operand means 0x00000000ffffffff,
   which is not a 64-bit minus one.  */

However, the ashq instruction patters do not use this properly.

This small test program triggers it:

#include <stdio.h>
#include <inttypes.h>

int main(int argc, char **argv)
{
        size_t i;
        uint64_t v, nv;

        for (i = 0; i < 16; i++) {
                v = ~(uint64_t)0 << i;
                nv = ~v;
                printf("%zu: mask %08llx not %08llx\n", i, v, nv);
        }

        return 0;
}

The relevant line from assembly:

    ashq %r6,$-1,%r2

which is misinterpreted by the assembler, so dissasembly is:

   10637:       79 56 8f ff     ashq r6,$0x00000000ffffffff,r2
   1063b:       ff ff ff 00 
   1063f:       00 00 00 52 


The attached patch fixes it.