[Bug middle-end/41738] New: optabs expands rotate using wrong mode

[hutchinsonandy at gcc dot gnu dot org]

I have been adding rotate capability to AVR port and have come this bug in 
optabs.c: expand_binop()

This occurs during a rotate expansion. For example

target  = op0  rotated by op1

In the particular situation (code extract below) it tries a reverse rotate of
(bits - op1). Where this expression is then expanded into an integer,
a negation or subtraction depending on type of op1 and target.

The expansion of the subtraction is using the mode of the target - it should be
using the mode of op1.
The mode of the rotation  amount need not be the same as the target.

target:DI = Op0:DI rotate op1:HI

In my testcase it is not and I get asserts latter in simplfy_rtx with the
mismatched inner/outer modes.

The negation mode looks equally wrong.

I found this on testcase 20010226-1.c with gcc 4.42 (line 55) - however, it
would appear to be on 4.5 and perhaps earlier versions.

Revision 126370 added this particular code to optabs.c in July 2007



 /* If we were trying to rotate, and that didn't work, try rotating
    the other direction before falling back to shifts and bitwise-or.  */
 if (((binoptab == rotl_optab
   && optab_handler (rotr_optab, mode)->insn_code != CODE_FOR_nothing)
      || (binoptab == rotr_optab
      && optab_handler (rotl_optab, mode)->insn_code != CODE_FOR_nothing))
     && mclass == MODE_INT)
   {
     optab otheroptab = (binoptab == rotl_optab ? rotr_optab : rotl_optab);
     rtx newop1;
     unsigned int bits = GET_MODE_BITSIZE (mode);

     if (CONST_INT_P (op1))
   newop1 = GEN_INT (bits - INTVAL (op1));
     else if (targetm.shift_truncation_mask (mode) == bits - 1)
   newop1 = negate_rtx (mode, op1);
     else
   newop1 = expand_binop (mode, sub_optab,
                  GEN_INT (bits), op1,
                  NULL_RTX, unsignedp, OPTAB_DIRECT);


-- 
           Summary: optabs expands rotate using wrong mode
           Product: gcc
           Version: 4.4.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: middle-end
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: hutchinsonandy at gcc dot gnu dot org
  GCC host triplet: i686-pc-linux-gnu
GCC target triplet: avr-unknown-none


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41738