SH-ELF: shift_cost[] indexed incorrectly and RTX_COST problems

[Toshi Morita]

version: CVS head, 3.0 branch, 2.95.x 
   host: i686-linux (RH 7.0)
 target: sh-elf

Here is are two nasty problems on the sh-elf target.
The first problem was found by Colin Burgess, and the
second problem I found while perusing the first problem.

Problem #1 
----------

The first problem is: shiftcosts() indexes past the end of
the shift_cost[] array.

This occurs because of three culprits:

1) shift_cost is defined as shift_cost[MAX_BITS_PER_WORD],

2) shiftcosts() in config/sh/sh.c assumes it will only be 
   called for SImode shifts and references shift_cost[] to 
   determine the cost of a shift,

3) init_expmed() contains this code:

  for (mode = GET_CLASS_NARROWEST_MODE (MODE_INT);
       mode != VOIDmode;
       mode = GET_MODE_WIDER_MODE (mode))
    {
      reg = gen_rtx_REG (mode, 10000);
      div_cost[(int) mode] = rtx_cost (gen_rtx_UDIV (mode, reg, reg), SET);
      mul_cost[(int) mode] = rtx_cost (gen_rtx_MULT (mode, reg, reg), SET);
      wider_mode = GET_MODE_WIDER_MODE (mode);
      if (wider_mode != VOIDmode)
        {
          mul_widen_cost[(int) wider_mode]
            = rtx_cost (gen_rtx_MULT (wider_mode,
                                      gen_rtx_ZERO_EXTEND (wider_mode, reg),
                                      gen_rtx_ZERO_EXTEND (wider_mode, reg)),
                        SET);
          mul_highpart_cost[(int) mode]
            = rtx_cost (gen_rtx_TRUNCATE
                        (mode,
                         gen_rtx_LSHIFTRT (wider_mode,
                                           gen_rtx_MULT (wider_mode,
                                                         gen_rtx_ZERO_EXTEND
                                                         (wider_mode, reg),
                                                         gen_rtx_ZERO_EXTEND
                                                         (wider_mode, reg)),
                                           GEN_INT (GET_MODE_BITSIZE (mode)))),
                        SET);
        }
    }

This code seems to determine the cost of shifting the XXmode types around,
and one of the rtxes it generates is a shift by 64 of a TImode rtx.

This problem can be easily demonstrated by placing a breakpoint on
shiftcosts() and compiling a null function.

rtx_cost() calls shiftcost() to determine the cost of the shift, and
shiftcost() references shift_cost[] to determine the cost, and retrieves
garbage past the end of the array.

Problem #2:
-----------

The second related problem is: on the SH, the RTX_COST macro appears to be
broken. It doesn't examine the type of the operands to determine the cost
of the operation.

For example, here is the addsubcosts() function called by RTX_COST to
determine the cost of an add/subtract:

int
addsubcosts (x)
     rtx x;
{
  /* Adding a register is a single cycle insn.  */
  if (GET_CODE (XEXP (x, 1)) == REG
      || GET_CODE (XEXP (x, 1)) == SUBREG)
    return 1;

  /* Likewise for small constants.  */
  if (GET_CODE (XEXP (x, 1)) == CONST_INT
      && CONST_OK_FOR_I (INTVAL (XEXP (x, 1))))
    return 1;

  /* Any other constant requires a 2 cycle pc-relative load plus an
     addition.  */
  return 3;
}

It doesn't check the operands types. It seems to assume it's always SImode
since the costs are reasonably consistent with that mode.

None of the cost calculation functions (addsubcosts, andcosts, multcosts,
or shiftcosts) seem to handle SFmode, DFmode, DImode, TImode, or OImode properly.
It always seems to assume the operands are SImode ... which leads to the first
problem of indexing past the end of shift_cost[].

Toshi