[Bug bootstrap/78390] [7 Regression] Bootstrap failure: match.pd: cannot determine type of operand

["matz at gcc dot gnu.org" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78390

--- Comment #18 from Michael Matz <matz at gcc dot gnu.org> ---
(In reply to Segher Boessenkool from comment #17)
> Combine should probably not try to generate this extract, I wonder if it
> can exist on any target.  So where is it coming from?

Easy:

  (subreg:SI (lshiftrt:DI (reg/v:DI 63 [ X ])
        (const_int 56 [0x38])) 4)

On s390x this is a lowpart subreg.  This is now rewritten as extraction
with pos==56 and len==32 (as the wanted mode is SImode).  Note that 56+32 is
larger than 64, i.e. outside the mode of the argument (DImode).  But that's 
supposed to be okay with make_extraction, especially if the arg is !MEM_P.
There even an explicit mention of this in make_extraction:

  if (BITS_BIT_ENDIAN)
    {...

      /* POS may be less than 0 now, but we check for that below.
         Note that it can only be less than 0 if !MEM_P (inner).  */
    }

Of course, pos here indeed is < 0 (it's -24).  But I see no sensible
checking of it for being less than zero.  There are some that were probably
intended to check this but they are guarded such that they don't work in this
case:

  else if (!MEM_P (inner))
    ...
      if (GET_MODE (inner) != wanted_inner_mode
          && (pos_rtx != 0
              || orig_pos + len > GET_MODE_BITSIZE (wanted_inner_mode)))
        return NULL_RTX;

The above would trigger, except that GET_MODE(inner) == wanted_inner_mode.

Now there are multiple ways to deal with this:
1) don't call make_extraction with pos+len>GET_MODE_PRECISION(inner)
2) Bail out in make_extraction more often, as the comment above claims

I think I'll try (1) first as it's going to generate more extracts.

> Of course the target should not "successfully" match it ;-)

True, but OTOH no target should probably match negative positions at which
points it seems more appropriate to not even ask the target to reject such :)