[PATCH] Fix gen_lowpart_if_possible (PR middle-end/85414)

[Jakub Jelinek]

Hi!

The following testcase FAILs, because cse_local sees
(zero_extend:TI (subreg/s/v:DI (reg:TI ...) 0))
inside of REG_EQUAL note, and simplify-rtx.c attempts to optimize it.
    case ZERO_EXTEND:
      /* Check for a zero extension of a subreg of a promoted
         variable, where the promotion is zero-extended, and the
         target mode is the same as the variable's promotion.  */
      if (GET_CODE (op) == SUBREG
          && SUBREG_PROMOTED_VAR_P (op)
          && SUBREG_PROMOTED_UNSIGNED_P (op)
          && !paradoxical_subreg_p (mode, GET_MODE (SUBREG_REG (op))))
        {
          temp = rtl_hooks.gen_lowpart_no_emit (mode, op);
          if (temp)
            return temp;
        }
This calls gen_lowpart_common which fails, because:
      /* MODE must occupy no more of the underlying registers than X.  */
      poly_uint64 regsize = REGMODE_NATURAL_SIZE (innermode);
      unsigned int mregs, xregs;
      if (!can_div_away_from_zero_p (msize, regsize, &mregs)
          || !can_div_away_from_zero_p (xsize, regsize, &xregs)
          || mregs > xregs)
        return 0;
and so gen_lowpart_if_possible emits (subreg:TI (subreg/s/v:DI (reg:TI ...) 0) 0)
which is invalid, unsurprisingly other passes have issues with it and DF in
the middle of reload discovers use of an unsaved call saved register that
way.

The following patch fixes it by never creating a subreg of subreg, other
spots avoid it similarly.  Bootstrapped/regtested on x86_64-linux and
i686-linux, ok for trunk?

Perhaps in GCC9 we might consider relaxing the above gen_lowpart_common
mregs/xregs restriction, say if the inner operand is a lowpart subreg of
a register with mode having at least xregs regs, allow it; not really sure
if it is desirable though.

2018-04-16  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/85414
	* rtlhooks.c (gen_lowpart_if_possible): Don't call gen_lowpart_SUBREG
	on a SUBREG.

	* gcc.dg/pr85414.c: New test.

--- gcc/rtlhooks.c.jj	2018-01-03 10:19:55.338533987 +0100
+++ gcc/rtlhooks.c	2018-04-16 15:16:54.257664102 +0200
@@ -123,9 +123,9 @@ gen_lowpart_if_possible (machine_mode mo
 
       return new_rtx;
     }
-  else if (mode != GET_MODE (x) && GET_MODE (x) != VOIDmode
+  else if (mode != GET_MODE (x) && GET_MODE (x) != VOIDmode && !SUBREG_P (x)
 	   && validate_subreg (mode, GET_MODE (x), x,
-			        subreg_lowpart_offset (mode, GET_MODE (x))))
+			       subreg_lowpart_offset (mode, GET_MODE (x))))
     return gen_lowpart_SUBREG (mode, x);
   else
     return 0;
--- gcc/testsuite/gcc.dg/pr85414.c.jj	2018-04-16 15:32:42.119137550 +0200
+++ gcc/testsuite/gcc.dg/pr85414.c	2018-04-16 15:31:39.332103633 +0200
@@ -0,0 +1,10 @@
+/* PR middle-end/85414 */
+/* { dg-do compile { target int128 } } */
+/* { dg-options "-Og -fgcse -Wno-uninitialized" } */
+
+int
+foo (void)
+{
+  unsigned __int128 c;
+  return __builtin_mul_overflow_p (59, -c, (short) 0);
+}

	Jakub