ICE in trunk due to MEM with address in vector mode

[Paulo Matos <pmatos at broadcom dot com>]

Hello,

I have found a strange case of an ICE due to a MEM with an address with vector mode.

The ICE looks like:
baaclc_block.c: In function 'fn2':
baaclc_block.c:22:1: internal compiler error: in trunc_int_for_mode, at explow.c:56
 }
 ^
0x64d050 trunc_int_for_mode(long, machine_mode)
        /home/pmatos/work/fp_gcc-master/gcc/explow.c:56
0x637148 gen_int_mode(long, machine_mode)
        /home/pmatos/work/fp_gcc-master/gcc/emit-rtl.c:422
0x64d61a plus_constant(machine_mode, rtx_def*, long)
        /home/pmatos/work/fp_gcc-master/gcc/explow.c:190
0x63b3e1 adjust_address_1(rtx_def*, machine_mode, long, int, int, int, long)
        /home/pmatos/work/fp_gcc-master/gcc/emit-rtl.c:2088
0x98f05b simplify_subreg(machine_mode, rtx_def*, machine_mode, unsigned int)
        /home/pmatos/work/fp_gcc-master/gcc/simplify-rtx.c:6061
0x98f381 simplify_gen_subreg(machine_mode, rtx_def*, machine_mode, unsigned int)
        /home/pmatos/work/fp_gcc-master/gcc/simplify-rtx.c:6123
0xf546ab propagate_rtx_1
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:565
0xf54c6c propagate_rtx
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:730
0xf56628 forward_propagate_and_simplify
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:1392
0xf568f4 forward_propagate_into
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:1465
0xf56b8d fwprop
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:1550
0xf56c30 execute
        /home/pmatos/work/fp_gcc-master/gcc/fwprop.c:1586

I cannot reproduce it with my host backend... however, I can try to explain what's happening. 
fwprop is calling propagate_rtx with rtxs of the form:
X: (subreg:DI (reg/v:V4SI 93 [ v4wspecExp ]) 8
mode: DImode
old_rtx: (reg/v:V4SI 93 [ v4wspecExp ])
new_rtx: (mem:V4SI (plus:V4SI (vec_concat:V4SI (vec_concat:V2SI (const_int 0 [0])
                (const_int 0 [0]))
            (vec_concat:V2SI (const_int 0 [0])
                (const_int 0 [0])))
        (reg:V4SI 92 [ D.2926 ])) [0  S16 A6

This will go through simplify_subreg that upon seeing a memory where the address is not mode dependent will call adjust_address_1 with adjust_address argument enabled and it will call plus_constant to adjust the address, however plus_constant won't find any constants in the address and will simply fallback to: x = gen_rtx_PLUS (mode, x, gen_int_mode (c, mode));

gen_int_mode (8, V4SI) will call trunc_int_for_mode with 8 and V4SI which will ICE because trunc_int_for_mode doesn't handle ICE.

I suggest the following patch to simplify-rtx.c:
diff --git a/gcc/simplify-rtx.c b/gcc/simplify-rtx.c
index 04af01e..b01211f5 100755
--- a/gcc/simplify-rtx.c
+++ b/gcc/simplify-rtx.c
@@ -6057,7 +6057,11 @@ simplify_subreg (enum machine_mode outermode, rtx op,
          have instruction to move the whole thing.  */
       && (! MEM_VOLATILE_P (op)
 	  || ! have_insn_for (SET, innermode))
-      && GET_MODE_SIZE (outermode) <= GET_MODE_SIZE (GET_MODE (op)))
+      && GET_MODE_SIZE (outermode) <= GET_MODE_SIZE (GET_MODE (op))
+      && !VECTOR_MODE_P (GET_MODE (XEXP (op, 0))))
     return adjust_address_nv (op, outermode, byte);
 
   /* Handle complex values represented as CONCAT


Comments?

Paulo Matos