[Bug target/102211] [12 regression] ICE introduced by r12-3277

[wilson at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102211

--- Comment #5 from Jim Wilson <wilson at gcc dot gnu.org> ---
I have a WIP fix that lets me build newlib and glibc via riscv-gnu-toolchain. 
I haven't tried a bootstrap yet.  I created a new predicate that uses the small
bit of deleted code I need from validate_subregs, and then modified most
patterns with an f constraint to use it.

+;; This predicate should be used for any pattern that has constraints that
+;; accept FP registers.
+(define_predicate "f_register_operand"
+  (match_operand 0 "register_operand")
+{
+  /* We can't allow a subreg that changes size in an FP reg, as that violates
+     the NaN-boxing requirement.  Copied from old validate_subregs code.  */
+  if (GET_CODE (op) == SUBREG)
+    {
+      machine_mode imode = GET_MODE (SUBREG_REG (op));
+      machine_mode omode = GET_MODE (op);
+      if (FLOAT_MODE_P (imode) || FLOAT_MODE_P (omode))
+       {
+         poly_uint64 isize = GET_MODE_SIZE (imode);
+         poly_uint64 osize = GET_MODE_SIZE (omode);
+
+         if (! (known_eq (isize, osize)
+                /* LRA can use subreg to store a floating point value in
+                   an integer mode.  Although the floating point and the
+                   integer modes need the same number of hard registers,
+                   the size of floating point mode can be less than the
+                   integer mode.  LRA also uses subregs for a register
+                   should be used in different mode in on insn.  */
+                || lra_in_progress))
+           return false;
+       }
+    }
+  return true;
+})

I haven't fixed the move patterns yet.  I need a few more new predicates for
that.

The riscv_hard_regno_mode_ok function looks odd as Hongtao Liu mentioned.  The
mov*i patterns have f constraints, but we don't allow FP values in integer
registers here.  I think this is a missed optimization, or maybe good register
allocation creates some no-op moves to hide the problem.  My current patch does
not need a riscv_hard_regno_mode_ok change to work.

The riscv_can_change_mode_class also looks a little odd.  The MIPS one that we
copied doesn't allow mode changes in FP regs, but the alpha one does allow mode
changes in FP regs if the modes have the same size.  I think the alpha one can
work for RISC-V and that this is another missed optimization, though again
maybe good regalloc hides the problem with no-op moves.

But I will worry about the possible missed optimizations after I get bootstrap
working again.