[Bug tree-optimization/91665] [8/9/10 Regression] ICE in build_vector_from_val, at tree.c:1904

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91665

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |NEW
           Assignee|jakub at gcc dot gnu.org           |unassigned at gcc dot gnu.org

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Ugh, this is a mess.
The problem is that because the reduction is done in short int (or could be
signed char), but the actual bumps in the corresponding unsigned type as
overflow is in that case well defined, analyze_scalar_evolution returns a
NOP_EXPR from an unsinged short chrec to short int, STRIP_NOPS (access_fn);
drops that and so
STMT_VINFO_LOOP_PHI_EVOLUTION_BASE_UNCHANGED and
STMT_VINFO_LOOP_PHI_EVOLUTION_PART are actually unsigned short.
I'm e.g. surprised that is_nonwrapping_integer_induction is happy about that,
it checks
if (TYPE_OVERFLOW_UNDEFINED (lhs_type))
and doesn't check
TYPE_OVERFLOW_UNDEFINED (TREE_TYPE (base)).
Though, max_stmt_executions is 65535 and thus maybe ok.
In any case, it seems vectorizable_reduction as well as
vect_create_epilog_for_reduction aren't prepared to cope, even when I added a
hack to fold_convert induc_val to the type it wants to create, there is another
ICE about short vs. unsigned short type mismatch.

I can offer a quick patch to punt in this case:
--- gcc/tree-vect-loop.c.jj     2019-07-10 15:52:56.793588202 +0200
+++ gcc/tree-vect-loop.c        2019-09-05 12:55:28.120545749 +0200
@@ -6656,10 +6656,12 @@ vectorizable_reduction (stmt_vec_info st
          gcc_assert (TREE_CODE (base) == INTEGER_CST
                      && TREE_CODE (step) == INTEGER_CST);
          cond_reduc_val = NULL_TREE;
+         if (!types_compatible_p (TREE_TYPE (reduc_def), TREE_TYPE (base)))
+           ;
          /* Find a suitable value, for MAX_EXPR below base, for MIN_EXPR
             above base; punt if base is the minimum value of the type for
             MAX_EXPR or maximum value of the type for MIN_EXPR for now.  */
-         if (tree_int_cst_sgn (step) == -1)
+         else if (tree_int_cst_sgn (step) == -1)
            {
              cond_reduc_op_code = MIN_EXPR;
              if (tree_int_cst_sgn (base) == -1)
although I'm not really sure if reduc_def is the right type to compare to.
But not really sure what needs to be done to resolve this properly, guess we
need to perform the reduction increase in vector unsigned short type somehow.