[PATCH] tree-optimization/100981 - fix SLP patterns involving reductions

[Richard Biener]

The following fixes the SLP FMA patterns to preserve reduction
info and the reduction vectorization to consider internal function
call defs for the reduction stmt.

Bootstrap & regtest running on x86_64-unknown-linux-gnu, Andre
verified we're not turning an ICE into a wrong-code bug
(.COMPLEX_MUL now appears in the reduction chain).

Note there's a testcase for the ICE which adds -march=armv8.3-a
and a testcase for correctness which doesn't since I didn't find
any dg effective target verifying armv8.3-a code can run.

2021-06-09  Richard Biener  <rguenther@suse.de>

	PR tree-optimization/100981
	* tree-vect-loop.c (vect_create_epilog_for_reduction): Use
	gimple_get_lhs to also handle calls.
	* tree-vect-slp-patterns.c (complex_pattern::build): Transfer
	reduction info.

	* gfortran.dg/vect/pr100981-1.f90: New testcase.
	* gfortran.dg/vect/pr100981-2.f90: Likewise.
---
 gcc/tree-vect-loop.c         | 2 +-
 gcc/tree-vect-slp-patterns.c | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index ba36348b835..ee79808472c 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -5247,7 +5247,7 @@ vect_create_epilog_for_reduction (loop_vec_info loop_vinfo,
       gcc_assert (STMT_VINFO_RELATED_STMT (orig_stmt_info) == stmt_info);
     }
   
-  scalar_dest = gimple_assign_lhs (orig_stmt_info->stmt);
+  scalar_dest = gimple_get_lhs (orig_stmt_info->stmt);
   scalar_type = TREE_TYPE (scalar_dest);
   scalar_results.create (group_size); 
   new_scalar_dest = vect_create_destination_var (scalar_dest, NULL);
diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c
index b25655c9876..2ed49cd9edc 100644
--- a/gcc/tree-vect-slp-patterns.c
+++ b/gcc/tree-vect-slp-patterns.c
@@ -544,6 +544,8 @@ complex_pattern::build (vec_info *vinfo)
     {
       /* Calculate the location of the statement in NODE to replace.  */
       stmt_info = SLP_TREE_REPRESENTATIVE (node);
+      stmt_vec_info reduc_def
+	= STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info));
       gimple* old_stmt = STMT_VINFO_STMT (stmt_info);
       tree lhs_old_stmt = gimple_get_lhs (old_stmt);
       tree type = TREE_TYPE (lhs_old_stmt);
@@ -568,9 +570,10 @@ complex_pattern::build (vec_info *vinfo)
 	= vinfo->add_pattern_stmt (call_stmt, stmt_info);
 
       /* Make sure to mark the representative statement pure_slp and
-	 relevant. */
+	 relevant and transfer reduction info. */
       STMT_VINFO_RELEVANT (call_stmt_info) = vect_used_in_scope;
       STMT_SLP_TYPE (call_stmt_info) = pure_slp;
+      STMT_VINFO_REDUC_DEF (call_stmt_info) = reduc_def;
 
       gimple_set_bb (call_stmt, gimple_bb (stmt_info->stmt));
       STMT_VINFO_VECTYPE (call_stmt_info) = SLP_TREE_VECTYPE (node);
-- 
2.26.2