Fix nb_iterations_estimate calculation in tree-vect-loop.c
Richard Sandiford
richard.sandiford@arm.com
Tue Nov 15 12:57:00 GMT 2016
vect_transform_loop has to reduce three iteration counts by
the vectorisation factor: nb_iterations_upper_bound,
nb_iterations_likely_upper_bound and nb_iterations_estimate.
All three are latch execution counts rather than loop body
execution counts. The calculations were taking that into
account for the first two, but not for nb_iterations_estimate.
This patch updates the way the calculations are done to fix
this and to add a bit more commentary about what is going on.
Tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install?
Thanks,
Richard
[ This patch is part of the SVE series posted here:
https://gcc.gnu.org/ml/gcc/2016-11/msg00030.html ]
gcc/
2016-11-15 Richard Sandiford <richard.sandiford@arm.com>
Alan Hayward <alan.hayward@arm.com>
David Sherwood <david.sherwood@arm.com>
* tree-vect-loop.c (vect_transform_loop): Protect the updates of
all three iteration counts with an any_* test. Use a single update
for each count. Fix the calculation of nb_iterations_estimate.
diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index 1cd9c72..53570f3 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -7043,27 +7043,25 @@ vect_transform_loop (loop_vec_info loop_vinfo)
/* Reduce loop iterations by the vectorization factor. */
scale_loop_profile (loop, GCOV_COMPUTE_SCALE (1, vf),
expected_iterations / vf);
- if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo))
- {
- if (loop->nb_iterations_upper_bound != 0)
- loop->nb_iterations_upper_bound = loop->nb_iterations_upper_bound - 1;
- if (loop->nb_iterations_likely_upper_bound != 0)
- loop->nb_iterations_likely_upper_bound
- = loop->nb_iterations_likely_upper_bound - 1;
- }
- loop->nb_iterations_upper_bound
- = wi::udiv_floor (loop->nb_iterations_upper_bound + 1, vf) - 1;
- loop->nb_iterations_likely_upper_bound
- = wi::udiv_floor (loop->nb_iterations_likely_upper_bound + 1, vf) - 1;
-
+ /* The minimum number of iterations performed by the epilogue. This
+ is 1 when peeling for gaps because we always need a final scalar
+ iteration. */
+ int min_epilogue_iters = LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) ? 1 : 0;
+ /* +1 to convert latch counts to loop iteration counts,
+ -min_epilogue_iters to remove iterations that cannot be performed
+ by the vector code. */
+ int bias = 1 - min_epilogue_iters;
+ /* In these calculations the "- 1" converts loop iteration counts
+ back to latch counts. */
+ if (loop->any_upper_bound)
+ loop->nb_iterations_upper_bound
+ = wi::udiv_floor (loop->nb_iterations_upper_bound + bias, vf) - 1;
+ if (loop->any_likely_upper_bound)
+ loop->nb_iterations_likely_upper_bound
+ = wi::udiv_floor (loop->nb_iterations_likely_upper_bound + bias, vf) - 1;
if (loop->any_estimate)
- {
- loop->nb_iterations_estimate
- = wi::udiv_floor (loop->nb_iterations_estimate, vf);
- if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)
- && loop->nb_iterations_estimate != 0)
- loop->nb_iterations_estimate = loop->nb_iterations_estimate - 1;
- }
+ loop->nb_iterations_estimate
+ = wi::udiv_floor (loop->nb_iterations_estimate + bias, vf) - 1;
if (dump_enabled_p ())
{
More information about the Gcc-patches
mailing list