Indeed loop vectorization throws if-converted bodies at the BB vectorizer as a last resort (because BB vectorization doesn't do if-conversion itself).  But
the BB vectorizer then uses the if-converted scalar code as the thing to
cost against (costing against the not if-converted loop body isn't really
possible).  To quote

      /* If we applied if-conversion then try to vectorize the
         BB of innermost loops.
         ???  Ideally BB vectorization would learn to vectorize
         control flow by applying if-conversion on-the-fly, the
         following retains the if-converted loop body even when
         only non-if-converted parts took part in BB vectorization.  */
      if (flag_tree_slp_vectorize != 0
          && loop_vectorized_call
          && ! loop->inner)
        {

as a "hack" we could see to scalar cost the always executed part of
the not if-converted loop body and apply the full bias of this cost
vs. the scalar cost of the if-converted body to the scalar cost of the
BB vectorization.  But that's really apples-to-oranges in the end
(as it is now).

Maybe we can cost the whole partly vectorized loop body in this mode
and compare it against the scalar cost of the original loop.  But even
the loop vectorizer costs the if-converted scalar loop, so it is off as well.

Long-term if-conversion needs to be integrated with vectorization so we
can at least keep track of what stmts were originally executed conditional
and what not.

Short-term I'm not sure we can do much.  Doing SLP on the if-converted
body does help in quite some cases.

GCC 11.1 has been released, retargeting bugs to GCC 11.2.

Is this really a costing issue, or should we instead reject the
BB fallback if it leaves any scalar COND_EXPRs around?  This would
be similar to the way that we reject IFN_MASK_LOAD/STORE calls,
except that the COND_EXPR tests would only apply to unvectorised
statements and so would need to be tested after SLP discovery
rather than before it.  (Ideally IFN_MASK_LOAD/STORE would work
like that too.)

(In reply to rsandifo@gcc.gnu.org from comment #3)
> Is this really a costing issue, or should we instead reject the
> BB fallback if it leaves any scalar COND_EXPRs around?  This would
> be similar to the way that we reject IFN_MASK_LOAD/STORE calls,
> except that the COND_EXPR tests would only apply to unvectorised
> statements and so would need to be tested after SLP discovery
> rather than before it.  (Ideally IFN_MASK_LOAD/STORE would work
> like that too.)

I suppose we could do that, but then I'm not sure how exactly we'd do it ;)

Good idea anyway.

GCC 11.2 is being released, retargeting bugs to GCC 11.3

Working on this now.

The master branch has been updated by hongtao Liu <liuhongt@gcc.gnu.org>:

https://gcc.gnu.org/g:819b7c3a339e3bdaf85cd55954c5536bd98aae09

commit r12-3103-g819b7c3a339e3bdaf85cd55954c5536bd98aae09
Author: liuhongt <hongtao.liu@intel.com>
Date:   Wed Aug 4 16:39:31 2021 +0800

    Disable slp in loop vectorizer when cost model is very-cheap.
    
    Performance impact for the commit with option:
    -march=x86-64 -O2 -ftree-vectorize -fvect-cost-model=very-cheap
    
    SPEC2017 fprate
    503.bwaves_r        BuildSame
    507.cactuBSSN_r         -0.04
    508.namd_r               0.14
    510.parest_r            -0.54
    511.povray_r             0.10
    519.lbm_r           BuildSame
    521.wrf_r                0.64
    526.blender_r           -0.32
    527.cam4_r               0.17
    538.imagick_r            0.09
    544.nab_r           BuildSame
    549.fotonik3d_r     BuildSame
    554.roms_r          BuildSame
    997.specrand_fr         -0.09
    Geometric mean:  0.02
    
    SPEC2017 intrate
    500.perlbench_r          0.26
    502.gcc_r                0.21
    505.mcf_r               -0.09
    520.omnetpp_r       BuildSame
    523.xalancbmk_r     BuildSame
    525.x264_r              -0.41
    531.deepsjeng_r     BuildSame
    541.leela_r              0.13
    548.exchange2_r     BuildSame
    557.xz_r            BuildSame
    999.specrand_ir     BuildSame
    Geometric mean:  0.02
    
    EEMBC: no regression, only improvement or build the same, the below is
    improved benchmarks.
    
    mp2decoddata1       7.59
    mp2decoddata2       31.80
    mp2decoddata3       12.15
    mp2decoddata4       11.16
    mp2decoddata5       11.19
    mp2decoddata1       7.06
    mp2decoddata2       24.12
    mp2decoddata3       10.83
    mp2decoddata4       10.04
    mp2decoddata5       10.07
    
    gcc/ChangeLog:
    
            PR tree-optimization/100089
            * tree-vectorizer.c (try_vectorize_loop_1): Disable slp in
            loop vectorizer when cost model is very-cheap.

So if we agree to a sane way to cost branchy code on the scalar side then it should be possible to compare the scalar cost of the not if-converted inner loop body against the full partially vectorized and if-converted inner loop body.

vect_bb_vectorization_profitable_p would have to add the cost of the scalar
stmts not covered by vectorization - this set is conveniently available as
the set of stmts not having the visited flag set before we clear it here:

vect_bb_vectorization_profitable_p (bb_vec_info bb_vinfo,
                                    vec<slp_instance> slp_instances)
{
...
  /* Unset visited flag.  */
  stmt_info_for_cost *cost;
  FOR_EACH_VEC_ELT (scalar_costs, i, cost)
    gimple_set_visited  (cost->stmt_info->stmt, false);

so we'd need to walk over all stmts in the BB and add the cost of the
not marked stmts to the vector cost.  We'd want to force a single
SLP "subgraph" in this mode to avoid going over the whole block
multiple times.

Created attachment 51350 [details]
patch

This implements scanning for not vectorized COND_EXPRs with -fvect-cost-model=very-cheap when vectorizing if-converted loop bodies.  It also gives
vect_bb_vectorization_profitable_p enough info to eventually do something
more fancy (namely the original not if-converted loop body).

The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:9216ee6d1195d48388f825cf1b072e570129cbbe

commit r12-3116-g9216ee6d1195d48388f825cf1b072e570129cbbe
Author: Richard Biener <rguenther@suse.de>
Date:   Tue Aug 24 12:25:25 2021 +0200

    tree-optimization/100089 - avoid leaving scalar if-converted code around
    
    This avoids leaving scalar if-converted code around for the case
    of BB vectorizing an if-converted loop body when using the very-cheap
    cost model.  In this case we scan not vectorized scalar stmts in
    the basic-block vectorized for COND_EXPRs and force the vectorization
    to be marked as not profitable.
    
    The patch also makes sure to always consider all BB vectorization
    subgraphs together for costing purposes when vectorizing an
    if-converted loop body.
    
    2021-08-24  Richard Biener  <rguenther@suse.de>
    
            PR tree-optimization/100089
            * tree-vectorizer.h (vect_slp_bb): Rename to ...
            (vect_slp_if_converted_bb): ... this and get the original
            loop as new argument.
            * tree-vectorizer.c (try_vectorize_loop_1): Revert previous fix,
            pass original loop to vect_slp_if_converted_bb.
            * tree-vect-slp.c (vect_bb_vectorization_profitable_p):
            If orig_loop was passed scan the not vectorized stmts
            for COND_EXPRs and force not profitable if found.
            (vect_slp_region): Pass down all SLP instances to costing
            if orig_loop was specified.
            (vect_slp_bbs): Pass through orig_loop.
            (vect_slp_bb): Rename to ...
            (vect_slp_if_converted_bb): ... this and get the original
            loop as new argument.
            (vect_slp_function): Adjust.

So this fixes it for -fvect-cost-model=very-cheap.  One could argue that we should enable the code for all cost models, fixing the -O3 regression (and backportable to the branch).

I'll see to experiment with "fancy" costing of the stray vectorizations.  I'll also note that the scanning for load/store (and other ifns not supported)
could be handled in the costing as well (but the costing would need to run
also for -fno-vect-cost-model then, just the result ignored if not forced).
I'm talking about

          bool require_loop_vectorize = false;
          for (gimple_stmt_iterator gsi = gsi_start_bb (bb);
               !gsi_end_p (gsi); gsi_next (&gsi))
            {
              gimple *stmt = gsi_stmt (gsi);
              gcall *call = dyn_cast <gcall *> (stmt);
              if (call && gimple_call_internal_p (call))
                {
                  internal_fn ifn = gimple_call_internal_fn (call);
                  if (ifn == IFN_MASK_LOAD || ifn == IFN_MASK_STORE
                      /* Don't keep the if-converted parts when the ifn with
                         specifc type is not supported by the backend.  */
                      || (direct_internal_fn_p (ifn)
                          && !direct_internal_fn_supported_p
                          (call, OPTIMIZE_FOR_SPEED)))
                    {
                      require_loop_vectorize = true;
                      break;
                    }
                }

The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:199cd0e0f8744ca1e61a95987b2d020a592a46d9

commit r12-6795-g199cd0e0f8744ca1e61a95987b2d020a592a46d9
Author: Richard Biener <rguenther@suse.de>
Date:   Fri Jan 21 13:29:06 2022 +0100

    tree-optimization/100089 - BB vectorization of if-converted loop bodies
    
    The PR complains that when we only partially BB vectorize an
    if-converted loop body that this can leave unvectorized code
    unconditionally executed and thus effectively slow down code.
    For -O2 we already mitigated the issue by not doing BB vectorization
    when not all if-converted stmts were covered but the issue is
    present with -O3 as well.  Thus the following simply extends the
    fix to cover all but the unlimited cost models.  It is after all
    very likely that we vectorize some stmts, if only a single
    paired store.
    
    2022-01-21  Richard Biener  <rguenther@suse.de>
    
            PR tree-optimization/100089
            * tree-vect-slp.cc (vect_slp_region): Reject BB vectorization
            of if-converted loops with unvectorized COND_EXPRs for
            all but the unlimited cost models.

Now fixed for GCC 12 even with -O3.

GCC 11.3 is being released, retargeting bugs to GCC 11.4.