[PATCH v5 2/2] x86: Add vec_duplicate<mode> expander

[H.J. Lu]

On Sun, Jun 27, 2021 at 1:43 AM Richard Sandiford
<richard.sandiford@arm.com> wrote:
>
> "H.J. Lu" <hjl.tools@gmail.com> writes:
> > 1. Update vec_duplicate to allow to fail so that backend can only allow
> > broadcasting an integer constant to a vector when broadcast instruction
> > is available.  This can be used by memset expander to avoid vec_duplicate
> > when loading from constant pool is more efficient.
>
> I don't see any changes in target-independent code though, other than
> the doc update.  It's still the case that (existing) uses of
> vec_duplicate_optab do not allow it to fail.

I have a followup patch set on

https://gitlab.com/x86-gcc/gcc/-/commits/users/hjl/pieces/broadcast

to use it to expand memset with vector broadcast:

https://gitlab.com/x86-gcc/gcc/-/commit/991c87f8a83ca736ae9ed92baa3ebadca289f6e3

For SSE2 which doesn't have vector broadcast, the constant vector broadcast
expander returns FAIL and load from constant pool will be used.

> Thanks,
> Richard
>
> > 2. Add vec_duplicate<mode> expander and enable vec_duplicate from a
> > non-standard SSE constant integer only if vector broadcast is available.
> >
> >       * config/i386/i386-expand.c (ix86_expand_integer_vec_duplicate):
> >       New function.
> >       * config/i386/i386-protos.h (ix86_expand_integer_vec_duplicat):
> >       New prototype.
> >       * config/i386/sse.md (INT_BROADCAST_MODE): New mode iterator.
> >       (vec_duplicate<mode>): New expander.
> >       * doc/md.texi: Update vec_duplicate.
> > ---
> >  gcc/config/i386/i386-expand.c | 24 ++++++++++++++++++++++++
> >  gcc/config/i386/i386-protos.h |  1 +
> >  gcc/config/i386/sse.md        | 28 ++++++++++++++++++++++++++++
> >  gcc/doc/md.texi               |  2 --
> >  4 files changed, 53 insertions(+), 2 deletions(-)
> >
> > diff --git a/gcc/config/i386/i386-expand.c b/gcc/config/i386/i386-expand.c
> > index e9e89c82764..75c160d4349 100644
> > --- a/gcc/config/i386/i386-expand.c
> > +++ b/gcc/config/i386/i386-expand.c
> > @@ -15742,6 +15742,30 @@ ix86_expand_vector_extract (bool mmx_ok, rtx target, rtx vec, int elt)
> >      }
> >  }
> >
> > +/* Expand integer vec_duplicate.  Return true if successful.  */
> > +
> > +bool
> > +ix86_expand_integer_vec_duplicate (rtx *operands)
> > +{
> > +  /* Enable VEC_DUPLICATE from a non-standard SSE constant integer only
> > +     if vector broadcast is available.  */
> > +  machine_mode mode = GET_MODE (operands[0]);
> > +  if (CONST_INT_P (operands[1])
> > +      && (!(TARGET_AVX2
> > +         || (TARGET_AVX
> > +             && (GET_MODE_INNER (mode) == SImode
> > +                 || GET_MODE_INNER (mode) == DImode)))
> > +       || standard_sse_constant_p (operands[1], mode)))
> > +    return false;
> > +
> > +  bool ok = ix86_expand_vector_init_duplicate (false, mode,
> > +                                            operands[0],
> > +                                            operands[1]);
> > +  gcc_assert (ok);
> > +
> > +  return true;
> > +}
> > +
> >  /* Generate code to copy vector bits i / 2 ... i - 1 from vector SRC
> >     to bits 0 ... i / 2 - 1 of vector DEST, which has the same mode.
> >     The upper bits of DEST are undefined, though they shouldn't cause
> > diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h
> > index 71745b9a1ea..a6cc09bb75b 100644
> > --- a/gcc/config/i386/i386-protos.h
> > +++ b/gcc/config/i386/i386-protos.h
> > @@ -258,6 +258,7 @@ extern void ix86_expand_mul_widen_hilo (rtx, rtx, rtx, bool, bool);
> >  extern void ix86_expand_sse2_mulv4si3 (rtx, rtx, rtx);
> >  extern void ix86_expand_sse2_mulvxdi3 (rtx, rtx, rtx);
> >  extern void ix86_expand_sse2_abs (rtx, rtx);
> > +extern bool ix86_expand_integer_vec_duplicate (rtx *);
> >
> >  /* In i386-c.c  */
> >  extern void ix86_target_macros (void);
> > diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md
> > index e4f01e64bc1..53a703fb466 100644
> > --- a/gcc/config/i386/sse.md
> > +++ b/gcc/config/i386/sse.md
> > @@ -24640,3 +24640,31 @@ (define_insn "*aes<aeswideklvariant>u8"
> >    "TARGET_WIDEKL"
> >    "aes<aeswideklvariant>\t{%0}"
> >    [(set_attr "type" "other")])
> > +
> > +;; Modes handled by broadcast patterns.  NB: Allow V64QI and V32HI with
> > +;; TARGET_AVX512F since ix86_expand_integer_vec_duplicate can expand
> > +;; without TARGET_AVX512BW which is used by memset vector broadcast
> > +;; expander to XI with:
> > +;;   vmovd           %edi, %xmm15
> > +;;   vpbroadcastb    %xmm15, %ymm15
> > +;;   vinserti64x4    $0x1, %ymm15, %zmm15, %zmm15
> > +
> > +(define_mode_iterator INT_BROADCAST_MODE
> > +  [(V64QI "TARGET_AVX512F") (V32QI "TARGET_AVX") V16QI
> > +   (V32HI "TARGET_AVX512F") (V16HI "TARGET_AVX") V8HI
> > +   (V16SI "TARGET_AVX512F") (V8SI "TARGET_AVX") V4SI
> > +   (V8DI "TARGET_AVX512F && TARGET_64BIT")
> > +   (V4DI "TARGET_AVX && TARGET_64BIT") (V2DI "TARGET_64BIT")])
> > +
> > +;; Broadcast from an integer.  NB: Enable broadcast only if we can move
> > +;; from GPR to SSE register directly.
> > +(define_expand "vec_duplicate<mode>"
> > +  [(set (match_operand:INT_BROADCAST_MODE 0 "register_operand")
> > +     (vec_duplicate:INT_BROADCAST_MODE
> > +       (match_operand:<ssescalarmode> 1 "general_operand")))]
> > +  "TARGET_SSE2 && TARGET_INTER_UNIT_MOVES_TO_VEC"
> > +{
> > +  if (!ix86_expand_integer_vec_duplicate (operands))
> > +    FAIL;
> > +  DONE;
> > +})
> > diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
> > index 1b918144330..a892c94d163 100644
> > --- a/gcc/doc/md.texi
> > +++ b/gcc/doc/md.texi
> > @@ -5077,8 +5077,6 @@ the mode appropriate for one element of @var{m}.
> >  This pattern only handles duplicates of non-constant inputs.  Constant
> >  vectors go through the @code{mov@var{m}} pattern instead.
> >
> > -This pattern is not allowed to @code{FAIL}.
> > -
> >  @cindex @code{vec_series@var{m}} instruction pattern
> >  @item @samp{vec_series@var{m}}
> >  Initialize vector output operand 0 so that element @var{i} is equal to



-- 
H.J.