[Bug c/80315] New: Calling __builtin_crypto_vshasigmaw with argument 3 out of range creates an unrecognizable insn

[gcc at svenkoehler dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80315

            Bug ID: 80315
           Summary: Calling __builtin_crypto_vshasigmaw with argument 3
                    out of range creates an unrecognizable insn
           Product: gcc
           Version: 6.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: gcc at svenkoehler dot de
  Target Milestone: ---

Created attachment 41126
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=41126&action=edit
Intermediate file of the source in question

Using the power8 crypto builtin with an argument 3 out of range 0..15 (thus
more that 4 bits) generates a further unrecognizable instruction:

        res = __builtin_crypto_vshasigmaw(test, 1, 0xff);

results in:

        breaking_sha.c: In function ‘main’:
        breaking_sha.c:14:1: error: unrecognizable insn:
         }
         ^
        (insn 10 9 11 2 (set (reg:V4SI 158)
                (unspec:V4SI [
                        (reg:V4SI 159)
                        (const_int 1 [0x1])
                        (reg:SI 161)
                    ] UNSPEC_VSHASIGMA)) breaking_sha.c:12 -1
             (nil))
        breaking_sha.c:14:1: internal compiler error: in extract_insn, at
recog.c:2287
        0x1067e9b3 _fatal_insn(char const*, rtx_def const*, char const*, int,
char const*)
                ../../src/gcc/rtl-error.c:108
        0x1067ea17 _fatal_insn_not_found(rtx_def const*, char const*, int, char
const*)
                ../../src/gcc/rtl-error.c:116
        0x10644fe7 extract_insn(rtx_insn*)
                ../../src/gcc/recog.c:2287
        0x10420567 instantiate_virtual_regs_in_insn
                ../../src/gcc/function.c:1582
        0x10420567 instantiate_virtual_regs
                ../../src/gcc/function.c:1950
        0x10420567 execute
                ../../src/gcc/function.c:1999
        Please submit a full bug report,
        with preprocessed source if appropriate.
        Please include the complete backtrace with any bug report.
        See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.

Trace as of: gcc version 6.2.0 20160901 (Ubuntu 6.2.0-3ubuntu11~16.04)
Likewise with: gcc version 5.4.0 20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.4)