[arm-8-branch] Backport spectre v1 patches

Ramana Radhakrishnan ramana.radhakrishnan@foss.arm.com
Wed Aug 15 13:06:00 GMT 2018


Hi,

Backport spectre mitigation patches into the arm-8-branch , these
appeared to be clean backports back to the branch without any conflicts.

Backported svn revisions

263169
263170
263171
263172
263173
263174
263175
263176
263180
263196
263197
263209
263253
263254
263255
263256
263258
263259
263263
263264
263266
263270
263272
263296
263301
263335
263344

Tested with arm-none-eabi, aarch64-none-elf, aarch64-none-linux-gnu,
bootstraps on x86_64 and a full build with config-list.mk.

Applied to the branch.

regards
Ramana
-------------- next part --------------
Index: gcc/ChangeLog.arm
===================================================================
--- gcc/ChangeLog.arm	(revision 263529)
+++ gcc/ChangeLog.arm	(working copy)
@@ -1,3 +1,190 @@
+2018-08-14  Ramana Radhakrishnan  <ramana.radhakrishnan@arm.com>
+
+	Backport spectre v1 mitigation patches.
+	2018-08-06  John David Anglin  <danglin@gcc.gnu.org>
+	PR target/86785
+	* config/pa/pa.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	2018-08-06  Ulrich Weigand  <uweigand@de.ibm.com>
+	PR target/86807
+	* config/spu/spu.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	2018-08-03  Sandra Loosemore  <sandra@codesourcery.com>
+	PR target/86799
+	* config/nios2/nios2.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define.
+
+	2018-08-03  Jeff Law  <law@redhat.com>
+	PR target/86795
+	* config/mn10300/mn10300.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	2018-08-02  Jeff Law  <law@redhat.com>
+	PR target/86790
+	* config/m68k/m68k.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86784
+	* config/h8300/h8300.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	2018-08-02  Nick Clifton  <nickc@redhat.com>
+	PR target/86813
+	* config/stormy16/stormy16.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86810
+	* config/v850/v850.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86810
+	* config/v850/v850.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86803
+	* config/rx/rx.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86797
+	* config/msp430/msp430.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86791
+	* config/mcore/mcore.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86789
+	* config/m32r/m32r.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86787
+	* config/iq2000/iq2000.c (TARGET_HAVE_SPECULATION_SAFE_VALUE):
+	Define to speculation_safe_value_not_needed.
+
+	PR target/86782
+	* config/frv/frv.c (TARGET_HAVE_SPECULATION_SAFE_VALUE): Define to
+	speculation_safe_value_not_needed.
+
+	PR target/86781
+	* config/fr30/fr30.c (TARGET_HAVE_SPECULATION_SAFE_VALUE): Define
+	to speculation_safe_value_not_needed.
+
+	2018-08-01  Tom de Vries  <tdevries@suse.de>
+	PR target/86800
+	* config/nvptx/nvptx.c (TARGET_HAVE_SPECULATION_SAFE_VALUE): Define to
+	speculation_safe_value_not_needed.
+
+	2018-08-01  Richard Earnshaw  <rearnsha@arm.com>
+	* config/rs6000/rs6000.md (speculation_barrier): Renamed from
+	rs6000_speculation_barrier.
+	* config/rs6000/rs6000.c (rs6000_expand_builtin): Adjust for
+	new barrier pattern name.
+
+	2018-08-01  Richard Earnshaw  <rearnsha@arm.com>
+
+	* config/i386/i386.md (unspecv): Add UNSPECV_SPECULATION_BARRIER.
+	(speculation_barrier): New insn.
+
+	2018-07-31  Ian Lance Taylor  <iant@golang.org>
+	* targhooks.c (default_have_speculation_safe_value): Add
+	ATTRIBUTE_UNUSED.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* config/pdp11/pdp11.c (TARGET_HAVE_SPECULATION_SAFE_VALUE): Redefine
+	to speculation_safe_value_not_needed.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* targhooks.h (speculation_safe_value_not_needed): New prototype.
+	* targhooks.c (speculation_safe_value_not_needed): New function.
+	* target.def (have_speculation_safe_value): Update documentation.
+	* doc/tm.texi: Regenerated.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+
+	* config/aarch64/iterators.md (ALLI_TI): New iterator.
+	* config/aarch64/aarch64.md (despeculate_copy<ALLI_TI:mode>): New
+	expand.
+	(despeculate_copy<ALLI:mode>_insn): New insn.
+	(despeculate_copyti_insn): New insn.
+	(despeculate_simple<ALLI:mode>): New insn
+	(despeculate_simpleti): New insn.
+	* config/aarch64/aarch64.c (aarch64_speculation_safe_value): New
+	function.
+	(TARGET_SPECULATION_SAFE_VALUE): Redefine to
+	aarch64_speculation_safe_value.
+	(aarch64_print_operand): Handle const0_rtx in modifier 'H'.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+
+	* config/aarch64/aarch64-speculation.cc: New file.
+	* config/aarch64/aarch64-passes.def (pass_track_speculation): Add
+	before pass_reorder_blocks.
+	* config/aarch64/aarch64-protos.h (make_pass_track_speculation): Add
+	prototype.
+	* config/aarch64/aarch64.c (aarch64_conditional_register_usage): Fix
+	X14 and X15 when tracking speculation.
+	* config/aarch64/aarch64.md (register name constants): Add
+	SPECULATION_TRACKER_REGNUM and SPECULATION_SCRATCH_REGNUM.
+	(unspec): Add UNSPEC_SPECULATION_TRACKER.
+	(speculation_barrier): New insn attribute.
+	(cmp<mode>): Allow SP in comparisons.
+	(speculation_tracker): New insn.
+	(speculation_barrier): Add speculation_barrier attribute.
+	* config/aarch64/t-aarch64: Add make rule for aarch64-speculation.o.
+	* config.gcc (aarch64*-*-*): Add aarch64-speculation.o to extra_objs.
+	* doc/invoke.texi (AArch64 Options): Document -mtrack-speculation.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* config/aarch64/aarch64.md (cb<optab><mode>1): Disable when
+	aarch64_track_speculation is true.
+	(tb<optab><mode>1): Likewise.
+	* config/aarch64/aarch64.c (aarch64_split_compare_regs): Do not
+	generate CB[N]Z when tracking speculation.
+	(aarch64_split_compare_and_swap): Likewise.
+	(aarch64_split_atomic_op): Likewise.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* config/aarch64/aarch64.opt (mtrack-speculation): New target option.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* config/aarch64.md (unspecv): Add UNSPECV_SPECULAION_BARRIER.
+	(speculation_barrier): New insn.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* config/arm/unspecs.md (unspecv): Add VUNSPEC_SPECULATION_BARRIER.
+	* config/arm/arm.md (speculation_barrier): New expand.
+	(speculation_barrier_insn): New pattern.
+
+	2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+	* builtin-types.def (BT_FN_PTR_PTR_VAR): New function type.
+	(BT_FN_I1_I1_VAR, BT_FN_I2_I2_VAR, BT_FN_I4_I4_VAR): Likewise.
+	(BT_FN_I8_I8_VAR, BT_FN_I16_I16_VAR): Likewise.
+	* builtin-attrs.def (ATTR_NOVOPS_NOTHROW_LEAF_LIST): New attribute
+	list.
+	* builtins.def (BUILT_IN_SPECULATION_SAFE_VALUE_N): New builtin.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_PTR): New internal builtin.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_1): Likewise.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_2): Likewise.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_4): Likewise.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_8): Likewise.
+	(BUILT_IN_SPECULATION_SAFE_VALUE_16): Likewise.
+	* builtins.c (expand_speculation_safe_value): New function.
+	(expand_builtin): Call it.
+	* doc/cpp.texi: Document predefine __HAVE_SPECULATION_SAFE_VALUE.
+	* doc/extend.texi: Document __builtin_speculation_safe_value.
+	* doc/md.texi: Document "speculation_barrier" pattern.
+	* doc/tm.texi.in: Pull in TARGET_SPECULATION_SAFE_VALUE and
+	TARGET_HAVE_SPECULATION_SAFE_VALUE.
+	* doc/tm.texi: Regenerated.
+	* target.def (have_speculation_safe_value, speculation_safe_value): New
+	hooks.
+	* targhooks.c (default_have_speculation_safe_value): New function.
+	(default_speculation_safe_value): New function.
+	* targhooks.h (default_have_speculation_safe_value): Add prototype.
+	(default_speculation_safe_value): Add prototype.
+
 2018-08-14  Yvan Roux  <yvan.roux@linaro.org>
 
 	* LINARO-VERSION: New file.
Index: gcc/builtin-attrs.def
===================================================================
--- gcc/builtin-attrs.def	(revision 263529)
+++ gcc/builtin-attrs.def	(working copy)
@@ -129,6 +129,8 @@
 
 DEF_ATTR_TREE_LIST (ATTR_NOTHROW_LEAF_LIST, ATTR_LEAF, ATTR_NULL, ATTR_NOTHROW_LIST)
 
+DEF_ATTR_TREE_LIST (ATTR_NOVOPS_NOTHROW_LEAF_LIST, ATTR_NOVOPS, \
+		        ATTR_NULL, ATTR_NOTHROW_LEAF_LIST)
 DEF_ATTR_TREE_LIST (ATTR_CONST_NOTHROW_LIST, ATTR_CONST,	\
 			ATTR_NULL, ATTR_NOTHROW_LIST)
 DEF_ATTR_TREE_LIST (ATTR_CONST_NOTHROW_LEAF_LIST, ATTR_CONST,	\
Index: gcc/builtin-types.def
===================================================================
--- gcc/builtin-types.def	(revision 263529)
+++ gcc/builtin-types.def	(working copy)
@@ -764,6 +764,12 @@
 			 BT_VOID, BT_LONG)
 DEF_FUNCTION_TYPE_VAR_1 (BT_FN_VOID_ULL_VAR,
 			 BT_VOID, BT_ULONGLONG)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_PTR_PTR_VAR, BT_PTR, BT_PTR)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_I1_I1_VAR, BT_I1, BT_I1)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_I2_I2_VAR, BT_I2, BT_I2)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_I4_I4_VAR, BT_I4, BT_I4)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_I8_I8_VAR, BT_I8, BT_I8)
+DEF_FUNCTION_TYPE_VAR_1 (BT_FN_I16_I16_VAR, BT_I16, BT_I16)
 
 DEF_FUNCTION_TYPE_VAR_2 (BT_FN_INT_FILEPTR_CONST_STRING_VAR,
 			 BT_INT, BT_FILEPTR, BT_CONST_STRING)
Index: gcc/builtins.c
===================================================================
--- gcc/builtins.c	(revision 263529)
+++ gcc/builtins.c	(working copy)
@@ -6629,6 +6629,55 @@
 }
 
 
+/* Expand a call to __builtin_speculation_safe_value_<N>.  MODE
+   represents the size of the first argument to that call, or VOIDmode
+   if the argument is a pointer.  IGNORE will be true if the result
+   isn't used.  */
+static rtx
+expand_speculation_safe_value (machine_mode mode, tree exp, rtx target,
+			       bool ignore)
+{
+  rtx val, failsafe;
+  unsigned nargs = call_expr_nargs (exp);
+
+  tree arg0 = CALL_EXPR_ARG (exp, 0);
+
+  if (mode == VOIDmode)
+    {
+      mode = TYPE_MODE (TREE_TYPE (arg0));
+      gcc_assert (GET_MODE_CLASS (mode) == MODE_INT);
+    }
+
+  val = expand_expr (arg0, NULL_RTX, mode, EXPAND_NORMAL);
+
+  /* An optional second argument can be used as a failsafe value on
+     some machines.  If it isn't present, then the failsafe value is
+     assumed to be 0.  */
+  if (nargs > 1)
+    {
+      tree arg1 = CALL_EXPR_ARG (exp, 1);
+      failsafe = expand_expr (arg1, NULL_RTX, mode, EXPAND_NORMAL);
+    }
+  else
+    failsafe = const0_rtx;
+
+  /* If the result isn't used, the behavior is undefined.  It would be
+     nice to emit a warning here, but path splitting means this might
+     happen with legitimate code.  So simply drop the builtin
+     expansion in that case; we've handled any side-effects above.  */
+  if (ignore)
+    return const0_rtx;
+
+  /* If we don't have a suitable target, create one to hold the result.  */
+  if (target == NULL || GET_MODE (target) != mode)
+    target = gen_reg_rtx (mode);
+
+  if (GET_MODE (val) != mode && GET_MODE (val) != VOIDmode)
+    val = convert_modes (mode, VOIDmode, val, false);
+
+  return targetm.speculation_safe_value (mode, target, val, failsafe);
+}
+
 /* Expand an expression EXP that calls a built-in function,
    with result going to TARGET if that's convenient
    (and in mode MODE if that's convenient).
@@ -7758,6 +7807,17 @@
 	 folding.  */
       break;
 
+    case BUILT_IN_SPECULATION_SAFE_VALUE_PTR:
+      return expand_speculation_safe_value (VOIDmode, exp, target, ignore);
+
+    case BUILT_IN_SPECULATION_SAFE_VALUE_1:
+    case BUILT_IN_SPECULATION_SAFE_VALUE_2:
+    case BUILT_IN_SPECULATION_SAFE_VALUE_4:
+    case BUILT_IN_SPECULATION_SAFE_VALUE_8:
+    case BUILT_IN_SPECULATION_SAFE_VALUE_16:
+      mode = get_builtin_sync_mode (fcode - BUILT_IN_SPECULATION_SAFE_VALUE_1);
+      return expand_speculation_safe_value (mode, exp, target, ignore);
+
     default:	/* just do library call, if unknown builtin */
       break;
     }
Index: gcc/builtins.def
===================================================================
--- gcc/builtins.def	(revision 263529)
+++ gcc/builtins.def	(working copy)
@@ -1017,6 +1017,28 @@
 	     true, true, true, ATTR_NOTHROW_LEAF_LIST, false,
 	     !targetm.have_tls)
 
+/* Suppressing speculation.  Users are expected to use the first (N)
+   variant, which will be translated internally into one of the other
+   types.  */
+
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_N, "speculation_safe_value",
+		 BT_FN_VOID_VAR, ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_PTR,
+		 "speculation_safe_value_ptr", BT_FN_PTR_PTR_VAR,
+		 ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_1, "speculation_safe_value_1",
+		 BT_FN_I1_I1_VAR, ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_2, "speculation_safe_value_2",
+		 BT_FN_I2_I2_VAR, ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_4, "speculation_safe_value_4",
+		 BT_FN_I4_I4_VAR, ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_8, "speculation_safe_value_8",
+		 BT_FN_I8_I8_VAR, ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_VALUE_16,
+		 "speculation_safe_value_16", BT_FN_I16_I16_VAR,
+		 ATTR_NOVOPS_NOTHROW_LEAF_LIST)
+
 /* Exception support.  */
 DEF_BUILTIN_STUB (BUILT_IN_UNWIND_RESUME, "__builtin_unwind_resume")
 DEF_BUILTIN_STUB (BUILT_IN_CXA_END_CLEANUP, "__builtin_cxa_end_cleanup")
Index: gcc/c-family/ChangeLog
===================================================================
--- gcc/c-family/ChangeLog	(revision 263529)
+++ gcc/c-family/ChangeLog	(working copy)
@@ -1,3 +1,12 @@
+2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+
+	* c-common.c (speculation_safe_resolve_call): New function.
+	(speculation_safe_resolve_params): New function.
+	(speculation_safe_resolve_return): New function.
+	(resolve_overloaded_builtin): Handle __builtin_speculation_safe_value.
+	* c-cppbuiltin.c (c_cpp_builtins): Add pre-define for
+	__HAVE_SPECULATION_SAFE_VALUE.
+
 2018-07-26  Release Manager
 
 	* GCC 8.2.0 released.
Index: gcc/c-family/c-common.c
===================================================================
--- gcc/c-family/c-common.c	(revision 263529)
+++ gcc/c-family/c-common.c	(working copy)
@@ -6456,6 +6456,122 @@
   return type ? type : error_mark_node;
 }
 
+/* Work out the size of the first argument of a call to
+   __builtin_speculation_safe_value.  Only pointers and integral types
+   are permitted.  Return -1 if the argument type is not supported or
+   the size is too large; 0 if the argument type is a pointer or the
+   size if it is integral.  */
+static enum built_in_function
+speculation_safe_value_resolve_call (tree function, vec<tree, va_gc> *params)
+{
+  /* Type of the argument.  */
+  tree type;
+  int size;
+
+  if (vec_safe_is_empty (params))
+    {
+      error ("too few arguments to function %qE", function);
+      return BUILT_IN_NONE;
+    }
+
+  type = TREE_TYPE ((*params)[0]);
+  if (TREE_CODE (type) == ARRAY_TYPE && c_dialect_cxx ())
+    {
+      /* Force array-to-pointer decay for C++.   */
+      (*params)[0] = default_conversion ((*params)[0]);
+      type = TREE_TYPE ((*params)[0]);
+    }
+
+  if (POINTER_TYPE_P (type))
+    return BUILT_IN_SPECULATION_SAFE_VALUE_PTR;
+
+  if (!INTEGRAL_TYPE_P (type))
+    goto incompatible;
+
+  if (!COMPLETE_TYPE_P (type))
+    goto incompatible;
+
+  size = tree_to_uhwi (TYPE_SIZE_UNIT (type));
+  if (size == 1 || size == 2 || size == 4 || size == 8 || size == 16)
+    return ((enum built_in_function)
+	    ((int) BUILT_IN_SPECULATION_SAFE_VALUE_1 + exact_log2 (size)));
+
+ incompatible:
+  /* Issue the diagnostic only if the argument is valid, otherwise
+     it would be redundant at best and could be misleading.  */
+  if (type != error_mark_node)
+    error ("operand type %qT is incompatible with argument %d of %qE",
+	   type, 1, function);
+
+  return BUILT_IN_NONE;
+}
+
+/* Validate and coerce PARAMS, the arguments to ORIG_FUNCTION to fit
+   the prototype for FUNCTION.  The first argument is mandatory, a second
+   argument, if present, must be type compatible with the first.  */
+static bool
+speculation_safe_value_resolve_params (location_t loc, tree orig_function,
+				       vec<tree, va_gc> *params)
+{
+  tree val;
+
+  if (params->length () == 0)
+    {
+      error_at (loc, "too few arguments to function %qE", orig_function);
+      return false;
+    }
+
+  else if (params->length () > 2)
+    {
+      error_at (loc, "too many arguments to function %qE", orig_function);
+      return false;
+    }
+
+  val = (*params)[0];
+  if (TREE_CODE (TREE_TYPE (val)) == ARRAY_TYPE)
+    val = default_conversion (val);
+  if (!(TREE_CODE (TREE_TYPE (val)) == POINTER_TYPE
+	|| TREE_CODE (TREE_TYPE (val)) == INTEGER_TYPE))
+    {
+      error_at (loc,
+		"expecting argument of type pointer or of type integer "
+		"for argument 1");
+      return false;
+    }
+  (*params)[0] = val;
+
+  if (params->length () == 2)
+    {
+      tree val2 = (*params)[1];
+      if (TREE_CODE (TREE_TYPE (val2)) == ARRAY_TYPE)
+	val2 = default_conversion (val2);
+      if (!(TREE_TYPE (val) == TREE_TYPE (val2)
+	    || useless_type_conversion_p (TREE_TYPE (val), TREE_TYPE (val2))))
+	{
+	  error_at (loc, "both arguments must be compatible");
+	  return false;
+	}
+      (*params)[1] = val2;
+    }
+
+  return true;
+}
+
+/* Cast the result of the builtin back to the type of the first argument,
+   preserving any qualifiers that it might have.  */
+static tree
+speculation_safe_value_resolve_return (tree first_param, tree result)
+{
+  tree ptype = TREE_TYPE (first_param);
+  tree rtype = TREE_TYPE (result);
+  ptype = TYPE_MAIN_VARIANT (ptype);
+
+  if (tree_int_cst_equal (TYPE_SIZE (ptype), TYPE_SIZE (rtype)))
+    return convert (ptype, result);
+
+  return result;
+}
+
 /* A helper function for resolve_overloaded_builtin in resolving the
    overloaded __sync_ builtins.  Returns a positive power of 2 if the
    first operand of PARAMS is a pointer to a supported data type.
@@ -7110,6 +7226,54 @@
   /* Handle BUILT_IN_NORMAL here.  */
   switch (orig_code)
     {
+    case BUILT_IN_SPECULATION_SAFE_VALUE_N:
+      {
+	tree new_function, first_param, result;
+	enum built_in_function fncode
+	  = speculation_safe_value_resolve_call (function, params);;
+
+	first_param = (*params)[0];
+	if (fncode == BUILT_IN_NONE
+	    || !speculation_safe_value_resolve_params (loc, function, params))
+	  return error_mark_node;
+
+	if (targetm.have_speculation_safe_value (true))
+	  {
+	    new_function = builtin_decl_explicit (fncode);
+	    result = build_function_call_vec (loc, vNULL, new_function, params,
+					      NULL);
+
+	    if (result == error_mark_node)
+	      return result;
+
+	    return speculation_safe_value_resolve_return (first_param, result);
+	  }
+	else
+	  {
+	    /* This target doesn't have, or doesn't need, active mitigation
+	       against incorrect speculative execution.  Simply return the
+	       first parameter to the builtin.  */
+	    if (!targetm.have_speculation_safe_value (false))
+	      /* The user has invoked __builtin_speculation_safe_value
+		 even though __HAVE_SPECULATION_SAFE_VALUE is not
+		 defined: emit a warning.  */
+	      warning_at (input_location, 0,
+			  "this target does not define a speculation barrier; "
+			  "your program will still execute correctly, "
+			  "but incorrect speculation may not be be "
+			  "restricted");
+
+	    /* If the optional second argument is present, handle any side
+	       effects now.  */
+	    if (params->length () == 2
+		&& TREE_SIDE_EFFECTS ((*params)[1]))
+	      return build2 (COMPOUND_EXPR, TREE_TYPE (first_param),
+			     (*params)[1], first_param);
+
+	    return first_param;
+	  }
+      }
+
     case BUILT_IN_ATOMIC_EXCHANGE:
     case BUILT_IN_ATOMIC_COMPARE_EXCHANGE:
     case BUILT_IN_ATOMIC_LOAD:
Index: gcc/c-family/c-cppbuiltin.c
===================================================================
--- gcc/c-family/c-cppbuiltin.c	(revision 263529)
+++ gcc/c-family/c-cppbuiltin.c	(working copy)
@@ -1361,7 +1361,12 @@
     cpp_define (pfile, "__WCHAR_UNSIGNED__");
 
   cpp_atomic_builtins (pfile);
-    
+
+  /* Show support for __builtin_speculation_safe_value () if the target
+     has been updated to fully support it.  */
+  if (targetm.have_speculation_safe_value (false))
+    cpp_define (pfile, "__HAVE_SPECULATION_SAFE_VALUE");
+
 #ifdef DWARF2_UNWIND_INFO
   if (dwarf2out_do_cfi_asm ())
     cpp_define (pfile, "__GCC_HAVE_DWARF2_CFI_ASM");
Index: gcc/config/aarch64/aarch64-passes.def
===================================================================
--- gcc/config/aarch64/aarch64-passes.def	(revision 263529)
+++ gcc/config/aarch64/aarch64-passes.def	(working copy)
@@ -19,3 +19,4 @@
    <http://www.gnu.org/licenses/>.  */
 
 INSERT_PASS_AFTER (pass_regrename, 1, pass_fma_steering);
+INSERT_PASS_BEFORE (pass_reorder_blocks, 1, pass_track_speculation);
Index: gcc/config/aarch64/aarch64-protos.h
===================================================================
--- gcc/config/aarch64/aarch64-protos.h	(revision 263529)
+++ gcc/config/aarch64/aarch64-protos.h	(working copy)
@@ -547,7 +547,8 @@
 std::string aarch64_get_extension_string_for_isa_flags (unsigned long,
 							unsigned long);
 
-rtl_opt_pass *make_pass_fma_steering (gcc::context *ctxt);
+rtl_opt_pass *make_pass_fma_steering (gcc::context *);
+rtl_opt_pass *make_pass_track_speculation (gcc::context *);
 
 poly_uint64 aarch64_regmode_natural_size (machine_mode);
 
Index: gcc/config/aarch64/aarch64-speculation.cc
===================================================================
--- gcc/config/aarch64/aarch64-speculation.cc	(nonexistent)
+++ gcc/config/aarch64/aarch64-speculation.cc	(working copy)
@@ -0,0 +1,494 @@
+/* Speculation tracking and mitigation (e.g. CVE 2017-5753) for AArch64.
+   Copyright (C) 2018 Free Software Foundation, Inc.
+   Contributed by ARM Ltd.
+
+   This file is part of GCC.
+
+   GCC is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3, or (at your option)
+   any later version.
+
+   GCC is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GCC; see the file COPYING3.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "target.h"
+#include "rtl.h"
+#include "tree-pass.h"
+#include "profile-count.h"
+#include "cfg.h"
+#include "cfgbuild.h"
+#include "print-rtl.h"
+#include "cfgrtl.h"
+#include "function.h"
+#include "basic-block.h"
+#include "memmodel.h"
+#include "emit-rtl.h"
+#include "insn-attr.h"
+#include "df.h"
+#include "tm_p.h"
+#include "insn-config.h"
+#include "recog.h"
+
+/* This pass scans the RTL just before the final branch
+   re-organisation pass.  The aim is to identify all places where
+   there is conditional control flow and to insert code that tracks
+   any speculative execution of a conditional branch.
+
+   To do this we reserve a call-clobbered register (so that it can be
+   initialized very early in the function prologue) that can then be
+   updated each time there is a conditional branch.  At each such
+   branch we then generate a code sequence that uses conditional
+   select operations that are not subject to speculation themselves
+   (we ignore for the moment situations where that might not always be
+   strictly true).  For example, a branch sequence such as:
+
+	B.EQ	<dst>
+	...
+   <dst>:
+
+   is transformed to:
+
+	B.EQ	<dst>
+	CSEL	tracker, tracker, XZr, ne
+	...
+   <dst>:
+	CSEL	tracker, tracker, XZr, eq
+
+   Since we start with the tracker initialized to all bits one, if at any
+   time the predicted control flow diverges from the architectural program
+   behavior, then the tracker will become zero (but not otherwise).
+
+   The tracker value can be used at any time at which a value needs
+   guarding against incorrect speculation.  This can be done in
+   several ways, but they all amount to the same thing.  For an
+   untrusted address, or an untrusted offset to a trusted address, we
+   can simply mask the address with the tracker with the untrusted
+   value.  If the CPU is not speculating, or speculating correctly,
+   then the value will remain unchanged, otherwise it will be clamped
+   to zero.  For more complex scenarios we can compare the tracker
+   against zero and use the flags to form a new selection with an
+   alternate safe value.
+
+   On implementations where the data processing instructions may
+   themselves produce speculative values, the architecture requires
+   that a CSDB instruction will resolve such data speculation, so each
+   time we use the tracker for protecting a vulnerable value we also
+   emit a CSDB: we do not need to do that each time the tracker itself
+   is updated.
+
+   At function boundaries, we need to communicate the speculation
+   tracking state with the caller or the callee.  This is tricky
+   because there is no register available for such a purpose without
+   creating a new ABI.  We deal with this by relying on the principle
+   that in all real programs the stack pointer, SP will never be NULL
+   at a function boundary; we can thus encode the speculation state in
+   SP by clearing SP if the speculation tracker itself is NULL.  After
+   the call we recover the tracking state back from SP into the
+   tracker register.  The results is that a function call sequence is
+   transformed to
+
+	MOV	tmp, SP
+	AND	tmp, tmp, tracker
+	MOV	SP, tmp
+	BL	<callee>
+	CMP	SP, #0
+	CSETM	tracker, ne
+
+   The additional MOV instructions in the pre-call sequence are needed
+   because SP cannot be used directly with the AND instruction.
+
+   The code inside a function body uses the post-call sequence in the
+   prologue to establish the tracker and the pre-call sequence in the
+   epilogue to re-encode the state for the return.
+
+   The code sequences have the nice property that if called from, or
+   calling a function that does not track speculation then the stack pointer
+   will always be non-NULL and hence the tracker will be initialized to all
+   bits one as we need: we lose the ability to fully track speculation in that
+   case, but we are still architecturally safe.
+
+   Tracking speculation in this way is quite expensive, both in code
+   size and execution time.  We employ a number of tricks to try to
+   limit this:
+
+   1) Simple leaf functions with no conditional branches (or use of
+   the tracker) do not need to establish a new tracker: they simply
+   carry the tracking state through SP for the duration of the call.
+   The same is also true for leaf functions that end in a tail-call.
+
+   2) Back-to-back function calls in a single basic block also do not
+   need to re-establish the tracker between the calls.  Again, we can
+   carry the tracking state in SP for this period of time unless the
+   tracker value is needed at that point in time.
+
+   We run the pass just before the final branch reorganization pass so
+   that we can handle most of the conditional branch cases using the
+   standard edge insertion code.  The reorg pass will hopefully clean
+   things up for afterwards so that the results aren't too
+   horrible.  */
+
+/* Generate a code sequence to clobber SP if speculating incorreclty.  */
+static rtx_insn *
+aarch64_speculation_clobber_sp ()
+{
+  rtx sp = gen_rtx_REG (DImode, SP_REGNUM);
+  rtx tracker = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+  rtx scratch = gen_rtx_REG (DImode, SPECULATION_SCRATCH_REGNUM);
+
+  start_sequence ();
+  emit_insn (gen_rtx_SET (scratch, sp));
+  emit_insn (gen_anddi3 (scratch, scratch, tracker));
+  emit_insn (gen_rtx_SET (sp, scratch));
+  rtx_insn *seq = get_insns ();
+  end_sequence ();
+  return seq;
+}
+
+/* Generate a code sequence to establish the tracker variable from the
+   contents of SP.  */
+static rtx_insn *
+aarch64_speculation_establish_tracker ()
+{
+  rtx sp = gen_rtx_REG (DImode, SP_REGNUM);
+  rtx tracker = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+  start_sequence ();
+  rtx cc = aarch64_gen_compare_reg (EQ, sp, const0_rtx);
+  emit_insn (gen_cstoredi_neg (tracker,
+			       gen_rtx_NE (CCmode, cc, const0_rtx), cc));
+  rtx_insn *seq = get_insns ();
+  end_sequence ();
+  return seq;
+}
+
+/* Main speculation tracking pass.  */
+unsigned int
+aarch64_do_track_speculation ()
+{
+  basic_block bb;
+  bool needs_tracking = false;
+  bool need_second_pass = false;
+  rtx_insn *insn;
+  int fixups_pending = 0;
+
+  FOR_EACH_BB_FN (bb, cfun)
+    {
+      insn = BB_END (bb);
+
+      if (dump_file)
+	fprintf (dump_file, "Basic block %d:\n", bb->index);
+
+      while (insn != BB_HEAD (bb)
+	     && NOTE_P (insn))
+	insn = PREV_INSN (insn);
+
+      if (control_flow_insn_p (insn))
+	{
+	  if (any_condjump_p (insn))
+	    {
+	      if (dump_file)
+		{
+		  fprintf (dump_file, "  condjump\n");
+		  dump_insn_slim (dump_file, insn);
+		}
+
+	      rtx src = SET_SRC (pc_set (insn));
+
+	      /* Check for an inverted jump, where the fall-through edge
+		 appears first.  */
+	      bool inverted = GET_CODE (XEXP (src, 2)) != PC;
+	      /* The other edge must be the PC (we assume that we don't
+		 have conditional return instructions).  */
+	      gcc_assert (GET_CODE (XEXP (src, 1 + !inverted)) == PC);
+
+	      rtx cond = copy_rtx (XEXP (src, 0));
+	      gcc_assert (COMPARISON_P (cond)
+			  && REG_P (XEXP (cond, 0))
+			  && REGNO (XEXP (cond, 0)) == CC_REGNUM
+			  && XEXP (cond, 1) == const0_rtx);
+	      enum rtx_code inv_cond_code
+		= reversed_comparison_code (cond, insn);
+	      /* We should be able to reverse all conditions.  */
+	      gcc_assert (inv_cond_code != UNKNOWN);
+	      rtx inv_cond = gen_rtx_fmt_ee (inv_cond_code, GET_MODE (cond),
+					     copy_rtx (XEXP (cond, 0)),
+					     copy_rtx (XEXP (cond, 1)));
+	      if (inverted)
+		std::swap (cond, inv_cond);
+
+	      insert_insn_on_edge (gen_speculation_tracker (cond),
+				   BRANCH_EDGE (bb));
+	      insert_insn_on_edge (gen_speculation_tracker (inv_cond),
+				   FALLTHRU_EDGE (bb));
+	      needs_tracking = true;
+	    }
+	  else if (GET_CODE (PATTERN (insn)) == RETURN)
+	    {
+	      /* If we already know we'll need a second pass, don't put
+		 out the return sequence now, or we might end up with
+		 two copies.  Instead, we'll do all return statements
+		 during the second pass.  However, if this is the
+		 first return insn we've found and we already
+		 know that we'll need to emit the code, we can save a
+		 second pass by emitting the code now.  */
+	      if (needs_tracking && ! need_second_pass)
+		{
+		  rtx_insn *seq = aarch64_speculation_clobber_sp ();
+		  emit_insn_before (seq, insn);
+		}
+	      else
+		{
+		  fixups_pending++;
+		  need_second_pass = true;
+		}
+	    }
+	  else if (find_reg_note (insn, REG_NON_LOCAL_GOTO, NULL_RTX))
+	    {
+	      rtx_insn *seq = aarch64_speculation_clobber_sp ();
+	      emit_insn_before (seq, insn);
+	      needs_tracking = true;
+	    }
+	}
+      else
+	{
+	  if (dump_file)
+	    {
+	      fprintf (dump_file, "  other\n");
+	      dump_insn_slim (dump_file, insn);
+	    }
+	}
+    }
+
+  FOR_EACH_BB_FN (bb, cfun)
+    {
+      rtx_insn *end = BB_END (bb);
+      rtx_insn *call_insn = NULL;
+
+      if (bb->flags & BB_NON_LOCAL_GOTO_TARGET)
+	{
+	  rtx_insn *label = NULL;
+	  /* For non-local goto targets we have to recover the
+	     speculation state from SP.  Find the last code label at
+	     the head of the block and place the fixup sequence after
+	     that.  */
+	  for (insn = BB_HEAD (bb); insn != end; insn = NEXT_INSN (insn))
+	    {
+	      if (LABEL_P (insn))
+		label = insn;
+	      /* Never put anything before the basic block note.  */
+	      if (NOTE_INSN_BASIC_BLOCK_P (insn))
+		label = insn;
+	      if (INSN_P (insn))
+		break;
+	    }
+
+	  gcc_assert (label);
+	  emit_insn_after (aarch64_speculation_establish_tracker (), label);
+	}
+
+      /* Scan the insns looking for calls.  We need to pass the
+	 speculation tracking state encoded in to SP.  After a call we
+	 restore the speculation tracking into the tracker register.
+	 To avoid unnecessary transfers we look for two or more calls
+	 within a single basic block and eliminate, where possible,
+	 any redundant operations.  */
+      for (insn = BB_HEAD (bb); ; insn = NEXT_INSN (insn))
+	{
+	  if (NONDEBUG_INSN_P (insn)
+	      && recog_memoized (insn) >= 0
+	      && (get_attr_speculation_barrier (insn)
+		  == SPECULATION_BARRIER_TRUE))
+	    {
+	      if (call_insn)
+		{
+		  /* This instruction requires the speculation
+		     tracking to be in the tracker register.  If there
+		     was an earlier call in this block, we need to
+		     copy the speculation tracking back there.  */
+		  emit_insn_after (aarch64_speculation_establish_tracker (),
+				   call_insn);
+		  call_insn = NULL;
+		}
+
+	      needs_tracking = true;
+	    }
+
+	  if (CALL_P (insn))
+	    {
+	      bool tailcall
+		= (SIBLING_CALL_P (insn)
+		   || find_reg_note (insn, REG_NORETURN, NULL_RTX));
+
+	      /* Tailcalls are like returns, we can eliminate the
+		 transfer between the tracker register and SP if we
+		 know that this function does not itself need
+		 tracking.  */
+	      if (tailcall && (need_second_pass || !needs_tracking))
+		{
+		  /* Don't clear call_insn if it is set - needs_tracking
+		     will be true in that case and so we will end
+		     up putting out mitigation sequences.  */
+		  fixups_pending++;
+		  need_second_pass = true;
+		  break;
+		}
+
+	      needs_tracking = true;
+
+	      /* We always need a transfer before the first call in a BB.  */
+	      if (!call_insn)
+		emit_insn_before (aarch64_speculation_clobber_sp (), insn);
+
+	      /* Tail-calls and no-return calls don't need any post-call
+		 reestablishment of the tracker.  */
+	      if (! tailcall)
+		call_insn = insn;
+	      else
+		call_insn = NULL;
+	    }
+
+	  if (insn == end)
+	    break;
+	}
+
+      if (call_insn)
+	{
+	  rtx_insn *seq = aarch64_speculation_establish_tracker ();
+
+	  /* Handle debug insns at the end of the BB.  Put the extra
+	     insns after them.  This ensures that we have consistent
+	     behaviour for the placement of the extra insns between
+	     debug and non-debug builds.  */
+	  for (insn = call_insn;
+	       insn != end && DEBUG_INSN_P (NEXT_INSN (insn));
+	       insn = NEXT_INSN (insn))
+	    ;
+
+	  if (insn == end)
+	    {
+	      edge e = find_fallthru_edge (bb->succs);
+	      /* We need to be very careful about some calls that
+		 appear at the end of a basic block.  If the call
+		 involves exceptions, then the compiler may depend on
+		 this being the last instruction in the block.  The
+		 easiest way to handle this is to commit the new
+		 instructions on the fall-through edge and to let
+		 commit_edge_insertions clean things up for us.
+
+		 Sometimes, eg with OMP, there may not even be an
+		 outgoing edge after the call.  In that case, there's
+		 not much we can do, presumably the compiler has
+		 decided that the call can never return in this
+		 context.  */
+	      if (e)
+		{
+		  /* We need to set the location lists explicitly in
+		     this case.  */
+		  if (! INSN_P (seq))
+		    {
+		      start_sequence ();
+		      emit_insn (seq);
+		      seq = get_insns ();
+		      end_sequence ();
+		    }
+
+		  for (rtx_insn *list = seq; list; list = NEXT_INSN (list))
+		    INSN_LOCATION (list) = INSN_LOCATION (call_insn);
+
+		  insert_insn_on_edge (seq, e);
+		}
+	    }
+	  else
+	    emit_insn_after (seq, call_insn);
+	}
+    }
+
+  if (needs_tracking)
+    {
+      if (need_second_pass)
+	{
+	  /* We found a return instruction before we found out whether
+	     or not we need to emit the tracking code, but we now
+	     know we do.  Run quickly over the basic blocks and
+	     fix up the return insns.  */
+	  FOR_EACH_BB_FN (bb, cfun)
+	    {
+	      insn = BB_END (bb);
+
+	      while (insn != BB_HEAD (bb)
+		     && NOTE_P (insn))
+		insn = PREV_INSN (insn);
+
+	      if ((control_flow_insn_p (insn)
+		   && GET_CODE (PATTERN (insn)) == RETURN)
+		  || (CALL_P (insn)
+		      && (SIBLING_CALL_P (insn)
+			  || find_reg_note (insn, REG_NORETURN, NULL_RTX))))
+		{
+		  rtx_insn *seq = aarch64_speculation_clobber_sp ();
+		  emit_insn_before (seq, insn);
+		  fixups_pending--;
+		}
+	    }
+	  gcc_assert (fixups_pending == 0);
+	}
+
+      /* Set up the initial value of the tracker, using the incoming SP.  */
+      insert_insn_on_edge (aarch64_speculation_establish_tracker (),
+			   single_succ_edge (ENTRY_BLOCK_PTR_FOR_FN (cfun)));
+      commit_edge_insertions ();
+    }
+
+  return 0;
+}
+
+namespace {
+
+const pass_data pass_data_aarch64_track_speculation =
+{
+  RTL_PASS,		/* type.  */
+  "speculation",	/* name.  */
+  OPTGROUP_NONE,	/* optinfo_flags.  */
+  TV_MACH_DEP,		/* tv_id.  */
+  0,			/* properties_required.  */
+  0,			/* properties_provided.  */
+  0,			/* properties_destroyed.  */
+  0,			/* todo_flags_start.  */
+  0			/* todo_flags_finish.  */
+};
+
+class pass_track_speculation : public rtl_opt_pass
+{
+ public:
+  pass_track_speculation(gcc::context *ctxt)
+    : rtl_opt_pass(pass_data_aarch64_track_speculation, ctxt)
+    {}
+
+  /* opt_pass methods:  */
+  virtual bool gate (function *)
+    {
+      return aarch64_track_speculation;
+    }
+
+  virtual unsigned int execute (function *)
+    {
+      return aarch64_do_track_speculation ();
+    }
+}; // class pass_track_speculation.
+} // anon namespace.
+
+/* Create a new pass instance.  */
+rtl_opt_pass *
+make_pass_track_speculation (gcc::context *ctxt)
+{
+  return new pass_track_speculation (ctxt);
+}
Index: gcc/config/aarch64/aarch64.c
===================================================================
--- gcc/config/aarch64/aarch64.c	(revision 263529)
+++ gcc/config/aarch64/aarch64.c	(working copy)
@@ -6684,6 +6684,12 @@
       break;
 
     case 'H':
+      if (x == const0_rtx)
+	{
+	  asm_fprintf (f, "xzr");
+	  break;
+	}
+
       if (!REG_P (x) || !GP_REGNUM_P (REGNO (x) + 1))
 	{
 	  output_operand_lossage ("invalid operand for '%%%c'", code);
@@ -12510,6 +12516,19 @@
 	fixed_regs[i] = 1;
 	call_used_regs[i] = 1;
       }
+
+  /* When tracking speculation, we need a couple of call-clobbered registers
+     to track the speculation state.  It would be nice to just use
+     IP0 and IP1, but currently there are numerous places that just
+     assume these registers are free for other uses (eg pointer
+     authentication).  */
+  if (aarch64_track_speculation)
+    {
+      fixed_regs[SPECULATION_TRACKER_REGNUM] = 1;
+      call_used_regs[SPECULATION_TRACKER_REGNUM] = 1;
+      fixed_regs[SPECULATION_SCRATCH_REGNUM] = 1;
+      call_used_regs[SPECULATION_SCRATCH_REGNUM] = 1;
+    }
 }
 
 /* Walk down the type tree of TYPE counting consecutive base elements.
@@ -14357,7 +14376,16 @@
 
   if (strong_zero_p)
     {
-      x = gen_rtx_NE (VOIDmode, rval, const0_rtx);
+      if (aarch64_track_speculation)
+	{
+	  /* Emit an explicit compare instruction, so that we can correctly
+	     track the condition codes.  */
+	  rtx cc_reg = aarch64_gen_compare_reg (NE, rval, const0_rtx);
+	  x = gen_rtx_NE (GET_MODE (cc_reg), cc_reg, const0_rtx);
+	}
+      else
+	x = gen_rtx_NE (VOIDmode, rval, const0_rtx);
+
       x = gen_rtx_IF_THEN_ELSE (VOIDmode, x,
 				gen_rtx_LABEL_REF (Pmode, label2), pc_rtx);
       aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x));
@@ -14375,7 +14403,16 @@
 
   if (!is_weak)
     {
-      x = gen_rtx_NE (VOIDmode, scratch, const0_rtx);
+      if (aarch64_track_speculation)
+	{
+	  /* Emit an explicit compare instruction, so that we can correctly
+	     track the condition codes.  */
+	  rtx cc_reg = aarch64_gen_compare_reg (NE, scratch, const0_rtx);
+	  x = gen_rtx_NE (GET_MODE (cc_reg), cc_reg, const0_rtx);
+	}
+      else
+	x = gen_rtx_NE (VOIDmode, scratch, const0_rtx);
+
       x = gen_rtx_IF_THEN_ELSE (VOIDmode, x,
 				gen_rtx_LABEL_REF (Pmode, label1), pc_rtx);
       aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x));
@@ -14711,7 +14748,16 @@
   aarch64_emit_store_exclusive (mode, cond, mem,
 				gen_lowpart (mode, new_out), model_rtx);
 
-  x = gen_rtx_NE (VOIDmode, cond, const0_rtx);
+  if (aarch64_track_speculation)
+    {
+      /* Emit an explicit compare instruction, so that we can correctly
+	 track the condition codes.  */
+      rtx cc_reg = aarch64_gen_compare_reg (NE, cond, const0_rtx);
+      x = gen_rtx_NE (GET_MODE (cc_reg), cc_reg, const0_rtx);
+    }
+  else
+    x = gen_rtx_NE (VOIDmode, cond, const0_rtx);
+
   x = gen_rtx_IF_THEN_ELSE (VOIDmode, x,
 			    gen_rtx_LABEL_REF (Pmode, label), pc_rtx);
   aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x));
@@ -17390,6 +17436,45 @@
     }
 }
 
+/* Override the default target speculation_safe_value.  */
+static rtx
+aarch64_speculation_safe_value (machine_mode mode,
+				rtx result, rtx val, rtx failval)
+{
+  /* Maybe we should warn if falling back to hard barriers.  They are
+     likely to be noticably more expensive than the alternative below.  */
+  if (!aarch64_track_speculation)
+    return default_speculation_safe_value (mode, result, val, failval);
+
+  if (!REG_P (val))
+    val = copy_to_mode_reg (mode, val);
+
+  if (!aarch64_reg_or_zero (failval, mode))
+    failval = copy_to_mode_reg (mode, failval);
+
+  switch (mode)
+    {
+    case E_QImode:
+      emit_insn (gen_despeculate_copyqi (result, val, failval));
+      break;
+    case E_HImode:
+      emit_insn (gen_despeculate_copyhi (result, val, failval));
+      break;
+    case E_SImode:
+      emit_insn (gen_despeculate_copysi (result, val, failval));
+      break;
+    case E_DImode:
+      emit_insn (gen_despeculate_copydi (result, val, failval));
+      break;
+    case E_TImode:
+      emit_insn (gen_despeculate_copyti (result, val, failval));
+      break;
+    default:
+      gcc_unreachable ();
+    }
+  return result;
+}
+
 /* Target-specific selftests.  */
 
 #if CHECKING_P
@@ -17859,6 +17944,9 @@
 #undef TARGET_SELECT_EARLY_REMAT_MODES
 #define TARGET_SELECT_EARLY_REMAT_MODES aarch64_select_early_remat_modes
 
+#undef TARGET_SPECULATION_SAFE_VALUE
+#define TARGET_SPECULATION_SAFE_VALUE aarch64_speculation_safe_value
+
 #if CHECKING_P
 #undef TARGET_RUN_TARGET_SELFTESTS
 #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests
Index: gcc/config/aarch64/aarch64.md
===================================================================
--- gcc/config/aarch64/aarch64.md	(revision 263529)
+++ gcc/config/aarch64/aarch64.md	(working copy)
@@ -88,6 +88,10 @@
     (P13_REGNUM		81)
     (P14_REGNUM		82)
     (P15_REGNUM		83)
+    ;; A couple of call-clobbered registers that we need to reserve when
+    ;; tracking speculation this is not ABI, so is subject to change.
+    (SPECULATION_TRACKER_REGNUM 15)
+    (SPECULATION_SCRATCH_REGNUM 14)
   ]
 )
 
@@ -189,6 +193,7 @@
     UNSPEC_CLASTB
     UNSPEC_FADDA
     UNSPEC_REV_SUBREG
+    UNSPEC_SPECULATION_TRACKER
 ])
 
 (define_c_enum "unspecv" [
@@ -199,6 +204,7 @@
     UNSPECV_SET_FPSR		; Represent assign of FPSR content.
     UNSPECV_BLOCKAGE		; Represent a blockage
     UNSPECV_PROBE_STACK_RANGE	; Represent stack range probing.
+    UNSPECV_SPECULATION_BARRIER ; Represent speculation barrier.
   ]
 )
 
@@ -275,6 +281,11 @@
 ;; no predicated insns.
 (define_attr "predicated" "yes,no" (const_string "no"))
 
+;; Set to true on an insn that requires the speculation tracking state to be
+;; in the tracking register before the insn issues.  Otherwise the compiler
+;; may chose to hold the tracking state encoded in SP.
+(define_attr "speculation_barrier" "true,false" (const_string "false"))
+
 ;; -------------------------------------------------------------------
 ;; Pipeline descriptions and scheduling
 ;; -------------------------------------------------------------------
@@ -678,7 +689,7 @@
 				(const_int 0))
 			   (label_ref (match_operand 1 "" ""))
 			   (pc)))]
-  ""
+  "!aarch64_track_speculation"
   {
     if (get_attr_length (insn) == 8)
       return aarch64_gen_far_branch (operands, 1, "Lcb", "<inv_cb>\\t%<w>0, ");
@@ -708,7 +719,7 @@
 	     (label_ref (match_operand 2 "" ""))
 	     (pc)))
    (clobber (reg:CC CC_REGNUM))]
-  ""
+  "!aarch64_track_speculation"
   {
     if (get_attr_length (insn) == 8)
       {
@@ -744,7 +755,7 @@
 			   (label_ref (match_operand 1 "" ""))
 			   (pc)))
    (clobber (reg:CC CC_REGNUM))]
-  ""
+  "!aarch64_track_speculation"
   {
     if (get_attr_length (insn) == 8)
       {
@@ -3133,7 +3144,7 @@
 
 (define_insn "cmp<mode>"
   [(set (reg:CC CC_REGNUM)
-	(compare:CC (match_operand:GPI 0 "register_operand" "r,r,r")
+	(compare:CC (match_operand:GPI 0 "register_operand" "rk,rk,rk")
 		    (match_operand:GPI 1 "aarch64_plus_operand" "r,I,J")))]
   ""
   "@
@@ -6081,6 +6092,21 @@
   DONE;
 })
 
+;; Track speculation through conditional branches.  We assume that
+;; SPECULATION_TRACKER_REGNUM is reserved for this purpose when necessary.
+(define_insn "speculation_tracker"
+  [(set (reg:DI SPECULATION_TRACKER_REGNUM)
+	(unspec [(reg:DI SPECULATION_TRACKER_REGNUM) (match_operand 0)]
+	 UNSPEC_SPECULATION_TRACKER))]
+  ""
+  {
+    operands[1] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+    output_asm_insn ("csel\\t%1, %1, xzr, %m0", operands);
+    return "";
+  }
+  [(set_attr "type" "csel")]
+)
+
 ;; Helper for aarch64.c code.
 (define_expand "set_clobber_cc"
   [(parallel [(set (match_operand 0)
@@ -6087,6 +6113,113 @@
 		   (match_operand 1))
 	      (clobber (reg:CC CC_REGNUM))])])
 
+;; Hard speculation barrier.
+(define_insn "speculation_barrier"
+  [(unspec_volatile [(const_int 0)] UNSPECV_SPECULATION_BARRIER)]
+  ""
+  "isb\;dsb\\tsy"
+  [(set_attr "length" "8")
+   (set_attr "type" "block")
+   (set_attr "speculation_barrier" "true")]
+)
+
+;; Support for __builtin_speculation_safe_value when we have speculation
+;; tracking enabled.  Use the speculation tracker to decide whether to
+;; copy operand 1 to the target, or to copy the fail value (operand 2).
+(define_expand "despeculate_copy<ALLI_TI:mode>"
+  [(set (match_operand:ALLI_TI 0 "register_operand" "=r")
+	(unspec_volatile:ALLI_TI
+	 [(match_operand:ALLI_TI 1 "register_operand" "r")
+	  (match_operand:ALLI_TI 2 "aarch64_reg_or_zero" "rZ")
+	  (use (reg:DI SPECULATION_TRACKER_REGNUM))
+	  (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))]
+  ""
+  "
+  {
+    if (operands[2] == const0_rtx)
+      {
+	rtx tracker;
+	if (<MODE>mode == TImode)
+	  tracker = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+	else
+	  tracker = gen_rtx_REG (<MODE>mode, SPECULATION_TRACKER_REGNUM);
+
+	emit_insn (gen_despeculate_simple<mode> (operands[0], operands[1],
+						 tracker));
+	DONE;
+      }
+  }
+  "
+)
+
+;; Patterns to match despeculate_copy<mode>.  Note that "hint 0x14" is the
+;; encoding for CSDB, but will work in older versions of the assembler.
+(define_insn "*despeculate_copy<ALLI:mode>_insn"
+  [(set (match_operand:ALLI 0 "register_operand" "=r")
+	(unspec_volatile:ALLI
+	 [(match_operand:ALLI 1 "register_operand" "r")
+	  (match_operand:ALLI 2 "aarch64_reg_or_zero" "rZ")
+	  (use (reg:DI SPECULATION_TRACKER_REGNUM))
+	  (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))]
+  ""
+  {
+    operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+    output_asm_insn ("cmp\\t%3, #0\;csel\\t%<w>0, %<w>1, %<w>2, ne\;hint\t0x14 // csdb",
+		     operands);
+    return "";
+  }
+  [(set_attr "length" "12")
+   (set_attr "type" "block")
+   (set_attr "speculation_barrier" "true")]
+)
+
+;; Pattern to match despeculate_copyti
+(define_insn "*despeculate_copyti_insn"
+  [(set (match_operand:TI 0 "register_operand" "=r")
+	(unspec_volatile:TI
+	 [(match_operand:TI 1 "register_operand" "r")
+	  (match_operand:TI 2 "aarch64_reg_or_zero" "rZ")
+	  (use (reg:DI SPECULATION_TRACKER_REGNUM))
+	  (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))]
+  ""
+  {
+    operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM);
+    output_asm_insn
+      ("cmp\\t%3, #0\;csel\\t%0, %1, %2, ne\;csel\\t%H0, %H1, %H2, ne\;hint\t0x14 // csdb",
+       operands);
+    return "";
+  }
+  [(set_attr "length" "16")
+   (set_attr "type" "block")
+   (set_attr "speculation_barrier" "true")]
+)
+
+(define_insn "despeculate_simple<ALLI:mode>"
+  [(set (match_operand:ALLI 0 "register_operand" "=r")
+	(unspec_volatile:ALLI
+	 [(match_operand:ALLI 1 "register_operand" "r")
+	  (use (match_operand:ALLI 2 "register_operand" ""))]
+	 UNSPECV_SPECULATION_BARRIER))]
+  ""
+  "and\\t%<w>0, %<w>1, %<w>2\;hint\t0x14 // csdb"
+  [(set_attr "type" "block")
+   (set_attr "length" "8")
+   (set_attr "speculation_barrier" "true")]
+)
+
+(define_insn "despeculate_simpleti"
+  [(set (match_operand:TI 0 "register_operand" "=r")
+	(unspec_volatile:TI
+	 [(match_operand:TI 1 "register_operand" "r")
+	  (use (match_operand:DI 2 "register_operand" ""))]
+	 UNSPECV_SPECULATION_BARRIER))]
+  ""
+  "and\\t%0, %1, %2\;and\\t%H0, %H1, %2\;hint\t0x14 // csdb"
+  [(set_attr "type" "block")
+   (set_attr "length" "12")
+   (set_attr "speculation_barrier" "true")]
+)
+
 ;; AdvSIMD Stuff
 (include "aarch64-simd.md")
 
Index: gcc/config/aarch64/aarch64.opt
===================================================================
--- gcc/config/aarch64/aarch64.opt	(revision 263529)
+++ gcc/config/aarch64/aarch64.opt	(working copy)
@@ -214,3 +214,7 @@
 mverbose-cost-dump
 Common Undocumented Var(flag_aarch64_verbose_cost)
 Enables verbose cost model dumping in the debug dump files.
+
+mtrack-speculation
+Target Var(aarch64_track_speculation)
+Generate code to track when the CPU might be speculating incorrectly.
Index: gcc/config/aarch64/iterators.md
===================================================================
--- gcc/config/aarch64/iterators.md	(revision 263529)
+++ gcc/config/aarch64/iterators.md	(working copy)
@@ -35,6 +35,9 @@
 ;; Iterator for all integer modes (up to 64-bit)
 (define_mode_iterator ALLI [QI HI SI DI])
 
+;; Iterator for all integer modes (up to 128-bit)
+(define_mode_iterator ALLI_TI [QI HI SI DI TI])
+
 ;; Iterator for all integer modes that can be extended (up to 64-bit)
 (define_mode_iterator ALLX [QI HI SI])
 
Index: gcc/config/aarch64/t-aarch64
===================================================================
--- gcc/config/aarch64/t-aarch64	(revision 263529)
+++ gcc/config/aarch64/t-aarch64	(working copy)
@@ -67,6 +67,16 @@
 	$(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
 		$(srcdir)/config/aarch64/cortex-a57-fma-steering.c
 
+aarch64-speculation.o: $(srcdir)/config/aarch64/aarch64-speculation.cc \
+    $(CONFIG_H) \
+    $(SYSTEM_H) \
+    $(TM_H) \
+    $(TARGET_H) \
+    $(RTL_BASE_H) \
+    $(TREE_PASS_H)
+	$(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_SPPFLAGS) $(INCLUDES) \
+	  $(srcdir)/config/aarch64/aarch64-speculation.cc
+
 comma=,
 MULTILIB_OPTIONS    = $(subst $(comma),/, $(patsubst %, mabi=%, $(subst $(comma),$(comma)mabi=,$(TM_MULTILIB_CONFIG))))
 MULTILIB_DIRNAMES   = $(subst $(comma), ,$(TM_MULTILIB_CONFIG))
Index: gcc/config/arm/arm.md
===================================================================
--- gcc/config/arm/arm.md	(revision 263529)
+++ gcc/config/arm/arm.md	(working copy)
@@ -12012,6 +12012,27 @@
   [(set_attr "length" "4")
    (set_attr "type" "coproc")])
 
+(define_expand "speculation_barrier"
+  [(unspec_volatile [(const_int 0)] VUNSPEC_SPECULATION_BARRIER)]
+  "TARGET_EITHER"
+  "
+    /* Don't emit anything for Thumb1 and suppress the warning from the
+       generic expansion.  */
+    if (!TARGET_32BIT)
+       DONE;
+  "
+)
+
+;; Generate a hard speculation barrier when we have not enabled speculation
+;; tracking.
+(define_insn "*speculation_barrier_insn"
+  [(unspec_volatile [(const_int 0)] VUNSPEC_SPECULATION_BARRIER)]
+  "TARGET_32BIT"
+  "isb\;dsb\\tsy"
+  [(set_attr "type" "block")
+   (set_attr "length" "8")]
+)
+
 ;; Vector bits common to IWMMXT and Neon
 (include "vec-common.md")
 ;; Load the Intel Wireless Multimedia Extension patterns
Index: gcc/config/arm/unspecs.md
===================================================================
--- gcc/config/arm/unspecs.md	(revision 263529)
+++ gcc/config/arm/unspecs.md	(working copy)
@@ -168,6 +168,7 @@
   VUNSPEC_MCRR2		; Represent the coprocessor mcrr2 instruction.
   VUNSPEC_MRRC		; Represent the coprocessor mrrc instruction.
   VUNSPEC_MRRC2		; Represent the coprocessor mrrc2 instruction.
+  VUNSPEC_SPECULATION_BARRIER ; Represents an unconditional speculation barrier.
 ])
 
 ;; Enumerators for NEON unspecs.
Index: gcc/config/fr30/fr30.c
===================================================================
--- gcc/config/fr30/fr30.c	(revision 263529)
+++ gcc/config/fr30/fr30.c	(working copy)
@@ -195,6 +195,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT constant_alignment_word_strings
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 

 
Index: gcc/config/frv/frv.c
===================================================================
--- gcc/config/frv/frv.c	(revision 263529)
+++ gcc/config/frv/frv.c	(working copy)
@@ -528,6 +528,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT constant_alignment_word_strings
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #define FRV_SYMBOL_REF_TLS_P(RTX) \
Index: gcc/config/h8300/h8300.c
===================================================================
--- gcc/config/h8300/h8300.c	(revision 263529)
+++ gcc/config/h8300/h8300.c	(working copy)
@@ -6148,4 +6148,7 @@
 #undef TARGET_MODE_DEPENDENT_ADDRESS_P
 #define TARGET_MODE_DEPENDENT_ADDRESS_P h8300_mode_dependent_address_p
 
+#undef TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
Index: gcc/config/i386/i386.md
===================================================================
--- gcc/config/i386/i386.md	(revision 263529)
+++ gcc/config/i386/i386.md	(working copy)
@@ -291,6 +291,9 @@
   UNSPECV_CLRSSBSY
   UNSPECV_MOVDIRI
   UNSPECV_MOVDIR64B
+
+  ;; For Speculation Barrier support
+  UNSPECV_SPECULATION_BARRIER
 ])
 
 ;; Constants to represent rounding modes in the ROUND instruction
@@ -20774,6 +20777,13 @@
   "movdir64b\t{%1, %0|%0, %1}"
   [(set_attr "type" "other")])
 
+(define_insn "speculation_barrier"
+  [(unspec_volatile [(const_int 0)] UNSPECV_SPECULATION_BARRIER)]
+  ""
+  "lfence"
+  [(set_attr "type" "other")
+   (set_attr "length" "3")])
+
 (include "mmx.md")
 (include "sse.md")
 (include "sync.md")
Index: gcc/config/iq2000/iq2000.c
===================================================================
--- gcc/config/iq2000/iq2000.c	(revision 263529)
+++ gcc/config/iq2000/iq2000.c	(working copy)
@@ -274,6 +274,9 @@
 #undef  TARGET_STARTING_FRAME_OFFSET
 #define TARGET_STARTING_FRAME_OFFSET	iq2000_starting_frame_offset
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 

 /* Return nonzero if we split the address into high and low parts.  */
Index: gcc/config/m32r/m32r.c
===================================================================
--- gcc/config/m32r/m32r.c	(revision 263529)
+++ gcc/config/m32r/m32r.c	(working copy)
@@ -226,6 +226,9 @@
 #undef TARGET_STARTING_FRAME_OFFSET
 #define TARGET_STARTING_FRAME_OFFSET m32r_starting_frame_offset
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 

 /* Called by m32r_option_override to initialize various things.  */
Index: gcc/config/m68k/m68k.c
===================================================================
--- gcc/config/m68k/m68k.c	(revision 263529)
+++ gcc/config/m68k/m68k.c	(working copy)
@@ -352,6 +352,9 @@
 #undef TARGET_PROMOTE_FUNCTION_MODE
 #define TARGET_PROMOTE_FUNCTION_MODE m68k_promote_function_mode
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 static const struct attribute_spec m68k_attribute_table[] =
 {
   /* { name, min_len, max_len, decl_req, type_req, fn_type_req,
Index: gcc/config/mcore/mcore.c
===================================================================
--- gcc/config/mcore/mcore.c	(revision 263529)
+++ gcc/config/mcore/mcore.c	(working copy)
@@ -253,6 +253,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT constant_alignment_word_strings
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 

 /* Adjust the stack and return the number of bytes taken to do it.  */
Index: gcc/config/mn10300/mn10300.c
===================================================================
--- gcc/config/mn10300/mn10300.c	(revision 263529)
+++ gcc/config/mn10300/mn10300.c	(working copy)
@@ -3437,4 +3437,7 @@
 #undef  TARGET_MODES_TIEABLE_P
 #define TARGET_MODES_TIEABLE_P mn10300_modes_tieable_p
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
Index: gcc/config/msp430/msp430.c
===================================================================
--- gcc/config/msp430/msp430.c	(revision 263529)
+++ gcc/config/msp430/msp430.c	(working copy)
@@ -3845,6 +3845,9 @@
   return true;
 }
 

+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-msp430.h"
Index: gcc/config/nios2/nios2.c
===================================================================
--- gcc/config/nios2/nios2.c	(revision 263529)
+++ gcc/config/nios2/nios2.c	(working copy)
@@ -5572,6 +5572,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT constant_alignment_word_strings
 
+#undef TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-nios2.h"
Index: gcc/config/nvptx/nvptx.c
===================================================================
--- gcc/config/nvptx/nvptx.c	(revision 263529)
+++ gcc/config/nvptx/nvptx.c	(working copy)
@@ -5864,6 +5864,9 @@
 #undef TARGET_CAN_CHANGE_MODE_CLASS
 #define TARGET_CAN_CHANGE_MODE_CLASS nvptx_can_change_mode_class
 
+#undef TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-nvptx.h"
Index: gcc/config/pa/pa.c
===================================================================
--- gcc/config/pa/pa.c	(revision 263529)
+++ gcc/config/pa/pa.c	(working copy)
@@ -428,6 +428,9 @@
 #undef TARGET_STARTING_FRAME_OFFSET
 #define TARGET_STARTING_FRAME_OFFSET pa_starting_frame_offset
 
+#undef TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 

 /* Parse the -mfixed-range= option string.  */
Index: gcc/config/pdp11/pdp11.c
===================================================================
--- gcc/config/pdp11/pdp11.c	(revision 263529)
+++ gcc/config/pdp11/pdp11.c	(working copy)
@@ -251,6 +251,9 @@
 
 #undef  TARGET_CAN_CHANGE_MODE_CLASS
 #define TARGET_CAN_CHANGE_MODE_CLASS pdp11_can_change_mode_class
+
+#undef TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
 

 /* A helper function to determine if REGNO should be saved in the
    current function's stack frame.  */
Index: gcc/config/rs6000/rs6000.c
===================================================================
--- gcc/config/rs6000/rs6000.c	(revision 263529)
+++ gcc/config/rs6000/rs6000.c	(working copy)
@@ -16838,7 +16838,7 @@
 
     case MISC_BUILTIN_SPEC_BARRIER:
       {
-	emit_insn (gen_rs6000_speculation_barrier ());
+	emit_insn (gen_speculation_barrier ());
 	return NULL_RTX;
       }
 
Index: gcc/config/rs6000/rs6000.md
===================================================================
--- gcc/config/rs6000/rs6000.md	(revision 263529)
+++ gcc/config/rs6000/rs6000.md	(working copy)
@@ -12604,7 +12604,7 @@
   return "ori 2,2,0";
 })
 
-(define_insn "rs6000_speculation_barrier"
+(define_insn "speculation_barrier"
   [(unspec_volatile:BLK [(const_int 0)] UNSPECV_SPEC_BARRIER)]
   ""
   "ori 31,31,0")
Index: gcc/config/rx/rx.c
===================================================================
--- gcc/config/rx/rx.c	(revision 263529)
+++ gcc/config/rx/rx.c	(working copy)
@@ -3785,6 +3785,9 @@
 #undef  TARGET_RTX_COSTS
 #define TARGET_RTX_COSTS rx_rtx_costs
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-rx.h"
Index: gcc/config/spu/spu.c
===================================================================
--- gcc/config/spu/spu.c	(revision 263529)
+++ gcc/config/spu/spu.c	(working copy)
@@ -7458,6 +7458,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT spu_constant_alignment
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-spu.h"
Index: gcc/config/stormy16/stormy16.c
===================================================================
--- gcc/config/stormy16/stormy16.c	(revision 263529)
+++ gcc/config/stormy16/stormy16.c	(working copy)
@@ -2728,6 +2728,9 @@
 #undef TARGET_CONSTANT_ALIGNMENT
 #define TARGET_CONSTANT_ALIGNMENT constant_alignment_word_strings
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-stormy16.h"
Index: gcc/config/v850/v850.c
===================================================================
--- gcc/config/v850/v850.c	(revision 263529)
+++ gcc/config/v850/v850.c	(working copy)
@@ -3374,6 +3374,9 @@
 #undef  TARGET_MODES_TIEABLE_P
 #define TARGET_MODES_TIEABLE_P v850_modes_tieable_p
 
+#undef  TARGET_HAVE_SPECULATION_SAFE_VALUE
+#define TARGET_HAVE_SPECULATION_SAFE_VALUE speculation_safe_value_not_needed
+
 struct gcc_target targetm = TARGET_INITIALIZER;
 
 #include "gt-v850.h"
Index: gcc/config.gcc
===================================================================
--- gcc/config.gcc	(revision 263529)
+++ gcc/config.gcc	(working copy)
@@ -304,7 +304,7 @@
 	extra_headers="arm_fp16.h arm_neon.h arm_acle.h"
 	c_target_objs="aarch64-c.o"
 	cxx_target_objs="aarch64-c.o"
-	extra_objs="aarch64-builtins.o aarch-common.o cortex-a57-fma-steering.o"
+	extra_objs="aarch64-builtins.o aarch-common.o cortex-a57-fma-steering.o aarch64-speculation.o"
 	target_gtfiles="\$(srcdir)/config/aarch64/aarch64-builtins.c"
 	target_has_targetm_common=yes
 	;;
Index: gcc/doc/cpp.texi
===================================================================
--- gcc/doc/cpp.texi	(revision 263529)
+++ gcc/doc/cpp.texi	(working copy)
@@ -2381,6 +2381,10 @@
 These macros are defined when the target processor supports atomic compare
 and swap operations on operands 1, 2, 4, 8 or 16 bytes in length, respectively.
 
+@item __HAVE_SPECULATION_SAFE_VALUE
+This macro is defined with the value 1 to show that this version of GCC
+supports @code{__builtin_speculation_safe_value}.
+
 @item __GCC_HAVE_DWARF2_CFI_ASM
 This macro is defined when the compiler is emitting DWARF CFI directives
 to the assembler.  When this is defined, it is possible to emit those same
Index: gcc/doc/extend.texi
===================================================================
--- gcc/doc/extend.texi	(revision 263529)
+++ gcc/doc/extend.texi	(working copy)
@@ -11065,6 +11065,7 @@
 @findex __builtin_powi
 @findex __builtin_powif
 @findex __builtin_powil
+@findex __builtin_speculation_safe_value
 @findex _Exit
 @findex _exit
 @findex abort
@@ -11708,6 +11709,96 @@
 
 @end deftypefn
 
+@deftypefn {Built-in Function} @var{type} __builtin_speculation_safe_value (@var{type} val, @var{type} failval)
+
+This built-in function can be used to help mitigate against unsafe
+speculative execution.  @var{type} may be any integral type or any
+pointer type.
+
+@enumerate
+@item
+If the CPU is not speculatively executing the code, then @var{val}
+is returned.
+@item
+If the CPU is executing speculatively then either:
+@itemize
+@item
+The function may cause execution to pause until it is known that the
+code is no-longer being executed speculatively (in which case
+@var{val} can be returned, as above); or
+@item
+The function may use target-dependent speculation tracking state to cause
+@var{failval} to be returned when it is known that speculative
+execution has incorrectly predicted a conditional branch operation.
+@end itemize
+@end enumerate
+
+The second argument, @var{failval}, is optional and defaults to zero
+if omitted.
+
+GCC defines the preprocessor macro
+@code{__HAVE_BUILTIN_SPECULATION_SAFE_VALUE} for targets that have been
+updated to support this builtin.
+
+The built-in function can be used where a variable appears to be used in a
+safe way, but the CPU, due to speculative execution may temporarily ignore
+the bounds checks.  Consider, for example, the following function:
+
+@smallexample
+int array[500];
+int f (unsigned untrusted_index)
+@{
+  if (untrusted_index < 500)
+    return array[untrusted_index];
+  return 0;
+@}
+@end smallexample
+
+If the function is called repeatedly with @code{untrusted_index} less
+than the limit of 500, then a branch predictor will learn that the
+block of code that returns a value stored in @code{array} will be
+executed.  If the function is subsequently called with an
+out-of-range value it will still try to execute that block of code
+first until the CPU determines that the prediction was incorrect
+(the CPU will unwind any incorrect operations at that point).
+However, depending on how the result of the function is used, it might be
+possible to leave traces in the cache that can reveal what was stored
+at the out-of-bounds location.  The built-in function can be used to
+provide some protection against leaking data in this way by changing
+the code to:
+
+@smallexample
+int array[500];
+int f (unsigned untrusted_index)
+@{
+  if (untrusted_index < 500)
+    return array[__builtin_speculation_safe_value (untrusted_index)];
+  return 0;
+@}
+@end smallexample
+
+The built-in function will either cause execution to stall until the
+conditional branch has been fully resolved, or it may permit
+speculative execution to continue, but using 0 instead of
+@code{untrusted_value} if that exceeds the limit.
+
+If accessing any memory location is potentially unsafe when speculative
+execution is incorrect, then the code can be rewritten as
+
+@smallexample
+int array[500];
+int f (unsigned untrusted_index)
+@{
+  if (untrusted_index < 500)
+    return *__builtin_speculation_safe_value (&array[untrusted_index], NULL);
+  return 0;
+@}
+@end smallexample
+
+which will cause a @code{NULL} pointer to be used for the unsafe case.
+
+@end deftypefn
+
 @deftypefn {Built-in Function} int __builtin_types_compatible_p (@var{type1}, @var{type2})
 
 You can use the built-in function @code{__builtin_types_compatible_p} to
Index: gcc/doc/invoke.texi
===================================================================
--- gcc/doc/invoke.texi	(revision 263529)
+++ gcc/doc/invoke.texi	(working copy)
@@ -605,7 +605,7 @@
 -mpc-relative-literal-loads @gol
 -msign-return-address=@var{scope} @gol
 -march=@var{name}  -mcpu=@var{name}  -mtune=@var{name}  @gol
--moverride=@var{string}  -mverbose-cost-dump}
+-moverride=@var{string}  -mverbose-cost-dump -mtrack-speculation} 
 
 @emph{Adapteva Epiphany Options}
 @gccoptlist{-mhalf-reg-file  -mprefer-short-insn-regs @gol
@@ -14699,6 +14699,14 @@
 precision of division results to about 16 bits for
 single precision and to 32 bits for double precision.
 
+@item -mtrack-speculation
+@itemx -mno-track-speculation
+Enable or disable generation of additional code to track speculative
+execution through conditional branches.  The tracking state can then
+be used by the compiler when expanding calls to
+@code{__builtin_speculation_safe_copy} to permit a more efficient code
+sequence to be generated.
+
 @item -march=@var{name}
 @opindex march
 Specify the name of the target architecture and, optionally, one or
Index: gcc/doc/md.texi
===================================================================
--- gcc/doc/md.texi	(revision 263529)
+++ gcc/doc/md.texi	(working copy)
@@ -6955,6 +6955,21 @@
 before the instruction with respect to loads and stores after the instruction.
 This pattern has no operands.
 
+@cindex @code{speculation_barrier} instruction pattern
+@item @samp{speculation_barrier}
+If the target can support speculative execution, then this pattern should
+be defined to an instruction that will block subsequent execution until
+any prior speculation conditions has been resolved.  The pattern must also
+ensure that the compiler cannot move memory operations past the barrier,
+so it needs to be an UNSPEC_VOLATILE pattern.  The pattern has no
+operands.
+
+If this pattern is not defined then the default expansion of
+@code{__builtin_speculation_safe_value} will emit a warning.  You can
+suppress this warning by defining this pattern with a final condition
+of @code{0} (zero), which tells the compiler that a speculation
+barrier is not needed for this target.
+
 @cindex @code{sync_compare_and_swap@var{mode}} instruction pattern
 @item @samp{sync_compare_and_swap@var{mode}}
 This pattern, if defined, emits code for an atomic compare-and-swap
Index: gcc/doc/tm.texi
===================================================================
--- gcc/doc/tm.texi	(revision 263529)
+++ gcc/doc/tm.texi	(working copy)
@@ -12010,6 +12010,42 @@
 
 @end defmac
 
+@deftypefn {Target Hook} bool TARGET_HAVE_SPECULATION_SAFE_VALUE (bool @var{active})
+This hook is used to determine the level of target support for
+ @code{__builtin_speculation_safe_value}.  If called with an argument
+ of false, it returns true if the target has been modified to support
+ this builtin.  If called with an argument of true, it returns true
+ if the target requires active mitigation execution might be speculative.
+ 
+ The default implementation returns false if the target does not define
+ a pattern named @code{speculation_barrier}.  Else it returns true
+ for the first case and whether the pattern is enabled for the current
+ compilation for the second case.
+ 
+ For targets that have no processors that can execute instructions
+ speculatively an alternative implemenation of this hook is available:
+ simply redefine this hook to @code{speculation_safe_value_not_needed}
+ along with your other target hooks.
+@end deftypefn
+
+@deftypefn {Target Hook} rtx TARGET_SPECULATION_SAFE_VALUE (machine_mode @var{mode}, rtx @var{result}, rtx @var{val}, rtx @var{failval})
+This target hook can be used to generate a target-specific code
+ sequence that implements the @code{__builtin_speculation_safe_value}
+ built-in function.  The function must always return @var{val} in
+ @var{result} in mode @var{mode} when the cpu is not executing
+ speculatively, but must never return that when speculating until it
+ is known that the speculation will not be unwound.  The hook supports
+ two primary mechanisms for implementing the requirements.  The first
+ is to emit a speculation barrier which forces the processor to wait
+ until all prior speculative operations have been resolved; the second
+ is to use a target-specific mechanism that can track the speculation
+ state and to return @var{failval} if it can determine that
+ speculation must be unwound at a later time.
+ 
+ The default implementation simply copies @var{val} to @var{result} and
+ emits a @code{speculation_barrier} instruction if that is defined.
+@end deftypefn
+
 @deftypefn {Target Hook} void TARGET_RUN_TARGET_SELFTESTS (void)
 If selftests are enabled, run any selftests for this target.
 @end deftypefn
Index: gcc/doc/tm.texi.in
===================================================================
--- gcc/doc/tm.texi.in	(revision 263529)
+++ gcc/doc/tm.texi.in	(working copy)
@@ -8112,4 +8112,8 @@
 
 @end defmac
 
+@hook TARGET_HAVE_SPECULATION_SAFE_VALUE
+
+@hook TARGET_SPECULATION_SAFE_VALUE
+
 @hook TARGET_RUN_TARGET_SELFTESTS
Index: gcc/target.def
===================================================================
--- gcc/target.def	(revision 263529)
+++ gcc/target.def	(working copy)
@@ -4256,6 +4256,46 @@
  hook_bool_void_true)
 
 DEFHOOK
+(have_speculation_safe_value,
+"This hook is used to determine the level of target support for\n\
+ @code{__builtin_speculation_safe_value}.  If called with an argument\n\
+ of false, it returns true if the target has been modified to support\n\
+ this builtin.  If called with an argument of true, it returns true\n\
+ if the target requires active mitigation execution might be speculative.\n\
+ \n\
+ The default implementation returns false if the target does not define\n\
+ a pattern named @code{speculation_barrier}.  Else it returns true\n\
+ for the first case and whether the pattern is enabled for the current\n\
+ compilation for the second case.\n\
+ \n\
+ For targets that have no processors that can execute instructions\n\
+ speculatively an alternative implemenation of this hook is available:\n\
+ simply redefine this hook to @code{speculation_safe_value_not_needed}\n\
+ along with your other target hooks.",
+bool, (bool active), default_have_speculation_safe_value)
+
+DEFHOOK
+(speculation_safe_value,
+"This target hook can be used to generate a target-specific code\n\
+ sequence that implements the @code{__builtin_speculation_safe_value}\n\
+ built-in function.  The function must always return @var{val} in\n\
+ @var{result} in mode @var{mode} when the cpu is not executing\n\
+ speculatively, but must never return that when speculating until it\n\
+ is known that the speculation will not be unwound.  The hook supports\n\
+ two primary mechanisms for implementing the requirements.  The first\n\
+ is to emit a speculation barrier which forces the processor to wait\n\
+ until all prior speculative operations have been resolved; the second\n\
+ is to use a target-specific mechanism that can track the speculation\n\
+ state and to return @var{failval} if it can determine that\n\
+ speculation must be unwound at a later time.\n\
+ \n\
+ The default implementation simply copies @var{val} to @var{result} and\n\
+ emits a @code{speculation_barrier} instruction if that is defined.",
+rtx, (machine_mode mode, rtx result, rtx val, rtx failval),
+ default_speculation_safe_value)
+ 
+
+DEFHOOK
 (can_use_doloop_p,
  "Return true if it is possible to use low-overhead loops (@code{doloop_end}\n\
 and @code{doloop_begin}) for a particular loop.  @var{iterations} gives the\n\
Index: gcc/targhooks.c
===================================================================
--- gcc/targhooks.c	(revision 263529)
+++ gcc/targhooks.c	(working copy)
@@ -2336,4 +2336,43 @@
 {
 }
 
+/* Default implementation of TARGET_HAVE_SPECULATION_SAFE_VALUE.  */
+bool
+default_have_speculation_safe_value (bool active ATTRIBUTE_UNUSED)
+{
+#ifdef HAVE_speculation_barrier
+  return active ? HAVE_speculation_barrier : true;
+#else
+  return false;
+#endif
+}
+/* Alternative implementation of TARGET_HAVE_SPECULATION_SAFE_VALUE
+   that can be used on targets that never have speculative execution.  */
+bool
+speculation_safe_value_not_needed (bool active)
+{
+  return !active;
+}
+
+/* Default implementation of the speculation-safe-load builtin.  This
+   implementation simply copies val to result and generates a
+   speculation_barrier insn, if such a pattern is defined.  */
+rtx
+default_speculation_safe_value (machine_mode mode ATTRIBUTE_UNUSED,
+				rtx result, rtx val,
+				rtx failval ATTRIBUTE_UNUSED)
+{
+  emit_move_insn (result, val);
+
+#ifdef HAVE_speculation_barrier
+  /* Assume the target knows what it is doing: if it defines a
+     speculation barrier, but it is not enabled, then assume that one
+     isn't needed.  */
+  if (HAVE_speculation_barrier)
+    emit_insn (gen_speculation_barrier ());
+#endif
+
+  return result;
+}
+
 #include "gt-targhooks.h"
Index: gcc/targhooks.h
===================================================================
--- gcc/targhooks.h	(revision 263529)
+++ gcc/targhooks.h	(working copy)
@@ -289,4 +289,8 @@
 extern bool default_stack_clash_protection_final_dynamic_probe (rtx);
 extern void default_select_early_remat_modes (sbitmap);
 
+extern bool default_have_speculation_safe_value (bool);
+extern bool speculation_safe_value_not_needed (bool);
+extern rtx default_speculation_safe_value (machine_mode, rtx, rtx, rtx);
+
 #endif /* GCC_TARGHOOKS_H */
Index: gcc/testsuite/ChangeLog
===================================================================
--- gcc/testsuite/ChangeLog	(revision 263529)
+++ gcc/testsuite/ChangeLog	(working copy)
@@ -1,3 +1,9 @@
+2018-07-31  Richard Earnshaw  <rearnsha@arm.com>
+
+	* c-c++-common/spec-barrier-1.c: New test.
+	* c-c++-common/spec-barrier-2.c: New test.
+	* gcc.dg/spec-barrier-3.c: New test.
+
 2018-08-01  Jakub Jelinek  <jakub@redhat.com>
 
 	PR c/85704
Index: gcc/testsuite/c-c++-common/spec-barrier-1.c
===================================================================
--- gcc/testsuite/c-c++-common/spec-barrier-1.c	(nonexistent)
+++ gcc/testsuite/c-c++-common/spec-barrier-1.c	(working copy)
@@ -0,0 +1,38 @@
+/* { dg-do run } */
+/* { dg-options "-O" } */
+
+/* Test that __builtin_speculation_safe_value returns the correct value.  */
+/* This test will cause an unfiltered warning to be emitted on targets
+   that have not implemented support for speculative execution
+   barriers.  They should fix that rather than disabling this
+   test.  */
+char a = 1;
+short b = 2;
+int c = 3;
+long d = 4;
+long long e = 5;
+int *f = (int*) &c;
+#ifdef __SIZEOF_INT128__
+__int128 g = 9;
+#endif
+
+int main ()
+{
+  if (__builtin_speculation_safe_value (a) != 1)
+    __builtin_abort ();
+  if (__builtin_speculation_safe_value (b) != 2)
+    __builtin_abort ();
+  if (__builtin_speculation_safe_value (c) != 3)
+    __builtin_abort ();
+  if (__builtin_speculation_safe_value (d) != 4)
+    __builtin_abort ();
+  if (__builtin_speculation_safe_value (e) != 5)
+    __builtin_abort ();
+  if (__builtin_speculation_safe_value (f) != &c)
+    __builtin_abort ();
+#ifdef __SIZEOF_INT128__
+  if (__builtin_speculation_safe_value (g) != 9)
+    __builtin_abort ();
+#endif
+  return 0;
+}
Index: gcc/testsuite/c-c++-common/spec-barrier-2.c
===================================================================
--- gcc/testsuite/c-c++-common/spec-barrier-2.c	(nonexistent)
+++ gcc/testsuite/c-c++-common/spec-barrier-2.c	(working copy)
@@ -0,0 +1,17 @@
+/* { dg-do run } */
+
+/* Even on targets that don't need the optional failval parameter,
+   side-effects on the operand should still be calculated.  */
+
+int x = 3;
+volatile int y = 9;
+
+int main ()
+{
+  int z = __builtin_speculation_safe_value (x, y++);
+  if (z != 3 || y != 10)
+    __builtin_abort ();
+  return 0;
+}
+
+/* { dg-prune-output "this target does not define a speculation barrier;" } */
Index: gcc/testsuite/gcc.dg/spec-barrier-3.c
===================================================================
--- gcc/testsuite/gcc.dg/spec-barrier-3.c	(nonexistent)
+++ gcc/testsuite/gcc.dg/spec-barrier-3.c	(working copy)
@@ -0,0 +1,13 @@
+/* { dg-do compile } */
+/* { dg-options "-Wpedantic" } */
+
+/* __builtin_speculation_safe_value returns a value with the same type
+   as its first argument.  There should be a warning if that isn't
+   type-compatible with the use.  */
+int *
+f (int x)
+{
+  return __builtin_speculation_safe_value (x);  /* { dg-warning "returning 'int' from a function with return type 'int \\*' makes pointer from integer without a cast" } */
+}
+
+/* { dg-prune-output "this target does not define a speculation barrier;" } */


More information about the Gcc-patches mailing list