This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Patch: Target support for IA-64 speculation
- From: Maxim Kuvyrkov <mkuvyrkov at ispras dot ru>
- To: gcc-patches at gcc dot gnu dot org
- Cc: Jim Wilson <wilson at specifixinc dot com>, "Vladimir N. Makarov" <vmakarov at redhat dot com>, Andrey Belevantsev <abel at ispras dot ru>
- Date: Fri, 30 Dec 2005 15:21:30 +0300
- Subject: Patch: Target support for IA-64 speculation
Hi,
This patch adds support for IA-64 data/control speculation to ia64 port.
The patch was bootstrapped and regtested on the 20051125 weekly snapshot
on ia64-linux and i686-linux, and bootstrapped and regtested on the
20051224 weekly snapshot on i686-linux.
Ok for mainline?
Thanks, Maxim
2005-12-30 Maxim Kuvyrkov <mkuvyrkov@ispras.ru>
* ia64.c (params.h): New include.
(ia64_first_cycle_multipass_dfa_lookahead_guard_spec,
ia64_h_i_d_extended, ia64_set_sched_flags, ia64_speculate_insn,
ia64_needs_block_p, ia64_gen_check,
ia64_sched_init_global):
New static functions to implement hooks from gcc_target.sched.
(ia64_mode_to_int, ia64_gen_spec_insn,
ia64_spec_check_p, ia64_spec_check_src_p): New static functions.
(ia64_adjust_cost): Renamed to ia64_adjust_cost_2.
(TARGET_SCHED_ADJUST_COST): Removed.
(TARGET_SCHED_ADJUST_COST_2, TARGET_SCHED_INIT_GLOBAL,
TARGET_SCHED_H_I_D_EXTENDED, TARGET_SCHED_SPECULATE_INSN,
TARGET_SCHED_NEEDS_BLOCK_P, TARGET_SCHED_GEN_CHECK,
TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC,
TARGET_SCHED_SET_SCHED_FLAGS): New macros.
(update_set_flags, group_barrier_needed): Fixed to handle
speculation checks.
(rtx_needs_barrier): Fixed to handle speculative loads and
their checks.
(stops_p): Added explicit initialization.
(pending_data_specs): New variable.
(ia64_variable_issue): Added code to count speculative loads and their
checks.
(ia64_first_cycle_multipass_dfa_lookahead_guard): Fixed to handle
speculative loads.
(enum SPEC_MODES, enum SPEC_GEN_LD_MAP, SPEC_GEN_CHECK_OFFSET):
New enumerations.
(SPEC_GEN_EXTEND_OFFSET, SPEC_N, SPEC_GEN_CHECK_MUTATION_OFFSET):
New constants.
(ia64_ld_address_bypass_p): Fixed to handle speculative loads.
* ia64.md (UNSPEC_LDA, UNSPEC_LDS, UNSPEC_LDSA, UNSPEC_LDCCLR,
UNSPEC_CHKACLR, UNSPEC_CHKS): New constants.
(itanium_class): chk_s renamed to chk_s_i. New constants: chk_s_f,
chk_a.
(data_speculative, control_speculative, check_load): New attributes.
(mov<mode>_advanced, mov<mode>_speculative,
mov<mode>_speculative_advanced, zero_extend<mode>di2_advanced,
zero_extend<mode>di2_speculative,
zero_extend<mode>di2_speculative_advanced): New patterns for
data and control speculative loads.
(mov<mode>_clr, zero_extend<mode>di2_clr): New patterns for
check loads.
(advanced_load_check_clr_<mode>, speculation_check_<mode>):
New pattern for data and control speculation checks.
* itanium1.md (1_fldc, 1_fldpc, 1_ldc, 1_chk_s_f, 1_chk_a,
1b_fldc, 1b_fldpc, 1b_ldc, 1b_chk_s_f, 1b_chk_a): New resource
constraints.
(1_chk_s, 1b_chk_s): Renamed to 1_chk_s_i, 1b_chk_s_i.
* itanium2.md (2_flda, 2_fldc, 2_fldpc, 2_ldc, 2_chk_s_f, 2_chk_a,
2b_flda, 2b_fldc, 2b_fldpc, 2b_ldc, 2b_chk_s_f, 2b_chk_a): New resource
constraints.
(2_chk_s, 2b_chk_s): Renamed to 2_chk_s_i, 2b_chk_s_i.
* ia64.opt (msched-br-data-spec, msched-ar-data-spec,
msched-control-spec, msched-br-in-data-spec, msched-ar-in-data-spec,
msched-in-control-spec, msched-ldc, msched-control-ldc,
msched-spec-verbose, msched-prefer-non-data-spec-insns,
msched-prefer-non-control-spec-insns,
msched-count-spec-in-critical-path, msched-issue-points,
msched-sort-points): New flags to control speculative scheduling.
Index: gcc/config/ia64/itanium1.md
===================================================================
--- gcc/config/ia64/itanium1.md (.../insn-tick) (revision 2240)
+++ gcc/config/ia64/itanium1.md (.../recovery) (revision 2240)
@@ -525,14 +525,29 @@
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "fcvtfx"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_F")
+
(define_insn_reservation "1_fld" 9
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "fld"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "no"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_fldc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "yes"))
+ (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
(define_insn_reservation "1_fldp" 9
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "fldp"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "no"))
+ (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_fldpc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "yes"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
(define_insn_reservation "1_fmac" 5
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "fmac"))
@@ -604,8 +619,14 @@
(eq (symbol_ref "bundling_p") (const_int 0)))
"1_I+1_not_ui1")
(define_insn_reservation "1_ld" 2
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "ld"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "no"))
+ (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_ldc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "yes"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
(define_insn_reservation "1_long_i" 1
(and (and (eq_attr "cpu" "itanium")
@@ -696,10 +717,19 @@
(eq_attr "itanium_class" "xtd"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_I")
-(define_insn_reservation "1_chk_s" 0
+(define_insn_reservation "1_chk_s_i" 0
(and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "chk_s"))
+ (eq_attr "itanium_class" "chk_s_i"))
(eq (symbol_ref "bundling_p") (const_int 0))) "1_A")
+(define_insn_reservation "1_chk_s_f" 0
+ (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "chk_s_f"))
+ (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_chk_a" 0
+ (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "chk_a"))
+ (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
(define_insn_reservation "1_lfetch" 0
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "lfetch"))
@@ -943,15 +973,15 @@
(define_bypass 2 "1_ilog,1_xtd" "1_ld" "ia64_ld_address_bypass_p")
(define_bypass 2 "1_ilog,1_xtd" "1_st" "ia64_st_address_bypass_p")
-(define_bypass 3 "1_ld" "1_mmmul,1_mmshf")
+(define_bypass 3 "1_ld,1_ldc" "1_mmmul,1_mmshf")
(define_bypass 3 "1_ld" "1_ld" "ia64_ld_address_bypass_p")
(define_bypass 3 "1_ld" "1_st" "ia64_st_address_bypass_p")
;; Intel docs say only LD, ST, IALU, ILOG, ISHF consumers have latency 4,
;; but HP engineers say any non-MM operation.
(define_bypass 4 "1_mmmul,1_mmshf,1_mmalua"
- "1_br,1_fcmp,1_fcvtfx,1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
- 1_frbr,1_frfr,1_frpr,1_ialu,1_icmp,1_ilog,1_ishf,1_ld,1_chk_s,\
+ "1_br,1_fcmp,1_fcvtfx,1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
+ 1_frbr,1_frfr,1_frpr,1_ialu,1_icmp,1_ilog,1_ishf,1_ld,1_ldc,1_chk_s_i,1_chk_s_f,1_chk_a,\
1_long_i,1_rse_m,1_sem,1_stf,1_st,1_syst_m0,1_syst_m,\
1_tbit,1_toar_i,1_toar_m,1_tobr,1_tofr,1_topr,1_xmpy,1_xtd")
@@ -965,15 +995,15 @@
(define_bypass 8 "1_fmisc,1_fcvtfx,1_fmac,1_xmpy" "1_stf")
;; We don't use here fcmp because scall may be predicated.
-(define_bypass 0 "1_fcvtfx,1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
+(define_bypass 0 "1_fcvtfx,1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
1_frbr,1_frfr,1_frpr,1_ialu,1_ialu_addr,1_ilog,1_ishf,\
- 1_ld,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_toar_m,\
- 1_tofr,1_xmpy,1_xtd" "1_scall")
+ 1_ld,1_ldc,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,\
+ 1_toar_m,1_tofr,1_xmpy,1_xtd" "1_scall")
(define_bypass 0 "1_unknown,1_ignore,1_stop_bit,1_br,1_fcmp,1_fcvtfx,\
- 1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,1_frbr,1_frfr,\
- 1_frpr,1_ialu,1_ialu_addr,1_icmp,1_ilog,1_ishf,1_ld,\
- 1_chk_s,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_nop,\
+ 1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,1_frbr,1_frfr,\
+ 1_frpr,1_ialu,1_ialu_addr,1_icmp,1_ilog,1_ishf,1_ld,1_ldc,\
+ 1_chk_s_i,1_chk_s_f,1_chk_a,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_nop,\
1_nop_b,1_nop_f,1_nop_i,1_nop_m,1_nop_x,1_rse_m,1_scall,\
1_sem,1_stf,1_st,1_syst_m0,1_syst_m,1_tbit,1_toar_i,\
1_toar_m,1_tobr,1_tofr,1_topr,1_xmpy,1_xtd,1_lfetch"
@@ -1407,14 +1437,29 @@
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "fcvtfx"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_F")
+
(define_insn_reservation "1b_fld" 9
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "fld"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "no"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_fldc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "yes"))
+ (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
(define_insn_reservation "1b_fldp" 9
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "fldp"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "no"))
+ (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_fldpc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "yes"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
(define_insn_reservation "1b_fmac" 5
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "fmac"))
@@ -1480,10 +1525,18 @@
(eq_attr "itanium_class" "ishf"))
(ne (symbol_ref "bundling_p") (const_int 0)))
"1b_I+1b_not_ui1")
+
(define_insn_reservation "1b_ld" 2
- (and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "ld"))
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "no"))
+ (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_ldc" 0
+ (and (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "yes"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
(define_insn_reservation "1b_long_i" 1
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "long_i"))
@@ -1566,10 +1619,20 @@
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "xtd"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_I")
-(define_insn_reservation "1b_chk_s" 0
+
+(define_insn_reservation "1b_chk_s_i" 0
(and (and (eq_attr "cpu" "itanium")
- (eq_attr "itanium_class" "chk_s"))
+ (eq_attr "itanium_class" "chk_s_i"))
(ne (symbol_ref "bundling_p") (const_int 0))) "1b_A")
+(define_insn_reservation "1b_chk_s_f" 0
+ (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "chk_s_f"))
+ (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_chk_a" 0
+ (and (and (eq_attr "cpu" "itanium")
+ (eq_attr "itanium_class" "chk_a"))
+ (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
(define_insn_reservation "1b_lfetch" 0
(and (and (eq_attr "cpu" "itanium")
(eq_attr "itanium_class" "lfetch"))
Index: gcc/config/ia64/ia64.opt
===================================================================
--- gcc/config/ia64/ia64.opt (.../insn-tick) (revision 2240)
+++ gcc/config/ia64/ia64.opt (.../recovery) (revision 2240)
@@ -96,3 +96,58 @@ mtune=
Target RejectNegative Joined
Schedule code for given CPU
+msched-br-data-spec
+Target Report Var(mflag_sched_br_data_spec) Init(0)
+Use data speculation before reload
+
+msched-ar-data-spec
+Target Report Var(mflag_sched_ar_data_spec) Init(1)
+Use data speculation after reload
+
+msched-control-spec
+Target Report Var(mflag_sched_control_spec) Init(0)
+Use control speculation
+
+msched-br-in-data-spec
+Target Report Var(mflag_sched_br_in_data_spec) Init(1)
+Use in block data speculation before reload
+
+msched-ar-in-data-spec
+Target Report Var(mflag_sched_ar_in_data_spec) Init(1)
+Use in block data speculation after reload
+
+msched-in-control-spec
+Target Report Var(mflag_sched_in_control_spec) Init(1)
+Use in block control speculation
+
+msched-ldc
+Target Report Var(mflag_sched_ldc) Init(1)
+Use simple data speculation check
+
+msched-control-ldc
+Target Report Var(mflag_control_ldc) Init(0)
+Use simple data speculation check for control speculation
+
+msched-spec-verbose
+Common Report Var(mflag_sched_spec_verbose) Init(0)
+Print information about speculative motions.
+
+msched-prefer-non-data-spec-insns
+Common Report Var(mflag_sched_prefer_non_data_spec_insns) Init(1)
+If non-zero, data speculative instructions will be choosen for schedule only if there are no other choices at the moment
+
+msched-prefer-non-control-spec-insns
+Common Report Var(mflag_sched_prefer_non_control_spec_insns) Init(1)
+If non-zero, control speculative instructions will be choosen for schedule only if there are no other choices at the moment
+
+msched-count-spec-in-critical-path
+Common Report Var(mflag_sched_count_spec_in_critical_path) Init(1)
+Count speculative dependencies while calculating priority of instructions
+
+msched-issue-points
+Common Report Var(mflag_sched_issue_points) Init(1)
+Enable points while choosing the best insn for schedule
+
+msched-sort-points
+Common Report Var(mflag_sched_sort_points) Init(1)
+Enable points while sorting ready list
Index: gcc/config/ia64/ia64.c
===================================================================
--- gcc/config/ia64/ia64.c (.../insn-tick) (revision 2240)
+++ gcc/config/ia64/ia64.c (.../recovery) (revision 2240)
@@ -38,6 +38,7 @@ Boston, MA 02110-1301, USA. */
#include "recog.h"
#include "expr.h"
#include "optabs.h"
+#include "params.h"
#include "except.h"
#include "function.h"
#include "ggc.h"
@@ -157,7 +158,17 @@ static void ia64_dependencies_evaluation
static void ia64_init_dfa_pre_cycle_insn (void);
static rtx ia64_dfa_pre_cycle_insn (void);
static int ia64_first_cycle_multipass_dfa_lookahead_guard (rtx);
+static int ia64_first_cycle_multipass_dfa_lookahead_guard_spec (rtx);
static int ia64_dfa_new_cycle (FILE *, int, rtx, int, int, int *);
+static void ia64_h_i_d_extended (void);
+static int ia64_mode_to_int (enum machine_mode);
+static void ia64_set_sched_flags (unsigned int *, spec_info_t);
+static int ia64_speculate_insn (rtx, int, rtx *);
+static rtx ia64_gen_spec_insn (rtx, int, int, int);
+static int ia64_needs_block_p (rtx);
+static rtx ia64_gen_check (rtx, rtx, int);
+static int ia64_spec_check_p (rtx);
+static int ia64_spec_check_src_p (rtx);
static rtx gen_tls_get_addr (void);
static rtx gen_thread_pointer (void);
static int find_gr_spill (int);
@@ -199,8 +210,9 @@ static void ia64_output_function_epilogu
static void ia64_output_function_end_prologue (FILE *);
static int ia64_issue_rate (void);
-static int ia64_adjust_cost (rtx, rtx, rtx, int);
+static int ia64_adjust_cost_2 (rtx, enum reg_note, rtx, int);
static void ia64_sched_init (FILE *, int, int);
+static void ia64_sched_init_global (FILE *, int, int);
static void ia64_sched_finish (FILE *, int);
static int ia64_dfa_sched_reorder (FILE *, int, rtx *, int *, int, int);
static int ia64_sched_reorder (FILE *, int, rtx *, int *, int);
@@ -312,8 +324,8 @@ static const struct attribute_spec ia64_
#undef TARGET_IN_SMALL_DATA_P
#define TARGET_IN_SMALL_DATA_P ia64_in_small_data_p
-#undef TARGET_SCHED_ADJUST_COST
-#define TARGET_SCHED_ADJUST_COST ia64_adjust_cost
+#undef TARGET_SCHED_ADJUST_COST_2
+#define TARGET_SCHED_ADJUST_COST_2 ia64_adjust_cost_2
#undef TARGET_SCHED_ISSUE_RATE
#define TARGET_SCHED_ISSUE_RATE ia64_issue_rate
#undef TARGET_SCHED_VARIABLE_ISSUE
@@ -322,6 +334,8 @@ static const struct attribute_spec ia64_
#define TARGET_SCHED_INIT ia64_sched_init
#undef TARGET_SCHED_FINISH
#define TARGET_SCHED_FINISH ia64_sched_finish
+#undef TARGET_SCHED_INIT_GLOBAL
+#define TARGET_SCHED_INIT_GLOBAL ia64_sched_init_global
#undef TARGET_SCHED_REORDER
#define TARGET_SCHED_REORDER ia64_sched_reorder
#undef TARGET_SCHED_REORDER2
@@ -345,6 +359,25 @@ static const struct attribute_spec ia64_
#undef TARGET_SCHED_DFA_NEW_CYCLE
#define TARGET_SCHED_DFA_NEW_CYCLE ia64_dfa_new_cycle
+#undef TARGET_SCHED_H_I_D_EXTENDED
+#define TARGET_SCHED_H_I_D_EXTENDED ia64_h_i_d_extended
+
+#undef TARGET_SCHED_SET_SCHED_FLAGS
+#define TARGET_SCHED_SET_SCHED_FLAGS ia64_set_sched_flags
+
+#undef TARGET_SCHED_SPECULATE_INSN
+#define TARGET_SCHED_SPECULATE_INSN ia64_speculate_insn
+
+#undef TARGET_SCHED_NEEDS_BLOCK_P
+#define TARGET_SCHED_NEEDS_BLOCK_P ia64_needs_block_p
+
+#undef TARGET_SCHED_GEN_CHECK
+#define TARGET_SCHED_GEN_CHECK ia64_gen_check
+
+#undef TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC
+#define TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC\
+ ia64_first_cycle_multipass_dfa_lookahead_guard_spec
+
#undef TARGET_FUNCTION_OK_FOR_SIBCALL
#define TARGET_FUNCTION_OK_FOR_SIBCALL ia64_function_ok_for_sibcall
#undef TARGET_ARG_PARTIAL_BYTES
@@ -5413,13 +5446,15 @@ update_set_flags (rtx x, struct reg_flag
return;
case IF_THEN_ELSE:
- /* There are three cases here:
+ /* There are four cases here:
(1) The destination is (pc), in which case this is a branch,
nothing here applies.
(2) The destination is ar.lc, in which case this is a
doloop_end_internal,
(3) The destination is an fp register, in which case this is
an fselect instruction.
+ (4) The condition has (unspec [(reg)] UNSPEC_LDC), in which case
+ this is a check load.
In all cases, nothing we do in this function applies. */
return;
@@ -5464,18 +5499,29 @@ set_src_needs_barrier (rtx x, struct reg
/* X is a conditional branch. */
/* ??? This seems redundant, as the caller sets this bit for
all JUMP_INSNs. */
- flags.is_branch = 1;
+ if (!ia64_spec_check_src_p (src))
+ flags.is_branch = 1;
return rtx_needs_barrier (src, flags, pred);
}
- need_barrier = rtx_needs_barrier (src, flags, pred);
+ if (ia64_spec_check_src_p (src))
+ /* Avoid checking one register twice (in condition
+ and in 'then' section) for ldc pattern. */
+ {
+ gcc_assert (REG_P (XEXP (src, 2)));
+ need_barrier = rtx_needs_barrier (XEXP (src, 2), flags, pred);
+
+ /* We process MEM below. */
+ src = XEXP (src, 1);
+ }
+
+ need_barrier |= rtx_needs_barrier (src, flags, pred);
dst = SET_DEST (x);
if (GET_CODE (dst) == ZERO_EXTRACT)
{
need_barrier |= rtx_needs_barrier (XEXP (dst, 1), flags, pred);
need_barrier |= rtx_needs_barrier (XEXP (dst, 2), flags, pred);
- dst = XEXP (dst, 0);
}
return need_barrier;
}
@@ -5724,6 +5770,11 @@ rtx_needs_barrier (rtx x, struct reg_fla
case UNSPEC_SETF_EXP:
case UNSPEC_ADDP4:
case UNSPEC_FR_SQRT_RECIP_APPROX:
+ case UNSPEC_LDA:
+ case UNSPEC_LDS:
+ case UNSPEC_LDSA:
+ case UNSPEC_CHKACLR:
+ case UNSPEC_CHKS:
need_barrier = rtx_needs_barrier (XVECEXP (x, 0, 0), flags, pred);
break;
@@ -5871,7 +5922,8 @@ group_barrier_needed (rtx insn)
break;
case JUMP_INSN:
- flags.is_branch = 1;
+ if (!ia64_spec_check_p (insn))
+ flags.is_branch = 1;
/* Don't bundle a jump following a call. */
if ((pat = prev_active_insn (insn))
@@ -6125,7 +6177,7 @@ static state_t prev_cycle_state = NULL;
/* The following array element values are TRUE if the corresponding
insn requires to add stop bits before it. */
-static char *stops_p;
+static char *stops_p = NULL;
/* The following variable is used to set up the mentioned above array. */
@@ -6147,6 +6199,9 @@ static int *clocks;
static int *add_cycles;
+/* The following variable value is number of data speculations in progress. */
+static int pending_data_specs = 0;
+
static rtx ia64_single_set (rtx);
static void ia64_emit_insn_before (rtx, rtx);
@@ -6196,16 +6251,17 @@ ia64_single_set (rtx insn)
return ret;
}
-/* Adjust the cost of a scheduling dependency. Return the new cost of
- a dependency LINK or INSN on DEP_INSN. COST is the current cost. */
+/* Adjust the cost of a scheduling dependency.
+ Return the new cost of a dependency of type DEP_TYPE or INSN on DEP_INSN.
+ COST is the current cost. */
static int
-ia64_adjust_cost (rtx insn, rtx link, rtx dep_insn, int cost)
+ia64_adjust_cost_2 (rtx insn, enum reg_note dep_type, rtx dep_insn, int cost)
{
enum attr_itanium_class dep_class;
enum attr_itanium_class insn_class;
- if (REG_NOTE_KIND (link) != REG_DEP_OUTPUT)
+ if (dep_type != REG_DEP_OUTPUT)
return cost;
insn_class = ia64_safe_itanium_class (insn);
@@ -6292,6 +6348,16 @@ ia64_sched_init (FILE *dump ATTRIBUTE_UN
init_insn_group_barriers ();
}
+/* We're beginning a scheduling pass. Check assertion. */
+
+static void
+ia64_sched_init_global (FILE *dump ATTRIBUTE_UNUSED,
+ int sched_verbose ATTRIBUTE_UNUSED,
+ int max_ready ATTRIBUTE_UNUSED)
+{
+ gcc_assert (!pending_data_specs);
+}
+
/* We are about to being issuing insns for this clock cycle.
Override the default sort algorithm to better slot instructions. */
@@ -6414,6 +6480,11 @@ ia64_variable_issue (FILE *dump ATTRIBUT
rtx insn ATTRIBUTE_UNUSED,
int can_issue_more ATTRIBUTE_UNUSED)
{
+ if (DONE_SPEC (insn) & BEGIN_DATA)
+ pending_data_specs++;
+ if (CHECK_SPEC (insn) & BEGIN_DATA)
+ pending_data_specs--;
+
last_scheduled_insn = insn;
memcpy (prev_cycle_state, curr_state, dfa_state_size);
if (reload_completed)
@@ -6436,8 +6507,22 @@ static int
ia64_first_cycle_multipass_dfa_lookahead_guard (rtx insn)
{
gcc_assert (insn && INSN_P (insn));
- return (!reload_completed
- || !safe_group_barrier_needed (insn));
+ return ((!reload_completed
+ || !safe_group_barrier_needed (insn))
+ && ia64_first_cycle_multipass_dfa_lookahead_guard_spec (insn));
+}
+
+/* We are choosing insn from the ready queue. Return nonzero if INSN
+ can be chosen. */
+
+static int
+ia64_first_cycle_multipass_dfa_lookahead_guard_spec (rtx insn)
+{
+ gcc_assert (insn && INSN_P (insn));
+ /* Size of ALAT is 32. As far as we perform conservative data speculation,
+ we keep ALAT half-empty. */
+ return (pending_data_specs < 16
+ || !(TODO_SPEC (insn) & BEGIN_DATA));
}
/* The following variable value is pseudo-insn used by the DFA insn
@@ -6525,6 +6610,541 @@ ia64_dfa_new_cycle (FILE *dump, int verb
return 0;
}
+/* Implement targetm.sched.h_i_d_extended hook.
+ Extend internal data structures. */
+static void
+ia64_h_i_d_extended (void)
+{
+ if (stops_p != NULL)
+ {
+ int new_clocks_length = get_max_uid () + 1;
+
+ stops_p = xrecalloc (stops_p, new_clocks_length, clocks_length, 1);
+
+ if (ia64_tune == PROCESSOR_ITANIUM)
+ {
+ clocks = xrecalloc (clocks, new_clocks_length, clocks_length,
+ sizeof (int));
+ add_cycles = xrecalloc (add_cycles, new_clocks_length, clocks_length,
+ sizeof (int));
+ }
+
+ clocks_length = new_clocks_length;
+ }
+}
+
+/* Constants that help mapping 'enum machine_mode' to int. */
+enum SPEC_MODES
+ {
+ SPEC_MODE_INVALID = -1,
+ SPEC_MODE_FIRST = 0,
+ SPEC_MODE_FOR_EXTEND_FIRST = 1,
+ SPEC_MODE_FOR_EXTEND_LAST = 3,
+ SPEC_MODE_LAST = 8
+ };
+
+/* Return index of the MODE. */
+static int
+ia64_mode_to_int (enum machine_mode mode)
+{
+ switch (mode)
+ {
+ case BImode: return 0; /* SPEC_MODE_FIRST */
+ case QImode: return 1; /* SPEC_MODE_FOR_EXTEND_FIRST */
+ case HImode: return 2;
+ case SImode: return 3; /* SPEC_MODE_FOR_EXTEND_LAST */
+ case DImode: return 4;
+ case SFmode: return 5;
+ case DFmode: return 6;
+ case XFmode: return 7;
+ case TImode:
+ /* This mode needs testing. Bypasses for ldfp8 instruction are not
+ mentioned in itanium[12].md. Predicate fp_register_operand is also
+ need to be defined. Overall: better disable for now. */
+ return SPEC_MODE_INVALID;
+ default: return SPEC_MODE_INVALID;
+ }
+}
+
+/* Provide information about speculation capabilities. */
+static void
+ia64_set_sched_flags (unsigned int *flags, spec_info_t spec_info)
+{
+ if (*flags & SCHED_RGN
+ || *flags & SCHED_EBB)
+ {
+ int mask = 0;
+
+ if ((mflag_sched_br_data_spec && !reload_completed)
+ || (mflag_sched_ar_data_spec && reload_completed))
+ {
+ mask |= BEGIN_DATA;
+
+ if ((mflag_sched_br_in_data_spec && !reload_completed)
+ || (mflag_sched_ar_in_data_spec && reload_completed))
+ mask |= BE_IN_DATA;
+ }
+
+ if (mflag_sched_control_spec)
+ {
+ mask |= BEGIN_CONTROL;
+
+ if (mflag_sched_in_control_spec)
+ mask |= BE_IN_CONTROL;
+ }
+
+ gcc_assert (*flags & USE_GLAT);
+
+ if (mask)
+ {
+ *flags |= USE_DEPS_LIST | DETACH_LIFE_INFO | DO_SPECULATION;
+
+ spec_info->mask = mask;
+ spec_info->flags = 0;
+
+ spec_info->spec_points_cutoff =
+ PARAM_VALUE (PARAM_SCHED_SPEC_POINTS_CUTOFF);
+
+ if ((mask & DATA_SPEC) && mflag_sched_prefer_non_data_spec_insns)
+ spec_info->flags |= PREFER_NON_DATA_SPEC;
+
+ if ((mask & CONTROL_SPEC)
+ && mflag_sched_prefer_non_control_spec_insns)
+ spec_info->flags |= PREFER_NON_CONTROL_SPEC;
+
+ if (mflag_sched_spec_verbose)
+ {
+ if (sched_verbose >= 1)
+ spec_info->dump = sched_dump;
+ else
+ spec_info->dump = stderr;
+ }
+ else
+ spec_info->dump = 0;
+
+ if (mflag_sched_count_spec_in_critical_path)
+ spec_info->flags |= COUNT_SPEC_IN_CRITICAL_PATH;
+
+ if (spec_info->spec_points_cutoff > MIN_SCHED_POINTS)
+ spec_info->flags |= USE_SPEC_POINTS;
+ if (mflag_sched_issue_points)
+ spec_info->flags |= USE_ISSUE_POINTS;
+ if (mflag_sched_sort_points)
+ spec_info->flags |= USE_SORT_POINTS;
+ }
+ }
+}
+
+/* Implement targetm.sched.speculate_insn hook.
+ Check if INSN can be TS speculative.
+ If 'no' - return -1.
+ If 'yes' - generate speculative pattern in NEW_PAT and return 1. */
+static int
+ia64_speculate_insn (rtx insn, int ts, rtx *new_pat)
+{
+ rtx pat, reg, mem, mem_reg;
+ int extend_p, mode_no;
+
+ gcc_assert (!(ts & ~BEGIN_SPEC) && ts);
+
+ pat = PATTERN (insn);
+
+ if (GET_CODE (pat) == COND_EXEC)
+ pat = COND_EXEC_CODE (pat);
+
+ if (GET_CODE (pat) != SET)
+ return -1;
+ reg = SET_DEST (pat);
+ if (!REG_P (reg))
+ return -1;
+
+ mem = SET_SRC (pat);
+ if (GET_CODE (mem) == ZERO_EXTEND)
+ {
+ mem = XEXP (mem, 0);
+ extend_p = 1;
+ }
+ else
+ extend_p = 0;
+ if (GET_CODE (mem) == UNSPEC)
+ {
+ int code;
+
+ code = XINT (mem, 1);
+ if (code != UNSPEC_LDA && code != UNSPEC_LDS && code != UNSPEC_LDSA)
+ return -1;
+
+ mem = XVECEXP (mem, 0, 0);
+ gcc_assert (MEM_P (mem));
+ }
+ if (!MEM_P (mem))
+ return -1;
+ mem_reg = XEXP (mem, 0);
+ if (!REG_P (mem_reg))
+ return -1;
+
+ /* We should use MEM's mode since REG's mode in presence of ZERO_EXTEND
+ will always be DImode. */
+ mode_no = ia64_mode_to_int (GET_MODE (mem));
+
+ if (mode_no == SPEC_MODE_INVALID
+ || (extend_p
+ && !(SPEC_MODE_FOR_EXTEND_FIRST <= mode_no
+ && mode_no <= SPEC_MODE_FOR_EXTEND_LAST)))
+ return -1;
+
+ extract_insn_cached (insn);
+ gcc_assert (reg == recog_data.operand[0] && mem == recog_data.operand[1]);
+ *new_pat = ia64_gen_spec_insn (insn, ts, mode_no, extend_p);
+ return 1;
+}
+
+enum
+ {
+ /* Offset to reach ZERO_EXTEND patterns. */
+ SPEC_GEN_EXTEND_OFFSET = SPEC_MODE_LAST - SPEC_MODE_FOR_EXTEND_FIRST + 1,
+ /* Number of patterns for each speculation mode. */
+ SPEC_N = (SPEC_MODE_LAST
+ + SPEC_MODE_FOR_EXTEND_LAST - SPEC_MODE_FOR_EXTEND_FIRST + 2)
+ };
+
+enum SPEC_GEN_LD_MAP
+ {
+ /* Offset to ld.a patterns. */
+ SPEC_GEN_A = 0 * SPEC_N,
+ /* Offset to ld.s patterns. */
+ SPEC_GEN_S = 1 * SPEC_N,
+ /* Offset to ld.sa patterns. */
+ SPEC_GEN_SA = 2 * SPEC_N,
+ /* Offset to ld.sa patterns. For this patterns corresponding ld.c will
+ mutate to chk.s. */
+ SPEC_GEN_SA_FOR_S = 3 * SPEC_N
+ };
+
+/* These offsets are used to get (4 * SPEC_N). */
+enum SPEC_GEN_CHECK_OFFSET
+ {
+ SPEC_GEN_CHKA_FOR_A_OFFSET = 4 * SPEC_N - SPEC_GEN_A,
+ SPEC_GEN_CHKA_FOR_SA_OFFSET = 4 * SPEC_N - SPEC_GEN_SA
+ };
+
+/* Return speculative pattern for INSN with speculative mode TS, machine mode
+ MODE_NO and with ZERO_EXTEND (if extend_p is nonzero). */
+static rtx
+ia64_gen_spec_insn (rtx insn, int ts, int mode_no, int extend_p)
+{
+ rtx pat, new_pat;
+ int load_no;
+ int shift = 0;
+
+ static rtx (* const gen_load[]) (rtx, rtx) = {
+ gen_movbi_advanced,
+ gen_movqi_advanced,
+ gen_movhi_advanced,
+ gen_movsi_advanced,
+ gen_movdi_advanced,
+ gen_movsf_advanced,
+ gen_movdf_advanced,
+ gen_movxf_advanced,
+ gen_movti_advanced,
+ gen_zero_extendqidi2_advanced,
+ gen_zero_extendhidi2_advanced,
+ gen_zero_extendsidi2_advanced,
+
+ gen_movbi_speculative,
+ gen_movqi_speculative,
+ gen_movhi_speculative,
+ gen_movsi_speculative,
+ gen_movdi_speculative,
+ gen_movsf_speculative,
+ gen_movdf_speculative,
+ gen_movxf_speculative,
+ gen_movti_speculative,
+ gen_zero_extendqidi2_speculative,
+ gen_zero_extendhidi2_speculative,
+ gen_zero_extendsidi2_speculative,
+
+ gen_movbi_speculative_advanced,
+ gen_movqi_speculative_advanced,
+ gen_movhi_speculative_advanced,
+ gen_movsi_speculative_advanced,
+ gen_movdi_speculative_advanced,
+ gen_movsf_speculative_advanced,
+ gen_movdf_speculative_advanced,
+ gen_movxf_speculative_advanced,
+ gen_movti_speculative_advanced,
+ gen_zero_extendqidi2_speculative_advanced,
+ gen_zero_extendhidi2_speculative_advanced,
+ gen_zero_extendsidi2_speculative_advanced,
+
+ gen_movbi_speculative_advanced,
+ gen_movqi_speculative_advanced,
+ gen_movhi_speculative_advanced,
+ gen_movsi_speculative_advanced,
+ gen_movdi_speculative_advanced,
+ gen_movsf_speculative_advanced,
+ gen_movdf_speculative_advanced,
+ gen_movxf_speculative_advanced,
+ gen_movti_speculative_advanced,
+ gen_zero_extendqidi2_speculative_advanced,
+ gen_zero_extendhidi2_speculative_advanced,
+ gen_zero_extendsidi2_speculative_advanced
+ };
+
+ load_no = extend_p ? mode_no + SPEC_GEN_EXTEND_OFFSET : mode_no;
+
+ if (ts & BEGIN_DATA)
+ {
+ /* We don't need recovery because even if this is ld.sa
+ ALAT entry will be allocated only if NAT bit is set to zero.
+ So it is enough to use ld.c here. */
+
+ if (ts & BEGIN_CONTROL)
+ {
+ load_no += SPEC_GEN_SA;
+
+ if (!mflag_sched_ldc)
+ shift = SPEC_GEN_CHKA_FOR_SA_OFFSET;
+ }
+ else
+ {
+ load_no += SPEC_GEN_A;
+
+ if (!mflag_sched_ldc)
+ shift = SPEC_GEN_CHKA_FOR_A_OFFSET;
+ }
+ }
+ else if (ts & BEGIN_CONTROL)
+ {
+ /* ld.sa can be used instead of ld.s to avoid basic block splitting. */
+ if (!mflag_control_ldc)
+ load_no += SPEC_GEN_S;
+ else
+ {
+ gcc_assert (mflag_sched_ldc);
+ load_no += SPEC_GEN_SA_FOR_S;
+ }
+ }
+ else
+ gcc_unreachable ();
+
+ CHECK_NO (insn) = load_no + shift;
+
+ new_pat = gen_load[load_no] (copy_rtx (recog_data.operand[0]),
+ copy_rtx (recog_data.operand[1]));
+
+ pat = PATTERN (insn);
+ if (GET_CODE (pat) == COND_EXEC)
+ new_pat = gen_rtx_COND_EXEC (VOIDmode, copy_rtx
+ (COND_EXEC_TEST (pat)), new_pat);
+
+ return new_pat;
+}
+
+/* Offset to branchy checks. */
+enum { SPEC_GEN_CHECK_MUTATION_OFFSET = 5 * SPEC_N };
+
+/* Return nonzero, if INSN needs branchy recovery check. */
+static int
+ia64_needs_block_p (rtx insn)
+{
+ int check_no;
+
+ check_no = CHECK_NO (insn);
+ gcc_assert (0 <= check_no && check_no < SPEC_GEN_CHECK_MUTATION_OFFSET);
+
+ return ((SPEC_GEN_S <= check_no && check_no < SPEC_GEN_S + SPEC_N)
+ || (4 * SPEC_N <= check_no && check_no < 4 * SPEC_N + SPEC_N));
+}
+
+/* Generate (or regenerate, if MUTATE_P != 0) recovery check for INSN.
+ If (LABEL != 0 || MUTATE_P != 0), generate branchy recovery check.
+ Otherwise, generate a simple check. */
+static rtx
+ia64_gen_check (rtx insn, rtx label, int mutate_p)
+{
+ rtx op1, pat, check_pat;
+
+ static rtx (* const gen_check[]) (rtx, rtx) = {
+ gen_movbi_clr,
+ gen_movqi_clr,
+ gen_movhi_clr,
+ gen_movsi_clr,
+ gen_movdi_clr,
+ gen_movsf_clr,
+ gen_movdf_clr,
+ gen_movxf_clr,
+ gen_movti_clr,
+ gen_zero_extendqidi2_clr,
+ gen_zero_extendhidi2_clr,
+ gen_zero_extendsidi2_clr,
+
+ gen_speculation_check_bi,
+ gen_speculation_check_qi,
+ gen_speculation_check_hi,
+ gen_speculation_check_si,
+ gen_speculation_check_di,
+ gen_speculation_check_sf,
+ gen_speculation_check_df,
+ gen_speculation_check_xf,
+ gen_speculation_check_ti,
+ gen_speculation_check_di,
+ gen_speculation_check_di,
+ gen_speculation_check_di,
+
+ gen_movbi_clr,
+ gen_movqi_clr,
+ gen_movhi_clr,
+ gen_movsi_clr,
+ gen_movdi_clr,
+ gen_movsf_clr,
+ gen_movdf_clr,
+ gen_movxf_clr,
+ gen_movti_clr,
+ gen_zero_extendqidi2_clr,
+ gen_zero_extendhidi2_clr,
+ gen_zero_extendsidi2_clr,
+
+ gen_movbi_clr,
+ gen_movqi_clr,
+ gen_movhi_clr,
+ gen_movsi_clr,
+ gen_movdi_clr,
+ gen_movsf_clr,
+ gen_movdf_clr,
+ gen_movxf_clr,
+ gen_movti_clr,
+ gen_zero_extendqidi2_clr,
+ gen_zero_extendhidi2_clr,
+ gen_zero_extendsidi2_clr,
+
+ gen_advanced_load_check_clr_bi,
+ gen_advanced_load_check_clr_qi,
+ gen_advanced_load_check_clr_hi,
+ gen_advanced_load_check_clr_si,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_sf,
+ gen_advanced_load_check_clr_df,
+ gen_advanced_load_check_clr_xf,
+ gen_advanced_load_check_clr_ti,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+
+ /* Following checks are generated during mutation. */
+ gen_advanced_load_check_clr_bi,
+ gen_advanced_load_check_clr_qi,
+ gen_advanced_load_check_clr_hi,
+ gen_advanced_load_check_clr_si,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_sf,
+ gen_advanced_load_check_clr_df,
+ gen_advanced_load_check_clr_xf,
+ gen_advanced_load_check_clr_ti,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+
+ 0,0,0,0,0,0,0,0,0,0,0,0,
+
+ gen_advanced_load_check_clr_bi,
+ gen_advanced_load_check_clr_qi,
+ gen_advanced_load_check_clr_hi,
+ gen_advanced_load_check_clr_si,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_sf,
+ gen_advanced_load_check_clr_df,
+ gen_advanced_load_check_clr_xf,
+ gen_advanced_load_check_clr_ti,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+ gen_advanced_load_check_clr_di,
+
+ gen_speculation_check_bi,
+ gen_speculation_check_qi,
+ gen_speculation_check_hi,
+ gen_speculation_check_si,
+ gen_speculation_check_di,
+ gen_speculation_check_sf,
+ gen_speculation_check_df,
+ gen_speculation_check_xf,
+ gen_speculation_check_ti,
+ gen_speculation_check_di,
+ gen_speculation_check_di,
+ gen_speculation_check_di
+ };
+
+ extract_insn_cached (insn);
+
+ if (label)
+ {
+ gcc_assert ((!mutate_p && ia64_needs_block_p (insn))
+ || (mutate_p && !ia64_needs_block_p (insn)));
+ op1 = label;
+ }
+ else
+ {
+ gcc_assert (!mutate_p && ! ia64_needs_block_p (insn));
+ op1 = copy_rtx (recog_data.operand[1]);
+ }
+
+ if (mutate_p)
+ CHECK_NO (insn) += SPEC_GEN_CHECK_MUTATION_OFFSET;
+
+ check_pat = gen_check[CHECK_NO (insn)] (copy_rtx
+ (recog_data.operand[0]), op1);
+
+ pat = PATTERN (insn);
+ if (GET_CODE (pat) == COND_EXEC)
+ check_pat = gen_rtx_COND_EXEC (VOIDmode, copy_rtx
+ (COND_EXEC_TEST (pat)), check_pat);
+
+ return check_pat;
+}
+
+/* Return nonzero, if X is branchy recovery check. */
+static int
+ia64_spec_check_p (rtx x)
+{
+ x = PATTERN (x);
+ if (GET_CODE (x) == COND_EXEC)
+ x = COND_EXEC_CODE (x);
+ if (GET_CODE (x) == SET)
+ return ia64_spec_check_src_p (SET_SRC (x));
+ return 0;
+}
+
+/* Return nonzero, if SRC belongs to recovery check. */
+static int
+ia64_spec_check_src_p (rtx src)
+{
+ if (GET_CODE (src) == IF_THEN_ELSE)
+ {
+ rtx t;
+
+ t = XEXP (src, 0);
+ if (GET_CODE (t) == NE)
+ {
+ t = XEXP (t, 0);
+
+ if (GET_CODE (t) == UNSPEC)
+ {
+ int code;
+
+ code = XINT (t, 1);
+
+ if (code == UNSPEC_CHKACLR
+ || code == UNSPEC_CHKS
+ || code == UNSPEC_LDCCLR)
+ {
+ gcc_assert (code != 0);
+ return code;
+ }
+ }
+ }
+ }
+ return 0;
+}
/* The following page contains abstract data `bundle states' which are
@@ -6995,7 +7615,7 @@ get_next_important_insn (rtx insn, rtx t
automata only says that we can issue an insn possibly inserting
some nops before it and using some template. Therefore insn
bundling in this function is implemented by using DFA
- (deterministic finite automata). We follows all possible insn
+ (deterministic finite automata). We follow all possible insn
sequences by inserting 0-2 nops (that is what the NDFA describe for
insn scheduling) before/after each insn being bundled. We know the
start of simulated processor cycle from insn scheduling (insn
@@ -7085,7 +7705,7 @@ bundling (FILE *dump, int verbose, rtx p
break;
}
}
- /* Froward pass: generation of bundle states. */
+ /* Forward pass: generation of bundle states. */
for (insn = get_next_important_insn (NEXT_INSN (prev_head_insn), tail);
insn != NULL_RTX;
insn = next_insn)
@@ -7618,11 +8238,27 @@ ia64_ld_address_bypass_p (rtx producer,
gcc_assert (src);
mem = SET_SRC (src);
gcc_assert (mem);
+
if (GET_CODE (mem) == UNSPEC && XVECLEN (mem, 0) > 0)
mem = XVECEXP (mem, 0, 0);
+ else if (GET_CODE (mem) == IF_THEN_ELSE)
+ /* ??? Is this bypass neccessary for ld.c? */
+ {
+ gcc_assert (XINT (XEXP (XEXP (mem, 0), 0), 1) == UNSPEC_LDCCLR);
+ mem = XEXP (mem, 1);
+ }
+
while (GET_CODE (mem) == SUBREG || GET_CODE (mem) == ZERO_EXTEND)
mem = XEXP (mem, 0);
+ if (GET_CODE (mem) == UNSPEC)
+ {
+ int c = XINT (mem, 1);
+
+ gcc_assert (c == UNSPEC_LDA || c == UNSPEC_LDS || c == UNSPEC_LDSA);
+ mem = XVECEXP (mem, 0, 0);
+ }
+
/* Note that LO_SUM is used for GOT loads. */
gcc_assert (GET_CODE (mem) == LO_SUM || GET_CODE (mem) == MEM);
@@ -7803,6 +8439,7 @@ ia64_reorg (void)
free (clocks);
}
free (stops_p);
+ stops_p = NULL;
emit_insn_group_barriers (dump_file);
ia64_final_schedule = 0;
Index: gcc/config/ia64/itanium2.md
===================================================================
--- gcc/config/ia64/itanium2.md (.../insn-tick) (revision 2240)
+++ gcc/config/ia64/itanium2.md (.../recovery) (revision 2240)
@@ -731,14 +731,38 @@
(eq_attr "itanium_class" "fcvtfx"))
(eq (symbol_ref "bundling_p") (const_int 0))) "2_F")
(define_insn_reservation "2_fld" 6
- (and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "fld"))
- (eq (symbol_ref "bundling_p") (const_int 0))) "2_M")
+ (and (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "data_speculative" "no"))
+ (eq_attr "check_load" "no"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M")
+(define_insn_reservation "2_flda" 6
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "data_speculative" "yes"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um01")
+(define_insn_reservation "2_fldc" 0
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "yes"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um01")
+
(define_insn_reservation "2_fldp" 6
- (and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "fldp"))
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "no"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um01")
+(define_insn_reservation "2_fldpc" 0
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "yes"))
(eq (symbol_ref "bundling_p") (const_int 0)))
"2_M_only_um01")
+
(define_insn_reservation "2_fmac" 4
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "fmac"))
@@ -801,11 +825,19 @@
(eq_attr "itanium_class" "ishf"))
(eq (symbol_ref "bundling_p") (const_int 0)))
"2_I+2_only_ui0")
+
(define_insn_reservation "2_ld" 1
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "no"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um01")
+(define_insn_reservation "2_ldc" 0
(and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "yes"))
(eq (symbol_ref "bundling_p") (const_int 0)))
"2_M_only_um01")
+
(define_insn_reservation "2_long_i" 1
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "long_i"))
@@ -909,11 +941,22 @@
(eq_attr "itanium_class" "xtd"))
(eq (symbol_ref "bundling_p") (const_int 0))) "2_I")
-(define_insn_reservation "2_chk_s" 0
+(define_insn_reservation "2_chk_s_i" 0
(and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "chk_s"))
+ (eq_attr "itanium_class" "chk_s_i"))
(eq (symbol_ref "bundling_p") (const_int 0)))
"2_I|2_M_only_um23")
+(define_insn_reservation "2_chk_s_f" 0
+ (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "chk_s_f"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um23")
+(define_insn_reservation "2_chk_a" 0
+ (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "chk_a"))
+ (eq (symbol_ref "bundling_p") (const_int 0)))
+ "2_M_only_um01")
+
(define_insn_reservation "2_lfetch" 0
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "lfetch"))
@@ -1025,23 +1068,23 @@
(define_bypass 0 "2_tbit" "2_br,2_scall")
(define_bypass 2 "2_ld" "2_ld" "ia64_ld_address_bypass_p")
(define_bypass 2 "2_ld" "2_st" "ia64_st_address_bypass_p")
-(define_bypass 2 "2_ld" "2_mmalua,2_mmmul,2_mmshf")
+(define_bypass 2 "2_ld,2_ldc" "2_mmalua,2_mmmul,2_mmshf")
(define_bypass 3 "2_ilog" "2_mmalua,2_mmmul,2_mmshf")
(define_bypass 3 "2_ialu" "2_mmalua,2_mmmul,2_mmshf")
-(define_bypass 3 "2_mmalua,2_mmmul,2_mmshf" "2_ialu,2_ilog,2_ishf,2_st,2_ld")
+(define_bypass 3 "2_mmalua,2_mmmul,2_mmshf" "2_ialu,2_ilog,2_ishf,2_st,2_ld,2_ldc")
(define_bypass 6 "2_tofr" "2_frfr,2_stf")
(define_bypass 7 "2_fmac" "2_frfr,2_stf")
;; We don't use here fcmp because scall may be predicated.
-(define_bypass 0 "2_fcvtfx,2_fld,2_fmac,2_fmisc,2_frar_i,2_frar_m,\
- 2_frbr,2_frfr,2_frpr,2_ialu,2_ilog,2_ishf,2_ld,2_long_i,\
+(define_bypass 0 "2_fcvtfx,2_fld,2_flda,2_fldc,2_fmac,2_fmisc,2_frar_i,2_frar_m,\
+ 2_frbr,2_frfr,2_frpr,2_ialu,2_ilog,2_ishf,2_ld,2_ldc,2_long_i,\
2_mmalua,2_mmmul,2_mmshf,2_mmshfi,2_toar_m,2_tofr,\
2_xmpy,2_xtd"
"2_scall")
-(define_bypass 0 "2_unknown,2_ignore,2_stop_bit,2_br,2_fcmp,2_fcvtfx,2_fld,\
+(define_bypass 0 "2_unknown,2_ignore,2_stop_bit,2_br,2_fcmp,2_fcvtfx,2_fld,2_flda,2_fldc,\
2_fmac,2_fmisc,2_frar_i,2_frar_m,2_frbr,2_frfr,2_frpr,\
- 2_ialu,2_icmp,2_ilog,2_ishf,2_ld,2_chk_s,2_long_i,\
+ 2_ialu,2_icmp,2_ilog,2_ishf,2_ld,2_ldc,2_chk_s_i,2_chk_s_f,2_chk_a,2_long_i,\
2_mmalua,2_mmmul,2_mmshf,2_mmshfi,2_nop,2_nop_b,2_nop_f,\
2_nop_i,2_nop_m,2_nop_x,2_rse_m,2_scall,2_sem,2_stf,2_st,\
2_syst_m0,2_syst_m,2_tbit,2_toar_i,2_toar_m,2_tobr,2_tofr,\
@@ -1543,14 +1586,38 @@
(eq_attr "itanium_class" "fcvtfx"))
(ne (symbol_ref "bundling_p") (const_int 0))) "2b_F")
(define_insn_reservation "2b_fld" 6
- (and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "fld"))
- (ne (symbol_ref "bundling_p") (const_int 0))) "2b_M")
+ (and (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "data_speculative" "no"))
+ (eq_attr "check_load" "no"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M")
+(define_insn_reservation "2b_flda" 6
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "data_speculative" "yes"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um01")
+(define_insn_reservation "2b_fldc" 0
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fld"))
+ (eq_attr "check_load" "yes"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um01")
+
(define_insn_reservation "2b_fldp" 6
- (and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "fldp"))
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "no"))
(ne (symbol_ref "bundling_p") (const_int 0)))
"2b_M_only_um01")
+(define_insn_reservation "2b_fldpc" 0
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "fldp"))
+ (eq_attr "check_load" "yes"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um01")
+
(define_insn_reservation "2b_fmac" 4
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "fmac"))
@@ -1611,11 +1678,20 @@
(eq_attr "itanium_class" "ishf"))
(ne (symbol_ref "bundling_p") (const_int 0)))
"2b_I+2b_only_ui0")
+
(define_insn_reservation "2b_ld" 1
- (and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "ld"))
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "no"))
(ne (symbol_ref "bundling_p") (const_int 0)))
"2b_M_only_um01")
+(define_insn_reservation "2b_ldc" 0
+ (and (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "ld"))
+ (eq_attr "check_load" "yes"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um01")
+
(define_insn_reservation "2b_long_i" 1
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "long_i"))
@@ -1714,11 +1790,23 @@
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "xtd"))
(ne (symbol_ref "bundling_p") (const_int 0))) "2b_I")
-(define_insn_reservation "2b_chk_s" 0
+
+(define_insn_reservation "2b_chk_s_i" 0
(and (and (eq_attr "cpu" "itanium2")
- (eq_attr "itanium_class" "chk_s"))
+ (eq_attr "itanium_class" "chk_s_i"))
(ne (symbol_ref "bundling_p") (const_int 0)))
"2b_I|2b_M_only_um23")
+(define_insn_reservation "2b_chk_s_f" 0
+ (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "chk_s_f"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um23")
+(define_insn_reservation "2b_chk_a" 0
+ (and (and (eq_attr "cpu" "itanium2")
+ (eq_attr "itanium_class" "chk_a"))
+ (ne (symbol_ref "bundling_p") (const_int 0)))
+ "2b_M_only_um01")
+
(define_insn_reservation "2b_lfetch" 0
(and (and (eq_attr "cpu" "itanium2")
(eq_attr "itanium_class" "lfetch"))
Index: gcc/config/ia64/ia64.md
===================================================================
--- gcc/config/ia64/ia64.md (.../insn-tick) (revision 2240)
+++ gcc/config/ia64/ia64.md (.../recovery) (revision 2240)
@@ -81,6 +81,12 @@
(UNSPEC_SHRP 29)
(UNSPEC_COPYSIGN 30)
(UNSPEC_VECT_EXTR 31)
+ (UNSPEC_LDA 40)
+ (UNSPEC_LDS 41)
+ (UNSPEC_LDSA 42)
+ (UNSPEC_LDCCLR 43)
+ (UNSPEC_CHKACLR 45)
+ (UNSPEC_CHKS 47)
])
(define_constants
@@ -124,18 +130,19 @@
(define_attr "itanium_class" "unknown,ignore,stop_bit,br,fcmp,fcvtfx,fld,
fldp,fmac,fmisc,frar_i,frar_m,frbr,frfr,frpr,ialu,icmp,ilog,ishf,
- ld,chk_s,long_i,mmalua,mmmul,mmshf,mmshfi,rse_m,scall,sem,stf,
+ ld,chk_s_i,chk_s_f,chk_a,long_i,mmalua,mmmul,mmshf,mmshfi,rse_m,scall,sem,stf,
st,syst_m0, syst_m,tbit,toar_i,toar_m,tobr,tofr,topr,xmpy,xtd,nop,
nop_b,nop_f,nop_i,nop_m,nop_x,lfetch,pre_cycle"
(const_string "unknown"))
-;; chk_s has an I and an M form; use type A for convenience.
+;; chk_s_i has an I and an M form; use type A for convenience.
(define_attr "type" "unknown,A,I,M,F,B,L,X,S"
(cond [(eq_attr "itanium_class" "ld,st,fld,fldp,stf,sem,nop_m") (const_string "M")
(eq_attr "itanium_class" "rse_m,syst_m,syst_m0") (const_string "M")
(eq_attr "itanium_class" "frar_m,toar_m,frfr,tofr") (const_string "M")
(eq_attr "itanium_class" "lfetch") (const_string "M")
- (eq_attr "itanium_class" "chk_s,ialu,icmp,ilog,mmalua")
+ (eq_attr "itanium_class" "chk_s_f,chk_a") (const_string "M")
+ (eq_attr "itanium_class" "chk_s_i,ialu,icmp,ilog,mmalua")
(const_string "A")
(eq_attr "itanium_class" "fmisc,fmac,fcmp,xmpy") (const_string "F")
(eq_attr "itanium_class" "fcvtfx,nop_f") (const_string "F")
@@ -170,6 +177,12 @@
;; when we have full intrinsics support.
(define_attr "first_insn" "no,yes" (const_string "no"))
+
+(define_attr "data_speculative" "no,yes" (const_string "no"))
+
+(define_attr "control_speculative" "no,yes" (const_string "no"))
+
+(define_attr "check_load" "no,yes" (const_string "no"))
;; DFA descriptions of ia64 processors used for insn scheduling and
;; bundling.
@@ -372,6 +385,191 @@
}
[(set_attr "itanium_class" "ialu,ialu,long_i,ld,st,frfr,tofr,fmisc,fld,stf,frbr,tobr,frar_i,toar_i,frar_m,toar_m,frpr,topr")])
+(define_mode_macro MODE [BI QI HI SI DI SF DF XF TI])
+(define_mode_macro MODE_FOR_EXTEND [QI HI SI])
+
+(define_mode_attr output_a [
+ (BI "ld1.a %0 = %1%P1")
+ (QI "ld1.a %0 = %1%P1")
+ (HI "ld2.a %0 = %1%P1")
+ (SI "ld4.a %0 = %1%P1")
+ (DI
+ "@
+ ld8.a %0 = %1%P1
+ ldf8.a %0 = %1%P1")
+ (SF
+ "@
+ ldfs.a %0 = %1%P1
+ ld4.a %0 = %1%P1")
+ (DF
+ "@
+ ldfd.a %0 = %1%P1
+ ld8.a %0 = %1%P1")
+ (XF "ldfe.a %0 = %1%P1")
+ (TI "ldfp8.a %X0 = %1%P1")])
+
+(define_mode_attr output_s [
+ (BI "ld1.s %0 = %1%P1")
+ (QI "ld1.s %0 = %1%P1")
+ (HI "ld2.s %0 = %1%P1")
+ (SI "ld4.s %0 = %1%P1")
+ (DI
+ "@
+ ld8.s %0 = %1%P1
+ ldf8.s %0 = %1%P1")
+ (SF
+ "@
+ ldfs.s %0 = %1%P1
+ ld4.s %0 = %1%P1")
+ (DF
+ "@
+ ldfd.s %0 = %1%P1
+ ld8.s %0 = %1%P1")
+ (XF "ldfe.s %0 = %1%P1")
+ (TI "ldfp8.s %X0 = %1%P1")])
+
+(define_mode_attr output_sa [
+ (BI "ld1.sa %0 = %1%P1")
+ (QI "ld1.sa %0 = %1%P1")
+ (HI "ld2.sa %0 = %1%P1")
+ (SI "ld4.sa %0 = %1%P1")
+ (DI
+ "@
+ ld8.sa %0 = %1%P1
+ ldf8.sa %0 = %1%P1")
+ (SF
+ "@
+ ldfs.sa %0 = %1%P1
+ ld4.sa %0 = %1%P1")
+ (DF
+ "@
+ ldfd.sa %0 = %1%P1
+ ld8.sa %0 = %1%P1")
+ (XF "ldfe.sa %0 = %1%P1")
+ (TI "ldfp8.sa %X0 = %1%P1")])
+
+(define_mode_attr output_c_clr [
+ (BI "ld1.c.clr%O1 %0 = %1%P1")
+ (QI "ld1.c.clr%O1 %0 = %1%P1")
+ (HI "ld2.c.clr%O1 %0 = %1%P1")
+ (SI "ld4.c.clr%O1 %0 = %1%P1")
+ (DI
+ "@
+ ld8.c.clr%O1 %0 = %1%P1
+ ldf8.c.clr %0 = %1%P1")
+ (SF
+ "@
+ ldfs.c.clr %0 = %1%P1
+ ld4.c.clr%O1 %0 = %1%P1")
+ (DF
+ "@
+ ldfd.c.clr %0 = %1%P1
+ ld8.c.clr%O1 %0 = %1%P1")
+ (XF "ldfe.c.clr %0 = %1%P1")
+ (TI "ldfp8.c.clr %X0 = %1%P1")])
+
+(define_mode_attr ld_reg_constr [(BI "=*r") (QI "=r") (HI "=r") (SI "=r") (DI "=r,*f") (SF "=f,*r") (DF "=f,*r") (XF "=f") (TI "=*x")])
+(define_mode_attr ldc_reg_constr [(BI "+*r") (QI "+r") (HI "+r") (SI "+r") (DI "+r,*f") (SF "+f,*r") (DF "+f,*r") (XF "+f") (TI "+*x")])
+(define_mode_attr chk_reg_constr [(BI "*r") (QI "r") (HI "r") (SI "r") (DI "r,*f") (SF "f,*r") (DF "f,*r") (XF "f") (TI "*x")])
+
+(define_mode_attr mem_constr [(BI "*m") (QI "m") (HI "m") (SI "m") (DI "m,Q") (SF "Q,m") (DF "Q,m") (XF "m") (TI "Q")])
+
+(define_mode_attr reg_pred_prefix [(BI "gr") (QI "gr") (HI "gr") (SI "gr") (DI "grfr") (SF "grfr") (DF "grfr") (XF "fr") (TI "fr")])
+
+(define_mode_attr ld_class [(BI "ld") (QI "ld") (HI "ld") (SI "ld") (DI "ld,fld") (SF "fld,ld") (DF "fld,ld") (XF "fld") (TI "fldp")])
+(define_mode_attr chka_class [(BI "chk_a") (QI "chk_a") (HI "chk_a") (SI "chk_a") (DI "chk_a,chk_a") (SF "chk_a,chk_a") (DF "chk_a,chk_a") (XF "chk_a") (TI "chk_a")])
+(define_mode_attr chks_class [(BI "chk_s_i") (QI "chk_s_i") (HI "chk_s_i") (SI "chk_s_i") (DI "chk_s_i,chk_s_f") (SF "chk_s_f,chk_s_i") (DF "chk_s_f,chk_s_i") (XF "chk_s_f") (TI "chk_s_i")])
+
+(define_mode_attr attr_yes [(BI "yes") (QI "yes") (HI "yes") (SI "yes") (DI "yes,yes") (SF "yes,yes") (DF "yes,yes") (XF "yes") (TI "yes")])
+
+(define_insn "mov<mode>_advanced"
+ [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+ (unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDA))]
+ "ia64_move_ok (operands[0], operands[1])"
+ "<output_a>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "data_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_advanced"
+ [(set (match_operand:DI 0 "gr_register_operand" "=r")
+ (zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDA)))]
+ ""
+ "<output_a>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "data_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_speculative"
+ [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+ (unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDS))]
+ "ia64_move_ok (operands[0], operands[1])"
+ "<output_s>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_speculative"
+ [(set (match_operand:DI 0 "gr_register_operand" "=r")
+ (zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDS)))]
+ ""
+ "<output_s>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_speculative_advanced"
+ [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+ (unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDSA))]
+ "ia64_move_ok (operands[0], operands[1])"
+ "<output_sa>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "data_speculative" "<attr_yes>")
+ (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_speculative_advanced"
+ [(set (match_operand:DI 0 "gr_register_operand" "=r")
+ (zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDSA)))]
+ ""
+ "<output_sa>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "data_speculative" "<attr_yes>")
+ (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_clr"
+ [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ldc_reg_constr>")
+ (if_then_else:MODE (ne (unspec [(match_dup 0)] UNSPEC_LDCCLR) (const_int 0))
+ (match_operand:MODE 1 "memory_operand" "<mem_constr>")
+ (match_dup 0)))]
+ "ia64_move_ok (operands[0], operands[1])"
+ "<output_c_clr>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "check_load" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_clr"
+ [(set (match_operand:DI 0 "gr_register_operand" "+r")
+ (if_then_else:DI (ne (unspec [(match_dup 0)] UNSPEC_LDCCLR) (const_int 0))
+ (zero_extend:DI (match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>"))
+ (match_dup 0)))]
+ ""
+ "<output_c_clr>"
+ [(set_attr "itanium_class" "<ld_class>")
+ (set_attr "check_load" "<attr_yes>")])
+
+(define_insn "advanced_load_check_clr_<mode>"
+ [(set (pc)
+ (if_then_else (ne (unspec [(match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<chk_reg_constr>")] UNSPEC_CHKACLR) (const_int 0))
+ (pc)
+ (label_ref (match_operand 1 "" ""))))]
+ ""
+ "chk.a.clr %0, %l1"
+ [(set_attr "itanium_class" "<chka_class>")])
+
+(define_insn "speculation_check_<mode>"
+ [(set (pc)
+ (if_then_else (ne (unspec [(match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<chk_reg_constr>")] UNSPEC_CHKS) (const_int 0))
+ (pc)
+ (label_ref (match_operand 1 "" ""))))]
+ ""
+ "chk.s %0, %l1"
+ [(set_attr "itanium_class" "<chks_class>")])
+
(define_split
[(set (match_operand 0 "register_operand" "")
(match_operand 1 "symbolic_operand" ""))]
@@ -6114,7 +6312,7 @@
[(trap_if (const_int 1) (match_operand 0 "const_int_operand" ""))]
""
"break %0"
- [(set_attr "itanium_class" "chk_s")])
+ [(set_attr "itanium_class" "chk_s_i")])
(define_expand "conditional_trap"
[(trap_if (match_operand 0 "" "") (match_operand 1 "" ""))]
@@ -6130,7 +6328,7 @@
(match_operand 2 "const_int_operand" ""))]
""
"(%J0) break %2"
- [(set_attr "itanium_class" "chk_s")
+ [(set_attr "itanium_class" "chk_s_i")
(set_attr "predicable" "no")])
(define_insn "break_f"