This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
[sel-sched] Limit renaming to the best insns only, compute target register availability on the fly
- From: Andrey Belevantsev <abel at ispras dot ru>
- To: GCC Patches <gcc-patches at gcc dot gnu dot org>
- Date: Wed, 26 Dec 2007 18:50:52 +0300
- Subject: [sel-sched] Limit renaming to the best insns only, compute target register availability on the fly
Hello,
This patch aims at computing the availability of the target register of
an insn on the fly. This is enough for insns which we are not going to
rename, and this saves us from the necessity of calling find_used_regs.
For the two best insns, if their original registers are not available,
we still try to rename them, i.e. we call find_used_regs to compute the
set of possible target registers for them. The number of insns that are
eligible for renaming is controlled by a param.
The net result of the patch is fewer calls to find_used_regs, which is
an expensive operation. This speedups the scheduler by 15-20% on the
heavy testcases.
To implement this, an extra field is added to struct expr which shows
whether the target register of this expression is available for
scheduling. The tricky issue is merging this field when expressions are
joined both along different code paths and along the same code path.
The compiler was bootstrapped and tested on ia64 with the assert in
find_used_regs verifying the consistency of the computed information
with the one we had before. Committed to sel-sched branch.
Andrey
2007-12-26 Andrey Belevantsev <abel@ispras.ru>
Compute availability of target registers on the fly.
Limit renaming to the best instructions.
* sel-sched.c (max_insns_to_rename): New variable.
(struct reg_rename): New structure that incapsulates the data
about hard registers available for renaming.
(expr_dest_reg, rhs_dest_regno): Kill.
(vinsn_writes_one_of_regs_p): Also check VINSN_REG_CLOBBERS.
(mark_unavailable_hard_regs): Use struct reg_rename instead of
separate hard reg sets.
(choose_best_reg_1): Ditto. Also check all hard_regno_nregs
registers, not only the first one.
(choose_best_reg, choose_best_pseudo_reg): Ditto.
(verify_target_availability): New.
(find_used_regs): Use it for checking that EXPR_TARGET_AVAILABLE
bit is consistent with find_used_regs_info. Also check for
VINSN_REG_CLOBBERS. Use struct reg_rename.
(apply_spec_to_expr): Properly compute EXPR_TARGET_AVAILABLE
for speculative insns.
(moveup_rhs): Fix formatting. Mark the target register of separate
instructions having dependencies in LHS as unavailable.
(moveup_set_rhs): Use merge_with_other_exprs when an expression
was changed.
(compute_av_set): Add comment. Compute other successors and
calculate whether target registers of expressions are available.
Use av_set_union_and_live and mark_unavailable_targets.
(propagate_lv_set): Handle INSN_REG_CLOBBERS too.
(compute_live): Export.
(find_used_regs_1): Use struct reg_rename. Handle INSN_REG_CLOBBERS.
(find_used_regs): Ditto.
(fill_vec_av_set): Calculate statistics about target availability
of expressions. Sort the vector before filtering out expressions.
Use target_available information for all expressions. Try to rename
only max_insns_to_rename expressions.
(find_best_expr): Record whether an instruction was renamed.
(generate_bookkeeping_insn, remove_insns_that_need_bookkeeping): Tidy.
(fill_insns): Tidy. Use expr_dest_regno. Set target availability
flag for the newly emitted insn to true.
(sel_global_init): Set max_insns_to_rename.
* sel-sched-ir.c (vinsn_delete): Also handle VINSN_REG_CLOBBERS.
(init_expr): Add target_available, was_renamed parameters.
Update all callers.
(merge_expr_data): New parameter JOIN_POINT_P. Compute target
availability also using this parameter. Handle EXPR_WAS_RENAMED.
(merge_expr): Ditto. Properly change vinsns of speculative exprs.
(set_unavailable_target_for_expr, expr_dest_reg, expr_dest_regno,
mark_unavailable_targets, av_set_add_nocopy, av_set_lookup_and_remove,
merge_with_other_exprs, av_set_union_and_live): New.
(av_set_lookup_other_equiv_rhs): New parameter laterp.
(deps_init_id_start_insn, deps_init_id_note_reg_clobber): Handle
IDATA_REG_CLOBBERS.
(cfg_succs_other): New.
(sel_split_edge): New variable other_bb. Assert that there is no more
than two blocks added during splitting. Put the correct lv set on the
other bb.
* sel-sched-ir.h (struct _expr): New fields target_available,
was_renamed.
(EXPR_TARGET_AVAILABLE, EXPR_WAS_RENAMED): New accessor macros.
(_list_remove_nofree, _list_iter_remove_nofree): New.
(struct idata_def): New field reg_clobbers.
(IDATA_REG_CLOBBERS, VINSN_REG_CLOBBERS, INSN_REG_CLOBBERS): New
accessor macros.
(cfg_succs_other, av_set_union_and_live, merge_with_other_exprs,
expr_dest_regno, expr_dest_reg. max_insns_to_rename): Export.
* sel-sched-dump.c (dump_expr_1): Handle EXPR_TARGET_AVAILABLE.
* sched-rgn.h (struct region): Kill has_renaming_p, was_pipelined_p,
needs_global_live_update fields. Kill all uses.
(RGN_HAS_RENAMING_P, RGN_WAS_PIPELINED_P,
RGN_NEEDS_GLOBAL_LIVE_UPDATE): Kill accessor macros. Kill all uses.
* params.def (PARAM_SELSCHED_INSNS_TO_RENAME): New.
Index: gcc/sel-sched.c
===================================================================
--- gcc/sel-sched.c (revision 131182)
+++ gcc/sel-sched.c (working copy)
@@ -86,6 +86,9 @@ bool bookkeeping_p;
/* True if we should make an aditional pass to set somewhat correct
sched cycles. */
bool reset_sched_cycles_p;
+
+/* Maximum number of insns that are eligible for renaming. */
+int max_insns_to_rename;
/* Definitions of local types and macros. */
@@ -153,6 +156,17 @@ struct hard_regs_data
#endif
};
+/* Holds the results of computation of available for renaming and
+ unavailable hard registers. */
+struct reg_rename
+{
+ /* These are unavailable due to calls crossing, globalness, etc. */
+ HARD_REG_SET unavailable_hard_regs;
+
+ /* These are *available* for renaming. */
+ HARD_REG_SET available_for_renaming;
+};
+
/* A global structure that contains the needed information about harg
regs. */
static struct hard_regs_data sel_hrd;
@@ -211,9 +225,8 @@ static int stat_substitutions_total;
static bool rtx_search (rtx, rtx);
static int sel_rank_for_schedule (const void *, const void *);
static bool equal_after_moveup_path_p (rhs_t, ilist_t, rhs_t);
-static regset compute_live (insn_t);
static basic_block generate_bookkeeping_insn (rhs_t, insn_t, edge, edge);
-static bool find_used_regs (insn_t, av_set_t, regset, HARD_REG_SET *,
+static bool find_used_regs (insn_t, av_set_t, regset, struct reg_rename *,
def_list_t *);
static bool move_op (insn_t, av_set_t, ilist_t, edge, edge, expr_t);
static void sel_sched_region_1 (void);
@@ -687,28 +700,6 @@ replace_dest_with_reg_in_rhs (rhs_t rhs,
change_vinsn_in_expr (rhs, vinsn);
}
-/* Return a destination register, if any, of EXPR. */
-static rtx
-expr_dest_reg (expr_t expr)
-{
- rtx dest = VINSN_LHS (RHS_VINSN (expr));
-
- if (dest != NULL_RTX && REG_P (dest))
- return dest;
-
- return NULL_RTX;
-}
-
-/* Returns the REGNO of the R's destination. */
-static unsigned
-rhs_dest_regno (rhs_t r)
-{
- rtx dest = expr_dest_reg (r);
-
- gcc_assert (dest != NULL_RTX);
- return REGNO (dest);
-}
-
/* Returns whether VI writes one of the REGS. */
static bool
vinsn_writes_one_of_regs_p (vinsn_t vi, regset used_regs,
@@ -726,6 +717,15 @@ vinsn_writes_one_of_regs_p (vinsn_t vi,
return true;
}
+ EXECUTE_IF_SET_IN_REG_SET (VINSN_REG_CLOBBERS (vi), 0, regno, rsi)
+ {
+ if (REGNO_REG_SET_P (used_regs, regno))
+ return true;
+ if (HARD_REGISTER_NUM_P (regno)
+ && TEST_HARD_REG_BIT (unavailable_hard_regs, regno))
+ return true;
+ }
+
return false;
}
@@ -917,7 +917,7 @@ init_hard_regs_data (void)
#endif
}
-/* Mark hardware regs in UNAVAILABLE_HARD_REGS that are not suitable
+/* Mark hardware regs in REG_RENAME_P that are not suitable
for renaming rhs in INSN due to hardware restrictions (register class,
modes compatibility etc). This doesn't affect original insn's dest reg,
if it isn't in USED_REGS. DEF is a definition insn of rhs for which the
@@ -926,17 +926,16 @@ init_hard_regs_data (void)
unavailable_hard_regs as well. */
static void
-mark_unavailable_hard_regs (def_t def, HARD_REG_SET *unavailable_hard_regs,
+mark_unavailable_hard_regs (def_t def, struct reg_rename *reg_rename_p,
regset used_regs ATTRIBUTE_UNUSED)
{
enum machine_mode mode;
enum reg_class cl = NO_REGS;
rtx orig_dest;
int cur_reg, regno;
- HARD_REG_SET hard_regs_ok;
gcc_assert (GET_CODE (PATTERN (def->orig_insn)) == SET);
- gcc_assert (unavailable_hard_regs);
+ gcc_assert (reg_rename_p);
orig_dest = SET_DEST (PATTERN (def->orig_insn));
@@ -953,7 +952,7 @@ mark_unavailable_hard_regs (def_t def, H
mode = GET_MODE (orig_dest);
- /* Stop when mode is not supported for renaming. Also Can't proceed
+ /* Stop when mode is not supported for renaming. Also can't proceed
if the original register is one of the fixed_regs, global_regs or
frame pointer. */
if (fixed_regs[regno]
@@ -965,10 +964,11 @@ mark_unavailable_hard_regs (def_t def, H
#endif
)
{
- SET_HARD_REG_SET (*unavailable_hard_regs);
+ SET_HARD_REG_SET (reg_rename_p->unavailable_hard_regs);
/* Give a chance for original register, if it isn't in used_regs. */
- CLEAR_HARD_REG_BIT (*unavailable_hard_regs, regno);
+ if (!def->crosses_call)
+ CLEAR_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs, regno);
return;
}
@@ -981,11 +981,12 @@ mark_unavailable_hard_regs (def_t def, H
int i;
for (i = hard_regno_nregs[FRAME_POINTER_REGNUM][Pmode]; i--;)
- SET_HARD_REG_BIT (*unavailable_hard_regs, FRAME_POINTER_REGNUM + i);
+ SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs,
+ FRAME_POINTER_REGNUM + i);
#if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM
for (i = hard_regno_nregs[HARD_FRAME_POINTER_REGNUM][Pmode]; i--;)
- SET_HARD_REG_BIT (*unavailable_hard_regs,
+ SET_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs,
HARD_FRAME_POINTER_REGNUM + i);
#endif
}
@@ -999,13 +1000,15 @@ mark_unavailable_hard_regs (def_t def, H
The HARD_REGNO_RENAME_OK covers other cases in condition below. */
if (IN_RANGE (REGNO (orig_dest), FIRST_STACK_REG, LAST_STACK_REG)
&& REGNO_REG_SET_P (used_regs, FIRST_STACK_REG))
- IOR_HARD_REG_SET (*unavailable_hard_regs, sel_hrd.stack_regs);
+ IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs,
+ sel_hrd.stack_regs);
#endif
/* If there's a call on this path, make regs from call_used_reg_set
unavailable. */
if (def->crosses_call)
- IOR_HARD_REG_SET (*unavailable_hard_regs, call_used_reg_set);
+ IOR_HARD_REG_SET (reg_rename_p->unavailable_hard_regs,
+ call_used_reg_set);
/* Stop here before reload: we need FRAME_REGS, STACK_REGS, and crosses_call,
but not register classes. */
@@ -1016,19 +1019,19 @@ mark_unavailable_hard_regs (def_t def, H
register class. */
cl = get_reg_class (def->orig_insn);
gcc_assert (cl != NO_REGS);
- IOR_COMPL_HARD_REG_SET (*unavailable_hard_regs, reg_class_contents[cl]);
+ COPY_HARD_REG_SET (reg_rename_p->available_for_renaming,
+ reg_class_contents[cl]);
+ /* Leave only registers available for this mode. */
if (!sel_hrd.regs_for_mode_ok[mode])
init_regs_for_mode (mode);
-
- /* Leave only registers available for this mode. */
- CLEAR_HARD_REG_SET (hard_regs_ok);
- IOR_HARD_REG_SET (hard_regs_ok, sel_hrd.regs_for_mode[mode]);
+ AND_HARD_REG_SET (reg_rename_p->available_for_renaming,
+ sel_hrd.regs_for_mode[mode]);
/* Exclude registers that are partially call clobbered. */
if (def->crosses_call
&& ! HARD_REGNO_CALL_PART_CLOBBERED (regno, mode))
- AND_COMPL_HARD_REG_SET (hard_regs_ok,
+ AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming,
sel_hrd.regs_for_call_clobbered[mode]);
/* Leave only those that are ok to rename. */
@@ -1037,7 +1040,8 @@ mark_unavailable_hard_regs (def_t def, H
int nregs;
int i;
- if (!TEST_HARD_REG_BIT (hard_regs_ok, cur_reg))
+ if (!TEST_HARD_REG_BIT (reg_rename_p->available_for_renaming,
+ cur_reg))
continue;
nregs = hard_regno_nregs[cur_reg][mode];
@@ -1048,16 +1052,16 @@ mark_unavailable_hard_regs (def_t def, H
break;
if (i >= 0)
- CLEAR_HARD_REG_BIT (hard_regs_ok, cur_reg);
+ CLEAR_HARD_REG_BIT (reg_rename_p->available_for_renaming,
+ cur_reg);
}
+ AND_COMPL_HARD_REG_SET (reg_rename_p->available_for_renaming,
+ reg_rename_p->unavailable_hard_regs);
/* Regno is always ok from the renaming part of view, but it really
could be in *unavailable_hard_regs already, so set it here instead
of there. */
- SET_HARD_REG_BIT (hard_regs_ok, regno);
-
- /* Exclude all hard regs but HARD_REGS_OK. */
- IOR_COMPL_HARD_REG_SET (*unavailable_hard_regs, hard_regs_ok);
+ SET_HARD_REG_BIT (reg_rename_p->available_for_renaming, regno);
}
/* reg_rename_tick[REG1] > reg_rename_tick[REG2] if REG1 was chosen as the
@@ -1092,19 +1096,21 @@ static int reg_rename_tick[FIRST_PSEUDO_
If no register satisfies the above conditions, NULL_RTX is returned. */
static rtx
-choose_best_reg_1 (HARD_REG_SET unavailable, def_list_t original_insns,
- bool *is_orig_reg_p_ptr)
+choose_best_reg_1 (HARD_REG_SET hard_regs_used,
+ struct reg_rename *reg_rename_p,
+ def_list_t original_insns, bool *is_orig_reg_p_ptr)
{
int best_new_reg;
int cur_reg;
enum machine_mode mode = VOIDmode;
- def_list_iterator i;
+ unsigned regno, i, n;
+ def_list_iterator di;
def_t def;
/* If original register is available, return it. */
*is_orig_reg_p_ptr = true;
- FOR_EACH_DEF (def, i, original_insns)
+ FOR_EACH_DEF (def, di, original_insns)
{
rtx orig_dest = SET_DEST (PATTERN (def->orig_insn));
@@ -1118,26 +1124,35 @@ choose_best_reg_1 (HARD_REG_SET unavaila
mode = GET_MODE (orig_dest);
gcc_assert (mode == GET_MODE (orig_dest));
- if (!TEST_HARD_REG_BIT (unavailable, REGNO (orig_dest)))
+ regno = REGNO (orig_dest);
+ for (i = 0, n = hard_regno_nregs[regno][mode]; i < n; i++)
+ if (TEST_HARD_REG_BIT (hard_regs_used, regno + i))
+ break;
+
+ /* All hard registers are available. */
+ if (i == n)
{
gcc_assert (mode != VOIDmode);
-
+
/* Hard registers should not be shared. */
- return gen_rtx_REG (mode, REGNO (orig_dest));
+ return gen_rtx_REG (mode, regno);
}
}
-
+
*is_orig_reg_p_ptr = false;
best_new_reg = -1;
-
+
/* Among all available regs choose the register that was
allocated earliest. */
for (cur_reg = 0; cur_reg < FIRST_PSEUDO_REGISTER; cur_reg++)
- if (!TEST_HARD_REG_BIT (unavailable, cur_reg))
+ if (TEST_HARD_REG_BIT (reg_rename_p->available_for_renaming,
+ cur_reg)
+ && !TEST_HARD_REG_BIT (hard_regs_used, cur_reg))
{
- if (best_new_reg < 0
- ||reg_rename_tick[cur_reg] < reg_rename_tick[best_new_reg])
+ /* All hard registers are available. */
+ if (best_new_reg < 0
+ || reg_rename_tick[cur_reg] < reg_rename_tick[best_new_reg])
best_new_reg = cur_reg;
}
@@ -1154,11 +1169,11 @@ choose_best_reg_1 (HARD_REG_SET unavaila
/* A wrapper around choose_best_reg_1 () to verify that we make correct
assumptions about available registers in the function. */
static rtx
-choose_best_reg (HARD_REG_SET unavailable, def_list_t original_insns,
- bool *is_orig_reg_p_ptr)
+choose_best_reg (HARD_REG_SET hard_regs_used, struct reg_rename *reg_rename_p,
+ def_list_t original_insns, bool *is_orig_reg_p_ptr)
{
- rtx best_reg = choose_best_reg_1 (unavailable, original_insns,
- is_orig_reg_p_ptr);
+ rtx best_reg = choose_best_reg_1 (hard_regs_used, reg_rename_p,
+ original_insns, is_orig_reg_p_ptr);
gcc_assert (best_reg == NULL_RTX
|| TEST_HARD_REG_BIT (sel_hrd.regs_ever_used, REGNO (best_reg)));
@@ -1175,7 +1190,7 @@ choose_best_reg (HARD_REG_SET unavailabl
not rely on this. */
static rtx
choose_best_pseudo_reg (regset used_regs,
- HARD_REG_SET unavailable_hard_regs,
+ struct reg_rename *reg_rename_p,
def_list_t original_insns, bool *is_orig_reg_p_ptr)
{
def_list_iterator i;
@@ -1211,7 +1226,8 @@ choose_best_pseudo_reg (regset used_regs
/* For hard registers, we have to check hardware imposed
limitations (frame/stack registers, calls crossed). */
- if (!TEST_HARD_REG_BIT (unavailable_hard_regs, orig_regno))
+ if (!TEST_HARD_REG_BIT (reg_rename_p->unavailable_hard_regs,
+ orig_regno))
return gen_rtx_REG (mode, orig_regno);
bad_hard_regs = true;
@@ -1234,6 +1250,41 @@ choose_best_pseudo_reg (regset used_regs
return gen_reg_rtx (mode);
}
+/* True when target of EXPR is available due to TARGET_AVAILABLE,
+ USED_REGS and UNAVAILABLE_HARD_REGS. */
+static void
+verify_target_availability (expr_t expr, regset used_regs,
+ HARD_REG_SET unavailable_hard_regs)
+{
+ unsigned n, i, regno;
+ enum machine_mode mode;
+ bool target_available, live_available, hard_available;
+
+ if (!REG_P (EXPR_LHS (expr)) || EXPR_TARGET_AVAILABLE (expr) < 0)
+ return;
+
+ regno = expr_dest_regno (expr);
+ mode = GET_MODE (EXPR_LHS (expr));
+ target_available = EXPR_TARGET_AVAILABLE (expr) == 1;
+ n = reload_completed ? hard_regno_nregs[regno][mode] : 1;
+
+ live_available = hard_available = true;
+ for (i = 0; i < n; i++)
+ {
+ if (bitmap_bit_p (used_regs, regno + i))
+ live_available = false;
+ if (TEST_HARD_REG_BIT (unavailable_hard_regs, regno + i))
+ hard_available = false;
+ }
+
+ /* When target is not available, it may be due to hard register
+ restrictions, e.g. crosses calls, so we check hard_available too. */
+ if (target_available)
+ gcc_assert (live_available);
+ else
+ gcc_assert (!live_available || !hard_available);
+}
+
/* Returns best register for given rhs, or NULL_RTX, if no register can be
chosen. The latter could happen when:
- RHS_SCHEDULE_AS_RHS is true but we were unable to find suitable
@@ -1243,10 +1294,12 @@ choose_best_pseudo_reg (regset used_regs
static bool
find_best_reg_for_rhs (rhs_t rhs, blist_t bnds, bool *is_orig_reg_p)
{
+ static struct reg_rename reg_rename_data;
+
av_set_iterator i2;
rhs_t rhs_orig;
regset used_regs;
- HARD_REG_SET hard_regs_used, unavailable_hard_regs;
+ HARD_REG_SET hard_regs_used;
rtx best_reg = NULL_RTX;
blist_t bnds1 = bnds;
def_list_t original_insns = NULL;
@@ -1256,11 +1309,12 @@ find_best_reg_for_rhs (rhs_t rhs, blist_
*is_orig_reg_p = false;
/* Don't bother to do anything if this insn doesn't set any registers. */
- if (bitmap_empty_p (VINSN_REG_SETS (EXPR_VINSN (rhs))))
+ if (bitmap_empty_p (VINSN_REG_SETS (EXPR_VINSN (rhs)))
+ && bitmap_empty_p (VINSN_REG_CLOBBERS (EXPR_VINSN (rhs))))
return true;
used_regs = get_clear_regset_from_pool ();
- CLEAR_HARD_REG_SET (unavailable_hard_regs);
+ CLEAR_HARD_REG_SET (reg_rename_data.unavailable_hard_regs);
/* Collect unavailable registers from all boundaries into USED_REGS. */
do
@@ -1285,27 +1339,11 @@ find_best_reg_for_rhs (rhs_t rhs, blist_
/* Compute used regs and OR it into the USED_REGS. */
res = find_used_regs (BND_TO (bnd), orig_ops, used_regs,
- &unavailable_hard_regs, &original_insns);
+ ®_rename_data, &original_insns);
+ /* FIXME: the assert is true until we'd have several boundaries. */
+ gcc_assert (res);
av_set_clear (&orig_ops);
-
- if (res)
- {
- if (false)
- /* FIXME: conditionalize dumping of regsets. */
- {
- dump_hard_reg_set ("unavailable hard regs: ",
- unavailable_hard_regs);
- print ("unavailable regs (live issues): ");
- dump_used_regs (used_regs);
- }
- }
- else
- {
- gcc_unreachable ();
- print ("Unable to find original op with find_used_regs.");
- break;
- }
}
while ((bnds1 = BLIST_NEXT (bnds1)));
@@ -1323,19 +1361,29 @@ find_best_reg_for_rhs (rhs_t rhs, blist_
if (EXPR_SEPARABLE_P (rhs))
{
+ /* Check that we have computed availability of a target register
+ correctly. */
+ verify_target_availability (rhs, used_regs,
+ reg_rename_data.unavailable_hard_regs);
+
/* Turn everything in hard regs after reload. */
if (reload_completed)
{
REG_SET_TO_HARD_REG_SET (hard_regs_used, used_regs);
+
/* Join hard registers unavailable due to register class
restrictions and live range intersection. */
- IOR_HARD_REG_SET (hard_regs_used, unavailable_hard_regs);
- best_reg = choose_best_reg (hard_regs_used, original_insns,
+ IOR_HARD_REG_SET (hard_regs_used,
+ reg_rename_data.unavailable_hard_regs);
+
+ best_reg = choose_best_reg (hard_regs_used,
+ ®_rename_data,
+ original_insns,
is_orig_reg_p);
}
else
best_reg = choose_best_pseudo_reg (used_regs,
- unavailable_hard_regs,
+ ®_rename_data,
original_insns,
is_orig_reg_p);
@@ -1371,13 +1419,12 @@ find_best_reg_for_rhs (rhs_t rhs, blist_
if (reg_ok)
{
- RGN_HAS_RENAMING_P (CONTAINING_RGN (BB_TO_BLOCK (0))) = 1;
-
/* Make sure that RHS has the right destination
register. */
- if (rhs_dest_regno (rhs) != REGNO (best_reg))
+ if (expr_dest_regno (rhs) != REGNO (best_reg))
{
replace_dest_with_reg_in_rhs (rhs, best_reg);
+ EXPR_WAS_RENAMED (rhs) = 1;
/* The resulting insn should be valid. */
if (!insn_rtx_valid (RHS_INSN (rhs)))
@@ -1392,12 +1439,17 @@ find_best_reg_for_rhs (rhs_t rhs, blist_
{
/* If !RHS_SCHEDULE_AS_RHS (RHS), just make sure INSN doesn't set
any of the HARD_REGS_USED set. */
- if (vinsn_writes_one_of_regs_p (RHS_VINSN (rhs), used_regs,
- unavailable_hard_regs))
- reg_ok = false;
+ if (vinsn_writes_one_of_regs_p
+ (RHS_VINSN (rhs), used_regs,
+ reg_rename_data.unavailable_hard_regs))
+ {
+ reg_ok = false;
+ gcc_assert (EXPR_TARGET_AVAILABLE (rhs) <= 0);
+ }
else
{
gcc_assert (reg_ok);
+ gcc_assert (EXPR_TARGET_AVAILABLE (rhs) != 0);
best_reg = NULL_RTX;
}
}
@@ -1447,6 +1499,7 @@ can_overcome_dep_p (ds_t ds)
}
static bool speculate_expr (expr_t, ds_t);
+static ds_t get_spec_check_type_for_insn (insn_t, expr_t);
/* Get a speculation check instruction.
C_RHS is a speculative expression,
@@ -1553,6 +1606,13 @@ apply_spec_to_expr (expr_t expr, ds_t ds
change_vinsn_in_expr (expr, spec_vinsn);
EXPR_SPEC_DONE_DS (expr) = ds;
+ /* Do not allow clobbering the address register of speculative
+ insns. */
+ if (res == 1
+ && bitmap_bit_p (VINSN_REG_USES (EXPR_VINSN (expr)),
+ expr_dest_regno (expr)))
+ EXPR_TARGET_AVAILABLE (expr) = false;
+
return true;
}
@@ -1707,7 +1767,7 @@ moveup_rhs_inside_insn_group (rhs_t insn
{
vinsn_t vi = RHS_VINSN (insn_to_move_up);
insn_t insn = VINSN_INSN (vi);
- ds_t *has_dep_p;
+ ds_t *has_dep_p;
ds_t full_ds;
full_ds = has_dependence_p (insn_to_move_up, through_insn, &has_dep_p);
@@ -1827,9 +1887,9 @@ moveup_rhs (rhs_t insn_to_move_up, insn_
}
else
{
+ /* We can move UNIQUE insn up only as a whole and unchanged,
+ so it shouldn't have any dependencies. */
if (VINSN_UNIQUE_P (vi))
- /* We can move UNIQUE insn up only as a whole and unchanged,
- so it shouldn't have any dependencies. */
return MOVEUP_RHS_NULL;
}
@@ -1847,8 +1907,16 @@ moveup_rhs (rhs_t insn_to_move_up, insn_
/* We have some dependency that cannot be discarded. */
return MOVEUP_RHS_NULL;
- if (has_dep_p[DEPS_IN_LHS] && !EXPR_SEPARABLE_P (insn_to_move_up))
- return MOVEUP_RHS_NULL;
+ if (has_dep_p[DEPS_IN_LHS])
+ {
+ /* Only separable insns can be moved up with the new register.
+ Anyways, we should mark that the original register is
+ unavailable. */
+ if (!EXPR_SEPARABLE_P (insn_to_move_up))
+ return MOVEUP_RHS_NULL;
+
+ EXPR_TARGET_AVAILABLE (insn_to_move_up) = false;
+ }
/* At this point we have either separable insns, that will be lifted
up only as RHSes, or non-separable insns with no dependency in lhs.
@@ -1918,7 +1986,6 @@ moveup_rhs (rhs_t insn_to_move_up, insn_
return MOVEUP_RHS_CHANGED;
}
-
/* Moves an av set AVP up through INSN, performing necessary
transformations. */
static void
@@ -1972,28 +2039,10 @@ moveup_set_rhs (av_set_t *avp, insn_t in
/* Mark that this insn changed this expr. */
insert_in_hash_vect (&EXPR_CHANGED_ON_INSNS (rhs),
VINSN_HASH (INSN_VINSN (insn)));
+ rhs = merge_with_other_exprs (avp, &i, rhs);
- {
- rhs_t rhs2 = av_set_lookup_other_equiv_rhs (*avp, RHS_VINSN (rhs));
-
- /* If the resulting insn after substitution is already in av_set,
- remove it. */
- if (rhs2 != NULL)
- {
- EXPR_USEFULNESS (rhs2) = 0;
- merge_expr (rhs2, rhs);
- /* Fix usefulness as it should be now REG_BR_PROB_BASE. */
- EXPR_USEFULNESS (rhs2) = REG_BR_PROB_BASE;
-
- av_set_iter_remove (&i);
- print (" and removed.");
-
- rhs = rhs2;
- }
-
- print (" result: ");
- dump_rhs (rhs);
- }
+ print (" result: ");
+ dump_rhs (rhs);
break;
case MOVEUP_RHS_SAME:
/* Cache that there is a no dependence. */
@@ -2112,8 +2161,8 @@ compute_av_set (insn_t insn, ilist_t p,
{
av_set_t av1;
av_set_t rhs_in_all_succ_branches;
- int succs_n, real_succs_n;
- insn_t *succs;
+ int succs_n, real_succs_n, other_succs_n;
+ insn_t *succs, *other_succs;
int *probs;
int succ, all_prob;
@@ -2164,6 +2213,8 @@ compute_av_set (insn_t insn, ilist_t p,
return NULL;
}
+ /* Find different kind of successors needed for correct computing of
+ SPEC and TARGET_AVAILABLE attributes. */
cfg_succs (insn, &succs, &probs, &succs_n);
/* Sometimes there are weird cases when sum of probabilities of outgoing
@@ -2173,6 +2224,8 @@ compute_av_set (insn_t insn, ilist_t p,
/* If there are successors that lead out of the region, then all rhses from
the below av_sets should be speculative. */
real_succs_n = cfg_succs_n (insn, SUCCS_ALL);
+ cfg_succs_other (insn, SUCCS_NORMAL, succs, succs_n, real_succs_n,
+ &other_succs, &other_succs_n);
/* Debug output. */
line_start ();
@@ -2228,10 +2281,40 @@ compute_av_set (insn_t insn, ilist_t p,
}
}
- /* Union the av_sets. */
- av_set_union_and_clear (&av1, &succ_set);
+ /* Union the av_sets. Check liveness restrictions on target registers
+ in special case of two successors. */
+ if (succs_n == 2 && succ == 1)
+ {
+ basic_block bb0 = BLOCK_FOR_INSN (succs[0]);
+ basic_block bb1 = BLOCK_FOR_INSN (succs[1]);
+
+ gcc_assert (BB_LV_SET_VALID_P (bb0) && BB_LV_SET_VALID_P (bb1));
+ av_set_union_and_live (&av1, &succ_set,
+ BB_LV_SET (bb0),
+ BB_LV_SET (bb1));
+ }
+ else
+ av_set_union_and_clear (&av1, &succ_set);
}
+ /* Check liveness restrictions via hard way when there are more than
+ two successors. */
+ if (succs_n > 2)
+ for (succ = 0; succ < succs_n; succ++)
+ {
+ basic_block succ_bb = BLOCK_FOR_INSN (succs[succ]);
+
+ gcc_assert (BB_LV_SET_VALID_P (succ_bb));
+ mark_unavailable_targets (av1, BB_AV_SET (succ_bb),
+ BB_LV_SET (succ_bb));
+ }
+
+ /* Finally, check liveness restrictions on paths leaving the region. */
+ if (other_succs_n > 0)
+ for (succ = 0; succ < other_succs_n; succ++)
+ mark_unavailable_targets
+ (av1, NULL, BB_LV_SET (BLOCK_FOR_INSN (other_succs[succ])));
+
if (real_succs_n > 1)
{
av_set_iterator i;
@@ -2294,7 +2377,9 @@ compute_av_set (insn_t insn, ilist_t p,
av_set_clear (&BB_AV_SET (bb));
print ("Save av(%d) in bb header", INSN_UID (insn));
-
+#if 0
+ mark_unavailable_targets (av1, BB_LV_SET (bb));
+#endif
BB_AV_SET (bb) = unique_p ? av_set_copy (av1) : av1;
BB_AV_LEVEL (bb) = global_level;
}
@@ -2324,8 +2409,11 @@ propagate_lv_set (regset lv, insn_t insn
/* LV1 = LV1 \ { DEST (insn) } */
if (GET_CODE (PATTERN (insn)) != COND_EXEC)
- /* May-defs should not kill other sets. */
- AND_COMPL_REG_SET (lv, INSN_REG_SETS (insn));
+ {
+ /* May-defs should not kill other sets. */
+ AND_COMPL_REG_SET (lv, INSN_REG_SETS (insn));
+ AND_COMPL_REG_SET (lv, INSN_REG_CLOBBERS (insn));
+ }
/* LV1 = LV1 U { SOURCES (insn) } */
/* FIXME: Should we consider the whole register clobbered in the cases of
@@ -2354,7 +2442,7 @@ compute_live_after_bb (basic_block bb)
/* Compute the set of all live registers at the point before INSN and save
it at INSN if INSN is bb header. */
-static regset
+regset
compute_live (insn_t insn)
{
if (sel_bb_head_p (insn) && !ignore_first)
@@ -2495,25 +2583,22 @@ get_spec_check_type_for_insn (insn_t ins
All the original operations found during the traversal are saved in the
ORIGINAL_INSNS list.
- UNAVAILABLE_HARD_REGS denotes the set of hardware registers that
+ REG_RENAME_P denotes the set of hardware registers that
can not be used with renaming due to the register class restrictions,
mode restrictions and other (the register we'll choose should be
compatible class with the original uses, shouldn't be in call_used_regs,
should be HARD_REGNO_RENAME_OK etc).
- This parameter can have NULL value. It should happen if we compute
- used registers before reload, and thus don't have information about hard
- registers.
CROSSES_CALL is true, if there is a call insn on the path from INSN to
- original insn. In this case CALL_USED_REG_SET will be added to the
- UNAVAILABLE_HARD_REGS at the point original operation is found.
+ original insn. In this case CALL_USED_REG_SET will be added to
+ unavailable hard regs at the point original operation is found.
Actually we need a complement set to the one computed by this routine,
but it's more natural to have the inverted set. */
static int
find_used_regs_1 (insn_t insn, av_set_t orig_ops, ilist_t path,
- regset used_regs, HARD_REG_SET *unavailable_hard_regs,
+ regset used_regs, struct reg_rename *reg_rename_p,
bool crosses_call, def_list_t *original_insns)
{
rhs_t rhs;
@@ -2610,6 +2695,7 @@ find_used_regs_1 (insn_t insn, av_set_t
compute_live_below_insn (insn, tmp);
AND_COMPL_REG_SET (tmp, INSN_REG_SETS (insn));
+ AND_COMPL_REG_SET (tmp, INSN_REG_CLOBBERS (insn));
IOR_REG_SET (used_regs, tmp);
return_regset_to_pool (tmp);
@@ -2670,7 +2756,7 @@ find_used_regs_1 (insn_t insn, av_set_t
}
b = find_used_regs_1 (succ, orig_ops, path,
- used_regs, unavailable_hard_regs, crosses_call,
+ used_regs, reg_rename_p, crosses_call,
original_insns);
if (b == 0)
@@ -2780,7 +2866,7 @@ find_used_regs_1 (insn_t insn, av_set_t
/* Find the set of registers that are unavailable for storing rhses
while moving ORIG_OPS up on the path starting from INSN due to
- liveness (USED_REGS) or hardware restrictions (UNAVAILABLE_HARD_REGS).
+ liveness (USED_REGS) or hardware restrictions (REG_RENAME_P).
All the original operations found during the traversal are saved in the
ORIGINAL_INSNS list.
@@ -2791,7 +2877,7 @@ find_used_regs_1 (insn_t insn, av_set_t
static bool
find_used_regs (insn_t insn, av_set_t orig_ops, regset used_regs,
- HARD_REG_SET *unavailable_hard_regs,
+ struct reg_rename *reg_rename_p,
def_list_t *original_insns)
{
def_list_iterator i;
@@ -2799,7 +2885,7 @@ find_used_regs (insn_t insn, av_set_t or
int res;
res = find_used_regs_1 (insn, orig_ops, NULL, used_regs,
- unavailable_hard_regs, false, original_insns);
+ reg_rename_p, false, original_insns);
if (res == -1)
return false;
@@ -2807,17 +2893,15 @@ find_used_regs (insn_t insn, av_set_t or
gcc_assert (res == 1);
gcc_assert (original_insns && *original_insns);
- /* Mark hardware regs in UNAVAILABLE_HARD_REGS that are not suitable
+ /* Mark hardware regs in REG_RENAME_P that are not suitable
for renaming rhs in INSN due to hardware restrictions (register class,
- modes compatibility etc). This doesn't affect original insn's dest regs.
- Registers that are in used_regs are always marked also in
- unavailable_hard_regs. */
+ modes compatibility etc). */
FOR_EACH_DEF (def, i, *original_insns)
{
vinsn_t vinsn = INSN_VINSN (def->orig_insn);
if (VINSN_SEPARABLE_P (vinsn))
- mark_unavailable_hard_regs (def, unavailable_hard_regs, used_regs);
+ mark_unavailable_hard_regs (def, reg_rename_p, used_regs);
if (def->needs_spec_check_p)
/* Do not allow clobbering of ld.[sa] address. */
@@ -3136,6 +3220,7 @@ fill_vec_av_set (av_set_t av, blist_t bn
av_set_iterator si;
rhs_t rhs;
int sched_next_worked = 0, stalled, n;
+ int succ, fail, sepsucc, sepfail;
deps_t dc = BND_DC (BLIST_BND (bnds));
bool av_contain_end_of_loop_p;
@@ -3156,16 +3241,52 @@ fill_vec_av_set (av_set_t av, blist_t bn
FOR_EACH_RHS (rhs, si, av)
VEC_safe_push (rhs_t, heap, vec_av_set, rhs);
+ if (sched_verbose >= 1)
+ {
+ int avail, unavail, dont_know, savail, sunavail, sdont_know;
+
+ avail = unavail = dont_know = 0;
+ savail = sunavail = sdont_know = 0;
+ for (n = 0; VEC_iterate (rhs_t, vec_av_set, n, rhs); n++)
+ if (EXPR_SEPARABLE_P (rhs))
+ switch (EXPR_TARGET_AVAILABLE (rhs))
+ {
+ case 0: unavail++; break;
+ case 1: avail++; break;
+ case -1: dont_know++; break;
+
+ default: gcc_unreachable ();
+ }
+ else
+ switch (EXPR_TARGET_AVAILABLE (rhs))
+ {
+ case 0: sunavail++; break;
+ case 1: savail++; break;
+ case -1: sdont_know++; break;
+
+ default: gcc_unreachable ();
+ }
+
+ print ("target-stats: %d %d %d %d %d %d\n", avail, unavail,
+ dont_know, savail, sunavail, sdont_know);
+ }
+
+ /* Sort the vector. */
+ qsort (VEC_address (rhs_t, vec_av_set), VEC_length (rhs_t, vec_av_set),
+ sizeof (rhs_t), sel_rank_for_schedule);
+
/* Filter out inappropriate expressions. */
+ succ = fail = sepsucc = sepfail = 0;
for (n = 0, stalled = 0; VEC_iterate (rhs_t, vec_av_set, n, rhs); )
{
insn_t insn = EXPR_INSN_RTX (rhs);
+ char target_available;
bool is_orig_reg_p = true;
/* Don't allow any insns other than from SCHED_GROUP if we have one. */
if (FENCE_SCHED_NEXT (fence) && insn != FENCE_SCHED_NEXT (fence))
{
- VEC_unordered_remove (rhs_t, vec_av_set, n);
+ VEC_ordered_remove (rhs_t, vec_av_set, n);
continue;
}
@@ -3175,15 +3296,39 @@ fill_vec_av_set (av_set_t av, blist_t bn
sched_next_worked++;
dump_insn_1 (insn, DUMP_INSN_UID);
-
+
/* Check all liveness requirements and try renaming.
FIXME: try to minimize calls to this. */
- if (! find_best_reg_for_rhs (rhs, bnds, &is_orig_reg_p))
+ target_available = EXPR_TARGET_AVAILABLE (rhs);
+
+ if (target_available == true)
+ /* Do nothing -- we can use an existing register. */
+ succ++;
+ else if (/* Non-separable instruction will never
+ get another register. */
+ (target_available == false
+ && !EXPR_SEPARABLE_P (rhs))
+ /* Don't try to find a register for low-priority expression. */
+ || (max_insns_to_rename >= 0
+ && n > max_insns_to_rename))
{
- VEC_unordered_remove (rhs_t, vec_av_set, n);
- print ("- no reg; ");
+ succ++;
+ VEC_ordered_remove (rhs_t, vec_av_set, n);
+ print ("- no register; ");
continue;
}
+ else
+ {
+ fail++;
+ if (/* Either we don't know the answer, or the register is
+ not available in separable RHS. Do it the hard way. */
+ ! find_best_reg_for_rhs (rhs, bnds, &is_orig_reg_p))
+ {
+ VEC_ordered_remove (rhs_t, vec_av_set, n);
+ print ("- no register; ");
+ continue;
+ }
+ }
if (av_contain_end_of_loop_p
&& check_stalling_p (rhs, is_orig_reg_p))
@@ -3199,7 +3344,7 @@ fill_vec_av_set (av_set_t av, blist_t bn
/* We need to know whether we do need to stall for any insns. */
stalled++;
print ("- not ready yet; ");
- VEC_unordered_remove (rhs_t, vec_av_set, n);
+ VEC_ordered_remove (rhs_t, vec_av_set, n);
continue;
}
@@ -3207,6 +3352,9 @@ fill_vec_av_set (av_set_t av, blist_t bn
n++;
}
+ if (sched_verbose >= 1)
+ print ("successes: %d %d\n", succ, fail);
+
if (VEC_empty (rhs_t, vec_av_set))
{
/* It seems possible that when no insns are ready, this could be
@@ -3223,10 +3371,6 @@ fill_vec_av_set (av_set_t av, blist_t bn
return false;
}
- /* Sort the vector. */
- qsort (VEC_address (rhs_t, vec_av_set), VEC_length (rhs_t, vec_av_set),
- sizeof (rhs_t), sel_rank_for_schedule);
-
line_start ();
print ("Sorted av set (%d): ", VEC_length (rhs_t, vec_av_set));
@@ -3236,7 +3380,7 @@ fill_vec_av_set (av_set_t av, blist_t bn
print ("\nreally_ready: %d stalled: %d \n",
VEC_length (rhs_t, vec_av_set), stalled);
-
+
/* Clear SCHED_NEXT. */
if (FENCE_SCHED_NEXT (fence))
{
@@ -3624,14 +3768,21 @@ find_best_expr (av_set_t *av_vliw_ptr, b
privileged_n = calculate_privileged_insns ();
can_issue = choose_best_insn (fence, privileged_n, &index);
if (can_issue)
- best = find_expr_for_ready (index, true);
+ {
+ best = find_expr_for_ready (index, true);
+ if (EXPR_WAS_RENAMED (best))
+ {
+ print ("renamed: %d %d\n", index, ready.n_ready);
+ EXPR_WAS_RENAMED (best) = 0;
+ }
+ }
else
need_stall = 1;
}
if (best != NULL)
- can_issue_more = invoke_aftermath_hooks (fence, EXPR_INSN_RTX (best),
- can_issue_more);
+ can_issue_more = invoke_aftermath_hooks (fence, EXPR_INSN_RTX (best),
+ can_issue_more);
return best;
}
@@ -3881,7 +4032,6 @@ generate_bookkeeping_insn (rhs_t c_rhs,
new_vinsn = create_vinsn_from_insn_rtx (new_insn_rtx);
copy_expr_onside (new_expr, c_rhs);
change_vinsn_in_expr (new_expr, new_vinsn);
-
new_insn = gen_insn_from_expr_after (new_expr, new_seqno, place_to_insert);
/* When inserting bookkeeping insn in new block, av sets should be
@@ -3949,8 +4099,8 @@ remove_insns_that_need_bookkeeping (fenc
&& (EXPR_SPEC (rhs)
|| !EXPR_ORIG_BB_INDEX (rhs)
|| !dominated_by_p (CDI_DOMINATORS,
- BLOCK_FOR_INSN (FENCE_INSN (fence)),
- BASIC_BLOCK (EXPR_ORIG_BB_INDEX (rhs)))))
+ BASIC_BLOCK (EXPR_ORIG_BB_INDEX (rhs)),
+ BLOCK_FOR_INSN (FENCE_INSN (fence)))))
{
line_start ();
print ("Removed expr that would need bookkeeping (but disabled or"
@@ -4131,12 +4281,14 @@ fill_insns (fence_t fence, int seqno, il
/* Assert that this can only happen with unscheduled code. */
gcc_assert (INSN_SCHED_TIMES (new_bnd_to) == 0);
}
-
BND_TO (bnd) = new_bnd_to;
- gcc_assert (NOTE_INSN_BASIC_BLOCK_P (PREV_INSN (new_bnd_to)));
av_set_clear (&BND_AV (bnd));
BND_AV (bnd) = compute_av_set (BND_TO (bnd), NULL, 0, true);
+#if 0
+ mark_unavailable_targets
+ (BND_AV (bnd), BB_LV_SET (BLOCK_FOR_INSN (BND_TO (bnd))));
+#endif
av_set_clear (&BND_AV1 (bnd));
BND_AV1 (bnd) = av_set_copy (BND_AV (bnd));
@@ -4150,7 +4302,7 @@ fill_insns (fence_t fence, int seqno, il
while ((bnds1 = BLIST_NEXT (bnds1)));
remove_insns_that_need_bookkeeping (fence, &av_vliw);
-
+
sel_dump_cfg ("after-compute_av");
/* If debug parameters tell us to ignore this attempt to move an insn,
@@ -4259,7 +4411,7 @@ fill_insns (fence_t fence, int seqno, il
to move_op except when renaming happened. Put the
correct register in RHS then. */
if (EXPR_SEPARABLE_P (rhs) && REG_P (EXPR_LHS (rhs))
- && rhs_dest_regno (rhs) != rhs_dest_regno (rhs_vliw))
+ && expr_dest_regno (rhs) != expr_dest_regno (rhs_vliw))
{
replace_dest_with_reg_in_rhs (rhs, EXPR_LHS (rhs_vliw));
stat_renamed_scheduled++;
@@ -4425,7 +4577,9 @@ fill_insns (fence_t fence, int seqno, il
clear_expr (c_rhs);
++INSN_SCHED_TIMES (insn);
+ EXPR_TARGET_AVAILABLE (INSN_EXPR (insn)) = true;
EXPR_ORIG_SCHED_CYCLE (INSN_EXPR (insn)) = fence->cycle;
+
if (INSN_NOP_P (place_to_insert))
/* Return the nop generated for preserving of data sets back
into pool. */
@@ -4889,7 +5043,8 @@ move_op (insn_t insn, av_set_t orig_ops,
int use = MAX (EXPR_USEFULNESS (c_rhs), EXPR_USEFULNESS (x));
gcc_assert (EXPR_USEFULNESS (c_rhs) == EXPR_USEFULNESS (x));
- merge_expr_data (c_rhs, x);
+ merge_expr_data (c_rhs, x, true);
+
EXPR_USEFULNESS (c_rhs) = use;
if (EXPR_SCHED_TIMES (x) == 0)
EXPR_SCHED_TIMES (c_rhs) = old_times;
@@ -5109,8 +5264,6 @@ add_region_head (void)
new_region_head = sel_create_basic_block_before (region_head);
can_add_real_insns_p = true;
- RGN_WAS_PIPELINED_P (CONTAINING_RGN (BB_TO_BLOCK (0))) = 1;
-
return true;
}
}
@@ -6000,6 +6153,7 @@ sel_global_init (void)
FIXME: move this in opts.c. */
if (!flag_sel_sched_pipelining)
flag_sel_sched_pipelining_outer_loops = 0;
+ max_insns_to_rename = PARAM_VALUE (PARAM_SELSCHED_INSNS_TO_RENAME);
calculate_dominance_info (CDI_DOMINATORS);
Index: gcc/sel-sched-ir.c
===================================================================
--- gcc/sel-sched-ir.c (revision 131182)
+++ gcc/sel-sched-ir.c (working copy)
@@ -1407,6 +1407,7 @@ vinsn_delete (vinsn_t vi)
return_regset_to_pool (VINSN_REG_SETS (vi));
return_regset_to_pool (VINSN_REG_USES (vi));
+ return_regset_to_pool (VINSN_REG_CLOBBERS (vi));
free (VINSN_ID (vi));
@@ -1662,7 +1663,8 @@ static void
init_expr (expr_t expr, vinsn_t vi, int spec, int use, int priority,
int sched_times, int orig_bb_index, ds_t spec_done_ds,
ds_t spec_to_check_ds, int orig_sched_cycle,
- VEC(unsigned, heap) *changed_on, bool was_substituted)
+ VEC(unsigned, heap) *changed_on, bool target_available,
+ bool was_substituted, bool was_renamed)
{
vinsn_attach (vi);
@@ -1681,7 +1683,9 @@ init_expr (expr_t expr, vinsn_t vi, int
else
EXPR_CHANGED_ON_INSNS (expr) = NULL;
+ EXPR_TARGET_AVAILABLE (expr) = target_available;
EXPR_WAS_SUBSTITUTED (expr) = was_substituted;
+ EXPR_WAS_RENAMED (expr) = was_renamed;
}
/* Make a copy of the rhs FROM into the rhs TO. */
@@ -1691,11 +1695,13 @@ copy_expr (expr_t to, expr_t from)
VEC(unsigned, heap) *temp;
temp = VEC_copy (unsigned, heap, EXPR_CHANGED_ON_INSNS (from));
- init_expr (to, EXPR_VINSN (from), EXPR_SPEC (from), EXPR_USEFULNESS (from),
- EXPR_PRIORITY (from), EXPR_SCHED_TIMES (from),
- EXPR_ORIG_BB_INDEX (from), EXPR_SPEC_DONE_DS (from),
- EXPR_SPEC_TO_CHECK_DS (from), EXPR_ORIG_SCHED_CYCLE (from),
- temp, EXPR_WAS_SUBSTITUTED (from));
+ init_expr (to, EXPR_VINSN (from), EXPR_SPEC (from),
+ EXPR_USEFULNESS (from), EXPR_PRIORITY (from),
+ EXPR_SCHED_TIMES (from), EXPR_ORIG_BB_INDEX (from),
+ EXPR_SPEC_DONE_DS (from), EXPR_SPEC_TO_CHECK_DS (from),
+ EXPR_ORIG_SCHED_CYCLE (from), temp,
+ EXPR_TARGET_AVAILABLE (from), EXPR_WAS_SUBSTITUTED (from),
+ EXPR_WAS_RENAMED (from));
}
/* Same, but the final expr will not ever be in av sets, so don't copy
@@ -1706,12 +1712,14 @@ copy_expr_onside (expr_t to, expr_t from
init_expr (to, EXPR_VINSN (from), EXPR_SPEC (from), EXPR_USEFULNESS (from),
EXPR_PRIORITY (from), EXPR_SCHED_TIMES (from), 0,
EXPR_SPEC_DONE_DS (from), EXPR_SPEC_TO_CHECK_DS (from), 0, NULL,
- EXPR_WAS_SUBSTITUTED (from));
+ EXPR_TARGET_AVAILABLE (from), EXPR_WAS_SUBSTITUTED (from),
+ EXPR_WAS_RENAMED (from));
}
-/* Merge bits of FROM rhs to TO rhs. */
+/* Merge bits of FROM rhs to TO rhs. When JOIN_POINT_P is true,
+ this is done along different paths. */
void
-merge_expr_data (expr_t to, expr_t from)
+merge_expr_data (expr_t to, expr_t from, bool join_point_p)
{
int i;
unsigned hash;
@@ -1729,6 +1737,46 @@ merge_expr_data (expr_t to, expr_t from)
if (RHS_SCHED_TIMES (to) > RHS_SCHED_TIMES (from))
RHS_SCHED_TIMES (to) = RHS_SCHED_TIMES (from);
+ if (EXPR_TARGET_AVAILABLE (to) < 0
+ || EXPR_TARGET_AVAILABLE (from) < 0)
+ EXPR_TARGET_AVAILABLE (to) = -1;
+ else
+ {
+ /* We try to detect the case when one of the expressions
+ can only be reached through another one. In this case,
+ we can do better. */
+ if (!join_point_p)
+ {
+ int toind, fromind;
+
+ toind = EXPR_ORIG_BB_INDEX (to);
+ fromind = EXPR_ORIG_BB_INDEX (from);
+
+ if (toind && toind == fromind)
+ /* Do nothing -- everything is done in
+ merge_with_other_exprs. */
+ ;
+#if 0
+ else if (dominated_by_p (CDI_DOMINATORS,
+ BASIC_BLOCK (toind),
+ BASIC_BLOCK (fromind)))
+ {
+ change_vinsn_in_expr (to, EXPR_VINSN (from));
+ EXPR_TARGET_AVAILABLE (to) = EXPR_TARGET_AVAILABLE (from);
+ }
+ else if (dominated_by_p (CDI_DOMINATORS,
+ BASIC_BLOCK (fromind),
+ BASIC_BLOCK (toind)))
+ /* Do nothing. */
+ ;
+#endif
+ else
+ EXPR_TARGET_AVAILABLE (to) = -1;
+ }
+ else
+ EXPR_TARGET_AVAILABLE (to) &= EXPR_TARGET_AVAILABLE (from);
+ }
+
if (EXPR_ORIG_BB_INDEX (to) != EXPR_ORIG_BB_INDEX (from))
EXPR_ORIG_BB_INDEX (to) = 0;
@@ -1747,11 +1795,12 @@ merge_expr_data (expr_t to, expr_t from)
insert_in_hash_vect (&EXPR_CHANGED_ON_INSNS (to), hash);
EXPR_WAS_SUBSTITUTED (to) |= EXPR_WAS_SUBSTITUTED (from);
+ EXPR_WAS_RENAMED (to) |= EXPR_WAS_RENAMED (to);
}
/* Merge bits of FROM rhs to TO rhs. Vinsns in the rhses should correlate. */
void
-merge_expr (expr_t to, expr_t from)
+merge_expr (expr_t to, expr_t from, bool join_point_p)
{
vinsn_t to_vi = EXPR_VINSN (to);
vinsn_t from_vi = EXPR_VINSN (from);
@@ -1762,10 +1811,11 @@ merge_expr (expr_t to, expr_t from)
/* Make sure that speculative pattern is propagated into exprs that
have non-speculative one. This will provide us with consistent
speculative bits and speculative patterns inside expr. */
- if (EXPR_SPEC_DONE_DS (to) == 0)
+ if (EXPR_SPEC_DONE_DS (to) == 0
+ && EXPR_SPEC_DONE_DS (from) != 0)
change_vinsn_in_expr (to, EXPR_VINSN (from));
- merge_expr_data (to, from);
+ merge_expr_data (to, from, join_point_p);
gcc_assert (EXPR_USEFULNESS (to) <= REG_BR_PROB_BASE);
}
@@ -1777,6 +1827,75 @@ clear_expr (rhs_t rhs)
RHS_VINSN (rhs) = NULL;
VEC_free (unsigned, heap, EXPR_CHANGED_ON_INSNS (rhs));
}
+
+/* For a given LV_SET, mark EXPR having unavailable target register. */
+static void
+set_unavailable_target_for_expr (expr_t expr, regset lv_set)
+{
+ if (EXPR_SEPARABLE_P (expr))
+ {
+ if (REG_P (EXPR_LHS (expr))
+ && bitmap_bit_p (lv_set, REGNO (EXPR_LHS (expr))))
+ EXPR_TARGET_AVAILABLE (expr) = false;
+ }
+ else
+ {
+ unsigned regno;
+ reg_set_iterator rsi;
+
+ EXECUTE_IF_SET_IN_REG_SET (VINSN_REG_SETS (EXPR_VINSN (expr)),
+ 0, regno, rsi)
+ if (bitmap_bit_p (lv_set, regno))
+ {
+ EXPR_TARGET_AVAILABLE (expr) = false;
+ break;
+ }
+
+ EXECUTE_IF_SET_IN_REG_SET (VINSN_REG_CLOBBERS (EXPR_VINSN (expr)),
+ 0, regno, rsi)
+ if (bitmap_bit_p (lv_set, regno))
+ {
+ EXPR_TARGET_AVAILABLE (expr) = false;
+ break;
+ }
+
+ }
+}
+
+/* Return a destination register, if any, of EXPR. */
+rtx
+expr_dest_reg (expr_t expr)
+{
+ rtx dest = VINSN_LHS (RHS_VINSN (expr));
+
+ if (dest != NULL_RTX && REG_P (dest))
+ return dest;
+
+ return NULL_RTX;
+}
+
+/* Returns the REGNO of the R's destination. */
+unsigned
+expr_dest_regno (expr_t expr)
+{
+ rtx dest = expr_dest_reg (expr);
+
+ gcc_assert (dest != NULL_RTX);
+ return REGNO (dest);
+}
+
+/* For a given LV_SET, mark all expressions in JOIN_SET, but not present in
+ AV_SET having unavailable target register. */
+void
+mark_unavailable_targets (av_set_t join_set, av_set_t av_set, regset lv_set)
+{
+ expr_t expr;
+ av_set_iterator avi;
+
+ FOR_EACH_RHS (expr, avi, join_set)
+ if (av_set_lookup (av_set, EXPR_VINSN (expr)) == NULL)
+ set_unavailable_target_for_expr (expr, lv_set);
+}
/* Av set functions. */
@@ -1789,6 +1908,14 @@ av_set_add (av_set_t *setp, expr_t expr)
copy_expr (_AV_SET_EXPR (*setp), expr);
}
+/* Same, but do not copy EXPR. */
+static void
+av_set_add_nocopy (av_set_t *setp, expr_t expr)
+{
+ _list_add (setp);
+ *_AV_SET_EXPR (*setp) = *expr;
+}
+
/* Remove expr pointed to by IP from the av_set. */
void
av_set_iter_remove (av_set_iterator *ip)
@@ -1818,29 +1945,96 @@ av_set_lookup (av_set_t set, vinsn_t sou
return NULL;
}
+/* Same, but also remove the RHS found. */
+static rhs_t
+av_set_lookup_and_remove (av_set_t *setp, vinsn_t sought_vinsn)
+{
+ rhs_t rhs;
+ av_set_iterator i;
+
+ FOR_EACH_RHS_1 (rhs, i, setp)
+ {
+ vinsn_t rhs_vinsn = RHS_VINSN (rhs);
+
+ if (rhs_vinsn == sought_vinsn
+ || vinsns_correlate_as_rhses_p (rhs_vinsn, sought_vinsn))
+ {
+ _list_iter_remove_nofree (&i);
+ return rhs;
+ }
+ }
+
+ return NULL;
+}
+
/* Search for an rhs in SET, such that it's equivalent to SOUGHT_VINSN in the
sense of vinsns_correlate_as_rhses_p function, but not SOUGHT_VINSN itself.
- Returns NULL if no such rhs is in SET was found. */
-rhs_t
-av_set_lookup_other_equiv_rhs (av_set_t set, vinsn_t sought_vinsn)
+ Returns NULL if no such rhs is in SET was found. Store in LATERP true
+ when other expression was found later than this, and false otherwise. */
+static rhs_t
+av_set_lookup_other_equiv_rhs (av_set_t set, vinsn_t sought_vinsn,
+ bool *laterp)
{
rhs_t rhs;
av_set_iterator i;
+ bool temp = false;
FOR_EACH_RHS (rhs, i, set)
{
vinsn_t rhs_vinsn = RHS_VINSN (rhs);
if (rhs_vinsn == sought_vinsn)
- continue;
+ {
+ temp = true;
+ continue;
+ }
if (vinsns_correlate_as_rhses_p (rhs_vinsn, sought_vinsn))
- return rhs;
+ {
+ *laterp = temp;
+ return rhs;
+ }
}
return NULL;
}
+/* If other expression is already in AVP, remove one of them. */
+expr_t
+merge_with_other_exprs (av_set_t *avp, av_set_iterator *ip, expr_t expr)
+{
+ bool later;
+ expr_t expr2;
+
+ expr2 = av_set_lookup_other_equiv_rhs (*avp, EXPR_VINSN (expr), &later);
+
+ if (expr2 != NULL)
+ {
+ /* Prefer the expression that comes earlier, as it will be the one
+ we will find. */
+ if (later && EXPR_ORIG_BB_INDEX (expr)
+ && EXPR_ORIG_BB_INDEX (expr) == EXPR_ORIG_BB_INDEX (expr2))
+ {
+ change_vinsn_in_expr (expr2, EXPR_VINSN (expr));
+ EXPR_TARGET_AVAILABLE (expr2) = EXPR_TARGET_AVAILABLE (expr);
+ }
+
+ EXPR_USEFULNESS (expr2) = 0;
+
+ merge_expr (expr2, expr, false);
+
+ /* Fix usefulness as it should be now REG_BR_PROB_BASE. */
+ EXPR_USEFULNESS (expr2) = REG_BR_PROB_BASE;
+
+ av_set_iter_remove (ip);
+ print (" and removed.");
+
+ return expr2;
+ }
+
+ return expr;
+}
+
/* Return true if there is an expr that correlates to VI in SET. */
bool
av_set_is_in_p (av_set_t set, vinsn_t vi)
@@ -1877,13 +2071,74 @@ av_set_union_and_clear (av_set_t *top, a
if (rhs2)
{
- merge_expr (rhs2, rhs1);
+ merge_expr (rhs2, rhs1, true);
av_set_iter_remove (&i);
}
}
/* Connect FROMP to the end of the TOP. */
*i.lp = *fromp;
+ *fromp = NULL;
+}
+
+/* Same as above, but also update availability of target register in
+ TOP judging by TO_LV_SET and FROM_LV_SET. */
+void
+av_set_union_and_live (av_set_t *top, av_set_t *fromp, regset to_lv_set,
+ regset from_lv_set)
+{
+ rhs_t rhs1;
+ av_set_iterator i;
+ _list_t *oldlp;
+ av_set_t in_both_set = NULL;
+
+ /* Delete from TOP all rhses, that present in FROMP. */
+ FOR_EACH_RHS_1 (rhs1, i, top)
+ {
+ rhs_t rhs2 = av_set_lookup_and_remove (fromp, RHS_VINSN (rhs1));
+
+ if (rhs2)
+ {
+ /* It may be that the expressions have different destination
+ registers, in which case we need to check liveness here. */
+ if (EXPR_SEPARABLE_P (rhs1))
+ {
+ int regno1 = (REG_P (EXPR_LHS (rhs1))
+ ? (int) expr_dest_regno (rhs1) : -1);
+ int regno2 = (REG_P (EXPR_LHS (rhs2))
+ ? (int) expr_dest_regno (rhs2) : -1);
+
+ /* ??? We don't have a way to check restrictions for
+ *other* register on the current path, we did it only
+ for the current target register. Give up. */
+ if (regno1 != regno2)
+ EXPR_TARGET_AVAILABLE (rhs2) = -1;
+ }
+ else if (EXPR_INSN_RTX (rhs1) != EXPR_INSN_RTX (rhs2))
+ EXPR_TARGET_AVAILABLE (rhs2) = -1;
+
+ merge_expr (rhs2, rhs1, true);
+ av_set_add_nocopy (&in_both_set, rhs2);
+ av_set_iter_remove (&i);
+ }
+ else
+ /* RHS1 is present in TOP, but not in FROMP. Check it on
+ FROM_LV_SET. */
+ set_unavailable_target_for_expr (rhs1, from_lv_set);
+ }
+
+ /* Save the old pointer to the end of the list. */
+ oldlp = i.lp;
+
+ /* These expressions are not present in TOP. Check liveness
+ restrictions on TO_LV_SET. */
+ FOR_EACH_RHS (rhs1, i, *fromp)
+ set_unavailable_target_for_expr (rhs1, to_lv_set);
+
+ /* Connect FROMP and in_both_set to the end of the TOP. */
+ *i.lp = in_both_set;
+ *oldlp = *fromp;
+
*fromp = NULL;
}
@@ -2007,6 +2262,7 @@ deps_init_id_start_insn (insn_t insn)
IDATA_REG_SETS (id) = get_clear_regset_from_pool ();
IDATA_REG_USES (id) = get_clear_regset_from_pool ();
+ IDATA_REG_CLOBBERS (id) = get_clear_regset_from_pool ();
deps_init_id_data.where = DEPS_IN_INSN;
}
@@ -2070,7 +2326,7 @@ deps_init_id_note_reg_clobber (int regno
deps_init_id_downgrade_to_use ();
if (IDATA_TYPE (deps_init_id_data.id) != PC)
- SET_REGNO_REG_SET (IDATA_REG_SETS (deps_init_id_data.id), regno);
+ SET_REGNO_REG_SET (IDATA_REG_CLOBBERS (deps_init_id_data.id), regno);
}
/* Note a use of REGNO. */
@@ -2307,7 +2563,7 @@ init_global_and_expr_for_insn (insn_t in
/* Initialize INSN's expr. */
init_expr (INSN_EXPR (insn), vinsn_create (insn, force_unique_p), 0,
REG_BR_PROB_BASE, INSN_PRIORITY (insn), 0, BLOCK_NUM (insn),
- spec_done_ds, 0, 0, NULL, false);
+ spec_done_ds, 0, 0, NULL, true, false, false);
}
init_first_time_insn_data (insn);
@@ -3261,7 +3517,7 @@ static void
init_simplejump (insn_t insn)
{
init_expr (INSN_EXPR (insn), vinsn_create (insn, false), 0,
- REG_BR_PROB_BASE, 0, 0, 0, 0, 0, 0, NULL, false);
+ REG_BR_PROB_BASE, 0, 0, 0, 0, 0, 0, NULL, true, false, false);
INSN_SEQNO (insn) = get_seqno_of_a_pred (insn);
@@ -3816,6 +4072,56 @@ cfg_succs (insn_t insn, insn_t **succsp,
cfg_succs_2 (insn, SUCCS_NORMAL, succsp, probs, np);
}
+/* Return the successors of INSN in SUCCSP and their number in NP,
+ which do NOT satisfy FLAGS. Empty blocks are skipped.
+ If N_ALL is non-negative, assume it is the number of total
+ successors. If N_OK is not negative, assume successors
+ satisfying flags are also precomputed. */
+void
+cfg_succs_other (insn_t insn, int flags,
+ insn_t *succsp_ok, int n_ok, int n_all,
+ insn_t **succsp, int *np)
+{
+ int i, no;
+ succ_iterator si;
+ insn_t succ;
+ bool free_ok = false;
+
+ if (n_ok < 0)
+ {
+ cfg_succs_1 (insn, flags, &succsp_ok, &n_ok);
+ free_ok = true;
+ }
+
+ if (n_all < 0)
+ n_all = cfg_succs_n (insn, SUCCS_ALL);
+
+ if (n_all == n_ok)
+ {
+ *np = 0;
+ if (free_ok)
+ free (succsp_ok);
+ return;
+ }
+
+ no = *np = n_all - n_ok;
+ *succsp = xmalloc (no * sizeof (**succsp));
+
+ FOR_EACH_SUCC_1 (succ, si, insn, SUCCS_ALL)
+ {
+ for (i = 0; i < n_ok; i++)
+ if (succsp_ok[i] == succ)
+ break;
+
+ if (i == n_ok)
+ (*succsp)[--no] = succ;
+ }
+
+ gcc_assert (no == 0);
+ if (free_ok)
+ free (succsp_ok);
+}
+
/* Return the only successor of INSN, honoring FLAGS. */
insn_t
cfg_succ_1 (insn_t insn, int flags)
@@ -4329,7 +4635,7 @@ sel_add_or_remove_bb (basic_block bb, in
{
if (add > 0)
{
- /* Extend luids so that new notes will recieve zero luids. */
+ /* Extend luids so that new notes will receive zero luids. */
sched_init_luids (NULL, NULL, NULL, NULL);
sched_init_bbs (last_added_blocks, NULL);
sel_init_bbs (last_added_blocks, NULL);
@@ -4707,6 +5013,7 @@ basic_block
sel_split_edge (edge e)
{
basic_block new_bb;
+ basic_block other_bb = NULL;
insn_init.what = INSN_INIT_WHAT_INSN;
@@ -4723,7 +5030,12 @@ sel_split_edge (edge e)
for (i = 0;
VEC_iterate (basic_block, last_added_blocks, i, bb); i++)
if (!bb->loop_father)
- add_bb_to_loop (bb, e->dest->loop_father);
+ {
+ add_bb_to_loop (bb, e->dest->loop_father);
+
+ gcc_assert (!other_bb && (new_bb->index != bb->index));
+ other_bb = bb;
+ }
}
/* Add all last_added_blocks to the region. */
@@ -4734,6 +5046,10 @@ sel_split_edge (edge e)
insn_init.todo = (INSN_INIT_TODO_LUID | INSN_INIT_TODO_SIMPLEJUMP);
sel_init_new_insns ();
+ /* Put the correct lv set on this block. */
+ if (other_bb && !sel_bb_empty_p (other_bb))
+ compute_live (sel_bb_head (other_bb));
+
return new_bb;
}
@@ -5090,9 +5406,6 @@ sel_add_block_to_region (basic_block bb,
RGN_NR_BLOCKS (rgn) += 1;
RGN_DONT_CALC_DEPS (rgn) = 0;
RGN_HAS_REAL_EBB (rgn) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
CONTAINING_RGN (bb->index) = rgn;
BLOCK_TO_BB (bb->index) = *bb_ord_index;
rgn_bb_table[RGN_BLOCKS (rgn) + *bb_ord_index] = bb->index;
@@ -5397,9 +5710,6 @@ make_regions_from_the_rest (void)
RGN_BLOCKS (nr_regions) = cur_rgn_blocks++;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
CONTAINING_RGN (bb->index) = nr_regions++;
BLOCK_TO_BB (bb->index) = 0;
}
Index: gcc/sel-sched-ir.h
===================================================================
--- gcc/sel-sched-ir.h (revision 131182)
+++ gcc/sel-sched-ir.h (working copy)
@@ -123,9 +123,17 @@ struct _expr
instead. */
VEC(unsigned, heap) *changed_on_insns;
+ /* True (1) when original target (register or memory) of this instruction
+ is available for scheduling, false otherwise. -1 means we're not sure;
+ please run find_used_regs to clarify. */
+ char target_available;
+
/* True when the expression was substituted. Used for statistical
purposes. */
- bool was_substituted;
+ BOOL_BITFIELD was_substituted : 1;
+
+ /* True when the expression was renamed. */
+ BOOL_BITFIELD was_renamed : 1;
};
typedef struct _expr expr_def;
@@ -151,7 +159,9 @@ typedef expr_t rhs_t;
#define EXPR_SPEC_DONE_DS(EXPR) ((EXPR)->spec_done_ds)
#define EXPR_SPEC_TO_CHECK_DS(EXPR) ((EXPR)->spec_to_check_ds)
#define EXPR_CHANGED_ON_INSNS(EXPR) ((EXPR)->changed_on_insns)
+#define EXPR_TARGET_AVAILABLE(EXPR) ((EXPR)->target_available)
#define EXPR_WAS_SUBSTITUTED(EXPR) ((EXPR)->was_substituted)
+#define EXPR_WAS_RENAMED(EXPR) ((EXPR)->was_renamed)
/* Obsolete. */
#define RHS_VINSN(RHS) ((RHS)->vinsn)
@@ -329,6 +339,14 @@ _list_add (_list_t *lp)
}
static inline void
+_list_remove_nofree (_list_t *lp)
+{
+ _list_t n = *lp;
+
+ *lp = _LIST_NEXT (n);
+}
+
+static inline void
_list_remove (_list_t *lp)
{
_list_t n = *lp;
@@ -378,6 +396,14 @@ _list_iter_remove (_list_iterator *ip)
ip->removed_p = true;
}
+static inline void
+_list_iter_remove_nofree (_list_iterator *ip)
+{
+ gcc_assert (!ip->removed_p && ip->can_remove_p);
+ _list_remove_nofree (ip->lp);
+ ip->removed_p = true;
+}
+
/* General macros to traverse a list. FOR_EACH_* interfaces are
implemented using these. */
#define _FOR_EACH(TYPE, ELEM, I, L) \
@@ -521,6 +547,8 @@ struct idata_def
for the best. */
regset reg_sets;
+ regset reg_clobbers;
+
regset reg_uses;
/* I hope that our renaming infrastructure handles this. */
@@ -532,6 +560,7 @@ struct idata_def
#define IDATA_RHS(ID) ((ID)->rhs)
#define IDATA_REG_SETS(ID) ((ID)->reg_sets)
#define IDATA_REG_USES(ID) ((ID)->reg_uses)
+#define IDATA_REG_CLOBBERS(ID) ((ID)->reg_clobbers)
/* Type to represent all needed info to emit an insn.
This is a virtual equivalent of the insn.
@@ -580,6 +609,7 @@ struct vinsn_def
#define VINSN_RHS(VI) (IDATA_RHS (VINSN_ID (VI)))
#define VINSN_REG_SETS(VI) (IDATA_REG_SETS (VINSN_ID (VI)))
#define VINSN_REG_USES(VI) (IDATA_REG_USES (VINSN_ID (VI)))
+#define VINSN_REG_CLOBBERS(VI) (IDATA_REG_CLOBBERS (VINSN_ID (VI)))
#define VINSN_COUNT(VI) ((VI)->count)
@@ -664,6 +694,7 @@ extern sel_insn_data_def insn_sid (insn_
#define INSN_LHS(INSN) (VINSN_LHS (INSN_VINSN (INSN)))
#define INSN_RHS(INSN) (VINSN_RHS (INSN_VINSN (INSN)))
#define INSN_REG_SETS(INSN) (VINSN_REG_SETS (INSN_VINSN (INSN)))
+#define INSN_REG_CLOBBERS(INSN) (VINSN_REG_CLOBBERS (INSN_VINSN (INSN)))
#define INSN_REG_USES(INSN) (VINSN_REG_USES (INSN_VINSN (INSN)))
#define INSN_SCHED_TIMES(INSN) (EXPR_SCHED_TIMES (INSN_EXPR (INSN)))
#define INSN_SEQNO(INSN) (SID (INSN)->seqno)
@@ -852,6 +883,7 @@ extern bool enable_moveup_set_path_p;
extern bool enable_schedule_as_rhs_p;
extern bool pipelining_p;
extern bool bookkeeping_p;
+extern int max_insns_to_rename;
extern bool preheader_removed;
/* Functions that are used in sel-sched.c. */
@@ -907,20 +939,24 @@ extern insn_t sel_gen_insn_from_expr_aft
extern bool vinsns_correlate_as_rhses_p (vinsn_t, vinsn_t);
extern void copy_expr (expr_t, expr_t);
extern void copy_expr_onside (expr_t, expr_t);
-extern void merge_expr_data (expr_t, expr_t);
-extern void merge_expr (expr_t, expr_t);
+extern void merge_expr_data (expr_t, expr_t, bool);
+extern void merge_expr (expr_t, expr_t, bool);
extern void clear_expr (expr_t);
+extern unsigned expr_dest_regno (expr_t);
+extern rtx expr_dest_reg (expr_t);
extern int find_in_hash_vect (VEC(unsigned, heap) *, unsigned);
extern void insert_in_hash_vect (VEC(unsigned, heap) **, unsigned);
+extern void mark_unavailable_targets (av_set_t, av_set_t, regset);
/* Av set functions. */
extern void av_set_add (av_set_t *, rhs_t);
extern void av_set_iter_remove (av_set_iterator *);
extern rhs_t av_set_lookup (av_set_t, vinsn_t);
-extern rhs_t av_set_lookup_other_equiv_rhs (av_set_t, vinsn_t);
+extern expr_t merge_with_other_exprs (av_set_t *, av_set_iterator *, expr_t);
extern bool av_set_is_in_p (av_set_t, vinsn_t);
extern av_set_t av_set_copy (av_set_t);
extern void av_set_union_and_clear (av_set_t *, av_set_t *);
+extern void av_set_union_and_live (av_set_t *, av_set_t *, regset, regset);
extern void av_set_clear (av_set_t *);
extern void av_set_leave_one (av_set_t *);
extern rhs_t av_set_element (av_set_t, int);
@@ -933,6 +969,8 @@ extern void sel_save_haifa_priorities (v
extern void sel_init_global_and_expr (bb_vec_t);
extern void sel_finish_global_and_expr (void);
+extern regset compute_live (insn_t);
+
/* Dependence analysis functions. */
extern void sel_clear_has_dependence (void);
extern ds_t has_dependence_p (rhs_t, insn_t, ds_t **);
@@ -977,6 +1015,7 @@ extern bool sel_insn_has_single_succ_p (
extern void cfg_succs_1 (insn_t, int, insn_t **, int *);
extern void cfg_succs_2 (insn_t, int, insn_t **, int **, int *);
extern void cfg_succs (insn_t, insn_t **, int **, int *);
+extern void cfg_succs_other (insn_t, int, insn_t *, int, int, insn_t **, int *);
extern insn_t cfg_succ_1 (insn_t, int);
extern insn_t cfg_succ (insn_t);
extern int overall_prob_of_succs (insn_t);
Index: gcc/sel-sched-dump.c
===================================================================
--- gcc/sel-sched-dump.c (revision 131182)
+++ gcc/sel-sched-dump.c (working copy)
@@ -390,9 +390,11 @@ dump_expr_1 (expr_t expr, int flags)
int orig_bb = EXPR_ORIG_BB_INDEX (expr);
if (orig_bb != 0)
- print ("orig_bb:%d", orig_bb);
+ print ("orig_bb:%d;", orig_bb);
}
-
+
+ if (!EXPR_TARGET_AVAILABLE (expr))
+ print ("targ_un;");
print ("]");
line_finish ();
}
Index: gcc/sched-rgn.c
===================================================================
--- gcc/sched-rgn.c (revision 131180)
+++ gcc/sched-rgn.c (working copy)
@@ -439,8 +439,6 @@ find_single_block_region (bool ebbs_p)
RGN_BLOCKS (nr_regions) = i;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
for (bb = ebb_start; ; bb = bb->next_bb)
{
@@ -466,11 +464,6 @@ find_single_block_region (bool ebbs_p)
break;
}
- if (RGN_NR_BLOCKS (nr_regions) == 1)
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
- else
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 1;
-
ebb_start = bb;
nr_regions++;
}
@@ -483,9 +476,6 @@ find_single_block_region (bool ebbs_p)
RGN_BLOCKS (nr_regions) = nr_regions;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
CONTAINING_RGN (bb->index) = nr_regions;
BLOCK_TO_BB (bb->index) = 0;
@@ -938,12 +928,6 @@ haifa_find_rgns (void)
RGN_BLOCKS (nr_regions) = idx++;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- if (num_bbs == 1)
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
- else
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 1;
CONTAINING_RGN (bb->index) = nr_regions;
BLOCK_TO_BB (bb->index) = count = 0;
@@ -1015,9 +999,6 @@ haifa_find_rgns (void)
RGN_BLOCKS (nr_regions) = idx++;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
CONTAINING_RGN (bb->index) = nr_regions++;
BLOCK_TO_BB (bb->index) = 0;
}
@@ -1265,8 +1246,6 @@ extend_rgns (int *degree, int *idxp, sbi
RGN_BLOCKS (nr_regions) = idx++;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
CONTAINING_RGN (bbn) = nr_regions;
BLOCK_TO_BB (bbn) = 0;
@@ -1293,7 +1272,6 @@ extend_rgns (int *degree, int *idxp, sbi
processed in the below cycle. */
{
RGN_NR_BLOCKS (nr_regions) = 1;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
nr_regions++;
}
@@ -1323,9 +1301,6 @@ extend_rgns (int *degree, int *idxp, sbi
RGN_NR_BLOCKS (nr_regions) = 1;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
RGN_HAS_REAL_EBB (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
nr_regions++;
}
@@ -1340,10 +1315,6 @@ extend_rgns (int *degree, int *idxp, sbi
if (!large)
{
RGN_NR_BLOCKS (nr_regions) = num_bbs;
- if (num_bbs == 1)
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
- else
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 1;
nr_regions++;
}
}
@@ -3265,9 +3236,6 @@ rgn_make_new_region_out_of_new_block (ba
RGN_NR_BLOCKS (nr_regions) = 1;
RGN_HAS_REAL_EBB (nr_regions) = 0;
RGN_DONT_CALC_DEPS (nr_regions) = 0;
- RGN_HAS_RENAMING_P (nr_regions) = 0;
- RGN_WAS_PIPELINED_P (nr_regions) = 0;
- RGN_NEEDS_GLOBAL_LIVE_UPDATE (nr_regions) = 0;
CONTAINING_RGN (bb->index) = nr_regions;
BLOCK_TO_BB (bb->index) = 0;
Index: gcc/sched-rgn.h
===================================================================
--- gcc/sched-rgn.h (revision 131180)
+++ gcc/sched-rgn.h (working copy)
@@ -36,9 +36,6 @@ typedef struct
unsigned int dont_calc_deps : 1;
/* This region has at least one non-trivial ebb. */
unsigned int has_real_ebb : 1;
- unsigned int has_renaming_p : 1;
- unsigned int was_pipelined_p : 1;
- unsigned int needs_global_live_update : 1;
}
region;
@@ -52,10 +49,6 @@ extern int *containing_rgn;
#define RGN_BLOCKS(rgn) (rgn_table[rgn].rgn_blocks)
#define RGN_DONT_CALC_DEPS(rgn) (rgn_table[rgn].dont_calc_deps)
#define RGN_HAS_REAL_EBB(rgn) (rgn_table[rgn].has_real_ebb)
-#define RGN_HAS_RENAMING_P(RGN) (rgn_table[RGN].has_renaming_p)
-#define RGN_WAS_PIPELINED_P(RGN) (rgn_table[RGN].was_pipelined_p)
-#define RGN_NEEDS_GLOBAL_LIVE_UPDATE(RGN) \
- (rgn_table[RGN].needs_global_live_update)
#define BLOCK_TO_BB(block) (block_to_bb[block])
#define CONTAINING_RGN(block) (containing_rgn[block])
Index: gcc/params.def
===================================================================
--- gcc/params.def (revision 131180)
+++ gcc/params.def (working copy)
@@ -595,6 +595,11 @@ DEFPARAM(PARAM_SELSCHED_MAX_SCHED_TIMES,
"Maximum number of times that an insn could be scheduled",
2, 0, 0)
+DEFPARAM(PARAM_SELSCHED_INSNS_TO_RENAME,
+ "selsched-insns-to-rename",
+ "Maximum number of instructions in the ready list that are considered eligible for renaming",
+ 2, 0, 0)
+
/* Minimal distance (in CPU cycles) between store and load targeting same
memory locations. */