This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
[ira] the initial patch
- From: "Vladimir N. Makarov" <vmakarov at redhat dot com>
- To: "gcc-patches at gcc dot gnu dot org" <gcc-patches at gcc dot gnu dot org>
- Date: Fri, 15 Dec 2006 09:50:42 -0500
- Subject: [ira] the initial patch
Here is the initial patch. To use IRA, just add
option -fira for gcc command-line. You can use additionaly option
-fira-algorithm=[regional,CB,priority] to choose the regional allocation
(default), Chaitin-Briggs, or Chow's priority-based allocation.
I'd recommend to use -flower-subreg option too for 32-bit targets (there
is no harm to use it for 64-bit machines too).
Vlad
2006-12-15 Vladimir Makarov <vmakarov@redhat.com>
* ira.h, ira-int.h, ira.c, ira-build.c, ira-costs.c
ira-conflicts.c, ira-color.c, ira-emit.c: New files.
* flags.h (ira_algorithm): New enumeration.
(flag_ira_algorithm): New external variable declaration.
* tree-pass.h (pass_ira): New external variable declaration.
* toplev.c (ira.h): New include.
(flag_ira_algorithm): New global variable.
(backend_init): Call init_ira_once.
* toplev.h (flag_ira, flag_ira_biased_coloring): New external
variable declarations.
* regs.h (contains_reg_of_mode, move_cost, may_move_in_cost,
may_move_out_cost): New external variable declarations.
* caller-save.c (no_caller_save_reg_set): New global variable.
(init_caller_save): Set up no_caller_save_reg_set.
* global.c (eliminable_regset): Make it external.
(gate_handle_global_alloc): New function.
(pass_global_alloc): Add the gate function.
* opts.c (decode_options): Print the warning for -fira.
(common_handle_option): Process -fira-algorithm option.
* timevar.def (TV_IRA): New pass.
* hard-reg-set.h (no_caller_save_reg_set): New external variable
declaration.
* regmove.c (regmove_optimize): Don't do replacement of output
operands by input operands.
(rest_of_handle_regmove): Don't do CFG cleanup for IRA.
* local-alloc.c (update_equiv_regs): Make it external. Return
true if jump label rebuilding should be done.
(gate_handle_local_alloc): New function.
(pass_local_alloc): Add the gate function.
* alias.c (stack_addr_p): New function.
(nonoverlapping_memrefs_p): Add code for IRA.
* common.opt (fira, fira-algorithm, fira-biased-coloring): New
options.
* regclass.c (contains_reg_of_mode, move_cost, may_move_in_cost,
may_move_out_cost): Make the variables external.
* rtl.h (eliminable_regset): New external variable declaration.
(update_equiv_regs): New external function definition.
* Makefile.in (IRA_INT_H): New definition.
(OBJS-common): Add ira.o, ira-build.o, ira-costs.o,
ira-conflicts.o, ira-color.o, and ira-emit.o.
(reload1.o, toplev.o): Add dependence on ira.h.
(ira.o, ira-build.o, ira-costs.o, ira-conflicts.o, ira-color.o,
ira-emit.o): New entries.
* passes.c (pass_ira): New pass.
* reload1.c (alter_reg): Add a new parameter.
(pseudo_reg_compare): New function.
(reload): Sort pseudos for IRA. Call alter_reg with the
additional parameter.
(count_spilled_pseudo): New variable freq. Use it.
(alter_reg): Add code for IRA.
(eliminate_regs_1, finish_spills, emit_input_reload_insns,
delete_output_reload): Use additional parameter for alter_reg.
(finish_spills, emit_input_reload_insns, delete_output_reload):
Call mark_allocation_change.
(finish_spills): Call retry_ira_color.
* doc/invoke.texi: Describe new options -fira,
-fira-biased-coloring, and -fira-algorithm.
* doc/passes.texi: Decribe IRA.
* doc/tm.texi: Decribe macro IRA_COVER_CLASSES.
* config/sparc/sparc.h (IRA_COVER_CLASSES): New macro.
* config/i386/i386.h (IRA_COVER_CLASSES): Ditto.
* config/ia64/ia64.h (IRA_COVER_CLASSES): Ditto.
* config/rs6000/rs6000.h (IRA_COVER_CLASSES): Ditto.
Richard Henderson <rth@redhat.com>
* tree-pass.h (pass_lower_subreg): New external definition.
* toplev.h (flag_lower_subreg): New external definition.
* rtl.def (CONCATN): New rtl expression.
* dwarf2out.c (concatn_loc_descriptor): New function.
(loc_descriptor): Process CONCATN.
* timevar.def (TV_LOWER_SUBREG): New definition.
* emit-rtl.c (gen_reg_rtx_offset): New function.
(gen_lowpart_common): Process CONCATN.
* simplify-rtx.c (simplify_subreg): Process CONCATN.
* common.opt (flower-subreg): New options.
* rtl.h (gen_reg_rtx_offset): New external definition.
* Makefile.in (OBJS-common): Add lower-subreg.o.
(lower-subreg.o): New entry.
* passes.c (pass_lower_subreg): Add new pass.
* lower-subreg.c: New file.
Index: doc/passes.texi
===================================================================
--- doc/passes.texi (revision 119879)
+++ doc/passes.texi (working copy)
@@ -822,6 +822,17 @@ Global register allocation. This pass a
the remaining pseudo registers (those whose life spans are not
contained in one basic block). The pass is located in @file{global.c}.
+@item
+The integrated register allocator (@acronym{IRA}). It can be used
+instead of the local and global allocator. It is called integrated
+because coalescing and register live range splitting are done
+on-the-fly during coloring. Source files of the allocator are
+@file{ira.c}, @file{ira-build.c}, @file{ira-costs.c},
+@file{ira-conflicts.c}, @file{ira-color.c}, @file{ira-emit.c}, plus
+header files @file{ira.h} and @file{ira-int.h} used for the
+communication between the allocator and the rest of the compiler and
+between the IRA files.
+
@cindex reloading
@item
Reloading. This pass renumbers pseudo registers with the hardware
Index: doc/tm.texi
===================================================================
--- doc/tm.texi (revision 119879)
+++ doc/tm.texi (working copy)
@@ -2730,6 +2730,19 @@ as below:
@end smallexample
@end defmac
+@defmac IRA_COVER_CLASSES
+The macro defines cover classes for the Integrated Register Allocator
+(@acronym{IRA}). Cover classes is a set of non-intersected register
+classes covering all hard registers used for register allocation
+purpose. Any move between two registers of a cover class should be
+cheaper than load or store of the registers. The macro value is array
+of register classes with @code{LIM_REG_CLASSES} used as the end
+marker.
+
+If the macro is defined, the integrated register allocator can be used
+for the target.
+@end defmac
+
@node Old Constraints
@section Obsolete Macros for Defining Constraints
@cindex defining constraints, obsolete method
Index: doc/invoke.texi
===================================================================
--- doc/invoke.texi (revision 119879)
+++ doc/invoke.texi (working copy)
@@ -313,7 +313,8 @@ Objective-C and Objective-C++ Dialects}.
-fgcse -fgcse-lm -fgcse-sm -fgcse-las -fgcse-after-reload @gol
-fcrossjumping -fif-conversion -fif-conversion2 @gol
-finline-functions -finline-functions-called-once @gol
--finline-limit=@var{n} -fkeep-inline-functions @gol
+-finline-limit=@var{n} -fira -fira-alogirthm=@var{algorithm} @gol
+-fira-biased-coloring -fkeep-inline-functions @gol
-fkeep-static-consts -fmerge-constants -fmerge-all-constants @gol
-fmodulo-sched -fno-branch-count-reg @gol
-fno-default-inline -fno-defer-pop -fmove-loop-invariants @gol
@@ -4999,6 +5000,29 @@ optimization.
Enabled at levels @option{-O2}, @option{-O3}, @option{-Os}.
+@item -fira
+@opindex fira
+If supported for the target machine, use the integrated register
+allocator (@acronym{IRA}) for the register allocation.
+
+@item -fira-biased-coloring
+@opindex fira-biased-coloring
+Use biased coloring for the integrated register allocator for possible
+improvement of the generated code. It makes register allocator
+slower.
+
+@item -fira-algorithm=@var{algorithm}
+Use specified algorithm for the integrated register allocator. The
+@var{algorithm} argument should be one of @code{regional}, @code{CB},
+or @code{priority}. The second algorithm specifies Chaitin-Briggs
+coloring, the third one specifies Chow's priority based coloring, and
+the third one which is the default specifies regional coloring based
+on Chaitin-Briggs coloring. The first algorithm is the best for the
+generated code quality especially for machines with small or moderate
+size register set, the second one is faster and generates decent code
+and the smallest size code, and the priority-based one is the fastest
+one.
+
@item -fdelayed-branch
@opindex fdelayed-branch
If supported for the target machine, attempt to reorder instructions
Index: flags.h
===================================================================
--- flags.h (revision 119879)
+++ flags.h (working copy)
@@ -192,6 +192,16 @@ extern int flag_dump_rtl_in_asm;
unused UIDs if there are a lot of instructions. If greater than
one, unconditionally renumber instruction UIDs. */
extern int flag_renumber_insns;
+
+/* The algorithm used for the integrated register allocator (IRA). */
+enum ira_algorithm {
+ IRA_ALGORITHM_REGIONAL,
+ IRA_ALGORITHM_CB,
+ IRA_ALGORITHM_PRIORITY
+};
+
+extern enum ira_algorithm flag_ira_algorithm;
+
/* Other basic status info about current function. */
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 119879)
+++ tree-pass.h (working copy)
@@ -332,6 +332,7 @@ extern struct tree_opt_pass pass_instant
extern struct tree_opt_pass pass_rtl_fwprop;
extern struct tree_opt_pass pass_rtl_fwprop_addr;
extern struct tree_opt_pass pass_jump2;
+extern struct tree_opt_pass pass_lower_subreg;
extern struct tree_opt_pass pass_cse;
extern struct tree_opt_pass pass_gcse;
extern struct tree_opt_pass pass_jump_bypass;
@@ -362,6 +363,7 @@ extern struct tree_opt_pass pass_sms;
extern struct tree_opt_pass pass_sched;
extern struct tree_opt_pass pass_local_alloc;
extern struct tree_opt_pass pass_global_alloc;
+extern struct tree_opt_pass pass_ira;
extern struct tree_opt_pass pass_postreload;
extern struct tree_opt_pass pass_clean_state;
extern struct tree_opt_pass pass_branch_prob;
Index: toplev.c
===================================================================
--- toplev.c (revision 119879)
+++ toplev.c (working copy)
@@ -67,6 +67,7 @@ Software Foundation, 51 Franklin Street,
#include "diagnostic.h"
#include "params.h"
#include "reload.h"
+#include "ira.h"
#include "dwarf2asm.h"
#include "integrate.h"
#include "real.h"
@@ -306,6 +307,10 @@ int flag_next_runtime = 0;
enum tls_model flag_tls_default = TLS_MODEL_GLOBAL_DYNAMIC;
+/* Set the default algorithm for the integrated register allocator. */
+
+enum ira_algorithm flag_ira_algorithm = IRA_ALGORITHM_REGIONAL;
+
/* Nonzero means change certain warnings into errors.
Usually these are warnings about failure to conform to some standard. */
@@ -1952,6 +1957,7 @@ backend_init (void)
init_alias_once ();
init_reload ();
init_varasm_once ();
+ init_ira_once ();
/* The following initialization functions need to generate rtl, so
provide a dummy function context for them. */
Index: toplev.h
===================================================================
--- toplev.h (revision 119879)
+++ toplev.h (working copy)
@@ -122,6 +122,7 @@ extern int flag_crossjumping;
extern int flag_if_conversion;
extern int flag_if_conversion2;
extern int flag_keep_static_consts;
+extern int flag_lower_subreg;
extern int flag_peel_loops;
extern int flag_rerun_cse_after_loop;
extern int flag_thread_jumps;
@@ -131,6 +132,8 @@ extern int flag_unroll_all_loops;
extern int flag_unswitch_loops;
extern int flag_cprop_registers;
extern int time_report;
+extern int flag_ira;
+extern int flag_ira_biased_coloring;
/* Things to do with target switches. */
extern void print_version (FILE *, const char *);
Index: regs.h
===================================================================
--- regs.h (revision 119879)
+++ regs.h (working copy)
@@ -237,7 +237,23 @@ extern int caller_save_needed;
/* Allocate reg_n_info tables */
extern void allocate_reg_info (size_t, int, int);
+/* 1 if the corresponding class does contain register of given
+ mode. */
+extern char contains_reg_of_mode [N_REG_CLASSES] [MAX_MACHINE_MODE];
+
+/* Maximum cost of moving from a register in one class to a register
+ in another class. */
+extern int move_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+
/* Specify number of hard registers given machine mode occupy. */
extern unsigned char hard_regno_nregs[FIRST_PSEUDO_REGISTER][MAX_MACHINE_MODE];
+/* Similar, but here we don't have to move if the first index is a
+ subset of the second so in that case the cost is zero. */
+extern int may_move_in_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+
+/* Similar, but here we don't have to move if the first index is a
+ superset of the second so in that case the cost is zero. */
+extern int may_move_out_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+
#endif /* GCC_REGS_H */
Index: caller-save.c
===================================================================
--- caller-save.c (revision 119879)
+++ caller-save.c (working copy)
@@ -37,6 +37,10 @@ Software Foundation, 51 Franklin Street,
#include "tm_p.h"
#include "addresses.h"
+/* Call used hard registers which can not be saved because there is no
+ insn for this. */
+HARD_REG_SET no_caller_save_reg_set;
+
#ifndef MAX_MOVE_MAX
#define MAX_MOVE_MAX MOVE_MAX
#endif
@@ -117,6 +121,7 @@ init_caller_save (void)
rtx test_reg, test_mem;
rtx saveinsn, restinsn;
+ CLEAR_HARD_REG_SET (no_caller_save_reg_set);
/* First find all the registers that we need to deal with and all
the modes that they can have. If we can't find a mode to use,
we can't have the register live over calls. */
@@ -244,6 +249,8 @@ init_caller_save (void)
{
call_fixed_regs[i] = 1;
SET_HARD_REG_BIT (call_fixed_reg_set, i);
+ if (call_used_regs[i])
+ SET_HARD_REG_BIT (no_caller_save_reg_set, i);
}
}
}
Index: rtl.def
===================================================================
--- rtl.def (revision 119879)
+++ rtl.def (working copy)
@@ -411,6 +411,13 @@ DEF_RTL_EXPR(SYMBOL_REF, "symbol_ref", "
pretend to be looking at the entire value and comparing it. */
DEF_RTL_EXPR(CC0, "cc0", "", RTX_OBJ)
+/* (CONCATN [a1 a2 .. an]) represents the virtual concatenation of all
+ An to make a value. This is an extension of the CONCAT to larger
+ numbers of components. This is used for decomposing large values
+ into register sized components. Like CONCAT, it should not appear
+ in the insn chain. */
+DEF_RTL_EXPR (CONCATN, "concatn", "E", RTX_OBJ)
+
/* ----------------------------------------------------------------------
Expressions for operators in an rtl pattern
---------------------------------------------------------------------- */
Index: global.c
===================================================================
--- global.c (revision 119879)
+++ global.c (working copy)
@@ -292,7 +292,7 @@ static int n_regs_set;
/* All registers that can be eliminated. */
-static HARD_REG_SET eliminable_regset;
+HARD_REG_SET eliminable_regset;
static int allocno_compare (const void *, const void *);
static void global_conflicts (void);
@@ -2501,6 +2501,13 @@ make_accurate_live_analysis (void)
}
free_bb_info ();
}
+
+static bool
+gate_handle_global_alloc (void)
+{
+ return ! flag_ira;
+}
+
/* Run old register allocator. Return TRUE if we must exit
rest_of_compilation upon return. */
static unsigned int
@@ -2534,7 +2541,7 @@ rest_of_handle_global_alloc (void)
struct tree_opt_pass pass_global_alloc =
{
"greg", /* name */
- NULL, /* gate */
+ gate_handle_global_alloc, /* gate */
rest_of_handle_global_alloc, /* execute */
NULL, /* sub */
NULL, /* next */
Index: dwarf2out.c
===================================================================
--- dwarf2out.c (revision 119879)
+++ dwarf2out.c (working copy)
@@ -4149,6 +4149,7 @@ static dw_loc_descr_ref based_loc_descr
static int is_based_loc (rtx);
static dw_loc_descr_ref mem_loc_descriptor (rtx, enum machine_mode mode);
static dw_loc_descr_ref concat_loc_descriptor (rtx, rtx);
+static dw_loc_descr_ref concatn_loc_descriptor (rtx);
static dw_loc_descr_ref loc_descriptor (rtx);
static dw_loc_descr_ref loc_descriptor_from_tree_1 (tree, int);
static dw_loc_descr_ref loc_descriptor_from_tree (tree);
@@ -9045,6 +9046,31 @@ concat_loc_descriptor (rtx x0, rtx x1)
return cc_loc_result;
}
+/* Return a descriptor that describes the concatenation of N locations. */
+
+static dw_loc_descr_ref
+concatn_loc_descriptor (rtx concatn)
+{
+ dw_loc_descr_ref cc_loc_result = NULL;
+ unsigned int i, n = XVECLEN (concatn, 0);
+
+ for (i = 0; i < n; ++i)
+ {
+ dw_loc_descr_ref ref;
+ rtx x = XVECEXP (concatn, 0, i);
+
+ ref = loc_descriptor (x);
+ if (ref == NULL)
+ return NULL;
+
+ add_loc_descr (&cc_loc_result, ref);
+ ref = new_loc_descr (DW_OP_piece, GET_MODE_SIZE (GET_MODE (x)), 0);
+ add_loc_descr (&cc_loc_result, ref);
+ }
+
+ return cc_loc_result;
+}
+
/* Output a proper Dwarf location descriptor for a variable or parameter
which is either allocated in a register or in a memory location. For a
register, we just generate an OP_REG and the register number. For a
@@ -9082,6 +9108,10 @@ loc_descriptor (rtx rtl)
loc_result = concat_loc_descriptor (XEXP (rtl, 0), XEXP (rtl, 1));
break;
+ case CONCATN:
+ loc_result = concatn_loc_descriptor (rtl);
+ break;
+
case VAR_LOCATION:
/* Single part. */
if (GET_CODE (XEXP (rtl, 1)) != PARALLEL)
Index: opts.c
===================================================================
--- opts.c (revision 119879)
+++ opts.c (working copy)
@@ -628,6 +628,14 @@ decode_options (unsigned int argc, const
flag_reorder_blocks_and_partition = 0;
flag_reorder_blocks = 1;
}
+
+#ifndef IRA_COVER_CLASSES
+ if (flag_ira)
+ {
+ inform ("-fira does not work on this architecture");
+ flag_ira = 0;
+ }
+#endif
}
/* Handle target- and language-independent options. Return zero to
@@ -946,6 +954,17 @@ common_handle_option (size_t scode, cons
warning (0, "unknown tls-model \"%s\"", arg);
break;
+ case OPT_fira_algorithm_:
+ if (!strcmp (arg, "regional"))
+ flag_ira_algorithm = IRA_ALGORITHM_REGIONAL;
+ else if (!strcmp (arg, "CB"))
+ flag_ira_algorithm = IRA_ALGORITHM_CB;
+ else if (!strcmp (arg, "priority"))
+ flag_ira_algorithm = IRA_ALGORITHM_PRIORITY;
+ else
+ warning (0, "unknown ira algorithm \"%s\"", arg);
+ break;
+
case OPT_ftracer:
flag_tracer_set = true;
break;
Index: timevar.def
===================================================================
--- timevar.def (revision 119879)
+++ timevar.def (working copy)
@@ -128,6 +128,7 @@ DEFTIMEVAR (TV_OVERLOAD , "
DEFTIMEVAR (TV_TEMPLATE_INSTANTIATION, "template instantiation")
DEFTIMEVAR (TV_EXPAND , "expand")
DEFTIMEVAR (TV_VARCONST , "varconst")
+DEFTIMEVAR (TV_LOWER_SUBREG , "lower subreg")
DEFTIMEVAR (TV_JUMP , "jump")
DEFTIMEVAR (TV_FWPROP , "forward prop")
DEFTIMEVAR (TV_CSE , "CSE")
@@ -154,6 +155,7 @@ DEFTIMEVAR (TV_SMS , "sms modulo s
DEFTIMEVAR (TV_SCHED , "scheduling")
DEFTIMEVAR (TV_LOCAL_ALLOC , "local alloc")
DEFTIMEVAR (TV_GLOBAL_ALLOC , "global alloc")
+DEFTIMEVAR (TV_IRA , "integrated RA")
DEFTIMEVAR (TV_RELOAD_CSE_REGS , "reload CSE regs")
DEFTIMEVAR (TV_SEQABSTR , "sequence abstraction")
DEFTIMEVAR (TV_GCSE_AFTER_RELOAD , "load CSE after reload")
Index: hard-reg-set.h
===================================================================
--- hard-reg-set.h (revision 119879)
+++ hard-reg-set.h (working copy)
@@ -446,6 +446,11 @@ extern char global_regs[FIRST_PSEUDO_REG
extern HARD_REG_SET regs_invalidated_by_call;
+/* Call used hard registers which can not be saved because there is no
+ insn for this. */
+
+extern HARD_REG_SET no_caller_save_reg_set;
+
#ifdef REG_ALLOC_ORDER
/* Table of register numbers in the order in which to try to use them. */
Index: regmove.c
===================================================================
--- regmove.c (revision 119879)
+++ regmove.c (working copy)
@@ -1082,7 +1082,8 @@ regmove_optimize (rtx f, int nregs)
for (pass = 0; pass <= 2; pass++)
{
- if (! flag_regmove && pass >= flag_expensive_optimizations)
+ if ((! flag_regmove || flag_ira)
+ && pass >= flag_expensive_optimizations)
goto done;
if (dump_file)
@@ -1130,7 +1131,7 @@ regmove_optimize (rtx f, int nregs)
}
}
}
- if (! flag_regmove)
+ if (! flag_regmove || flag_ira)
continue;
if (! find_matches (insn, &match))
@@ -2486,7 +2487,8 @@ static unsigned int
rest_of_handle_regmove (void)
{
regmove_optimize (get_insns (), max_reg_num ());
- cleanup_cfg (CLEANUP_EXPENSIVE | CLEANUP_UPDATE_LIFE);
+ if (! flag_ira)
+ cleanup_cfg (CLEANUP_EXPENSIVE | CLEANUP_UPDATE_LIFE);
return 0;
}
Index: local-alloc.c
===================================================================
--- local-alloc.c (revision 119879)
+++ local-alloc.c (working copy)
@@ -81,6 +81,7 @@ Software Foundation, 51 Franklin Street,
#include "ggc.h"
#include "timevar.h"
#include "tree-pass.h"
+#include "params.h"
/* Next quantity number available for allocation. */
@@ -292,7 +293,6 @@ static int equiv_init_movable_p (rtx, in
static int contains_replace_regs (rtx);
static int memref_referenced_p (rtx, rtx);
static int memref_used_between_p (rtx, rtx, rtx);
-static void update_equiv_regs (void);
static void no_equiv (rtx, rtx, void *);
static void block_alloc (int);
static int qty_sugg_compare (int, int);
@@ -788,9 +788,11 @@ memref_used_between_p (rtx memref, rtx s
into the using insn. If it succeeds, we can eliminate the register
completely.
- Initialize the REG_EQUIV_INIT array of initializing insns. */
+ Initialize the REG_EQUIV_INIT array of initializing insns.
-static void
+ Return non-zero if jump label rebuilding should be done. */
+
+int
update_equiv_regs (void)
{
rtx insn;
@@ -1240,6 +1242,7 @@ update_equiv_regs (void)
end_alias_analysis ();
CLEAR_REG_SET (&cleared_regs);
free (reg_equiv);
+ return recorded_label_ref;
}
/* Mark REG as having no known equivalence.
@@ -2522,6 +2525,12 @@ dump_local_alloc (FILE *file)
fprintf (file, ";; Register %d in %d.\n", i, reg_renumber[i]);
}
+static bool
+gate_handle_local_alloc (void)
+{
+ return ! flag_ira;
+}
+
/* Run old register allocator. Return TRUE if we must exit
rest_of_compilation upon return. */
static unsigned int
@@ -2575,7 +2584,7 @@ rest_of_handle_local_alloc (void)
struct tree_opt_pass pass_local_alloc =
{
"lreg", /* name */
- NULL, /* gate */
+ gate_handle_local_alloc, /* gate */
rest_of_handle_local_alloc, /* execute */
NULL, /* sub */
NULL, /* next */
Index: alias.c
===================================================================
--- alias.c (revision 119879)
+++ alias.c (working copy)
@@ -162,6 +162,7 @@ static alias_set_entry get_alias_set_ent
static rtx fixed_scalar_and_varying_struct_p (rtx, rtx, rtx, rtx,
int (*) (rtx, int));
static int aliases_everything_p (rtx);
+static int stack_addr_p (rtx);
static bool nonoverlapping_component_refs_p (tree, tree);
static tree decl_for_component_ref (tree);
static rtx adjust_offset_for_component_ref (tree, rtx);
@@ -2008,6 +2009,23 @@ adjust_offset_for_component_ref (tree x,
return GEN_INT (ioffset);
}
+/* The function returns nonzero if X is a stack address. */
+static int
+stack_addr_p (rtx x)
+{
+ if (x == hard_frame_pointer_rtx || x == frame_pointer_rtx
+ || x == arg_pointer_rtx || x == stack_pointer_rtx)
+ return 1;
+ if (GET_CODE (x) == PLUS
+ && (XEXP (x, 0) == hard_frame_pointer_rtx
+ || XEXP (x, 0) == frame_pointer_rtx
+ || XEXP (x, 0) == arg_pointer_rtx
+ || XEXP (x, 0) == stack_pointer_rtx)
+ && CONSTANT_P (XEXP (x, 1)))
+ return 1;
+ return 0;
+}
+
/* Return nonzero if we can determine the exprs corresponding to memrefs
X and Y and they do not overlap. */
@@ -2017,9 +2035,24 @@ nonoverlapping_memrefs_p (rtx x, rtx y)
tree exprx = MEM_EXPR (x), expry = MEM_EXPR (y);
rtx rtlx, rtly;
rtx basex, basey;
+ rtx x_addr, y_addr;
rtx moffsetx, moffsety;
HOST_WIDE_INT offsetx = 0, offsety = 0, sizex, sizey, tem;
+ if (flag_ira)
+ {
+ /* We need this code for IRA because stack slot sharing. RTL in
+ decl can be different than RTL used in insns. It is safe
+ code although it can be conservative sometime. */
+ x_addr = canon_rtx (get_addr (XEXP (x, 0)));
+ y_addr = canon_rtx (get_addr (XEXP (y, 0)));
+
+ if (stack_addr_p (x_addr) && stack_addr_p (y_addr)
+ && memrefs_conflict_p (SIZE_FOR_MODE (y), y_addr,
+ SIZE_FOR_MODE (x), x_addr, 0))
+ return 0;
+ }
+
/* Unless both have exprs, we can't tell anything. */
if (exprx == 0 || expry == 0)
return 0;
Index: emit-rtl.c
===================================================================
--- emit-rtl.c (revision 119879)
+++ emit-rtl.c (working copy)
@@ -812,13 +812,12 @@ gen_reg_rtx (enum machine_mode mode)
return val;
}
-/* Generate a register with same attributes as REG, but offsetted by OFFSET.
+/* Update NEW with same attributes as REG, but offsetted by OFFSET.
Do the big endian correction if needed. */
-rtx
-gen_rtx_REG_offset (rtx reg, enum machine_mode mode, unsigned int regno, int offset)
+static void
+update_reg_offset (rtx new, rtx reg, int offset)
{
- rtx new = gen_rtx_REG (mode, regno);
tree decl;
HOST_WIDE_INT var_size;
@@ -860,7 +859,7 @@ gen_rtx_REG_offset (rtx reg, enum machin
if ((BYTES_BIG_ENDIAN || WORDS_BIG_ENDIAN)
&& decl != NULL
&& offset > 0
- && GET_MODE_SIZE (GET_MODE (reg)) > GET_MODE_SIZE (mode)
+ && GET_MODE_SIZE (GET_MODE (reg)) > GET_MODE_SIZE (GET_MODE (new))
&& ((var_size = int_size_in_bytes (TREE_TYPE (decl))) > 0
&& var_size < GET_MODE_SIZE (GET_MODE (reg))))
{
@@ -904,6 +903,27 @@ gen_rtx_REG_offset (rtx reg, enum machin
REG_ATTRS (new) = get_reg_attrs (REG_EXPR (reg),
REG_OFFSET (reg) + offset);
+}
+
+/* Generate a register with same attributes as REG, but offsetted by OFFSET. */
+
+rtx
+gen_rtx_REG_offset (rtx reg, enum machine_mode mode,
+ unsigned int regno, int offset)
+{
+ rtx new = gen_rtx_REG (mode, regno);
+ update_reg_offset (new, reg, offset);
+ return new;
+}
+
+/* Generate a new pseudo register with same attributes as REG, but
+ offsetted by OFFSET. */
+
+rtx
+gen_reg_rtx_offset (rtx reg, enum machine_mode mode, int offset)
+{
+ rtx new = gen_reg_rtx (mode);
+ update_reg_offset (new, reg, offset);
return new;
}
@@ -1153,8 +1173,9 @@ gen_lowpart_common (enum machine_mode mo
return gen_rtx_fmt_e (GET_CODE (x), mode, XEXP (x, 0));
}
else if (GET_CODE (x) == SUBREG || REG_P (x)
- || GET_CODE (x) == CONCAT || GET_CODE (x) == CONST_VECTOR
- || GET_CODE (x) == CONST_DOUBLE || GET_CODE (x) == CONST_INT)
+ || GET_CODE (x) == CONCAT || GET_CODE (x) == CONCATN
+ || GET_CODE (x) == CONST_VECTOR || GET_CODE (x) == CONST_DOUBLE
+ || GET_CODE (x) == CONST_INT)
return simplify_gen_subreg (mode, x, innermode, offset);
/* Otherwise, we can't do this. */
Index: simplify-rtx.c
===================================================================
--- simplify-rtx.c (revision 119879)
+++ simplify-rtx.c (working copy)
@@ -4646,13 +4646,23 @@ simplify_subreg (enum machine_mode outer
/* Handle complex values represented as CONCAT
of real and imaginary part. */
- if (GET_CODE (op) == CONCAT)
+ if (GET_CODE (op) == CONCAT || GET_CODE (op) == CONCATN)
{
unsigned int inner_size, final_offset;
rtx part, res;
- inner_size = GET_MODE_UNIT_SIZE (innermode);
- part = byte < inner_size ? XEXP (op, 0) : XEXP (op, 1);
+ if (GET_CODE (op) == CONCAT)
+ {
+ inner_size = GET_MODE_SIZE (innermode) / 2;
+ part = byte < inner_size ? XEXP (op, 0) : XEXP (op, 1);
+ }
+ else
+ {
+ /* ??? We've got room; perhaps we should store the inner size
+ of the CONCATN in one of the subsequent unused fields. */
+ inner_size = GET_MODE_SIZE (innermode) / XVECLEN (op, 0);
+ part = XVECEXP (op, 0, byte / inner_size);
+ }
final_offset = byte % inner_size;
if (final_offset + GET_MODE_SIZE (outermode) > inner_size)
return NULL_RTX;
Index: common.opt
===================================================================
--- common.opt (revision 119879)
+++ common.opt (working copy)
@@ -557,6 +557,18 @@ fipa-type-escape
Common Report Var(flag_ipa_type_escape) Init(0)
Type based escape and alias analysis
+fira
+Common Report Var(flag_ira)
+Use integrated register allocator.
+
+fira-algorithm=
+Common Joined RejectNegative
+-fira-algorithm=[regional|CB|priority] Set the used IRA algorithm
+
+fira-biased-coloring
+Common Report Var(flag_ira_biased_coloring)
+Use biased coloring for the integrated register allocator.
+
fivopts
Common Report Var(flag_ivopts) Init(1)
Optimize induction variables on trees
@@ -581,6 +593,10 @@ floop-optimize
Common
Does nothing. Preserved for backward compatibility.
+flower-subreg
+Common Report Var(flag_lower_subreg)
+Subreg lowering
+
fmath-errno
Common Report Var(flag_errno_math) Init(1)
Set errno after built-in math functions
Index: regclass.c
===================================================================
--- regclass.c (revision 119879)
+++ regclass.c (working copy)
@@ -206,22 +206,22 @@ bool have_regs_of_mode [MAX_MACHINE_MODE
/* 1 if class does contain register of given mode. */
-static char contains_reg_of_mode [N_REG_CLASSES] [MAX_MACHINE_MODE];
+char contains_reg_of_mode [N_REG_CLASSES] [MAX_MACHINE_MODE];
/* Maximum cost of moving from a register in one class to a register in
another class. Based on REGISTER_MOVE_COST. */
-static int move_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+int move_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
/* Similar, but here we don't have to move if the first index is a subset
of the second so in that case the cost is zero. */
-static int may_move_in_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+int may_move_in_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
/* Similar, but here we don't have to move if the first index is a superset
of the second so in that case the cost is zero. */
-static int may_move_out_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
+int may_move_out_cost[MAX_MACHINE_MODE][N_REG_CLASSES][N_REG_CLASSES];
#ifdef FORBIDDEN_INC_DEC_CLASSES
Index: rtl.h
===================================================================
--- rtl.h (revision 119879)
+++ rtl.h (working copy)
@@ -1474,6 +1474,7 @@ extern int rtx_equal_p (rtx, rtx);
extern rtvec gen_rtvec_v (int, rtx *);
extern rtx gen_reg_rtx (enum machine_mode);
extern rtx gen_rtx_REG_offset (rtx, enum machine_mode, unsigned int, int);
+extern rtx gen_reg_rtx_offset (rtx, enum machine_mode, int);
extern rtx gen_label_rtx (void);
extern rtx gen_lowpart_common (enum machine_mode, rtx);
@@ -2147,6 +2148,9 @@ extern bool can_copy_p (enum machine_mod
extern rtx fis_get_condition (rtx);
/* In global.c */
+#ifdef HARD_CONST
+extern HARD_REG_SET eliminable_regset;
+#endif
extern void mark_elimination (int, int);
extern void dump_global_regs (FILE *);
#ifdef HARD_CONST
@@ -2181,6 +2185,7 @@ extern void dbr_schedule (rtx);
/* In local-alloc.c */
extern void dump_local_alloc (FILE *);
+extern int update_equiv_regs (void);
/* In reload1.c */
extern int function_invariant_p (rtx);
Index: Makefile.in
===================================================================
--- Makefile.in (revision 119879)
+++ Makefile.in (working copy)
@@ -814,6 +814,7 @@ TREE_DATA_REF_H = tree-data-ref.h $(LAMB
VARRAY_H = varray.h $(MACHMODE_H) $(SYSTEM_H) coretypes.h $(TM_H)
TREE_INLINE_H = tree-inline.h $(VARRAY_H) $(SPLAY_TREE_H)
REAL_H = real.h $(MACHMODE_H)
+IRA_INT_H = ira.h ira-int.h $(CFGLOOP_H)
#
# Now figure out from those variables how to compile and link.
@@ -1018,7 +1019,8 @@ OBJS-common = \
lambda-trans.o lambda-code.o tree-loop-linear.o tree-ssa-sink.o \
tree-vrp.o tree-stdarg.o tree-cfgcleanup.o tree-ssa-reassoc.o \
tree-ssa-structalias.o tree-object-size.o \
- rtl-factoring.o
+ rtl-factoring.o ira.o ira-build.o ira-costs.o ira-conflicts.o \
+ ira-color.o ira-emit.o lower-subreg.o
OBJS-md = $(out_object_file)
@@ -2154,7 +2156,7 @@ toplev.o : toplev.c $(CONFIG_H) $(SYSTEM
$(INSN_ATTR_H) output.h $(DIAGNOSTIC_H) debug.h insn-config.h intl.h \
$(RECOG_H) Makefile toplev.h dwarf2out.h sdbout.h dbxout.h $(EXPR_H) \
hard-reg-set.h $(BASIC_BLOCK_H) graph.h except.h $(REGS_H) $(TIMEVAR_H) \
- value-prof.h $(PARAMS_H) $(TM_P_H) reload.h dwarf2asm.h $(TARGET_H) \
+ value-prof.h $(PARAMS_H) $(TM_P_H) reload.h ira.h dwarf2asm.h $(TARGET_H) \
langhooks.h insn-flags.h $(CFGLAYOUT_H) $(CFGLOOP_H) hosthooks.h \
$(CGRAPH_H) $(COVERAGE_H) alloc-pool.h $(GGC_H) $(INTEGRATE_H) \
$(CPPLIB_H) opts.h params.def tree-mudflap.h $(REAL_H)
@@ -2525,7 +2527,7 @@ reload1.o : reload1.c $(CONFIG_H) $(SYST
$(EXPR_H) $(OPTABS_H) reload.h $(REGS_H) hard-reg-set.h insn-config.h \
$(BASIC_BLOCK_H) $(RECOG_H) output.h $(FUNCTION_H) toplev.h $(TM_P_H) \
addresses.h except.h $(TREE_H) $(REAL_H) $(FLAGS_H) $(MACHMODE_H) \
- $(OBSTACK_H) $(TARGET_H)
+ $(OBSTACK_H) $(TARGET_H) ira.h
rtlhooks.o : rtlhooks.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
rtlhooks-def.h $(EXPR_H) $(RECOG_H)
postreload.o : postreload.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
@@ -2554,6 +2556,33 @@ alias.o : alias.c $(CONFIG_H) $(SYSTEM_H
$(ALIAS_H) $(EMIT_RTL_H) $(GGC_H) $(FUNCTION_H) cselib.h $(TREE_H) $(TM_P_H) \
langhooks.h $(TARGET_H) gt-alias.h $(TIMEVAR_H) $(CGRAPH_H) \
$(SPLAY_TREE_H) $(VARRAY_H) $(IPA_TYPE_ESCAPE_H) tree-pass.h
+ira-build.o: ira-build.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(TARGET_H) $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) \
+ insn-config.h $(RECOG_H) $(BASIC_BLOCK_H) toplev.h $(TM_P_H) \
+ $(PARAMS_H) $(DF_H) $(IRA_INT_H)
+ira-costs.o: ira-costs.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(TARGET_H) $(RTL_H) insn-config.h $(RECOG_H) \
+ $(REGS_H) hard-reg-set.h $(FLAGS_H) errors.h \
+ $(EXPR_H) $(BASIC_BLOCK_H) $(TM_P_H) \
+ $(IRA_INT_H)
+ira-conflicts.o: ira-conflicts.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(TARGET_H) $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) \
+ insn-config.h $(RECOG_H) $(BASIC_BLOCK_H) toplev.h $(TM_P_H) $(PARAMS_H) \
+ $(DF_H) $(IRA_INT_H)
+ira-color.o: ira-color.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(TARGET_H) $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) \
+ $(EXPR_H) $(BASIC_BLOCK_H) toplev.h $(TM_P_H) $(PARAMS_H) \
+ $(DF_H) $(IRA_INT_H)
+ira-emit.o: ira-emit.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(TARGET_H) $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) \
+ $(EXPR_H) $(BASIC_BLOCK_H) toplev.h $(TM_P_H) $(PARAMS_H) \
+ $(IRA_INT_H)
+ira.o: ira.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(TARGET_H) $(TM_H) $(RTL_H) $(RECOG_H) \
+ $(REGS_H) hard-reg-set.h $(FLAGS_H) $(OBSTACK_H) \
+ $(EXPR_H) $(BASIC_BLOCK_H) toplev.h $(TM_P_H) \
+ $(DF_H) $(IRA_INT_H) $(PARAMS_H) $(TIMEVAR_H) $(INTEGRATE_H) \
+ tree-pass.h output.h
regmove.o : regmove.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
insn-config.h $(TIMEVAR_H) tree-pass.h \
$(RECOG_H) output.h $(REGS_H) hard-reg-set.h $(FLAGS_H) $(FUNCTION_H) \
@@ -2649,6 +2678,8 @@ hooks.o: hooks.c $(CONFIG_H) $(SYSTEM_H)
pretty-print.o: $(CONFIG_H) $(SYSTEM_H) coretypes.h intl.h $(PRETTY_PRINT_H) \
$(TREE_H)
errors.o : errors.c $(CONFIG_H) $(SYSTEM_H) errors.h $(BCONFIG_H)
+lower-subreg.o : lower-subreg.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(MACHMODE_H) $(RTL_H) bitmap.h $(TM_H)
$(out_object_file): $(out_file) $(CONFIG_H) coretypes.h $(TM_H) $(TREE_H) \
$(RTL_H) $(REGS_H) hard-reg-set.h insn-config.h conditions.h \
Index: passes.c
===================================================================
--- passes.c (revision 119879)
+++ passes.c (working copy)
@@ -634,6 +634,7 @@ init_optimization_passes (void)
NEXT_PASS (pass_unshare_all_rtl);
NEXT_PASS (pass_instantiate_virtual_regs);
NEXT_PASS (pass_jump2);
+ NEXT_PASS (pass_lower_subreg);
NEXT_PASS (pass_cse);
NEXT_PASS (pass_rtl_fwprop);
NEXT_PASS (pass_gcse);
@@ -660,6 +661,7 @@ init_optimization_passes (void)
NEXT_PASS (pass_sched);
NEXT_PASS (pass_local_alloc);
NEXT_PASS (pass_global_alloc);
+ NEXT_PASS (pass_ira);
NEXT_PASS (pass_postreload);
*p = NULL;
Index: config/sparc/sparc.h
===================================================================
--- config/sparc/sparc.h (revision 119879)
+++ config/sparc/sparc.h (working copy)
@@ -1061,6 +1061,19 @@ enum reg_class { NO_REGS, FPCC_REGS, I64
{-1, -1, -1, 0x20}, /* GENERAL_OR_EXTRA_FP_REGS */ \
{-1, -1, -1, 0x3f}} /* ALL_REGS */
+/* The following macro defines cover classes for Integrated Register
+ Allocator. Cover classes is a set of non-intersected register
+ classes covering all hard registers used for register allocation
+ purpose. Any move between two registers of a cover class should be
+ cheaper than load or store of the registers. The macro value is
+ array of register classes with LIM_REG_CLASSES used as the end
+ marker. */
+
+#define IRA_COVER_CLASSES \
+{ \
+ GENERAL_REGS, EXTRA_FP_REGS, FPCC_REGS, LIM_REG_CLASSES \
+}
+
/* Defines invalid mode changes. Borrowed from pa64-regs.h.
SImode loads to floating-point registers are not zero-extended.
Index: config/i386/i386.h
===================================================================
--- config/i386/i386.h (revision 119879)
+++ config/i386/i386.h (working copy)
@@ -1217,6 +1217,19 @@ enum reg_class
{ 0xffffffff,0x1fffff } \
}
+/* The following macro defines cover classes for Integrated Register
+ Allocator. Cover classes is a set of non-intersected register
+ classes covering all hard registers used for register allocation
+ purpose. Any move between two registers of a cover class should be
+ cheaper than load or store of the registers. The macro value is
+ array of register classes with LIM_REG_CLASSES used as the end
+ marker. */
+
+#define IRA_COVER_CLASSES \
+{ \
+ GENERAL_REGS, FLOAT_REGS, MMX_REGS, SSE_REGS, LIM_REG_CLASSES \
+}
+
/* The same information, inverted:
Return the class number of the smallest class containing
reg number REGNO. This could be a conditional expression
Index: config/ia64/ia64.h
===================================================================
--- config/ia64/ia64.h (revision 119879)
+++ config/ia64/ia64.h (working copy)
@@ -797,6 +797,19 @@ enum reg_class
0xFFFFFFFF, 0xFFFFFFFF, 0x3FFF }, \
}
+/* The following macro defines cover classes for Integrated Register
+ Allocator. Cover classes is a set of non-intersected register
+ classes covering all hard registers used for register allocation
+ purpose. Any move between two registers of a cover class should be
+ cheaper than load or store of the registers. The macro value is
+ array of register classes with LIM_REG_CLASSES used as the end
+ marker. */
+
+#define IRA_COVER_CLASSES \
+{ \
+ PR_REGS, BR_REGS, AR_M_REGS, AR_I_REGS, GR_REGS, FR_REGS, LIM_REG_CLASSES \
+}
+
/* A C expression whose value is a register class containing hard register
REGNO. In general there is more than one such class; choose a class which
is "minimal", meaning that no smaller class also contains the register. */
Index: config/rs6000/rs6000.h
===================================================================
--- config/rs6000/rs6000.h (revision 119879)
+++ config/rs6000/rs6000.h (working copy)
@@ -1061,6 +1061,22 @@ enum reg_class
{ 0xffffffff, 0xffffffff, 0xffffffff, 0x0003ffff } /* ALL_REGS */ \
}
+/* The following macro defines cover classes for Integrated Register
+ Allocator. Cover classes is a set of non-intersected register
+ classes covering all hard registers used for register allocation
+ purpose. Any move between two registers of a cover class should be
+ cheaper than load or store of the registers. The macro value is
+ array of register classes with LIM_REG_CLASSES used as the end
+ marker. */
+
+#define IRA_COVER_CLASSES \
+{ \
+ SPEC_OR_GEN_REGS /* GENERAL_REGS */, FLOAT_REGS, ALTIVEC_REGS, \
+ /*VRSAVE_REGS,*/ VSCR_REGS, SPE_ACC_REGS, SPEFSCR_REGS, \
+ /* MQ_REGS, LINK_REGS, CTR_REGS, */ \
+ CR_REGS, XER_REGS, LIM_REG_CLASSES \
+}
+
/* The same information, inverted:
Return the class number of the smallest class containing
reg number REGNO. This could be a conditional expression
Index: reload1.c
===================================================================
--- reload1.c (revision 119879)
+++ reload1.c (working copy)
@@ -45,6 +45,8 @@ Software Foundation, 51 Franklin Street,
#include "toplev.h"
#include "except.h"
#include "tree.h"
+#include "ira.h"
+#include "params.h"
#include "target.h"
/* This file contains the reload pass of the compiler, which is
@@ -388,7 +390,7 @@ static void delete_caller_save_insns (vo
static void spill_failure (rtx, enum reg_class);
static void count_spilled_pseudo (int, int, int);
static void delete_dead_insn (rtx);
-static void alter_reg (int, int);
+static void alter_reg (int, int, bool);
static void set_label_offsets (rtx, rtx, int);
static void check_eliminable_occurrences (rtx);
static void elimination_effects (rtx, enum machine_mode);
@@ -633,6 +635,20 @@ static int something_needs_operands_chan
/* Nonzero means we couldn't get enough spill regs. */
static int failure;
+/* The function is used to sort pseudos according their usage
+ frequencies (putting most frequently ones first). */
+static int
+pseudo_reg_compare (const void *v1p, const void *v2p)
+{
+ int regno1 = *(int *) v1p;
+ int regno2 = *(int *) v2p;
+ int diff;
+
+ if ((diff = REG_FREQ (regno2) - REG_FREQ (regno1)) != 0)
+ return diff;
+ return regno1 - regno2;
+}
+
/* Main entry point for the reload pass.
FIRST is the first insn of the function being compiled.
@@ -827,12 +843,27 @@ reload (rtx first, int global)
offsets_known_at = XNEWVEC (char, num_labels);
offsets_at = (HOST_WIDE_INT (*)[NUM_ELIMINABLE_REGS]) xmalloc (num_labels * NUM_ELIMINABLE_REGS * sizeof (HOST_WIDE_INT));
- /* Alter each pseudo-reg rtx to contain its hard reg number.
- Assign stack slots to the pseudos that lack hard regs or equivalents.
- Do not touch virtual registers. */
+ {
+ int n, *pseudo_regs;
- for (i = LAST_VIRTUAL_REGISTER + 1; i < max_regno; i++)
- alter_reg (i, -1);
+ /* Alter each pseudo-reg rtx to contain its hard reg number.
+ Assign stack slots to the pseudos that lack hard regs or
+ equivalents. Do not touch virtual registers. */
+
+ pseudo_regs = XNEWVEC (int, max_regno - LAST_VIRTUAL_REGISTER - 1);
+ for (n = 0, i = LAST_VIRTUAL_REGISTER + 1; i < max_regno; i++)
+ pseudo_regs [n++] = i;
+
+ if (flag_ira)
+ qsort (pseudo_regs, n, sizeof (int), pseudo_reg_compare);
+ if (frame_pointer_needed || ! flag_ira)
+ for (i = 0; i < n; i++)
+ alter_reg (pseudo_regs [i], -1, false);
+ else
+ for (i = n - 1; i >= 0; i--)
+ alter_reg (pseudo_regs [i], -1, false);
+ free (pseudo_regs);
+ }
/* If we have some registers we think can be eliminated, scan all insns to
see if there is an insn that sets one of these registers to something
@@ -954,7 +985,7 @@ reload (rtx first, int global)
the loop. */
reg_equiv_memory_loc[i] = 0;
reg_equiv_init[i] = 0;
- alter_reg (i, -1);
+ alter_reg (i, -1, true);
}
}
@@ -1689,6 +1720,7 @@ static HARD_REG_SET used_spill_regs_loca
static void
count_spilled_pseudo (int spilled, int spilled_nregs, int reg)
{
+ int freq = REG_FREQ (reg);
int r = reg_renumber[reg];
int nregs = hard_regno_nregs[r][PSEUDO_REGNO_MODE (reg)];
@@ -1698,9 +1730,9 @@ count_spilled_pseudo (int spilled, int s
SET_REGNO_REG_SET (&spilled_pseudos, reg);
- spill_add_cost[r] -= REG_FREQ (reg);
+ spill_add_cost[r] -= freq;
while (nregs-- > 0)
- spill_cost[r + nregs] -= REG_FREQ (reg);
+ spill_cost[r + nregs] -= freq;
}
/* Find reload register to use for reload number ORDER. */
@@ -1969,7 +2001,7 @@ delete_dead_insn (rtx insn)
can share one stack slot. */
static void
-alter_reg (int i, int from_reg)
+alter_reg (int i, int from_reg, bool dont_share_p)
{
/* When outputting an inline function, this can happen
for a reg that isn't actually used. */
@@ -2002,7 +2034,13 @@ alter_reg (int i, int from_reg)
unsigned int total_size = MAX (inherent_size, reg_max_ref_width[i]);
unsigned int min_align = reg_max_ref_width[i] * BITS_PER_UNIT;
int adjust = 0;
+ bool shared_p = false;
+
+ x = (dont_share_p || ! flag_ira
+ ? NULL_RTX : reuse_stack_slot (i, inherent_size, total_size));
+ if (x)
+ shared_p = true;
/* Each pseudo reg has an inherent size which comes from its own mode,
and a total size which provides room for paradoxical subregs
which refer to the pseudo reg in wider modes.
@@ -2011,7 +2049,7 @@ alter_reg (int i, int from_reg)
enough inherent space and enough total space.
Otherwise, we allocate a new slot, making sure that it has no less
inherent space, and no less total space, then the previous slot. */
- if (from_reg == -1)
+ else if (from_reg == -1 || (! dont_share_p && flag_ira))
{
/* No known place to spill from => no slot to reuse. */
x = assign_stack_local (mode, total_size,
@@ -2026,6 +2064,9 @@ alter_reg (int i, int from_reg)
/* Nothing can alias this slot except this pseudo. */
set_mem_alias_set (x, new_alias_set ());
+
+ if (! dont_share_p && flag_ira)
+ mark_new_stack_slot (x, i, total_size);
}
/* Reuse a stack slot if possible. */
@@ -2096,8 +2137,13 @@ alter_reg (int i, int from_reg)
/* If we have a decl for the original register, set it for the
memory. If this is a shared MEM, make a copy. */
- if (REG_EXPR (regno_reg_rtx[i])
- && DECL_P (REG_EXPR (regno_reg_rtx[i])))
+ if (shared_p)
+ {
+ x = copy_rtx (x);
+ set_mem_attrs_from_reg (x, regno_reg_rtx[i]);
+ }
+ else if (REG_EXPR (regno_reg_rtx[i])
+ && DECL_P (REG_EXPR (regno_reg_rtx[i])))
{
rtx decl = DECL_RTL_IF_SET (REG_EXPR (regno_reg_rtx[i]));
@@ -2361,7 +2407,7 @@ eliminate_regs_1 (rtx x, enum machine_mo
/* There exists at least one use of REGNO that cannot be
eliminated. Prevent the defining insn from being deleted. */
reg_equiv_init[regno] = NULL_RTX;
- alter_reg (regno, -1);
+ alter_reg (regno, -1, true);
}
return x;
@@ -3710,6 +3756,8 @@ finish_spills (int global)
SET_HARD_REG_BIT (pseudo_previous_regs[i], reg_renumber[i]);
/* Mark it as no longer having a hard register home. */
reg_renumber[i] = -1;
+ if (flag_ira)
+ mark_allocation_change (i);
/* We will need to scan everything again. */
something_changed = 1;
}
@@ -3746,10 +3794,23 @@ finish_spills (int global)
if (reg_old_renumber[i] != reg_renumber[i])
{
HARD_REG_SET forbidden;
+
COPY_HARD_REG_SET (forbidden, bad_spill_regs_global);
IOR_HARD_REG_SET (forbidden, pseudo_forbidden_regs[i]);
IOR_HARD_REG_SET (forbidden, pseudo_previous_regs[i]);
- retry_global_alloc (i, forbidden);
+ if (flag_ira)
+ {
+ /* We might migrate pseudo to another hard register on
+ previous iteration. So check this. */
+ if (reg_renumber [i] < 0)
+ {
+ retry_ira_color (i, forbidden);
+ if (reg_renumber[i] >= 0)
+ something_changed = 1;
+ }
+ }
+ else
+ retry_global_alloc (i, forbidden);
if (reg_renumber[i] >= 0)
CLEAR_REGNO_REG_SET (&spilled_pseudos, i);
}
@@ -3796,7 +3857,7 @@ finish_spills (int global)
if (reg_old_renumber[i] == regno)
continue;
- alter_reg (i, reg_old_renumber[i]);
+ alter_reg (i, reg_old_renumber[i], false);
reg_old_renumber[i] = regno;
if (dump_file)
{
@@ -6610,7 +6671,9 @@ emit_input_reload_insns (struct insn_cha
&& REG_N_SETS (REGNO (old)) == 1)
{
reg_renumber[REGNO (old)] = REGNO (rl->reg_rtx);
- alter_reg (REGNO (old), -1);
+ if (flag_ira)
+ mark_allocation_change (REGNO (old));
+ alter_reg (REGNO (old), -1, false);
}
special = 1;
}
@@ -8081,7 +8144,9 @@ delete_output_reload (rtx insn, int j, i
/* For the debugging info, say the pseudo lives in this reload reg. */
reg_renumber[REGNO (reg)] = REGNO (rld[j].reg_rtx);
- alter_reg (REGNO (reg), -1);
+ if (flag_ira)
+ mark_allocation_change (REGNO (reg));
+ alter_reg (REGNO (reg), -1, false);
}
else
{
/* Integrated Register Allocator entry point.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
/* The integrated register allocator (IRA) is called integrated
because register coalescing and register live range splitting are
done on-the-fly during coloring. Register coalescing is done by
hard register preferencing during hard register assigning. The
live range splitting is a byproduct of the regional register
allocation.
The regional allocation is top-down process. The first we do
allocation for all function then we improve it for loops then their
subloops and so on. To reduce register shuffling, the same
mechanism of hard register prefrencing is used. This approach
works as good as Callahan-Koblentz algorithm but it is simpler.
We use Chaitin-Briggs coloring for each loop (or function) with
optional biased coloring. If pseudo-registers got different
location on loop borders we rename them inside the loop and
generate pseudo-register move insns. Some optimizations (like
removing redundant stores, moving register shuffling to less
frequent points, and code duplication reducing) to minimize effect
of register shuffling is done
If we don't improve register allocation for loops we get classic
Chaitin-Briggs coloring (only instead of separate pass of
coalescing, we use hard register preferencing).
Optionally we implements Chow's priority coloring only for all
function. It is quite analogous to the current gcc global register
allocator only we use more sophisticated hard register
preferencing.
Literature is worth to read for better understanding the code:
o Preston Briggs, Keith D. Cooper, Linda Torczon. Improvements to
Graph Coloring Register Allocation.
o David Callahan, Brian Koblenz. Register allocation via
hierarchical graph coloring
o Keith Cooper, Anshuman Dasgupta, Jason Eckhardt. Revisiting Graph
Coloring Register Allocation: A Study of the Chaitin-Briggs and
Callahan-Koblenz Algorithms.
o Guei-Yuan Lueh, Thomas Gross, and Ali-Reza Adl-Tabatabai. Global
Register Allocation Based on Graph Fusion.
*/
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "regs.h"
#include "rtl.h"
#include "tm_p.h"
#include "target.h"
#include "flags.h"
#include "obstack.h"
#include "bitmap.h"
#include "hard-reg-set.h"
#include "basic-block.h"
#include "expr.h"
#include "recog.h"
#include "params.h"
#include "timevar.h"
#include "tree-pass.h"
#include "output.h"
#include "reload.h"
#include "errors.h"
#include "integrate.h"
#include "df.h"
#include "ggc.h"
#include "ira-int.h"
static void setup_inner_mode (void);
static void setup_reg_mode_hard_regset (void);
static void setup_class_subset_and_move_costs (void);
static void setup_class_hard_regs (void);
static void setup_available_class_regs (void);
static void setup_alloc_regs (int);
static void setup_reg_subclasses (void);
static void setup_cover_classes (void);
static void setup_class_translate (void);
static void print_class_cover (FILE *);
static void find_reg_class_closure (void);
static void setup_reg_class_nregs (void);
static void setup_prohibited_class_mode_regs (void);
static void setup_eliminable_regset (void);
static void find_reg_equiv_invariant_const (void);
static void setup_reg_renumber (void);
static void calculate_allocation_cost (void);
#ifdef ENABLE_IRA_CHECKING
static void check_allocation (void);
#endif
static void fix_reg_equiv_init (void);
#ifdef ENABLE_IRA_CHECKING
static void print_redundant_copies (void);
#endif
static bool gate_ira (void);
static unsigned int rest_of_handle_ira (void);
/* Dump file for IRA. */
FILE *ira_dump_file;
/* The number of elements in the following array. */
int spilled_reg_stack_slots_num;
/* The following array contains description of spilled registers stack
slots have been used in the current function so far. */
struct spilled_reg_stack_slot *spilled_reg_stack_slots;
/* The following variable values are correspondingly overall cost of
the allocation, cost of hard register usage for the pseudos, cost
of memory usage for the pseudos, cost of loads, stores and register
move insns generated for register live range splitting. */
int overall_cost;
int reg_cost, mem_cost;
int load_cost, store_cost, shuffle_cost;
int move_loops_num, additional_jumps_num;
/* A mode whose value is immediately contained in given mode
value. */
unsigned char mode_inner_mode [NUM_MACHINE_MODES];
/* The following array is a map hard regs X modes -> number registers
for store value of given mode starting with given hard
register. */
HARD_REG_SET reg_mode_hard_regset [FIRST_PSEUDO_REGISTER] [NUM_MACHINE_MODES];
/* The following two variables are array analog of macros
MEMORY_MOVE_COST and REGISTER_MOVE_COST. */
int memory_move_cost [MAX_MACHINE_MODE] [N_REG_CLASSES] [2];
int register_move_cost [MAX_MACHINE_MODE] [N_REG_CLASSES] [N_REG_CLASSES];
/* Nonzero value of element of the following array means that the
1st class is a subset of the 2nd class. */
int class_subset_p [N_REG_CLASSES] [N_REG_CLASSES];
/* Temporary hard reg set used for different calculation. */
static HARD_REG_SET temp_hard_regset;
/* The function sets up mode_inner_mode array. */
static void
setup_inner_mode (void)
{
int i;
enum machine_mode wider;
for (i = 0; i < NUM_MACHINE_MODES; i++)
mode_inner_mode [i] = VOIDmode;
for (i = 0; i < NUM_MACHINE_MODES; i++)
{
wider = GET_MODE_WIDER_MODE (i);
if (wider != VOIDmode)
{
ira_assert (mode_inner_mode [wider] == VOIDmode);
mode_inner_mode [wider] = i;
}
}
}
/* The function sets up map REG_MODE_HARD_REGSET. */
static void
setup_reg_mode_hard_regset (void)
{
int i, m, hard_regno;
for (m = 0; m < NUM_MACHINE_MODES; m++)
for (hard_regno = 0; hard_regno < FIRST_PSEUDO_REGISTER; hard_regno++)
{
CLEAR_HARD_REG_SET (reg_mode_hard_regset [hard_regno] [m]);
for (i = hard_regno_nregs [hard_regno] [m] - 1; i >= 0; i--)
if (hard_regno + i < FIRST_PSEUDO_REGISTER)
SET_HARD_REG_BIT (reg_mode_hard_regset [hard_regno] [m],
hard_regno + i);
}
}
/* The function sets up MEMORY_MOVE_COST, REGISTER_MOVE_COST and
CLASS_SUBSET_P. */
static void
setup_class_subset_and_move_costs (void)
{
int cl, cl2;
enum machine_mode mode;
for (cl = (int) N_REG_CLASSES - 1; cl >= 0; cl--)
{
for (mode = 0; mode < MAX_MACHINE_MODE; mode++)
{
memory_move_cost [mode] [cl] [0] = MEMORY_MOVE_COST (mode, cl, 0);
memory_move_cost [mode] [cl] [1] = MEMORY_MOVE_COST (mode, cl, 1);
}
for (cl2 = (int) N_REG_CLASSES - 1; cl2 >= 0; cl2--)
{
if (cl != NO_REGS && cl2 != NO_REGS)
for (mode = 0; mode < MAX_MACHINE_MODE; mode++)
register_move_cost [mode] [cl] [cl2]
= REGISTER_MOVE_COST (mode, cl, cl2);
GO_IF_HARD_REG_SUBSET (reg_class_contents[cl],
reg_class_contents[cl2],
subset);
class_subset_p [cl] [cl2] = FALSE;
continue;
subset:
class_subset_p [cl] [cl2] = TRUE;
}
}
}
/* Hard registers which can be used for the allocation of given
register class. The order is defined by the allocation order. */
short class_hard_regs [N_REG_CLASSES] [FIRST_PSEUDO_REGISTER];
/* The size of the above array for given register class. */
int class_hard_regs_num [N_REG_CLASSES];
/* Index (in class_hard_regs) for given register class and hard
register (in general case a hard register can belong to several
register classes). */
short class_hard_reg_index [N_REG_CLASSES] [FIRST_PSEUDO_REGISTER];
/* The function sets up the three arrays declared above. */
static void
setup_class_hard_regs (void)
{
int cl, i, hard_regno, n;
HARD_REG_SET processed_hard_reg_set;
ira_assert (SHRT_MAX >= FIRST_PSEUDO_REGISTER);
/* We could call ORDER_REGS_FOR_LOCAL_ALLOC here (it is usually
putting hard callee-used hard registers first). But our
heuristics work better. */
for (cl = (int) N_REG_CLASSES - 1; cl >= 0; cl--)
{
COPY_HARD_REG_SET (temp_hard_regset, reg_class_contents [cl]);
AND_COMPL_HARD_REG_SET (temp_hard_regset, no_alloc_regs);
CLEAR_HARD_REG_SET (processed_hard_reg_set);
for (n = 0, i = 0; i < FIRST_PSEUDO_REGISTER; i++)
{
#ifdef REG_ALLOC_ORDER
hard_regno = reg_alloc_order [i];
#else
hard_regno = i;
#endif
if (TEST_HARD_REG_BIT (processed_hard_reg_set, hard_regno))
continue;
SET_HARD_REG_BIT (processed_hard_reg_set, hard_regno);
if (! TEST_HARD_REG_BIT (temp_hard_regset, hard_regno))
class_hard_reg_index [cl] [hard_regno] = -1;
else
{
class_hard_reg_index [cl] [hard_regno] = n;
class_hard_regs [cl] [n++] = hard_regno;
}
}
class_hard_regs_num [cl] = n;
}
}
/* Number of class hard registers available for the register
allocation for given classes. */
int available_class_regs [N_REG_CLASSES];
/* Function setting up AVAILABLE_CLASS_REGS. */
static void
setup_available_class_regs (void)
{
int i, j;
memset (available_class_regs, 0, sizeof (available_class_regs));
for (i = 0; i < N_REG_CLASSES; i++)
{
COPY_HARD_REG_SET (temp_hard_regset, reg_class_contents [i]);
AND_COMPL_HARD_REG_SET (temp_hard_regset, no_alloc_regs);
for (j = 0; j < FIRST_PSEUDO_REGISTER; j++)
if (TEST_HARD_REG_BIT (temp_hard_regset, j))
available_class_regs [i]++;
}
}
/* Hard registers can not be used for the register allocator. */
HARD_REG_SET no_alloc_regs;
/* The function setting up different global variables defining hard
registers for the allocation. It depends on USE_HARD_FRAME_P whose
nonzero value means that we can use hard frame pointer for the
allocation. */
static void
setup_alloc_regs (int use_hard_frame_p)
{
COPY_HARD_REG_SET (no_alloc_regs, fixed_reg_set);
if (! use_hard_frame_p)
SET_HARD_REG_BIT (no_alloc_regs, HARD_FRAME_POINTER_REGNUM);
setup_class_hard_regs ();
setup_available_class_regs ();
}
/* Define the following macro if allocation through malloc if
preferable. */
/*#define IRA_NO_OBSTACK*/
#ifndef IRA_NO_OBSTACK
/* Obstack used for storing all dynamic data (except bitmaps) of the
IRA. */
static struct obstack ira_obstack;
#endif
/* Obstack used for storing all bitmaps of the IRA. */
static struct bitmap_obstack ira_bitmap_obstack;
/* The function allocates memory of size LEN for IRA data. */
void *
ira_allocate (size_t len)
{
void *res;
#ifndef IRA_NO_OBSTACK
res = obstack_alloc (&ira_obstack, len);
#else
res = xmalloc (len);
#endif
return res;
}
/* The function free memory ADDR allocated for IRA data. */
void
ira_free (void *addr ATTRIBUTE_UNUSED)
{
#ifndef IRA_NO_OBSTACK
/* do nothing */
#else
free (addr);
#endif
}
/* The function allocates bitmap for IRA. */
bitmap
ira_allocate_bitmap (void)
{
return BITMAP_ALLOC (&ira_bitmap_obstack);
}
/* The function frees bitmap B allocated for IRA. */
void
ira_free_bitmap (bitmap b ATTRIBUTE_UNUSED)
{
/* do nothing */
}
/* The function allocates regset for IRA. */
regset
ira_allocate_regset (void)
{
return ALLOC_REG_SET (&ira_bitmap_obstack);
}
/* The function frees regset R allocated for IRA. */
void
ira_free_regset (regset r ATTRIBUTE_UNUSED)
{
/* do nothing */
}
/* The function returns nonzero if hard registers starting with
HARD_REGNO and containing value of MODE are not in set
HARD_REGSET. */
int
hard_reg_not_in_set_p (int hard_regno, enum machine_mode mode,
HARD_REG_SET hard_regset)
{
int i;
ira_assert (hard_regno >= 0);
for (i = hard_regno_nregs [hard_regno] [mode] - 1; i >= 0; i--)
if (TEST_HARD_REG_BIT (hard_regset, hard_regno + i))
return FALSE;
return TRUE;
}
/* The function outputs information about allocation of all pseudos
into file F. */
void
print_disposition (FILE *f)
{
int i, n, max_regno;
pseudo_t p;
basic_block bb;
fprintf (f, "Disposition:");
max_regno = max_reg_num ();
for (n = 0, i = FIRST_PSEUDO_REGISTER; i < max_regno; i++)
for (p = regno_pseudo_map [i]; p != NULL; p = PSEUDO_NEXT_REGNO_PSEUDO (p))
{
if (n % 4 == 0)
fprintf (f, "\n");
n++;
fprintf (f, " %4d:r%-4d", PSEUDO_NUM (p), PSEUDO_REGNO (p));
if ((bb = PSEUDO_LOOP_TREE_NODE (p)->bb) != NULL)
fprintf (f, "b%-3d", bb->index);
else
fprintf (f, "l%-3d", PSEUDO_LOOP_TREE_NODE (p)->loop->num);
if (PSEUDO_HARD_REGNO (p) >= 0)
fprintf (f, " %3d", PSEUDO_HARD_REGNO (p));
else
fprintf (f, " mem");
}
fprintf (f, "\n");
}
/* The function outputs information about allocation of all pseudos
into stderr. */
void
debug_disposition (void)
{
print_disposition (stderr);
}
/* For each reg class, table listing all the classes contained in it
(excluding the class itself. Fixed registers are excluded from the
consideration). */
static enum reg_class alloc_reg_class_subclasses[N_REG_CLASSES][N_REG_CLASSES];
/* The function initializes the tables of subclasses of each reg
class. */
static void
setup_reg_subclasses (void)
{
int i, j;
for (i = 0; i < N_REG_CLASSES; i++)
for (j = 0; j < N_REG_CLASSES; j++)
alloc_reg_class_subclasses [i] [j] = LIM_REG_CLASSES;
for (i = 0; i < N_REG_CLASSES; i++)
{
if (i == (int) NO_REGS)
continue;
COPY_HARD_REG_SET (temp_hard_regset, reg_class_contents [i]);
AND_COMPL_HARD_REG_SET (temp_hard_regset, fixed_reg_set);
GO_IF_HARD_REG_EQUAL (temp_hard_regset, zero_hard_reg_set, cont);
goto ok;
cont:
continue;
ok:
for (j = 0; j < N_REG_CLASSES; j++)
if (i != j)
{
enum reg_class *p;
GO_IF_HARD_REG_SUBSET (reg_class_contents [i],
reg_class_contents [j], subclass);
continue;
subclass:
p = &alloc_reg_class_subclasses [j] [0];
while (*p != LIM_REG_CLASSES) p++;
*p = (enum reg_class) i;
}
}
}
/* The value is size of the subsequent array. */
int reg_class_cover_size;
/* The array containing cover classes whose hard registers are used
for the allocation -- see also comments for macro
IRA_COVER_CLASSES. */
enum reg_class reg_class_cover [N_REG_CLASSES];
/* The function checks IRA_COVER_CLASSES and sets the two global
variables defined above. */
static void
setup_cover_classes (void)
{
int i, j;
enum reg_class cl;
static enum reg_class classes [] = IRA_COVER_CLASSES;
reg_class_cover_size = 0;
for (i = 0; (cl = classes [i]) != LIM_REG_CLASSES; i++)
{
for (j = 0; j < i; j++)
if (reg_classes_intersect_p (cl, classes [j]))
gcc_unreachable ();
COPY_HARD_REG_SET (temp_hard_regset, reg_class_contents [cl]);
AND_COMPL_HARD_REG_SET (temp_hard_regset, fixed_reg_set);
GO_IF_HARD_REG_EQUAL (temp_hard_regset, zero_hard_reg_set, cont);
reg_class_cover [reg_class_cover_size++] = cl;
cont:
;
}
}
/* Map of register classes to corresponding cover class containing the
given class. If given class is not a subset of a cover class, we
translate it into the cheapest cover class. */
enum reg_class class_translate [N_REG_CLASSES];
/* The function sets up array CLASS_TRANSLATE. */
static void
setup_class_translate (void)
{
enum reg_class cl, cover_class, best_class, *cl_ptr;
enum machine_mode mode;
int i, cost, min_cost, best_cost;
for (cl = 0; cl < N_REG_CLASSES; cl++)
class_translate [cl] = NO_REGS;
for (i = 0; i < reg_class_cover_size; i++)
{
cover_class = reg_class_cover [i];
for (cl_ptr = &alloc_reg_class_subclasses [cover_class] [0];
(cl = *cl_ptr) != LIM_REG_CLASSES;
cl_ptr++)
{
if (class_translate [cl] == NO_REGS)
class_translate [cl] = cover_class;
#ifdef ENABLE_IRA_CHECKING
else
{
COPY_HARD_REG_SET (temp_hard_regset, reg_class_contents [cl]);
AND_COMPL_HARD_REG_SET (temp_hard_regset, fixed_reg_set);
GO_IF_HARD_REG_SUBSET (temp_hard_regset, zero_hard_reg_set, ok);
gcc_unreachable ();
ok:
;
}
#endif
}
class_translate [cover_class] = cover_class;
}
/* For classes which are not fully covered by a cover class (in
other words covered by more one cover class), use the cheapest
cover class. */
for (cl = 0; cl < N_REG_CLASSES; cl++)
{
if (cl == NO_REGS || class_translate [cl] != NO_REGS)
continue;
best_class = NO_REGS;
best_cost = INT_MAX;
for (i = 0; i < reg_class_cover_size; i++)
{
cover_class = reg_class_cover [i];
COPY_HARD_REG_SET (temp_hard_regset,
reg_class_contents [cover_class]);
AND_HARD_REG_SET (temp_hard_regset, reg_class_contents [cl]);
GO_IF_HARD_REG_EQUAL (temp_hard_regset, zero_hard_reg_set,
no_intersection);
min_cost = INT_MAX;
for (mode = 0; mode < MAX_MACHINE_MODE; mode++)
{
cost = (memory_move_cost [mode] [cl] [0]
+ memory_move_cost [mode] [cl] [1]);
if (min_cost > cost)
min_cost = cost;
}
if (best_class == NO_REGS || best_cost > min_cost)
{
best_class = cover_class;
best_cost = min_cost;
}
no_intersection:
;
}
class_translate [cl] = best_class;
}
}
/* The function outputs all cover classes and the translation map into
file F. */
static void
print_class_cover (FILE *f)
{
static const char *const reg_class_names[] = REG_CLASS_NAMES;
int i;
fprintf (f, "Class cover:\n");
for (i = 0; i < reg_class_cover_size; i++)
fprintf (f, " %s", reg_class_names [reg_class_cover [i]]);
fprintf (f, "\nClass translation:\n");
for (i = 0; i < N_REG_CLASSES; i++)
fprintf (f, " %s -> %s\n", reg_class_names [i],
reg_class_names [class_translate [i]]);
}
/* The function outputs all cover classes and the translation map into
stderr. */
void
debug_class_cover (void)
{
print_class_cover (stderr);
}
/* Function setting up different arrays concerning class subsets and
cover classes. */
static void
find_reg_class_closure (void)
{
setup_reg_subclasses ();
setup_cover_classes ();
setup_class_translate ();
}
/* Map: register class x machine mode -> number of hard registers of
given class needed to store value of given mode. If the number is
different, the size will be negative. */
int reg_class_nregs [N_REG_CLASSES] [MAX_MACHINE_MODE];
/* Maximal value of the previous array elements. */
int max_nregs;
/* Function forming REG_CLASS_NREGS map. */
static void
setup_reg_class_nregs (void)
{
int m;
enum reg_class cl;
max_nregs = -1;
for (cl = 0; cl < N_REG_CLASSES; cl++)
for (m = 0; m < MAX_MACHINE_MODE; m++)
{
reg_class_nregs [cl] [m] = CLASS_MAX_NREGS (cl, m);
if (max_nregs < reg_class_nregs [cl] [m])
max_nregs = reg_class_nregs [cl] [m];
}
}
/* Array whose values are hard regset of hard registers of given
register class whose HARD_REGNO_MODE_OK values are zero. */
HARD_REG_SET prohibited_class_mode_regs [N_REG_CLASSES] [NUM_MACHINE_MODES];
/* The function setting up PROHIBITED_CLASS_MODE_REGS. */
static void
setup_prohibited_class_mode_regs (void)
{
int i, j, k, hard_regno;
enum reg_class cl;
for (i = 0; i < reg_class_cover_size; i++)
{
cl = reg_class_cover [i];
for (j = 0; j < NUM_MACHINE_MODES; j++)
{
CLEAR_HARD_REG_SET (prohibited_class_mode_regs [cl] [j]);
for (k = class_hard_regs_num [cl] - 1; k >= 0; k--)
{
hard_regno = class_hard_regs [cl] [k];
if (! HARD_REGNO_MODE_OK (hard_regno, j))
SET_HARD_REG_BIT (prohibited_class_mode_regs [cl] [j],
hard_regno);
}
}
}
}
/* Hard regsets whose all bits are correspondingly zero or one. */
HARD_REG_SET zero_hard_reg_set;
HARD_REG_SET one_hard_reg_set;
/* Function called once during compiler work. It sets up different
arrays whose values don't depend on the compiled function. */
void
init_ira_once (void)
{
CLEAR_HARD_REG_SET (zero_hard_reg_set);
SET_HARD_REG_SET (one_hard_reg_set);
setup_inner_mode ();
setup_reg_mode_hard_regset ();
setup_class_subset_and_move_costs ();
setup_alloc_regs (flag_omit_frame_pointer != 0);
find_reg_class_closure ();
setup_reg_class_nregs ();
setup_prohibited_class_mode_regs ();
init_ira_costs_once ();
}
/* The function sets up ELIMINABLE_REGSET and REGS_EVER_LIVE. */
static void
setup_eliminable_regset (void)
{
int i;
#ifdef ELIMINABLE_REGS
static const struct {const int from, to; } eliminables[] = ELIMINABLE_REGS;
#endif
int need_fp
= (! flag_omit_frame_pointer
|| (current_function_calls_alloca && EXIT_IGNORE_STACK)
|| FRAME_POINTER_REQUIRED);
CLEAR_HARD_REG_SET (eliminable_regset);
/* Build the regset of all eliminable registers and show we can't
use those that we already know won't be eliminated. */
#ifdef ELIMINABLE_REGS
for (i = 0; i < (int) ARRAY_SIZE (eliminables); i++)
{
bool cannot_elim
= (! CAN_ELIMINATE (eliminables [i].from, eliminables [i].to)
|| (eliminables [i].to == STACK_POINTER_REGNUM && need_fp));
if (! regs_asm_clobbered [eliminables [i].from])
{
if (! cannot_elim)
SET_HARD_REG_BIT (eliminable_regset, eliminables [i].from);
}
else if (cannot_elim)
error ("%s cannot be used in asm here",
reg_names [eliminables [i].from]);
else
regs_ever_live [eliminables [i].from] = 1;
}
#if FRAME_POINTER_REGNUM != HARD_FRAME_POINTER_REGNUM
if (! regs_asm_clobbered [HARD_FRAME_POINTER_REGNUM])
{
if (! need_fp)
SET_HARD_REG_BIT (eliminable_regset, HARD_FRAME_POINTER_REGNUM);
}
else if (need_fp)
error ("%s cannot be used in asm here",
reg_names [HARD_FRAME_POINTER_REGNUM]);
else
regs_ever_live [HARD_FRAME_POINTER_REGNUM] = 1;
#endif
#else
if (! regs_asm_clobbered [FRAME_POINTER_REGNUM])
{
if (! need_fp)
SET_HARD_REG_BIT (eliminable_regset, FRAME_POINTER_REGNUM);
}
else if (need_fp)
error ("%s cannot be used in asm here", reg_names [FRAME_POINTER_REGNUM]);
else
regs_ever_live [FRAME_POINTER_REGNUM] = 1;
#endif
}
/* The element value is nonzero if the corresponding regno value is
invariant. */
int *reg_equiv_invariant_p;
/* The element value is equiv constant or NULL_RTX. */
rtx *reg_equiv_const;
/* The function sets up the two array declaraed above. */
static void
find_reg_equiv_invariant_const (void)
{
int i, invariant_p;
rtx list, insn, note, constant, x;
for (i = FIRST_PSEUDO_REGISTER; i < reg_equiv_init_size; i++)
{
constant = NULL_RTX;
invariant_p = FALSE;
for (list = reg_equiv_init [i]; list != NULL_RTX; list = XEXP (list, 1))
{
insn = XEXP (list, 0);
note = find_reg_note (insn, REG_EQUIV, NULL_RTX);
if (note == NULL_RTX)
continue;
x = XEXP (note, 0);
if (! function_invariant_p (x)
|| ! flag_pic
/* A function invariant is often CONSTANT_P but may
include a register. We promise to only pass CONSTANT_P
objects to LEGITIMATE_PIC_OPERAND_P. */
|| (CONSTANT_P (x) && LEGITIMATE_PIC_OPERAND_P (x)))
{
/* It can happen that a REG_EQUIV note contains a MEM that
is not a legitimate memory operand. As later stages of
reload assume that all addresses found in the
reg_equiv_* arrays were originally legitimate, we
ignore such REG_EQUIV notes. */
if (memory_operand (x, VOIDmode))
continue;
else if (function_invariant_p (x))
{
if (GET_CODE (x) == PLUS
|| x == frame_pointer_rtx || x == arg_pointer_rtx)
invariant_p = TRUE;
else
constant = x;
}
}
}
reg_equiv_invariant_p [i] = invariant_p;
reg_equiv_const [i] = constant;
}
}
/* The function sets up REG_RENUMBER and CALLER_SAVE_NEEDED used by
reload from the allocation found by IRA. */
static void
setup_reg_renumber (void)
{
int i, hard_regno;
pseudo_t p;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
if (PSEUDO_REGNO (p) < 0)
/* It is a cap. */
continue;
hard_regno = PSEUDO_HARD_REGNO (p);
reg_renumber [REGNO (PSEUDO_REG (p))]
= (hard_regno < 0 ? -1 : hard_regno);
if (hard_regno >= 0 && PSEUDO_CALLS_CROSSED_NUM (p) != 0
&& ! hard_reg_not_in_set_p (hard_regno, PSEUDO_MODE (p),
call_used_reg_set))
{
ira_assert (flag_caller_saves);
caller_save_needed = 1;
}
}
}
/* The function calculates cost of the found register allocation. */
static void
calculate_allocation_cost (void)
{
int i, hard_regno, cost;
pseudo_t p;
overall_cost = reg_cost = mem_cost = 0;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
hard_regno = PSEUDO_HARD_REGNO (p) = reg_renumber [PSEUDO_REGNO (p)];
PSEUDO_ASSIGNED_P (p) = TRUE;
if (hard_regno < 0)
{
cost = PSEUDO_MEMORY_COST (p);
mem_cost += cost;
}
else
{
cost = (PSEUDO_HARD_REG_COSTS (p)
[class_hard_reg_index
[PSEUDO_COVER_CLASS (p)] [hard_regno]]);
reg_cost += cost;
}
overall_cost += cost;
}
if (ira_dump_file != NULL)
{
fprintf (ira_dump_file,
"+++Costs: overall %d, reg %d, mem %d, ld %d, st %d, move %d\n",
overall_cost, reg_cost, mem_cost,
load_cost, store_cost, shuffle_cost);
fprintf (ira_dump_file, "+++ move loops %d, new jumps %d\n",
move_loops_num, additional_jumps_num);
}
}
#ifdef ENABLE_IRA_CHECKING
/* The function checks correctness of the allocation. */
static void
check_allocation (void)
{
pseudo_t p, conflict_p, *pseudo_vec;
int i, hard_regno, conflict_hard_regno, j, nregs, conflict_nregs;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
if (PSEUDO_REGNO (p) < 0 || (hard_regno = PSEUDO_HARD_REGNO (p)) < 0)
continue;
nregs = hard_regno_nregs [hard_regno] [PSEUDO_MODE (p)];
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (p);
for (j = 0; (conflict_p = pseudo_vec [j]) != NULL; j++)
if ((conflict_hard_regno = PSEUDO_HARD_REGNO (conflict_p)) >= 0)
{
conflict_nregs
= (hard_regno_nregs
[conflict_hard_regno] [PSEUDO_MODE (conflict_p)]);
if ((conflict_hard_regno <= hard_regno
&& hard_regno < conflict_hard_regno + conflict_nregs)
|| (hard_regno <= conflict_hard_regno
&& conflict_hard_regno < hard_regno + nregs))
{
fprintf (stderr, "bad allocation for %d and %d\n",
PSEUDO_REGNO (p), PSEUDO_REGNO (conflict_p));
gcc_unreachable ();
}
}
}
}
#endif
/* The function fixes values of array REG_EQUIV_INIT after live range
splitting done by IRA. */
static void
fix_reg_equiv_init (void)
{
int max_regno = max_reg_num ();
int i, new_regno;
rtx x, prev, next, insn, set;
if (reg_equiv_init_size < max_regno)
{
reg_equiv_init = ggc_realloc (reg_equiv_init, max_regno * sizeof (rtx));
while (reg_equiv_init_size < max_regno)
reg_equiv_init [reg_equiv_init_size++] = NULL_RTX;
for (i = FIRST_PSEUDO_REGISTER; i < reg_equiv_init_size; i++)
for (prev = NULL_RTX, x = reg_equiv_init [i]; x != NULL_RTX; x = next)
{
next = XEXP (x, 1);
insn = XEXP (x, 0);
set = single_set (insn);
ira_assert (set != NULL_RTX
&& (REG_P (SET_DEST (set)) || REG_P (SET_SRC (set))));
if (REG_P (SET_DEST (set))
&& ((int) REGNO (SET_DEST (set)) == i
|| (int) ORIGINAL_REGNO (SET_DEST (set)) == i))
new_regno = REGNO (SET_DEST (set));
else if (REG_P (SET_SRC (set))
&& ((int) REGNO (SET_SRC (set)) == i
|| (int) ORIGINAL_REGNO (SET_SRC (set)) == i))
new_regno = REGNO (SET_SRC (set));
else
gcc_unreachable ();
if (new_regno == i)
prev = x;
else
{
if (prev == NULL_RTX)
reg_equiv_init [i] = next;
else
XEXP (prev, 1) = next;
XEXP (x, 1) = reg_equiv_init [new_regno];
reg_equiv_init [new_regno] = x;
}
}
}
}
#ifdef ENABLE_IRA_CHECKING
/* The function prints redundant memory memory copies. */
static void
print_redundant_copies (void)
{
int i, hard_regno;
pseudo_t p;
struct pseudo_copy *cp, *next_cp;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
if (PSEUDO_REGNO (p) < 0)
/* It is a cap. */
continue;
hard_regno = PSEUDO_HARD_REGNO (p);
if (hard_regno >= 0)
continue;
for (cp = PSEUDO_COPIES (p); cp != NULL; cp = next_cp)
if (cp->first == p)
next_cp = cp->next_first_pseudo_copy;
else
{
next_cp = cp->next_second_pseudo_copy;
if (ira_dump_file != NULL && cp->move_insn != NULL_RTX
&& PSEUDO_HARD_REGNO (cp->first) == hard_regno)
fprintf (ira_dump_file, "move %d(freq %d):%d\n",
INSN_UID (cp->move_insn), cp->freq, hard_regno);
}
}
}
#endif
/* This is the main entry of IRA. */
void
ira (FILE *f)
{
int overall_cost_before;
int rebuild_p;
ira_dump_file = f;
no_new_pseudos = 0;
rebuild_p = update_equiv_regs ();
#ifndef IRA_NO_OBSTACK
gcc_obstack_init (&ira_obstack);
#endif
bitmap_obstack_initialize (&ira_bitmap_obstack);
reg_equiv_invariant_p = ira_allocate (max_reg_num () * sizeof (int));
memset (reg_equiv_invariant_p, 0, max_reg_num () * sizeof (int));
reg_equiv_const = ira_allocate (max_reg_num () * sizeof (rtx));
memset (reg_equiv_const, 0, max_reg_num () * sizeof (rtx));
find_reg_equiv_invariant_const ();
if (rebuild_p)
{
rebuild_jump_labels (get_insns ());
purge_all_dead_edges ();
delete_unreachable_blocks ();
}
setup_eliminable_regset ();
overall_cost = reg_cost = mem_cost = 0;
load_cost = store_cost = shuffle_cost = 0;
move_loops_num = additional_jumps_num = 0;
ira_build (flag_ira_algorithm != IRA_ALGORITHM_CB
&& flag_ira_algorithm != IRA_ALGORITHM_PRIORITY);
ira_color ();
ira_emit ();
max_regno = max_reg_num ();
/* Allocate the reg_renumber array. */
allocate_reg_info (max_regno, FALSE, TRUE);
setup_reg_renumber ();
no_new_pseudos = 1;
if (flag_ira_algorithm == IRA_ALGORITHM_REGIONAL)
{
/* Even if new registers are not created rebuild IRA internal
representation to use correct regno pseudo map. */
ira_destroy ();
ira_build (FALSE);
}
calculate_allocation_cost ();
#ifdef ENABLE_IRA_CHECKING
check_allocation ();
#endif
max_regno = max_reg_num ();
delete_trivially_dead_insns (get_insns (), max_regno);
/* The allocator makes live register information inaccurate. */
life_analysis (PROP_DEATH_NOTES | PROP_LOG_LINKS | PROP_REG_INFO);
max_regno = max_reg_num ();
/* Determine if the current function is a leaf before running IRA
since this can impact optimizations done by the prologue and
epilogue thus changing register elimination offsets. */
current_function_is_leaf = leaf_function_p ();
/* Allocate the reg_renumber array. */
allocate_reg_info (max_regno, FALSE, TRUE);
/* And the reg_equiv_memory_loc array. */
VEC_safe_grow (rtx, gc, reg_equiv_memory_loc_vec, max_regno);
memset (VEC_address (rtx, reg_equiv_memory_loc_vec), 0,
sizeof (rtx) * max_regno);
reg_equiv_memory_loc = VEC_address (rtx, reg_equiv_memory_loc_vec);
allocate_initial_values (reg_equiv_memory_loc);
fix_reg_equiv_init ();
/* ??? We need it only because subsequent optimization like
post-reload needs it. */
regclass (get_insns (), max_regno);
#ifdef ENABLE_IRA_CHECKING
print_redundant_copies ();
#endif
overall_cost_before = overall_cost;
spilled_reg_stack_slots_num = 0;
spilled_reg_stack_slots
= ira_allocate (max_regno * sizeof (struct spilled_reg_stack_slot));
build_insn_chain (get_insns ());
reload_completed = ! reload (get_insns (), 1);
ira_free (spilled_reg_stack_slots);
if (ira_dump_file != NULL && overall_cost_before != overall_cost)
fprintf (ira_dump_file, "+++Overall after reload %d\n", overall_cost);
ira_destroy ();
cleanup_cfg (CLEANUP_EXPENSIVE);
ira_free (reg_equiv_invariant_p);
ira_free (reg_equiv_const);
bitmap_obstack_release (&ira_bitmap_obstack);
#ifndef IRA_NO_OBSTACK
obstack_free (&ira_obstack, NULL);
#endif
reload_completed = 1;
}
static bool
gate_ira (void)
{
return flag_ira != 0;
}
/* Run the integrated register allocator. */
static unsigned int
rest_of_handle_ira (void)
{
ira (dump_file);
return 0;
}
struct tree_opt_pass pass_ira =
{
"ira", /* name */
gate_ira, /* gate */
rest_of_handle_ira, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_IRA, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
0, /* todo_flags_start */
TODO_dump_func |
TODO_ggc_collect, /* todo_flags_finish */
'y' /* letter */
};
/* Communication between the Integrated Register Allocator and rest of
the compiler.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
/* If secondary reloads are the same for inputs and outputs, define those
macros here. */
#ifdef SECONDARY_RELOAD_CLASS
#define SECONDARY_INPUT_RELOAD_CLASS(CLASS, MODE, X) \
SECONDARY_RELOAD_CLASS (CLASS, MODE, X)
#define SECONDARY_OUTPUT_RELOAD_CLASS(CLASS, MODE, X) \
SECONDARY_RELOAD_CLASS (CLASS, MODE, X)
#endif
/* If either macro is defined, show that we need secondary reloads. */
#if defined(SECONDARY_INPUT_RELOAD_CLASS) || defined(SECONDARY_OUTPUT_RELOAD_CLASS)
#define HAVE_SECONDARY_RELOADS
#endif
/* If MEMORY_MOVE_COST isn't defined, give it a default here. */
#ifndef MEMORY_MOVE_COST
#ifdef HAVE_SECONDARY_RELOADS
#define MEMORY_MOVE_COST(MODE,CLASS,IN) \
(4 + memory_move_secondary_cost ((MODE), (CLASS), (IN)))
#else
#define MEMORY_MOVE_COST(MODE,CLASS,IN) 4
#endif
#endif
extern GTY (()) struct varray_head_tag *reg_equiv_memory_loc_varray;
extern rtx *reg_equiv_constant;
extern rtx *reg_equiv_memory_loc;
extern rtx *reg_equiv_address;
extern rtx *reg_equiv_mem;
extern void init_ira_once (void);
extern rtx ira_eliminate_regs (rtx, enum machine_mode);
extern void ira (FILE *);
extern void mark_allocation_change (int);
extern void try_to_migrate (int, HARD_REG_SET, void *, HARD_REG_SET *);
extern void retry_ira_color (int, HARD_REG_SET);
extern rtx reuse_stack_slot (int, unsigned int, unsigned int);
extern void mark_new_stack_slot (rtx, int, unsigned int);
/* Building pseudos for IRA.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "rtl.h"
#include "tm_p.h"
#include "target.h"
#include "regs.h"
#include "flags.h"
#include "hard-reg-set.h"
#include "basic-block.h"
#include "insn-config.h"
#include "recog.h"
#include "toplev.h"
#include "params.h"
#include "df.h"
#include "output.h"
#include "reload.h"
#include "ira-int.h"
static void create_loop_tree_nodes (int);
static void finish_loop_tree_nodes (void);
static void add_loop_to_tree (struct loop *);
static void form_loop_tree (void);
static void initiate_pseudos (void);
static void check_pseudo_conflict_vec (pseudo_t, int);
static void add_pseudo_conflict (pseudo_t, pseudo_t);
static pseudo_t create_cap_pseudo (pseudo_t);
static void finish_pseudos (void);
static void initiate_copies (void);
static void finish_copies (void);
static void create_insn_pseudos (rtx, int);
static void create_bb_pseudos (struct ira_loop_tree_node *);
static void create_loop_pseudos (edge);
static void create_loop_tree_node_pseudos (struct ira_loop_tree_node *);
static void create_pseudos (void);
static void create_loop_tree_node_caps (struct ira_loop_tree_node *);
/* All natural loops. */
struct loops ira_loops;
/* The root of the loop tree corresponding to the all function. */
struct ira_loop_tree_node *ira_loop_tree_root;
/* All basic block data are referred through the following array. We
can not use member `aux' for this because it is used for insertion
of insns on edges. */
struct ira_loop_tree_node *ira_bb_nodes;
/* All loop data are referred through the following array. */
struct ira_loop_tree_node *ira_loop_nodes;
/* Map regno -> pseudo for the current loop tree node. */
pseudo_t *regno_pseudo_map;
/* Array of references to all pseudos and its size. The order number
of the pseudo corresponds to the index in the array. */
pseudo_t *pseudos;
int pseudos_num;
/* Array of references to copies and its size. The order number of
the copy corresponds to the index in the array. */
copy_t *copies;
int copies_num;
/* Data flow data used for IRA data flow analysis. */
struct df *build_df;
/* LAST_BASIC_BLOCK before generating additional insns because of live
range splitting. Emitting insns on a critical edge creates a new
basic block. */
static int last_basic_block_before_change;
/* The following function creates the loop tree nodes. If LOOPS_P is
zero, the nodes corresponding to the loops (except the root which
corresponds the all function) will be not created (it will be done
only for basic blocks). */
static void
create_loop_tree_nodes (int loops_p)
{
unsigned int i, j;
int max_regno, skip_p;
edge_iterator ei;
edge e;
VEC (edge, heap) *edges;
loop_p loop;
ira_bb_nodes
= ira_allocate (sizeof (struct ira_loop_tree_node) * last_basic_block);
last_basic_block_before_change = last_basic_block;
for (i = 0; i < (unsigned int) last_basic_block; i++)
{
ira_bb_nodes [i].regno_pseudo_map = NULL;
ira_bb_nodes [i].mentioned_pseudos = NULL;
ira_bb_nodes [i].modified_regnos = NULL;
ira_bb_nodes [i].border_pseudos = NULL;
ira_bb_nodes [i].local_copies = NULL;
}
ira_loop_nodes = ira_allocate (sizeof (struct ira_loop_tree_node)
* VEC_length (loop_p, ira_loops.larray));
max_regno = max_reg_num ();
for (i = 0; VEC_iterate (loop_p, ira_loops.larray, i, loop); i++)
{
if (loop != ira_loops.tree_root)
{
ira_loop_nodes [i].regno_pseudo_map = NULL;
if (! loops_p)
continue;
skip_p = FALSE;
FOR_EACH_EDGE (e, ei, loop->header->preds)
if (e->src != loop->latch
&& (e->flags & EDGE_ABNORMAL) && EDGE_CRITICAL_P (e))
{
skip_p = TRUE;
break;
}
if (skip_p)
continue;
edges = get_loop_exit_edges (loop);
for (j = 0; VEC_iterate (edge, edges, j, e); j++)
if ((e->flags & EDGE_ABNORMAL) && EDGE_CRITICAL_P (e))
{
skip_p = TRUE;
break;
}
VEC_free (edge, heap, edges);
if (skip_p)
continue;
}
ira_loop_nodes [i].regno_pseudo_map
= ira_allocate (sizeof (pseudo_t) * max_regno);
memset (ira_loop_nodes [i].regno_pseudo_map, 0,
sizeof (pseudo_t) * max_regno);
ira_loop_nodes [i].mentioned_pseudos = ira_allocate_bitmap ();
ira_loop_nodes [i].modified_regnos = ira_allocate_bitmap ();
ira_loop_nodes [i].border_pseudos = ira_allocate_bitmap ();
ira_loop_nodes [i].local_copies = ira_allocate_bitmap ();
}
}
/* The function frees the loop tree nodes. */
static void
finish_loop_tree_nodes (void)
{
unsigned int i;
loop_p loop;
for (i = 0; VEC_iterate (loop_p, ira_loops.larray, i, loop); i++)
if (ira_loop_nodes [i].regno_pseudo_map != NULL)
{
ira_free_bitmap (ira_loop_nodes [i].local_copies);
ira_free_bitmap (ira_loop_nodes [i].border_pseudos);
ira_free_bitmap (ira_loop_nodes [i].modified_regnos);
ira_free_bitmap (ira_loop_nodes [i].mentioned_pseudos);
ira_free (ira_loop_nodes [i].regno_pseudo_map);
}
ira_free (ira_loop_nodes);
for (i = 0; i < (unsigned int) last_basic_block_before_change; i++)
{
if (ira_bb_nodes [i].local_copies != NULL)
ira_free_bitmap (ira_bb_nodes [i].local_copies);
if (ira_bb_nodes [i].border_pseudos != NULL)
ira_free_bitmap (ira_bb_nodes [i].border_pseudos);
if (ira_bb_nodes [i].modified_regnos != NULL)
ira_free_bitmap (ira_bb_nodes [i].modified_regnos);
if (ira_bb_nodes [i].mentioned_pseudos != NULL)
ira_free_bitmap (ira_bb_nodes [i].mentioned_pseudos);
if (ira_bb_nodes [i].regno_pseudo_map != NULL)
ira_free (ira_bb_nodes [i].regno_pseudo_map);
}
ira_free (ira_bb_nodes);
}
/* The following recursive functions adds loop to the loop tree
hierarchy. The loop is added only once. */
static void
add_loop_to_tree (struct loop *loop)
{
struct loop *father;
struct ira_loop_tree_node *loop_node, *father_node;
/* Can not use loop node access macros because of potential checking
and because the nodes are not initialized yet. */
if (loop->outer != NULL)
add_loop_to_tree (loop->outer);
if (ira_loop_nodes [loop->num].regno_pseudo_map != NULL
&& ira_loop_nodes [loop->num].inner == NULL)
{
/* We have not added loop node to the tree yet. */
loop_node = &ira_loop_nodes [loop->num];
loop_node->loop = loop;
loop_node->bb = NULL;
for (father = loop->outer; father != NULL; father = father->outer)
if (ira_loop_nodes [father->num].regno_pseudo_map != NULL)
break;
if (father == NULL)
{
loop_node->next = NULL;
loop_node->father = NULL;
}
else
{
father_node = &ira_loop_nodes [father->num];
loop_node->next = father_node->inner;
father_node->inner = loop_node;
loop_node->father = father_node;
}
}
}
/* The following function creates the loop tree. The algorithm is
designed to provide correct order of loops (by the last loop BB)
and basic blocks in chain formed by next. */
static void
form_loop_tree (void)
{
unsigned int i;
basic_block bb;
struct loop *father;
struct ira_loop_tree_node *bb_node, *loop_node;
loop_p loop;
/* Can not use loop/bb node access macros because of potential
checking and the nodes are not initialized yet. */
for (i = 0; VEC_iterate (loop_p, ira_loops.larray, i, loop); i++)
if (ira_loop_nodes [i].regno_pseudo_map != NULL)
ira_loop_nodes [i].inner = NULL;
FOR_EACH_BB_REVERSE (bb)
{
bb_node = &ira_bb_nodes [bb->index];
bb_node->bb = bb;
bb_node->loop = NULL;
bb_node->inner = NULL;
bb_node->next = NULL;
for (father = bb->loop_father; father != NULL; father = father->outer)
if (ira_loop_nodes [father->num].regno_pseudo_map != NULL)
break;
add_loop_to_tree (father);
loop_node = &ira_loop_nodes [father->num];
bb_node->next = loop_node->inner;
bb_node->father = loop_node;
loop_node->inner = bb_node;
}
ira_loop_tree_root = IRA_LOOP_NODE_BY_INDEX (ira_loops.tree_root->num);
ira_assert (ira_loop_tree_root->regno_pseudo_map != NULL);
}
/* Varray containing references to all created pseudos. It is a
container of array pseudos. */
static varray_type pseudo_varray;
/* The function initializes data concerning pseudos. */
static void
initiate_pseudos (void)
{
VARRAY_GENERIC_PTR_NOGC_INIT (pseudo_varray, max_reg_num () * 2, "pseudos");
pseudos = NULL;
pseudos_num = 0;
regno_pseudo_map = ira_allocate (max_reg_num () * sizeof (pseudo_t));
memset (regno_pseudo_map, 0, max_reg_num () * sizeof (pseudo_t));
}
/* The function creates and returns pseudo corresponding to REGNO in
LOOP_TREE_NODE. */
pseudo_t
create_pseudo (int regno, struct ira_loop_tree_node *loop_tree_node)
{
pseudo_t p;
p = ira_allocate (sizeof (struct pseudo));
PSEUDO_REGNO (p) = regno;
PSEUDO_LOOP_TREE_NODE (p) = loop_tree_node;
if (regno >= 0)
{
PSEUDO_NEXT_REGNO_PSEUDO (p) = regno_pseudo_map [regno];
regno_pseudo_map [regno] = p;
if (loop_tree_node->regno_pseudo_map [regno] == NULL)
/* Remember that we can create temporary pseudos to break
cycles in register shuffle. */
loop_tree_node->regno_pseudo_map [regno] = p;
}
PSEUDO_CAP (p) = NULL;
PSEUDO_CAP_MEMBER (p) = NULL;
PSEUDO_NUM (p) = pseudos_num;
PSEUDO_CONFLICT_PSEUDO_VEC (p) = NULL;
PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p) = 0;
CLEAR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p));
PSEUDO_FREQ (p) = 1;
PSEUDO_HARD_REGNO (p) = -1;
PSEUDO_CALL_FREQ (p) = 0;
PSEUDO_CALLS_CROSSED_NUM (p) = 0;
PSEUDO_CALLS_CROSSED (p) = NULL;
/* ??? Too conservative. */
if (regno >= 0 && REG_N_CALLS_CROSSED (regno) != 0)
PSEUDO_CALLS_CROSSED (p)
= ira_allocate (sizeof (rtx) * REG_N_CALLS_CROSSED (regno));
#ifdef STACK_REGS
PSEUDO_NO_STACK_REG_P (p) = FALSE;
#endif
PSEUDO_IN_GRAPH_P (p) = FALSE;
PSEUDO_ASSIGNED_P (p) = FALSE;
PSEUDO_MAY_BE_SPILLED_P (p) = FALSE;
PSEUDO_MODE (p) = (regno < 0 ? VOIDmode : PSEUDO_REGNO_MODE (regno));
PSEUDO_COPIES (p) = NULL;
PSEUDO_HARD_REG_COSTS (p) = NULL;
PSEUDO_CONFLICT_HARD_REG_COSTS (p) = NULL;
PSEUDO_CURR_HARD_REG_COSTS (p) = NULL;
PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p) = NULL;
PSEUDO_LEFT_CONFLICTS_NUM (p) = -1;
PSEUDO_COVER_CLASS (p) = NO_REGS;
PSEUDO_COVER_CLASS_COST (p) = 0;
PSEUDO_MEMORY_COST (p) = 0;
PSEUDO_NEXT_BUCKET_PSEUDO (p) = NULL;
PSEUDO_PREV_BUCKET_PSEUDO (p) = NULL;
VARRAY_PUSH_GENERIC_PTR (pseudo_varray, p);
pseudos = (pseudo_t *) &VARRAY_GENERIC_PTR (pseudo_varray, 0);
pseudos_num = VARRAY_ACTIVE_SIZE (pseudo_varray);
return p;
}
/* The function allocates conflict vector of P for NUM pseudos. */
void
allocate_pseudo_conflicts (pseudo_t p, int num)
{
ira_assert (PSEUDO_CONFLICT_PSEUDO_VEC (p) == NULL);
PSEUDO_CONFLICT_PSEUDO_VEC (p)
= ira_allocate (sizeof (pseudo_t) * (num + 1));
PSEUDO_CONFLICT_PSEUDO_VEC (p) [0] = NULL;
PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p) = 0;
PSEUDO_CONFLICT_PSEUDO_VEC_SIZE (p) = num;
}
/* The function checks that conflict vector of P has enough space to
contain NUM pseudo references. If the space is not enough, the
function expands the conflict vector. */
static void
check_pseudo_conflict_vec (pseudo_t p, int num)
{
int size;
pseudo_t *vec;
ira_assert (PSEUDO_CONFLICT_PSEUDO_VEC (p) != NULL);
if (PSEUDO_CONFLICT_PSEUDO_VEC_SIZE (p) >= num)
return;
size = 3 * num / 2 + 1;
vec = ira_allocate (sizeof (pseudo_t) * (size + 1));
memcpy (vec, PSEUDO_CONFLICT_PSEUDO_VEC (p),
sizeof (pseudo_t)
* (PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p) + 1));
ira_free (PSEUDO_CONFLICT_PSEUDO_VEC (p));
PSEUDO_CONFLICT_PSEUDO_VEC (p) = vec;
PSEUDO_CONFLICT_PSEUDO_VEC_SIZE (p) = size;
}
/* The function adds P1 to conflict vector of P2 and vise versa. */
static void
add_pseudo_conflict (pseudo_t p1, pseudo_t p2)
{
int size1, size2;
pseudo_t *vec1, *vec2;
size1 = PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p1);
size2 = PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p2);
check_pseudo_conflict_vec (p1, size1 + 1);
check_pseudo_conflict_vec (p2, size2 + 1);
vec1 = PSEUDO_CONFLICT_PSEUDO_VEC (p1);
vec2 = PSEUDO_CONFLICT_PSEUDO_VEC (p2);
vec1 [size1] = p2;
vec2 [size2] = p1;
vec1 [size1 + 1] = NULL;
vec2 [size2 + 1] = NULL;
PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p1)++;
PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p2)++;
}
/* This recursive function outputs pseudo P and if it is cap the
function outputs members. */
void
print_expanded_pseudo (pseudo_t p)
{
basic_block bb;
fprintf (ira_dump_file, " p%d(r%d", PSEUDO_NUM (p), PSEUDO_REGNO (p));
if ((bb = PSEUDO_LOOP_TREE_NODE (p)->bb) != NULL)
fprintf (ira_dump_file, ",b%d", bb->index);
else
fprintf (ira_dump_file, ",l%d", PSEUDO_LOOP_TREE_NODE (p)->loop->num);
if (PSEUDO_REGNO (p) < 0)
{
fprintf (ira_dump_file, ":");
print_expanded_pseudo (PSEUDO_CAP_MEMBER (p));
}
fprintf (ira_dump_file, ")");
}
/* The function creates and returns cap representing pseudo P in the
father loop. */
static pseudo_t
create_cap_pseudo (pseudo_t p)
{
int i, regno, hard_regs_num, conflicts_num;
int *reg_costs, *conflict_reg_costs;
basic_block bb;
pseudo_t cap, conflict_pseudo, conflict_father_pseudo;
pseudo_t *pseudo_vec;
struct ira_loop_tree_node *father;
father = PSEUDO_LOOP_TREE_NODE (p)->father;
cap = create_pseudo (-1, father);
/* We just need to set a mode giving the same size. */
PSEUDO_MODE (cap) = PSEUDO_MODE (p);
PSEUDO_COVER_CLASS (cap) = PSEUDO_COVER_CLASS (p);
PSEUDO_AVAILABLE_REGS_NUM (cap) = PSEUDO_AVAILABLE_REGS_NUM (p);
PSEUDO_CAP_MEMBER (cap) = p;
hard_regs_num = class_hard_regs_num [PSEUDO_COVER_CLASS (p)];
PSEUDO_HARD_REG_COSTS (cap) = reg_costs
= ira_allocate (hard_regs_num * sizeof (int));
memcpy (reg_costs, PSEUDO_HARD_REG_COSTS (p), hard_regs_num * sizeof (int));
PSEUDO_CONFLICT_HARD_REG_COSTS (cap) = conflict_reg_costs
= ira_allocate (hard_regs_num * sizeof (int));
memcpy (conflict_reg_costs, PSEUDO_CONFLICT_HARD_REG_COSTS (p),
hard_regs_num * sizeof (int));
PSEUDO_CURR_HARD_REG_COSTS (cap)
= ira_allocate (hard_regs_num * sizeof (int));
PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (cap)
= ira_allocate (hard_regs_num * sizeof (int));
/* ??? costs, call_p etc. */
bitmap_set_bit (father->mentioned_pseudos, PSEUDO_NUM (cap));
PSEUDO_CAP (p) = cap;
PSEUDO_AVAILABLE_REGS_NUM (cap) = PSEUDO_AVAILABLE_REGS_NUM (p);
PSEUDO_COVER_CLASS_COST (cap) = PSEUDO_COVER_CLASS_COST (p);
PSEUDO_MEMORY_COST (cap) = PSEUDO_MEMORY_COST (p);
PSEUDO_FREQ (cap) = PSEUDO_FREQ (p);
PSEUDO_CALL_FREQ (cap) = PSEUDO_CALL_FREQ (p);
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (cap),
PSEUDO_CONFLICT_HARD_REGS (p));
PSEUDO_CALLS_CROSSED_NUM (cap) = PSEUDO_CALLS_CROSSED_NUM (p);
PSEUDO_CALLS_CROSSED (cap)
= ira_allocate (sizeof (rtx) * PSEUDO_CALLS_CROSSED_NUM (p));
memcpy (PSEUDO_CALLS_CROSSED (cap), PSEUDO_CALLS_CROSSED (p),
sizeof (rtx) * PSEUDO_CALLS_CROSSED_NUM (p));
#ifdef STACK_REGS
PSEUDO_NO_STACK_REG_P (cap) = PSEUDO_NO_STACK_REG_P (p);
#endif
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (p);
for (conflicts_num = i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
conflicts_num++;
allocate_pseudo_conflicts (cap, conflicts_num);
for (conflicts_num = i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
{
regno = PSEUDO_REGNO (conflict_pseudo);
if (regno < 0)
conflict_father_pseudo = PSEUDO_CAP (conflict_pseudo);
else
{
conflict_father_pseudo = father->regno_pseudo_map [regno];
if (conflict_father_pseudo == NULL)
conflict_father_pseudo = PSEUDO_CAP (conflict_pseudo);
}
if (conflict_father_pseudo != NULL)
add_pseudo_conflict (cap, conflict_father_pseudo);
}
if (ira_dump_file != NULL)
{
fprintf (ira_dump_file, " Creating cap ");
print_expanded_pseudo (cap);
fprintf (ira_dump_file, "\n Conflicts:");
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (cap);
for (i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
{
fprintf (ira_dump_file, " p%d(r%d,", PSEUDO_NUM (conflict_pseudo),
PSEUDO_REGNO (conflict_pseudo));
if ((bb = PSEUDO_LOOP_TREE_NODE (conflict_pseudo)->bb) != NULL)
fprintf (ira_dump_file, "b%d)", bb->index);
else
fprintf (ira_dump_file, "l%d)",
PSEUDO_LOOP_TREE_NODE (conflict_pseudo)->loop->num);
}
fprintf (ira_dump_file, "\n");
}
return cap;
}
/* The function frees memory allocated for all pseudos. */
static void
finish_pseudos (void)
{
int i;
pseudo_t p;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
if (PSEUDO_CONFLICT_PSEUDO_VEC (p) != NULL)
ira_free (PSEUDO_CONFLICT_PSEUDO_VEC (p));
if (PSEUDO_HARD_REG_COSTS (p) != NULL)
ira_free (PSEUDO_HARD_REG_COSTS (p));
if (PSEUDO_CONFLICT_HARD_REG_COSTS (p) != NULL)
ira_free (PSEUDO_CONFLICT_HARD_REG_COSTS (p));
if (PSEUDO_CURR_HARD_REG_COSTS (p) != NULL)
ira_free (PSEUDO_CURR_HARD_REG_COSTS (p));
if (PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p) != NULL)
ira_free (PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p));
if (PSEUDO_CALLS_CROSSED (p) != NULL)
ira_free (PSEUDO_CALLS_CROSSED (p));
ira_free (p);
}
ira_free (regno_pseudo_map);
VARRAY_FREE (pseudo_varray);
}
/* Varray containing references to all created copies. It is a
container of array copies. */
static varray_type copy_varray;
/* The function initializes data concerning pseudo copies. */
static void
initiate_copies (void)
{
VARRAY_GENERIC_PTR_NOGC_INIT (copy_varray, get_max_uid (), "copies");
copies = NULL;
copies_num = 0;
}
/* The function creates and returns copy of pseudos FIRST and SECOND
with frequency FREQ and move insn MOVE_INSN. */
copy_t
create_copy (pseudo_t first, pseudo_t second, int freq, rtx move_insn)
{
copy_t cp;
cp = ira_allocate (sizeof (struct pseudo_copy));
cp->num = copies_num;
cp->first = first;
cp->second = second;
cp->freq = freq;
cp->move_insn = move_insn;
VARRAY_PUSH_GENERIC_PTR (copy_varray, cp);
copies = (copy_t *) &VARRAY_GENERIC_PTR (copy_varray, 0);
copies_num = VARRAY_ACTIVE_SIZE (copy_varray);
return cp;
}
/* The function frees memory allocated for all copies. */
static void
finish_copies (void)
{
int i;
for (i = 0; i < copies_num; i++)
ira_free (copies [i]);
VARRAY_FREE (copy_varray);
}
/* The current loop tree node. */
struct ira_loop_tree_node *ira_curr_loop_tree_node;
/* The recursive function traverses loop tree node with root LOOP_NODE
calling non-null functions PREORDER_FUNC and POSTORDER_FUNC
correspondingly in preorder and postorder. The function sets up
IRA_CURR_LOOP_TREE_NODE. */
void
traverse_loop_tree (struct ira_loop_tree_node *loop_node,
void (*preorder_func) (struct ira_loop_tree_node *),
void (*postorder_func) (struct ira_loop_tree_node *))
{
struct ira_loop_tree_node *subloop_node;
if (loop_node->bb == NULL)
ira_curr_loop_tree_node = loop_node;
if (preorder_func != NULL)
(*preorder_func) (loop_node);
for (subloop_node = loop_node->inner;
subloop_node != NULL;
subloop_node = subloop_node->next)
{
traverse_loop_tree (subloop_node, preorder_func, postorder_func);
ira_curr_loop_tree_node = loop_node;
}
if (postorder_func != NULL)
(*postorder_func) (loop_node);
}
/* The current basic block. */
static basic_block curr_bb;
/* This recursive function creates pseudos corresponding to
pseudo-registers containing in X. Nonzero OUTPUT_P means that X is
lvalue. */
static void
create_insn_pseudos (rtx x, int output_p)
{
int i, j;
const char *fmt;
enum rtx_code code = GET_CODE (x);
if (code == REG)
{
int regno;
if ((regno = REGNO (x)) >= FIRST_PSEUDO_REGISTER)
{
pseudo_t p;
if ((p = ira_curr_loop_tree_node->regno_pseudo_map [regno]) == NULL)
p = create_pseudo (regno, ira_curr_loop_tree_node);
PSEUDO_FREQ (p) += REG_FREQ_FROM_BB (curr_bb);
bitmap_set_bit (ira_curr_loop_tree_node->mentioned_pseudos,
PSEUDO_NUM (p));
if (output_p)
bitmap_set_bit (ira_curr_loop_tree_node->modified_regnos, regno);
}
return;
}
else if (code == SET)
{
create_insn_pseudos (SET_DEST (x), TRUE);
create_insn_pseudos (SET_SRC (x), FALSE);
return;
}
else if (code == CLOBBER)
{
create_insn_pseudos (XEXP (x, 0), TRUE);
return;
}
else if (code == MEM)
{
create_insn_pseudos (XEXP (x, 0), FALSE);
return;
}
else if (code == PRE_DEC || code == POST_DEC || code == PRE_INC ||
code == POST_INC || code == POST_MODIFY || code == PRE_MODIFY)
{
create_insn_pseudos (XEXP (x, 0), TRUE);
create_insn_pseudos (XEXP (x, 0), FALSE);
return;
}
fmt = GET_RTX_FORMAT (code);
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
{
if (fmt[i] == 'e')
create_insn_pseudos (XEXP (x, i), output_p);
else if (fmt[i] == 'E')
for (j = 0; j < XVECLEN (x, i); j++)
create_insn_pseudos (XVECEXP (x, i, j), output_p);
}
}
/* The function creates pseudos corresponding to pseudo-registers
living in basic blocks represented by the corresponding loop tree
node BB_NODE. */
static void
create_bb_pseudos (struct ira_loop_tree_node *bb_node)
{
basic_block bb;
rtx insn;
unsigned int i;
bitmap_iterator bi;
pseudo_t *map;
curr_bb = bb = bb_node->bb;
ira_assert (bb != NULL);
FOR_BB_INSNS (bb, insn)
if (INSN_P (insn))
create_insn_pseudos (PATTERN (insn), FALSE);
/* It might be a pseudo living through from one subloop to
another. */
map = ira_curr_loop_tree_node->regno_pseudo_map;
EXECUTE_IF_SET_IN_REG_SET (DF_UPWARD_LIVE_IN (build_df, bb),
FIRST_PSEUDO_REGISTER, i, bi)
if (map [i] == NULL)
create_pseudo (i, ira_curr_loop_tree_node);
}
/* The function creates pseudos corresponding to pseudo-registers
living on edge E (a loop enter or exit). It also finds pseudos
living on the loop border. */
static void
create_loop_pseudos (edge e)
{
unsigned int i;
bitmap live_in_regs, border_pseudos;
bitmap_iterator bi;
pseudo_t *map;
live_in_regs = DF_UPWARD_LIVE_IN (build_df, e->dest);
map = ira_curr_loop_tree_node->regno_pseudo_map;
border_pseudos = ira_curr_loop_tree_node->border_pseudos;
EXECUTE_IF_SET_IN_REG_SET (DF_UPWARD_LIVE_OUT (build_df, e->src),
FIRST_PSEUDO_REGISTER, i, bi)
if (bitmap_bit_p (live_in_regs, i))
{
if (map [i] == NULL)
create_pseudo (i, ira_curr_loop_tree_node);
bitmap_set_bit (border_pseudos, PSEUDO_NUM (map [i]));
}
}
/* The function creates pseudos corresponding to pseudo-registers
living in loop represented by the corresponding loop tree node
LOOP_NODE. */
static void
create_loop_tree_node_pseudos (struct ira_loop_tree_node *loop_node)
{
if (loop_node->bb != NULL)
create_bb_pseudos (loop_node);
else if (loop_node != ira_loop_tree_root)
{
int i;
edge_iterator ei;
edge e;
VEC (edge, heap) *edges;
FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds)
if (e->src != loop_node->loop->latch)
create_loop_pseudos (e);
edges = get_loop_exit_edges (loop_node->loop);
for (i = 0; VEC_iterate (edge, edges, i, e); i++)
create_loop_pseudos (e);
VEC_free (edge, heap, edges);
}
}
/* The function creates pseudos corresponding to pseudo-registers in
the current function. The function traverses the loop tree for
this. */
static void
create_pseudos (void)
{
int i;
pseudo_t p, father_p;
struct ira_loop_tree_node *father;
traverse_loop_tree (ira_loop_tree_root, create_loop_tree_node_pseudos, NULL);
if (flag_ira_algorithm != IRA_ALGORITHM_REGIONAL)
return;
/* Propagate frequencies for regional register allocator. */
for (i = max_reg_num () - 1; i >= FIRST_PSEUDO_REGISTER; i--)
for (p = regno_pseudo_map [i]; p != NULL; p = PSEUDO_NEXT_REGNO_PSEUDO (p))
if ((father = PSEUDO_LOOP_TREE_NODE (p)->father) != NULL
&& (father_p = father->regno_pseudo_map [i]) != NULL)
PSEUDO_FREQ (father_p) += PSEUDO_FREQ (p);
}
/* Bitmap of pseudos living only inside the current loop. */
static bitmap local_pseudos_bitmap;
/* The function creates caps representing pseudos living only inside
the loop given by LOOP_NODE on higher loop level. */
static void
create_loop_tree_node_caps (struct ira_loop_tree_node *loop_node)
{
unsigned int j;
bitmap_iterator bi;
pseudo_t p, cap, another_p, father_p;
struct pseudo_copy *cp, *next_cp;
struct ira_loop_tree_node *father;
if (loop_node->bb != NULL || loop_node == ira_loop_tree_root)
return;
bitmap_and_compl (local_pseudos_bitmap, loop_node->mentioned_pseudos,
loop_node->border_pseudos);
EXECUTE_IF_SET_IN_BITMAP (local_pseudos_bitmap, 0, j, bi)
create_cap_pseudo (pseudos [j]);
father = loop_node->father;
EXECUTE_IF_SET_IN_BITMAP (local_pseudos_bitmap, 0, j, bi)
{
p = pseudos [j];
cap = PSEUDO_CAP (p);
for (cp = PSEUDO_COPIES (p); cp != NULL; cp = next_cp)
{
if (cp->first == p)
{
next_cp = cp->next_first_pseudo_copy;
another_p = cp->second;
}
else if (cp->second == p)
{
next_cp = cp->next_second_pseudo_copy;
another_p = cp->first;
}
else
gcc_unreachable ();
if ((father_p = PSEUDO_CAP (another_p)) == NULL)
father_p = father->regno_pseudo_map [PSEUDO_REGNO (another_p)];
if (father_p != NULL)
/* Upper level pseudo might be not existing because it is
not mentioned or lived on the border. It is just
living on bb start of the loop. */
add_pseudo_copy (cap, father_p, cp->freq, cp->move_insn);
}
}
}
/* This entry function creates internal representation for IRA
(pseudos, copies, loop tree nodes). If LOOP_P is zero the nodes
corresponding to the loops (except the root which corresponds the
all function) and correspondingly pseudos for the loops will be not
created (it will be done only for basic blocks). Such value is
used for Chaitin-Briggs and Chow's priority coloring. */
void
ira_build (int loop_p)
{
build_df = df_init (DF_HARD_REGS);
df_lr_add_problem (build_df, 0);
df_ri_add_problem (build_df, DF_RI_LIFE);
df_analyze (build_df);
initiate_pseudos ();
initiate_copies ();
ira_assert (current_loops == NULL);
flow_loops_find (&ira_loops);
current_loops = &ira_loops;
create_loop_tree_nodes (loop_p);
form_loop_tree ();
free_dominance_info (CDI_DOMINATORS);
create_pseudos ();
ira_costs ();
ira_build_conflicts ();
tune_pseudo_costs_and_cover_classes ();
if (flag_ira_algorithm == IRA_ALGORITHM_REGIONAL)
{
local_pseudos_bitmap = ira_allocate_bitmap ();
traverse_loop_tree (ira_loop_tree_root, NULL,
create_loop_tree_node_caps);
ira_free_bitmap (local_pseudos_bitmap);
}
}
/* The function releases data created by function ira_build. */
void
ira_destroy (void)
{
basic_block bb;
finish_loop_tree_nodes ();
ira_assert (current_loops == &ira_loops);
flow_loops_free (&ira_loops);
FOR_ALL_BB (bb)
bb->loop_father = NULL;
current_loops = NULL;
finish_copies ();
finish_pseudos ();
df_finish (build_df);
}
/* conflict costs for conflict hard regs ??? */
/* IRA allocation based on graph coloring.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "rtl.h"
#include "tm_p.h"
#include "target.h"
#include "varray.h"
#include "regs.h"
#include "flags.h"
#include "sbitmap.h"
#include "bitmap.h"
#include "hard-reg-set.h"
#include "basic-block.h"
#include "expr.h"
#include "toplev.h"
#include "reload.h"
#include "params.h"
#include "df.h"
#include "ira-int.h"
/* We use optimistic colouring with optional biased colouring. */
static void update_copy_costs (pseudo_t, int);
static int assign_hard_reg (pseudo_t, int);
static void add_pseudo_to_bucket (pseudo_t, pseudo_t *);
static void add_pseudo_to_ordered_bucket (pseudo_t, pseudo_t *);
static void delete_pseudo_from_bucket (pseudo_t, pseudo_t *);
static void push_pseudo_to_stack (pseudo_t);
static void remove_pseudo_from_bucket_and_push (pseudo_t, int);
static void push_only_colorable (void);
static void push_pseudo_to_spill (pseudo_t);
static int loop_edge_freq (struct ira_loop_tree_node *, int, int);
static int calculate_pseudo_spill_cost (pseudo_t);
static void push_pseudos_to_stack (void);
static void pop_pseudos_from_stack (void);
static void put_pseudo_into_bucket (pseudo_t);
static void color_pseudos (void);
static void print_loop_title (struct ira_loop_tree_node *);
static void color_pass (struct ira_loop_tree_node *);
static int pseudo_compare_func (const void *, const void *);
static void priority_coloring (void);
static void do_coloring (void);
static void move_spill_restore (void);
static void update_pseudo_avaialable_regs (void);
/* Bitmap of pseudos which should be colored. */
static bitmap coloring_pseudo_bitmap;
/* Bitmap of pseudos which should be taken into account during
coloring. In general case it contains pseudos from
coloring_pseudo_bitmap plus other already colored conflicting
pseudos. */
static bitmap consideration_pseudo_bitmap;
/* This page contains function to choose hard register for pseudos. */
/* Array whose element value is TRUE if the corresponding hard
register already allocated for a pseudo. */
static int allocated_hardreg_p [FIRST_PSEUDO_REGISTER];
/* The function updates costs (decrease if DECR_P) of the pseudos
connected by copies with PSEUDO. */
static void
update_copy_costs (pseudo_t pseudo, int decr_p)
{
int i, hard_regno, cost;
enum machine_mode mode;
enum reg_class class;
pseudo_t another_pseudo;
struct pseudo_copy *cp, *next_cp;
if (PSEUDO_COVER_CLASS (pseudo) == NO_REGS)
return;
hard_regno = PSEUDO_HARD_REGNO (pseudo);
ira_assert (hard_regno >= 0 && PSEUDO_COVER_CLASS (pseudo) != NO_REGS);
i = class_hard_reg_index [PSEUDO_COVER_CLASS (pseudo)] [hard_regno];
ira_assert (i >= 0);
class = REGNO_REG_CLASS (hard_regno);
mode = PSEUDO_MODE (pseudo);
for (cp = PSEUDO_COPIES (pseudo); cp != NULL; cp = next_cp)
{
if (cp->first == pseudo)
{
next_cp = cp->next_first_pseudo_copy;
another_pseudo = cp->second;
}
else if (cp->second == pseudo)
{
next_cp = cp->next_second_pseudo_copy;
another_pseudo = cp->first;
}
else
gcc_unreachable ();
if (PSEUDO_COVER_CLASS (pseudo) != PSEUDO_COVER_CLASS (another_pseudo))
continue;
cost = (cp->second == pseudo
? register_move_cost [mode] [class]
[PSEUDO_COVER_CLASS (another_pseudo)]
: register_move_cost [mode] [PSEUDO_COVER_CLASS (another_pseudo)]
[class]);
if (decr_p)
cost = -cost;
PSEUDO_CURR_HARD_REG_COSTS (another_pseudo) [i] += cp->freq * cost;
PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (another_pseudo) [i]
+= cp->freq * cost;
}
}
/* Function choosing a hard register for PSEUDO. If RETRY_P is
nonzero, it means that the function called from
`retry_ira_color'. */
static int
assign_hard_reg (pseudo_t pseudo, int retry_p)
{
HARD_REG_SET conflicting_regs, biased_regs;
int i, j, hard_regno, best_hard_regno, class_size;
int cost, mem_cost, min_cost, full_cost, min_full_cost, add_cost, *costs;
int *conflict_costs;
enum reg_class cover_class, class;
enum machine_mode mode;
pseudo_t conflict_pseudo, conflict_pseudo2;
pseudo_t *pseudo_vec, *pseudo_vec2;
pseudo_t another_pseudo;
struct pseudo_copy *cp, *next_cp;
static int full_costs [FIRST_PSEUDO_REGISTER];
ira_assert (! PSEUDO_ASSIGNED_P (pseudo));
cover_class = PSEUDO_COVER_CLASS (pseudo);
class_size = class_hard_regs_num [cover_class];
CLEAR_HARD_REG_SET (biased_regs);
memset (full_costs, 0, sizeof (int) * class_size);
mem_cost = PSEUDO_MEMORY_COST (pseudo);
mode = PSEUDO_MODE (pseudo);
costs = PSEUDO_CURR_HARD_REG_COSTS (pseudo);
memcpy (full_costs, costs, sizeof (int) * class_size);
COPY_HARD_REG_SET (conflicting_regs, PSEUDO_CONFLICT_HARD_REGS (pseudo));
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (pseudo);
for (i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
/* Reload can give another class so we need to check all
pseudos. */
if (retry_p
|| (cover_class == PSEUDO_COVER_CLASS (conflict_pseudo)
&& bitmap_bit_p (consideration_pseudo_bitmap,
PSEUDO_NUM (conflict_pseudo))))
{
if (PSEUDO_ASSIGNED_P (conflict_pseudo))
{
if ((hard_regno = PSEUDO_HARD_REGNO (conflict_pseudo)) >= 0)
IOR_HARD_REG_SET
(conflicting_regs,
reg_mode_hard_regset
[hard_regno] [PSEUDO_MODE (conflict_pseudo)]);
continue;
}
else
{
conflict_costs
= PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (conflict_pseudo);
if (conflict_costs != NULL)
for (j = class_size - 1; j >= 0; j--)
full_costs [j] -= conflict_costs [j];
}
if (retry_p || ! flag_ira_biased_coloring)
continue;
pseudo_vec2 = PSEUDO_CONFLICT_PSEUDO_VEC (conflict_pseudo);
for (j = 0; (conflict_pseudo2 = pseudo_vec2 [j]) != NULL; j++)
if (cover_class == PSEUDO_COVER_CLASS (conflict_pseudo2)
&& PSEUDO_ASSIGNED_P (conflict_pseudo2)
&& (hard_regno = PSEUDO_HARD_REGNO (conflict_pseudo2)) >= 0
&& (retry_p
|| bitmap_bit_p (consideration_pseudo_bitmap,
PSEUDO_NUM (conflict_pseudo2))))
IOR_HARD_REG_SET (biased_regs,
reg_mode_hard_regset
[hard_regno] [PSEUDO_MODE (conflict_pseudo2)]);
}
for (cp = PSEUDO_COPIES (pseudo); cp != NULL; cp = next_cp)
{
if (cp->first == pseudo)
{
next_cp = cp->next_first_pseudo_copy;
another_pseudo = cp->second;
}
else if (cp->second == pseudo)
{
next_cp = cp->next_second_pseudo_copy;
another_pseudo = cp->first;
}
else
gcc_unreachable ();
if (PSEUDO_ASSIGNED_P (another_pseudo))
continue;
conflict_costs = PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (another_pseudo);
if (conflict_costs != NULL)
for (j = class_size - 1; j >= 0; j--)
full_costs [j] += conflict_costs [j];
}
IOR_HARD_REG_SET (conflicting_regs, no_alloc_regs);
IOR_COMPL_HARD_REG_SET (conflicting_regs, reg_class_contents [cover_class]);
best_hard_regno = -1;
GO_IF_HARD_REG_SUBSET (reg_class_contents [cover_class], conflicting_regs,
fail);
min_cost = min_full_cost = INT_MAX;
/* We don't care about giving callee saved registers to pseudos no
living through calls because call used register are allocated
first (it is usual practice to put them first in
REG_ALLOC_ORDER). */
for (i = 0; i < class_size; i++)
{
hard_regno = class_hard_regs [cover_class] [i];
#ifdef STACK_REGS
if (PSEUDO_NO_STACK_REG_P (pseudo)
&& FIRST_STACK_REG <= hard_regno && hard_regno <= LAST_STACK_REG)
continue;
#endif
if (TEST_HARD_REG_BIT (prohibited_class_mode_regs
[cover_class] [mode], hard_regno))
continue;
if (hard_reg_not_in_set_p (hard_regno, mode, conflicting_regs))
{
cost = costs [i];
full_cost = full_costs [i];
if (! allocated_hardreg_p [hard_regno]
&& hard_reg_not_in_set_p (hard_regno, mode, call_used_reg_set))
/* We need to save/restore the register in
epilogue/prologue. Therefore we increase the cost. */
{
/* ??? If only part is call clobbered. */
class = REGNO_REG_CLASS (hard_regno);
add_cost = (memory_move_cost [mode] [class] [0]
+ memory_move_cost [mode] [class] [1] - 1);
cost += add_cost;
full_cost += add_cost;
}
if (min_cost > cost)
min_cost = cost;
if (min_full_cost > full_cost)
{
min_full_cost = full_cost;
best_hard_regno = hard_regno;
ira_assert (hard_regno >= 0);
}
else if (min_full_cost == full_cost && flag_ira_biased_coloring != 0
&& TEST_HARD_REG_BIT (biased_regs, hard_regno)
&& ! TEST_HARD_REG_BIT (biased_regs, best_hard_regno))
best_hard_regno = hard_regno;
}
}
if (min_cost > mem_cost)
best_hard_regno = -1;
fail:
PSEUDO_HARD_REGNO (pseudo) = best_hard_regno;
PSEUDO_ASSIGNED_P (pseudo) = TRUE;
if (best_hard_regno >= 0)
{
allocated_hardreg_p [best_hard_regno] = TRUE;
update_copy_costs (pseudo, TRUE);
}
return best_hard_regno >= 0;
}
/* This page contains allocator based on Chaitin algorithm. */
/* Bucket of pseudos pseudo be colored currently without spilling. */
static pseudo_t colorable_pseudo_bucket;
/* Bucket of pseudos pseudo might be not colored currently without
spilling. */
static pseudo_t uncolorable_pseudo_bucket;
/* Add PSEUDO to *BUCKET_PTR bucket. PSEUDO should be not in a bucket
before the call. */
static void
add_pseudo_to_bucket (pseudo_t pseudo, pseudo_t *bucket_ptr)
{
pseudo_t first_pseudo;
first_pseudo = *bucket_ptr;
PSEUDO_NEXT_BUCKET_PSEUDO (pseudo) = first_pseudo;
PSEUDO_PREV_BUCKET_PSEUDO (pseudo) = NULL;
if (first_pseudo != NULL)
PSEUDO_PREV_BUCKET_PSEUDO (first_pseudo) = pseudo;
*bucket_ptr = pseudo;
}
/* Add PSEUDO to *BUCKET_PTR bucket maintaining the order according
their frequency. PSEUDO should be not in a bucket before the
call. */
static void
add_pseudo_to_ordered_bucket (pseudo_t pseudo, pseudo_t *bucket_ptr)
{
pseudo_t before, after;
enum reg_class cover_class;
int freq, nregs;
freq = PSEUDO_FREQ (pseudo);
cover_class = PSEUDO_COVER_CLASS (pseudo);
nregs = reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)];
for (before = *bucket_ptr, after = NULL;
before != NULL;
after = before, before = PSEUDO_NEXT_BUCKET_PSEUDO (before))
if (PSEUDO_FREQ (before) > freq)
break;
PSEUDO_NEXT_BUCKET_PSEUDO (pseudo) = before;
PSEUDO_PREV_BUCKET_PSEUDO (pseudo) = after;
if (after == NULL)
*bucket_ptr = pseudo;
else
PSEUDO_NEXT_BUCKET_PSEUDO (after) = pseudo;
if (before != NULL)
PSEUDO_PREV_BUCKET_PSEUDO (before) = pseudo;
}
/* Delete PSEUDO from *BUCKET_PTR bucket. It should be there before
the call. */
static void
delete_pseudo_from_bucket (pseudo_t pseudo, pseudo_t *bucket_ptr)
{
pseudo_t prev_pseudo, next_pseudo;
prev_pseudo = PSEUDO_PREV_BUCKET_PSEUDO (pseudo);
next_pseudo = PSEUDO_NEXT_BUCKET_PSEUDO (pseudo);
if (prev_pseudo != NULL)
PSEUDO_NEXT_BUCKET_PSEUDO (prev_pseudo) = next_pseudo;
else
{
ira_assert (*bucket_ptr == pseudo);
*bucket_ptr = next_pseudo;
}
if (next_pseudo != NULL)
PSEUDO_PREV_BUCKET_PSEUDO (next_pseudo) = prev_pseudo;
}
/* Varray representing the stack of pseudos used during coloring. */
static varray_type pseudo_stack_varray;
/* The function puts PSEUDO onto the coloring stack without removing
it from the bucket. Such action can result in moving conflicting
pseudos from the uncolorable bucket to the colorable one. */
static void
push_pseudo_to_stack (pseudo_t pseudo)
{
int i, conflicts_num, conflict_size, size;
pseudo_t conflict_pseudo;
pseudo_t *pseudo_vec;
enum reg_class cover_class;
PSEUDO_IN_GRAPH_P (pseudo) = FALSE;
VARRAY_PUSH_GENERIC_PTR (pseudo_stack_varray, pseudo);
cover_class = PSEUDO_COVER_CLASS (pseudo);
if (cover_class == NO_REGS)
return;
size = reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)];
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (pseudo);
for (i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
if (cover_class == PSEUDO_COVER_CLASS (conflict_pseudo)
&& bitmap_bit_p (coloring_pseudo_bitmap, PSEUDO_NUM (conflict_pseudo)))
{
if (PSEUDO_IN_GRAPH_P (conflict_pseudo)
&& ! PSEUDO_ASSIGNED_P (conflict_pseudo))
{
conflicts_num = PSEUDO_LEFT_CONFLICTS_NUM (conflict_pseudo);
conflict_size
= (reg_class_nregs
[cover_class] [PSEUDO_MODE (conflict_pseudo)]);
ira_assert
(PSEUDO_LEFT_CONFLICTS_NUM (conflict_pseudo) >= size);
PSEUDO_LEFT_CONFLICTS_NUM (conflict_pseudo) -= size;
if (conflicts_num + conflict_size
<= PSEUDO_AVAILABLE_REGS_NUM (conflict_pseudo))
continue;
conflicts_num = PSEUDO_LEFT_CONFLICTS_NUM (conflict_pseudo);
if (conflicts_num + conflict_size
<= PSEUDO_AVAILABLE_REGS_NUM (conflict_pseudo))
{
delete_pseudo_from_bucket
(conflict_pseudo, &uncolorable_pseudo_bucket);
add_pseudo_to_ordered_bucket (conflict_pseudo,
&colorable_pseudo_bucket);
}
}
}
}
/* The function puts PSEUDO onto the coloring stack and removes it
from the bucket. The pseudo is in the colorable bucket if
COLORABLE_P is nonzero. */
static void
remove_pseudo_from_bucket_and_push (pseudo_t pseudo, int colorable_p)
{
enum reg_class cover_class;
pseudo_t *bucket_ptr;
bucket_ptr = (colorable_p
? &colorable_pseudo_bucket : &uncolorable_pseudo_bucket);
delete_pseudo_from_bucket (pseudo, bucket_ptr);
if (ira_dump_file != NULL)
{
fprintf (ira_dump_file, " Pushing");
print_expanded_pseudo (pseudo);
fprintf (ira_dump_file, "%s\n", colorable_p ? "" : "(potential spill)");
}
cover_class = PSEUDO_COVER_CLASS (pseudo);
ira_assert ((colorable_p
&& (PSEUDO_LEFT_CONFLICTS_NUM (pseudo)
+ reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)]
<= PSEUDO_AVAILABLE_REGS_NUM (pseudo)))
|| (! colorable_p
&& (PSEUDO_LEFT_CONFLICTS_NUM (pseudo)
+ reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)]
> PSEUDO_AVAILABLE_REGS_NUM (pseudo))));
if (! colorable_p)
PSEUDO_MAY_BE_SPILLED_P (pseudo) = TRUE;
push_pseudo_to_stack (pseudo);
}
/* The function puts all pseudos from colorable bucket onto the
coloring stack. */
static void
push_only_colorable (void)
{
for (;colorable_pseudo_bucket != NULL;)
remove_pseudo_from_bucket_and_push (colorable_pseudo_bucket, TRUE);
}
/* The function puts PSEUDO chosen for potential spilling onto the
coloring stack. */
static void
push_pseudo_to_spill (pseudo_t pseudo)
{
delete_pseudo_from_bucket (pseudo, &uncolorable_pseudo_bucket);
PSEUDO_MAY_BE_SPILLED_P (pseudo) = TRUE;
if (ira_dump_file != NULL)
fprintf (ira_dump_file, " Pushing p%d(%d) (potential spill)\n",
PSEUDO_NUM (pseudo), PSEUDO_REGNO (pseudo));
push_pseudo_to_stack (pseudo);
}
/* The function returns frequency of exit edges (if EXIT_P) or enter
from/to the loop given by its LOOP_NODE. */
static int
loop_edge_freq (struct ira_loop_tree_node *loop_node, int regno, int exit_p)
{
int freq, i;
edge_iterator ei;
edge e;
VEC (edge, heap) *edges;
ira_assert (loop_node->loop != NULL
&& (regno < 0 || regno >= FIRST_PSEUDO_REGISTER));
freq = 0;
if (! exit_p)
{
FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds)
if (e->src != loop_node->loop->latch
&& (regno < 0
|| (bitmap_bit_p (DF_UPWARD_LIVE_OUT (build_df, e->src), regno)
&& bitmap_bit_p (DF_UPWARD_LIVE_IN (build_df, e->dest),
regno))))
freq += EDGE_FREQUENCY (e);
}
else
{
edges = get_loop_exit_edges (loop_node->loop);
for (i = 0; VEC_iterate (edge, edges, i, e); i++)
if (regno < 0
|| (bitmap_bit_p (DF_UPWARD_LIVE_OUT (build_df, e->src), regno)
&& bitmap_bit_p (DF_UPWARD_LIVE_IN (build_df, e->dest),
regno)))
freq += EDGE_FREQUENCY (e);
VEC_free (edge, heap, edges);
}
return REG_FREQ_FROM_EDGE_FREQ (freq);
}
/* The function calculates and returns cost of putting pseudo P into
memory. */
static int
calculate_pseudo_spill_cost (pseudo_t p)
{
int regno, cost;
enum machine_mode mode;
enum reg_class class;
pseudo_t subloop_pseudo;
struct ira_loop_tree_node *subloop_node;
regno = PSEUDO_REGNO (p);
cost = PSEUDO_MEMORY_COST (p) - PSEUDO_COVER_CLASS_COST (p);
if (regno < 0)
return cost;
mode = PSEUDO_MODE (p);
class = PSEUDO_COVER_CLASS (p);
for (subloop_node = PSEUDO_LOOP_TREE_NODE (p)->inner;
subloop_node != NULL;
subloop_node = subloop_node->next)
if (subloop_node->bb == NULL)
{
subloop_pseudo = subloop_node->regno_pseudo_map [regno];
if (subloop_pseudo == NULL)
continue;
if (PSEUDO_HARD_REGNO (subloop_pseudo) < 0)
cost -= (memory_move_cost [mode] [class] [0]
* loop_edge_freq (subloop_node, regno, TRUE)
+ memory_move_cost [mode] [class] [1]
* loop_edge_freq (subloop_node, regno, FALSE));
else
/* ??? move register to register. How often do we have
different hard registers. */
cost += 2 * (memory_move_cost [mode] [class] [1]
* loop_edge_freq (subloop_node, regno, FALSE)
+ memory_move_cost [mode] [class] [0]
* loop_edge_freq (subloop_node, regno, TRUE));
}
return cost;
}
/* All pseudos sorted accoring their priorities. */
static pseudo_t *sorted_pseudos;
/* Push pseudos on the coloring stack. The order of pseudos in the
stack defines the order for the subsequent coloring. */
static void
push_pseudos_to_stack (void)
{
int i, j;
double pseudo_pri, i_pseudo_pri;
pseudo_t pseudo, i_pseudo;
pseudo_t *pseudo_vec;
enum reg_class cover_class;
int num, cover_class_pseudos_num [N_REG_CLASSES];
pseudo_t *cover_class_pseudos [N_REG_CLASSES];
/* Initialize. */
for (i = 0; i < reg_class_cover_size; i++)
{
cover_class = reg_class_cover [i];
cover_class_pseudos_num [cover_class] = 0;
cover_class_pseudos [cover_class] = NULL;
}
/* Calculate uncolorable pseudos of each cover class. */
for (pseudo = uncolorable_pseudo_bucket;
pseudo != NULL;
pseudo = PSEUDO_NEXT_BUCKET_PSEUDO (pseudo))
if ((cover_class = PSEUDO_COVER_CLASS (pseudo)) != NO_REGS)
{
cover_class_pseudos_num [cover_class]++;
PSEUDO_TEMP (pseudo) = INT_MAX;
}
/* Define place where to put uncolorable pseudos of the same cover
class. */
for (num = i = 0; i < reg_class_cover_size; i++)
{
cover_class = reg_class_cover [i];
if (cover_class_pseudos_num [cover_class] != 0)
{
cover_class_pseudos [cover_class] = sorted_pseudos + num;
num += cover_class_pseudos_num [cover_class];
cover_class_pseudos_num [cover_class] = 0;
}
}
ira_assert (num <= pseudos_num);
/* Put uncolorable pseudos of the same cover class together. */
for (pseudo = uncolorable_pseudo_bucket;
pseudo != NULL;
pseudo = PSEUDO_NEXT_BUCKET_PSEUDO (pseudo))
if ((cover_class = PSEUDO_COVER_CLASS (pseudo)) != NO_REGS)
cover_class_pseudos
[cover_class] [cover_class_pseudos_num [cover_class]++] = pseudo;
for (;;)
{
push_only_colorable ();
pseudo = uncolorable_pseudo_bucket;
if (pseudo == NULL)
break;
cover_class = PSEUDO_COVER_CLASS (pseudo);
if (cover_class == NO_REGS)
{
push_pseudo_to_spill (pseudo);
continue;
}
/* Potential spilling. */
ira_assert (reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)] > 0);
num = cover_class_pseudos_num [cover_class];
ira_assert (num > 0);
pseudo_vec = cover_class_pseudos [cover_class];
pseudo = NULL;
pseudo_pri = 0;
/* Sort uncolorable pseudo to find the one with the lowest spill
cost. */
for (i = 0, j = num - 1; i <= j;)
{
i_pseudo = pseudo_vec [i];
if (! PSEUDO_IN_GRAPH_P (i_pseudo)
&& PSEUDO_IN_GRAPH_P (pseudo_vec [j]))
{
i_pseudo = pseudo_vec [j];
pseudo_vec [j] = pseudo_vec [i];
pseudo_vec [i] = i_pseudo;
}
if (PSEUDO_IN_GRAPH_P (i_pseudo))
{
i++;
if (PSEUDO_TEMP (i_pseudo) == INT_MAX)
PSEUDO_TEMP (i_pseudo)
= calculate_pseudo_spill_cost (i_pseudo);
i_pseudo_pri
= ((double) PSEUDO_TEMP (i_pseudo)
/ (PSEUDO_LEFT_CONFLICTS_NUM (i_pseudo)
* reg_class_nregs [PSEUDO_COVER_CLASS (i_pseudo)]
[PSEUDO_MODE (i_pseudo)] + 1));
if (pseudo == NULL || pseudo_pri > i_pseudo_pri
|| (pseudo_pri == i_pseudo_pri
&& (PSEUDO_NUM (pseudo) > PSEUDO_NUM (i_pseudo))))
{
pseudo = i_pseudo;
pseudo_pri = i_pseudo_pri;
}
}
if (! PSEUDO_IN_GRAPH_P (pseudo_vec [j]))
j--;
}
ira_assert (pseudo != NULL && j >= 0);
cover_class_pseudos_num [cover_class] = j + 1;
ira_assert (PSEUDO_IN_GRAPH_P (pseudo)
&& PSEUDO_COVER_CLASS (pseudo) == cover_class
&& (PSEUDO_LEFT_CONFLICTS_NUM (pseudo)
+ reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)]
> PSEUDO_AVAILABLE_REGS_NUM (pseudo)));
remove_pseudo_from_bucket_and_push (pseudo, FALSE);
}
}
/* Assign hard registers to pseudos on the coloring stack. */
static void
pop_pseudos_from_stack (void)
{
pseudo_t pseudo;
enum reg_class cover_class;
for (;VARRAY_ACTIVE_SIZE (pseudo_stack_varray) != 0;)
{
pseudo = VARRAY_TOP_GENERIC_PTR (pseudo_stack_varray);
VARRAY_POP (pseudo_stack_varray);
cover_class = PSEUDO_COVER_CLASS (pseudo);
if (ira_dump_file != NULL)
{
fprintf (ira_dump_file, " Popping");
print_expanded_pseudo (pseudo);
fprintf (ira_dump_file, " -- ");
}
if (cover_class == NO_REGS)
{
PSEUDO_HARD_REGNO (pseudo) = -1;
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "assign memory\n");
}
else if (assign_hard_reg (pseudo, FALSE))
{
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "assign reg %d\n",
PSEUDO_HARD_REGNO (pseudo));
}
else
{
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "spill\n");
}
PSEUDO_IN_GRAPH_P (pseudo) = TRUE;
}
}
/* The function put PSEUDO in a bucket corresponding to its number and
size of its conflicting pseudos and hard registers. */
static void
put_pseudo_into_bucket (pseudo_t pseudo)
{
int i, hard_regno, conflict_pseudos_size, hard_regs_num;
pseudo_t conflict_pseudo;
pseudo_t *pseudo_vec;
enum reg_class cover_class;
HARD_REG_SET temp_set;
cover_class = PSEUDO_COVER_CLASS (pseudo);
hard_regs_num = class_hard_regs_num [cover_class];
if (hard_regs_num != 0)
{
memcpy (PSEUDO_CURR_HARD_REG_COSTS (pseudo),
PSEUDO_HARD_REG_COSTS (pseudo), sizeof (int) * hard_regs_num);
memcpy (PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (pseudo),
PSEUDO_CONFLICT_HARD_REG_COSTS (pseudo),
sizeof (int) * hard_regs_num);
}
COPY_HARD_REG_SET (temp_set, PSEUDO_CONFLICT_HARD_REGS (pseudo));
AND_HARD_REG_SET (temp_set, reg_class_contents [cover_class]);
AND_COMPL_HARD_REG_SET (temp_set, no_alloc_regs);
conflict_pseudos_size = 0;
GO_IF_HARD_REG_EQUAL (temp_set, zero_hard_reg_set, skip);
for (i = 0; i < (int) hard_regs_num; i++)
{
hard_regno = class_hard_regs [cover_class] [i];
if (TEST_HARD_REG_BIT (temp_set, hard_regno))
{
conflict_pseudos_size++;
CLEAR_HARD_REG_BIT (temp_set, hard_regno);
GO_IF_HARD_REG_EQUAL (temp_set, zero_hard_reg_set, skip);
}
}
skip:
pseudo_vec = PSEUDO_CONFLICT_PSEUDO_VEC (pseudo);
CLEAR_HARD_REG_SET (temp_set);
if (cover_class != NO_REGS)
for (i = 0; (conflict_pseudo = pseudo_vec [i]) != NULL; i++)
if (cover_class == PSEUDO_COVER_CLASS (conflict_pseudo)
&& bitmap_bit_p (consideration_pseudo_bitmap,
PSEUDO_NUM (conflict_pseudo)))
{
if (! PSEUDO_ASSIGNED_P (conflict_pseudo))
conflict_pseudos_size
+= reg_class_nregs [cover_class] [PSEUDO_MODE (conflict_pseudo)];
else if ((hard_regno = PSEUDO_HARD_REGNO (conflict_pseudo)) >= 0)
{
int last = (hard_regno
+ hard_regno_nregs
[hard_regno] [PSEUDO_MODE (conflict_pseudo)]);
while (hard_regno < last)
{
if (! TEST_HARD_REG_BIT (temp_set, hard_regno))
{
conflict_pseudos_size++;
SET_HARD_REG_BIT (temp_set, hard_regno);
}
hard_regno++;
}
}
}
PSEUDO_IN_GRAPH_P (pseudo) = TRUE;
PSEUDO_LEFT_CONFLICTS_NUM (pseudo) = conflict_pseudos_size;
if (conflict_pseudos_size
+ reg_class_nregs [cover_class] [PSEUDO_MODE (pseudo)]
<= PSEUDO_AVAILABLE_REGS_NUM (pseudo))
add_pseudo_to_ordered_bucket (pseudo, &colorable_pseudo_bucket);
else
add_pseudo_to_bucket (pseudo, &uncolorable_pseudo_bucket);
}
/* Function implements Chaitin-Briggs coloring for pseudos in
COLORING_PSEUDO_BITMAP taking into account pseudos in
CONSIDERATION_PSEUDO_BITMAP. */
static void
color_pseudos (void)
{
unsigned int i;
bitmap_iterator bi;
/* Put the pseudos into the corresponding buckets. */
colorable_pseudo_bucket = NULL;
uncolorable_pseudo_bucket = NULL;
EXECUTE_IF_SET_IN_BITMAP (coloring_pseudo_bitmap, 0, i, bi)
put_pseudo_into_bucket (pseudos [i]);
push_pseudos_to_stack ();
pop_pseudos_from_stack ();
}
/* The function outputs information about the loop given by its
LOOP_TREE_NODE. */
static void
print_loop_title (struct ira_loop_tree_node *loop_tree_node)
{
unsigned int j;
bitmap_iterator bi;
ira_assert (loop_tree_node->loop != NULL);
fprintf (ira_dump_file,
"\n Loop %d (father %d, header bb%d, depth %d)\n ref:",
loop_tree_node->loop->num,
(loop_tree_node->father == NULL
? -1 : loop_tree_node->father->loop->num),
loop_tree_node->loop->header->index,
loop_tree_node->loop->depth);
EXECUTE_IF_SET_IN_BITMAP (loop_tree_node->mentioned_pseudos, 0, j, bi)
fprintf (ira_dump_file, " %dr%d", j, PSEUDO_REGNO (pseudos [j]));
fprintf (ira_dump_file, "\n modified regnos:");
EXECUTE_IF_SET_IN_BITMAP (loop_tree_node->modified_regnos, 0, j, bi)
fprintf (ira_dump_file, " %d", j);
fprintf (ira_dump_file, "\n border:");
EXECUTE_IF_SET_IN_BITMAP (loop_tree_node->border_pseudos, 0, j, bi)
fprintf (ira_dump_file, " %dr%d", j, PSEUDO_REGNO (pseudos [j]));
fprintf (ira_dump_file, "\n");
}
/* The function implements Chaitin-Briggs coloring for pseudos inside
loop (in extreme case it can be all function) given by the
corresponding LOOP_TREE_NODE. */
static void
color_pass (struct ira_loop_tree_node *loop_tree_node)
{
int regno, hard_regno, index = -1;
int cost, exit_freq, enter_freq;
unsigned int j;
bitmap_iterator bi;
enum machine_mode mode;
enum reg_class class;
pseudo_t p, subloop_pseudo;
struct ira_loop_tree_node *subloop_node;
if (loop_tree_node->loop == NULL)
return;
if (ira_dump_file != NULL)
print_loop_title (loop_tree_node);
bitmap_copy (coloring_pseudo_bitmap, loop_tree_node->mentioned_pseudos);
bitmap_ior_into (coloring_pseudo_bitmap, loop_tree_node->border_pseudos);
bitmap_copy (consideration_pseudo_bitmap, coloring_pseudo_bitmap);
EXECUTE_IF_SET_IN_BITMAP (consideration_pseudo_bitmap, 0, j, bi)
{
p = pseudos [j];
if (! PSEUDO_ASSIGNED_P (p))
continue;
bitmap_clear_bit (coloring_pseudo_bitmap, PSEUDO_NUM (p));
}
/* Color all mentioned including transparent. */
color_pseudos ();
/* Update costs for subloops. */
for (subloop_node = loop_tree_node->inner;
subloop_node != NULL;
subloop_node = subloop_node->next)
if (subloop_node->bb == NULL)
EXECUTE_IF_SET_IN_BITMAP (consideration_pseudo_bitmap, 0, j, bi)
{
p = pseudos [j];
mode = PSEUDO_MODE (p);
class = PSEUDO_COVER_CLASS (p);
hard_regno = PSEUDO_HARD_REGNO (p);
if (hard_regno >= 0)
{
index = class_hard_reg_index [class] [hard_regno];
ira_assert (index >= 0);
}
regno = PSEUDO_REGNO (p);
/* ??? conflict costs */
if (regno >= 0)
{
subloop_pseudo = subloop_node->regno_pseudo_map [regno];
if (subloop_pseudo == NULL)
continue;
exit_freq = loop_edge_freq (subloop_node, regno, TRUE);
enter_freq = loop_edge_freq (subloop_node, regno, FALSE);
if (reg_equiv_invariant_p [regno]
|| reg_equiv_const [regno] != NULL_RTX)
{
if (! PSEUDO_ASSIGNED_P (subloop_pseudo))
{
PSEUDO_HARD_REGNO (subloop_pseudo) = hard_regno;
PSEUDO_ASSIGNED_P (subloop_pseudo) = TRUE;
if (hard_regno >= 0)
update_copy_costs (subloop_pseudo, TRUE);
}
}
else if (hard_regno < 0)
{
PSEUDO_MEMORY_COST (subloop_pseudo)
-= ((memory_move_cost [mode] [class] [1] * enter_freq)
+ (memory_move_cost [mode] [class] [0] * exit_freq));
}
else
{
cost = (register_move_cost [mode] [class] [class]
* (exit_freq + enter_freq));
PSEUDO_HARD_REG_COSTS (subloop_pseudo) [index] -= cost;
PSEUDO_CONFLICT_HARD_REG_COSTS (subloop_pseudo) [index]
-= cost;
PSEUDO_MEMORY_COST (subloop_pseudo)
+= (memory_move_cost [mode] [class] [0] * enter_freq
+ memory_move_cost [mode] [class] [1] * exit_freq);
if (PSEUDO_COVER_CLASS_COST (subloop_pseudo)
> PSEUDO_HARD_REG_COSTS (subloop_pseudo) [index])
PSEUDO_COVER_CLASS_COST (subloop_pseudo)
= PSEUDO_HARD_REG_COSTS (subloop_pseudo) [index];
}
}
else
{
exit_freq = loop_edge_freq (subloop_node, -1, TRUE);
enter_freq = loop_edge_freq (subloop_node, -1, FALSE);
cost = (register_move_cost [mode] [class] [class]
* (exit_freq + enter_freq));
}
}
}
/* The function is used to sort pseudos according to their priorities
which are calculated analogous to ones in file `global.c'. */
static int
pseudo_compare_func (const void *v1p, const void *v2p)
{
pseudo_t p1 = *(const pseudo_t *) v1p, p2 = *(const pseudo_t *) v2p;
int pri1, pri2;
pri1 = (((double) (floor_log2 (REG_N_REFS (PSEUDO_REGNO (p1)))
* PSEUDO_FREQ (p1))
/ REG_LIVE_LENGTH (PSEUDO_REGNO (p1)))
* (10000 / REG_FREQ_MAX) * PSEUDO_REGNO_SIZE (PSEUDO_REGNO (p1)));
pri2 = (((double) (floor_log2 (REG_N_REFS (PSEUDO_REGNO (p2)))
* PSEUDO_FREQ (p2))
/ REG_LIVE_LENGTH (PSEUDO_REGNO (p2)))
* (10000 / REG_FREQ_MAX) * PSEUDO_REGNO_SIZE (PSEUDO_REGNO (p2)));
if (pri2 - pri1)
return pri2 - pri1;
/* If regs are equally good, sort by pseudos, so that the results of
qsort leave nothing to chance. */
return PSEUDO_NUM (p1) - PSEUDO_NUM (p2);
}
/* The function implements Chow's prioity-based coloring. */
static void
priority_coloring (void)
{
int i, hard_regs_num;
pseudo_t p;
memcpy (sorted_pseudos, pseudos, pseudos_num * sizeof (pseudo_t));
for (i = 0; i < pseudos_num; i++)
{
bitmap_set_bit (coloring_pseudo_bitmap, i);
p = pseudos [i];
hard_regs_num = class_hard_regs_num [PSEUDO_COVER_CLASS (p)];
if (hard_regs_num == 0)
continue;
memcpy (PSEUDO_CURR_HARD_REG_COSTS (p),
PSEUDO_HARD_REG_COSTS (p), sizeof (int) * hard_regs_num);
memcpy (PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p),
PSEUDO_CONFLICT_HARD_REG_COSTS (p),
sizeof (int) * hard_regs_num);
}
bitmap_copy (consideration_pseudo_bitmap, coloring_pseudo_bitmap);
qsort (sorted_pseudos, pseudos_num, sizeof (pseudo_t), pseudo_compare_func);
for (i = 0; i < pseudos_num; i++)
{
p = sorted_pseudos [i];
assign_hard_reg (p, FALSE);
PSEUDO_IN_GRAPH_P (p) = TRUE;
}
}
/* The function initialized common data for cloring and calls
functions to do Chaitin-Briggs, regional, and Chow's priority-based
coloring. */
static void
do_coloring (void)
{
coloring_pseudo_bitmap = ira_allocate_bitmap ();
consideration_pseudo_bitmap = ira_allocate_bitmap ();
if (flag_ira_algorithm == IRA_ALGORITHM_PRIORITY)
priority_coloring ();
else
{
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "\n**** Pseudos coloring:\n\n");
traverse_loop_tree (ira_loop_tree_root, color_pass, NULL);
}
if (ira_dump_file != NULL)
print_disposition (ira_dump_file);
ira_free_bitmap (consideration_pseudo_bitmap);
ira_free_bitmap (coloring_pseudo_bitmap);
}
/* The functions moves future spill/restore code to less frequent
points (if it is profitable) by reassigning some pseudos to memory
which means make longer live-range where the corresponding
pseudo-registers will be in memory. */
static void
move_spill_restore (void)
{
int i, cost, changed_p, regno, hard_regno, index;
enum machine_mode mode;
enum reg_class class;
pseudo_t p, father_pseudo, grandfather_pseudo;
struct ira_loop_tree_node *father, *grandfather;
for (;;)
{
changed_p = FALSE;
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "New iteration of spill/restore move\n");
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
regno = PSEUDO_REGNO (p);
if (regno < 0
|| PSEUDO_HARD_REGNO (p) >= 0
|| (father = PSEUDO_LOOP_TREE_NODE (p)->father) == NULL)
continue;
father_pseudo = father->regno_pseudo_map [regno];
if (father_pseudo == NULL)
continue;
if ((hard_regno = PSEUDO_HARD_REGNO (father_pseudo)) < 0)
continue;
mode = PSEUDO_MODE (p);
class = PSEUDO_COVER_CLASS (father_pseudo);
index = class_hard_reg_index [class] [hard_regno];
ira_assert (index >= 0);
cost = (PSEUDO_ORIGINAL_MEMORY_COST (father_pseudo)
- PSEUDO_HARD_REG_COSTS (p) [index]);
cost -= (memory_move_cost [mode] [class] [0]
* loop_edge_freq (PSEUDO_LOOP_TREE_NODE (p), regno, FALSE)
+ memory_move_cost [mode] [class] [1]
* loop_edge_freq (PSEUDO_LOOP_TREE_NODE (p), regno, TRUE));
if ((grandfather = father->father) != NULL)
{
grandfather_pseudo = father->regno_pseudo_map [regno];
if (grandfather_pseudo != NULL)
{
if (PSEUDO_HARD_REGNO (grandfather_pseudo) < 0)
cost -= (memory_move_cost [mode] [class] [1]
* loop_edge_freq (father, regno, FALSE)
+ memory_move_cost [mode] [class] [0]
* loop_edge_freq (father, regno, TRUE));
else
cost += (memory_move_cost [mode] [class] [0]
* loop_edge_freq (father, regno, FALSE)
+ memory_move_cost [mode] [class] [1]
* loop_edge_freq (father, regno, TRUE));
}
}
if (cost < 0)
{
PSEUDO_HARD_REGNO (father_pseudo) = -1;
changed_p = TRUE;
}
}
if (! changed_p)
break;
}
}
/* The function called from the reload to mark changes in the
allocation of REGNO made by the reload. */
void
mark_allocation_change (int regno)
{
pseudo_t p = regno_pseudo_map [regno];
int old_hard_regno, hard_regno, cost;
enum reg_class cover_class = PSEUDO_COVER_CLASS (p);
ira_assert (p != NULL);
hard_regno = reg_renumber [regno];
if ((old_hard_regno = PSEUDO_HARD_REGNO (p)) == hard_regno)
return;
if (old_hard_regno < 0)
cost = -PSEUDO_MEMORY_COST (p);
else
{
ira_assert (class_hard_reg_index [cover_class] [old_hard_regno] >= 0);
cost = -(PSEUDO_HARD_REG_COSTS (p)
[class_hard_reg_index [cover_class] [old_hard_regno]]);
update_copy_costs (p, FALSE);
}
overall_cost -= cost;
PSEUDO_HARD_REGNO (p) = hard_regno;
if (hard_regno < 0)
cost += PSEUDO_MEMORY_COST (p);
else if (class_hard_reg_index [cover_class] [hard_regno] >= 0)
{
cost += (PSEUDO_HARD_REG_COSTS (p)
[class_hard_reg_index [cover_class] [hard_regno]]);
update_copy_costs (p, TRUE);
}
else
/* Reload chages class of the pseudo. */
cost = 0;
overall_cost += cost;
}
/* The function is analog of function `retry_global_alloc'. It is
called by reload to try to put spilled pseudo-register REGNO into a
hard register which is not in FORBIDDEN_REGS. */
void
retry_ira_color (int regno, HARD_REG_SET forbidden_regs)
{
pseudo_t p;
int hard_regno, hard_regs_num;
enum reg_class cover_class;
p = regno_pseudo_map [regno];
mark_allocation_change (regno);
ira_assert (reg_renumber [regno] < 0);
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "spill %d(p%d), cost=%d", regno, PSEUDO_NUM (p),
PSEUDO_MEMORY_COST (p) - PSEUDO_COVER_CLASS_COST (p));
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p), forbidden_regs);
PSEUDO_ASSIGNED_P (p) = FALSE;
cover_class = PSEUDO_COVER_CLASS (p);
hard_regs_num = class_hard_regs_num [cover_class];
if (hard_regs_num != 0)
{
memcpy (PSEUDO_CURR_HARD_REG_COSTS (p), PSEUDO_HARD_REG_COSTS (p),
sizeof (int) * hard_regs_num);
memcpy (PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p),
PSEUDO_CONFLICT_HARD_REG_COSTS (p),
sizeof (int) * hard_regs_num);
}
assign_hard_reg (p, TRUE);
hard_regno = PSEUDO_HARD_REGNO (p);
reg_renumber [regno] = hard_regno;
if (hard_regno >= 0)
{
ira_assert (class_hard_reg_index [cover_class] [hard_regno] >= 0);
overall_cost -= (PSEUDO_MEMORY_COST (p)
- (PSEUDO_HARD_REG_COSTS (p)
[class_hard_reg_index
[cover_class] [hard_regno]]));
if (PSEUDO_CALLS_CROSSED_NUM (p) != 0
&& ! hard_reg_not_in_set_p (hard_regno, PSEUDO_MODE (p),
call_used_reg_set))
{
ira_assert (flag_caller_saves);
caller_save_needed = 1;
}
}
/* If we found a register, modify the RTL for the register to show
the hard register, and mark that register live. */
if (reg_renumber[regno] >= 0)
{
if (ira_dump_file != NULL)
fprintf (ira_dump_file, ": reassign to %d", reg_renumber[regno]);
REGNO (regno_reg_rtx[regno]) = reg_renumber[regno];
mark_home_live (regno);
}
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "\n");
}
/* The function called by the reload returns already allocated stack
slot (if any) for REGNO with given INHERENT_SIZE and
TOTAL_SIZE. */
rtx
reuse_stack_slot (int regno, unsigned int inherent_size,
unsigned int total_size)
{
unsigned int i;
int n;
rtx x;
bitmap_iterator bi;
struct spilled_reg_stack_slot *slot;
ira_assert (flag_ira && inherent_size == PSEUDO_REGNO_BYTES (regno)
&& inherent_size <= total_size);
x = NULL_RTX;
if (flag_omit_frame_pointer)
n = spilled_reg_stack_slots_num - 1;
else
n = 0;
for (;;)
{
if (flag_omit_frame_pointer)
{
if (n < 0)
break;
slot = &spilled_reg_stack_slots [n--];
}
else if (n >= spilled_reg_stack_slots_num)
break;
else
slot = &spilled_reg_stack_slots [n++];
if (slot->width < total_size
|| GET_MODE_SIZE (GET_MODE (slot->mem)) < inherent_size)
continue;
EXECUTE_IF_SET_IN_BITMAP (&slot->spilled_regs,
FIRST_PSEUDO_REGISTER, i, bi)
{
if (pseudo_reg_conflict_p (regno, i))
goto cont;
}
SET_REGNO_REG_SET (&slot->spilled_regs, regno);
x = slot->mem;
break;
cont:
;
}
if (x)
{
if (ira_dump_file)
{
fprintf (ira_dump_file, " Assigning %d slot of", regno);
EXECUTE_IF_SET_IN_BITMAP (&slot->spilled_regs,
FIRST_PSEUDO_REGISTER, i, bi)
{
if ((unsigned) regno != i)
fprintf (ira_dump_file, " %d", i);
}
fprintf (ira_dump_file, "\n");
}
}
return x;
}
/* The function called by the reload when a new stack slot X with
TOTAL_SIZE was allocated for REGNO. */
void
mark_new_stack_slot (rtx x, int regno, unsigned int total_size)
{
struct spilled_reg_stack_slot *slot;
ira_assert (flag_ira && PSEUDO_REGNO_BYTES (regno) <= total_size);
slot = &spilled_reg_stack_slots [spilled_reg_stack_slots_num++];
INIT_REG_SET (&slot->spilled_regs);
SET_REGNO_REG_SET (&slot->spilled_regs, regno);
slot->mem = x;
slot->width = total_size;
if (ira_dump_file)
fprintf (ira_dump_file, " Assigning %d a new slot\n", regno);
}
/* Update avaliable hard registers for each pseudo. */
static void
update_pseudo_avaialable_regs (void)
{
int i, j, n, cost;
enum reg_class cover_class;
pseudo_t p;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
cover_class = PSEUDO_COVER_CLASS (p);
PSEUDO_AVAILABLE_REGS_NUM (p) = available_class_regs [cover_class];
if (cover_class == NO_REGS)
continue;
cost = PSEUDO_MEMORY_COST (p);
for (n = 0, j = class_hard_regs_num [cover_class] - 1; j >= 0; j--)
if (TEST_HARD_REG_BIT (PSEUDO_CONFLICT_HARD_REGS (p),
class_hard_regs [cover_class] [j]))
n++;
if (n > 0 && ira_dump_file != NULL)
fprintf (ira_dump_file, "reg %d of %s has %d regs less\n",
PSEUDO_REGNO (p), reg_class_names [cover_class], n);
PSEUDO_AVAILABLE_REGS_NUM (p) -= n;
}
}
/* Entry function doing color-based register allocation. */
void
ira_color (void)
{
update_pseudo_avaialable_regs ();
sorted_pseudos = ira_allocate (sizeof (pseudo_t) * pseudos_num);
VARRAY_GENERIC_PTR_NOGC_INIT (pseudo_stack_varray, pseudos_num,
"stack of pseudos");
memset (allocated_hardreg_p, 0, sizeof (allocated_hardreg_p));
do_coloring ();
VARRAY_FREE (pseudo_stack_varray);
ira_free (sorted_pseudos);
move_spill_restore ();
}
/* IRA conflict builder.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "regs.h"
#include "rtl.h"
#include "tm_p.h"
#include "target.h"
#include "flags.h"
#include "hard-reg-set.h"
#include "basic-block.h"
#include "insn-config.h"
#include "recog.h"
#include "toplev.h"
#include "params.h"
#include "df.h"
#include "ira-int.h"
/* The file contains code is analogous to one in global but the code
works on the pseudo basis. */
static void set_pseudo_live (pseudo_t);
static void clear_pseudo_live (pseudo_t);
static void record_regno_conflict (int);
static void mark_reg_store (rtx, rtx, void *);
static void mark_reg_clobber (rtx, rtx, void *);
static void mark_reg_conflicts (rtx);
static void mark_reg_death (rtx);
static int commutative_constraint_p (const char *);
static int get_dup_num (int, int);
static rtx get_dup (int, int);
static void add_pseudo_copy_to_list (copy_t);
static void remove_pseudo_copy_from_list (copy_t);
static void swap_pseudo_copy_ends_if_necessary (copy_t);
static void add_pseudo_copies (rtx);
static enum reg_class single_reg_class (const char *, rtx op, rtx);
static enum reg_class single_reg_operand_class (int);
static void process_single_reg_class_operands (int);
static void process_bb_node_for_conflicts (struct ira_loop_tree_node *);
static void build_conflict_bit_table (void);
static void propagate_pseudo_info (pseudo_t);
static void propagate_info (void);
static void mirror_conflicts (void);
static void remove_conflict_pseudo_copies (void);
static void build_pseudo_conflict_vects (void);
static void propagate_modified_regnos (struct ira_loop_tree_node *);
static void print_conflicts (FILE *);
/* The number of bits in each element of `conflicts' and what
type that element has. We use the largest integer format on the
host machine. */
#define INT_BITS HOST_BITS_PER_WIDE_INT
#define INT_TYPE HOST_WIDE_INT
/* Bit mask for pseudos live at current point in the scan. */
static INT_TYPE *pseudos_live;
/* The same as previous but as bitmap. */
static bitmap pseudos_live_bitmap;
/* Set, clear or test bit number I in R, a bit vector indexed by
pseudo number. */
#define SET_PSEUDO_CONFLICT_ROW(R, I) \
((R)[(unsigned) (I) / INT_BITS] \
|= ((INT_TYPE) 1 << ((unsigned) (I) % INT_BITS)))
#define CLEAR_PSEUDO_CONFLICT_ROW(R, I) \
((R) [(unsigned) (I) / INT_BITS] \
&= ~((INT_TYPE) 1 << ((unsigned) (I) % INT_BITS)))
#define TEST_PSEUDO_CONFLICT_ROW(R, I) \
((R) [(unsigned) (I) / INT_BITS] \
& ((INT_TYPE) 1 << ((unsigned) (I) % INT_BITS)))
/* Set, clear or test bit number I in `pseudos_live',
a bit vector indexed by pseudo number. */
#define SET_PSEUDO_LIVE(I) SET_PSEUDO_CONFLICT_ROW (pseudos_live, I)
#define CLEAR_PSEUDO_LIVE(I) CLEAR_PSEUDO_CONFLICT_ROW (pseudos_live, I)
#define TEST_PSEUDO_LIVE(I) TEST_PSEUDO_CONFLICT_ROW (pseudos_live, I)
/* pseudos_num by pseudos_num array of bits, recording whether two
pseudo's conflict (can't go in the same hardware register).
`conflicts' is symmetric after the call to mirror_conflicts. */
static INT_TYPE *conflicts;
/* Number of ints required to hold pseudos_num bits. This is the
length of a row in `conflicts'. */
static int pseudo_row_words;
/* Two macros to test 1 in an element of `conflicts'. */
#define CONFLICTP(I, J) \
(conflicts[(I) * pseudo_row_words + (unsigned) (J) / INT_BITS] \
& ((INT_TYPE) 1 << ((unsigned) (J) % INT_BITS)))
/* For each pseudo set in PSEUDO_SET, set PSEUDO to that pseudo, and
execute CODE. */
#define EXECUTE_IF_SET_IN_PSEUDO_SET(PSEUDO_SET, PSEUDO, CODE) \
do { \
int i_; \
int pseudo_; \
INT_TYPE *p_ = (PSEUDO_SET); \
\
for (i_ = pseudo_row_words - 1, pseudo_ = 0; i_ >= 0; \
i_--, pseudo_ += INT_BITS) \
{ \
unsigned INT_TYPE word_ = (unsigned INT_TYPE) *p_++; \
\
for ((PSEUDO) = pseudo_; word_; word_ >>= 1, (PSEUDO)++) \
{ \
if (word_ & 1) \
{CODE;} \
} \
} \
} while (0)
/* Set of hard regs (except eliminable ones) currently live (during
scan of all insns). */
static HARD_REG_SET hard_regs_live;
/* Loop tree node corresponding to the current basic block. */
static struct ira_loop_tree_node *curr_bb_node;
/* Current map regno -> pseudo. */
static pseudo_t *curr_regno_pseudo_map;
/* The function marks pseudo P as currently living. */
static void
set_pseudo_live (pseudo_t p)
{
if (TEST_PSEUDO_LIVE (PSEUDO_NUM (p)))
return;
SET_PSEUDO_LIVE (PSEUDO_NUM (p));
bitmap_set_bit (pseudos_live_bitmap, PSEUDO_NUM (p));
}
/* The function marks pseudo P as currently not living. */
static void
clear_pseudo_live (pseudo_t p)
{
CLEAR_PSEUDO_LIVE (PSEUDO_NUM (p));
bitmap_clear_bit (pseudos_live_bitmap, PSEUDO_NUM (p));
}
/* Record all regs that are set in any one insn. Communication from
mark_reg_{store,clobber} and build_conflict_bit_table. */
static rtx *regs_set;
/* Number elelments in the previous array. */
static int n_regs_set;
/* Record a conflict between hard register REGNO or pseudo
corresponding to pseudo-register REGNO and everything currently
live. */
static void
record_regno_conflict (int regno)
{
int j;
if (regno < FIRST_PSEUDO_REGISTER)
{
/* When a hard register becomes live, record conflicts with live
pseudo regs. */
EXECUTE_IF_SET_IN_PSEUDO_SET (pseudos_live, j,
{
SET_HARD_REG_BIT (PSEUDO_CONFLICT_HARD_REGS (pseudos [j]), regno);
});
}
else
{
/* When a pseudo-register becomes live, record conflicts first
with hard regs, then with other pseudos. */
pseudo_t p = curr_regno_pseudo_map [regno];
int pn, pn_prod;
ira_assert (p != NULL || REG_N_REFS (regno) == 0);
if (p == NULL)
return;
pn = PSEUDO_NUM (p);
pn_prod = pn * pseudo_row_words;
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p), hard_regs_live);
for (j = pseudo_row_words - 1; j >= 0; j--)
conflicts [pn_prod + j] |= pseudos_live [j];
/* Don't set up conflict for the pseudo with itself. */
CLEAR_PSEUDO_CONFLICT_ROW (conflicts + pn_prod, pn);
}
}
/* Handle the case where REG is set by the insn being scanned, during
the scan to accumulate conflicts. Store a 1 in hard_regs_live or
pseudos_live for this register or the corresponding pseudo, record
how many consecutive hardware registers it actually needs, and
record a conflict with all other pseudos already live.
Note that even if REG does not remain alive after this insn, we
must mark it here as live, to ensure a conflict between REG and any
other pseudos set in this insn that really do live. This is
because those other pseudos could be considered after this.
REG might actually be something other than a register; if so, we do
nothing.
SETTER is 0 if this register was modified by an auto-increment
(i.e., a REG_INC note was found for it). */
static void
mark_reg_store (rtx reg, rtx setter ATTRIBUTE_UNUSED,
void *data ATTRIBUTE_UNUSED)
{
int regno;
if (GET_CODE (reg) == SUBREG)
reg = SUBREG_REG (reg);
if (! REG_P (reg))
return;
regs_set [n_regs_set++] = reg;
regno = REGNO (reg);
if (regno >= FIRST_PSEUDO_REGISTER)
{
pseudo_t p = curr_regno_pseudo_map [regno];
ira_assert (p != NULL || REG_N_REFS (regno) == 0);
if (p != NULL)
set_pseudo_live (p);
record_regno_conflict (regno);
}
else if (! TEST_HARD_REG_BIT (no_alloc_regs, regno))
{
int last = regno + hard_regno_nregs [regno] [GET_MODE (reg)];
while (regno < last)
{
record_regno_conflict (regno);
if (! TEST_HARD_REG_BIT (eliminable_regset, regno))
SET_HARD_REG_BIT (hard_regs_live, regno);
regno++;
}
}
}
/* Like mark_reg_store except notice just CLOBBERs; ignore SETs. */
static void
mark_reg_clobber (rtx reg, rtx setter, void *data)
{
if (GET_CODE (setter) == CLOBBER)
mark_reg_store (reg, setter, data);
}
/* Record that REG (or the corresponding pseudo) has conflicts with
all the pseudo currently live. Do not mark REG (or the pseudo)
itself as live. */
static void
mark_reg_conflicts (rtx reg)
{
int regno;
if (GET_CODE (reg) == SUBREG)
reg = SUBREG_REG (reg);
if (! REG_P (reg))
return;
regno = REGNO (reg);
if (regno >= FIRST_PSEUDO_REGISTER)
record_regno_conflict (regno);
else if (! TEST_HARD_REG_BIT (no_alloc_regs, regno))
{
int last = regno + hard_regno_nregs [regno] [GET_MODE (reg)];
while (regno < last)
{
record_regno_conflict (regno);
regno++;
}
}
}
/* Mark REG (or the corresponding pseudo) as being dead (following the
insn being scanned now). Store a 0 in hard_regs_live or
pseudos_live for this register. */
static void
mark_reg_death (rtx reg)
{
int regno = REGNO (reg);
if (regno >= FIRST_PSEUDO_REGISTER)
{
pseudo_t p = curr_regno_pseudo_map [regno];
ira_assert (p != NULL || REG_N_REFS (regno) == 0);
if (p != NULL)
clear_pseudo_live (p);
}
else if (! TEST_HARD_REG_BIT (no_alloc_regs, regno))
{
int last = regno + hard_regno_nregs [regno] [GET_MODE (reg)];
while (regno < last)
{
CLEAR_HARD_REG_BIT (hard_regs_live, regno);
regno++;
}
}
}
/* The function returns nonzero, if the operand constraint STR is
commutative. */
static int
commutative_constraint_p (const char *str)
{
int ignore_p;
int c;
for (ignore_p = FALSE;;)
{
c = *str;
if (c == '\0')
break;
str += CONSTRAINT_LEN (c, str);
if (c == '#')
ignore_p = TRUE;
else if (c == ',')
ignore_p = FALSE;
else if (! ignore_p)
{
if (c == '%')
return TRUE;
}
}
return FALSE;
}
/* The function returns number of the operand which should be the same
in any case as operand with number OP_NUM. If USE_COMMUT_OP_P is
non-zero, the function makes temporarily commutative operand
exchange before this. The function takes only really possible
alternatives into consideration. */
static int
get_dup_num (int op_num, int use_commut_op_p)
{
int curr_alt, c, original, dup;
int ignore_p, commut_op_used_p;
const char *str;
rtx op, equiv_const = NULL_RTX; /* ??? */
if (op_num < 0 || recog_data.n_alternatives == 0)
return -1;
op = recog_data.operand [op_num];
ira_assert (REG_P (op));
commut_op_used_p = TRUE;
if (use_commut_op_p)
{
if (commutative_constraint_p (recog_data.constraints [op_num]))
op_num++;
else if (op_num > 0 && commutative_constraint_p (recog_data.constraints
[op_num - 1]))
op_num--;
else
commut_op_used_p = FALSE;
}
str = recog_data.constraints [op_num];
for (ignore_p = FALSE, original = -1, curr_alt = 0;;)
{
c = *str;
if (c == '\0')
break;
if (c == '#')
ignore_p = TRUE;
else if (c == ',')
{
curr_alt++;
ignore_p = FALSE;
}
else if (! ignore_p)
switch (c)
{
case 'X':
return -1;
case 'i':
if (equiv_const != NULL_RTX && CONSTANT_P (equiv_const))
return -1;
break;
case 'n':
if (equiv_const != NULL_RTX
&& (GET_CODE (equiv_const) == CONST_INT
|| (GET_CODE (equiv_const) == CONST_DOUBLE
&& GET_MODE (equiv_const) == VOIDmode)))
return -1;
break;
case 's':
if (equiv_const != NULL_RTX
&& CONSTANT_P (equiv_const)
&& GET_CODE (equiv_const) != CONST_INT
&& (GET_CODE (equiv_const) != CONST_DOUBLE
|| GET_MODE (equiv_const) != VOIDmode))
return -1;
break;
case 'I':
case 'J':
case 'K':
case 'L':
case 'M':
case 'N':
case 'O':
case 'P':
if ((equiv_const != NULL_RTX
&& GET_CODE (equiv_const) == CONST_INT
&& CONST_OK_FOR_CONSTRAINT_P (INTVAL (equiv_const),
c, str)))
return -1;
break;
case 'E':
case 'F':
if (equiv_const != NULL_RTX
&& (GET_CODE (equiv_const) == CONST_DOUBLE
|| (GET_CODE (equiv_const) == CONST_VECTOR
&& (GET_MODE_CLASS (GET_MODE (equiv_const))
== MODE_VECTOR_FLOAT))))
return -1;
break;
case 'G':
case 'H':
if (equiv_const != NULL_RTX
&& GET_CODE (equiv_const) == CONST_DOUBLE
&& CONST_DOUBLE_OK_FOR_CONSTRAINT_P (equiv_const, c, str))
return -1;
break;
case 'm':
case 'o':
/* Accept a register which might be placed in memory. */
return -1;
break;
case 'V':
case '<':
case '>':
break;
case 'p':
GO_IF_LEGITIMATE_ADDRESS (VOIDmode, op, win_p);
break;
win_p:
return -1;
case 'g':
return -1;
case 'r':
case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
case 'h': case 'j': case 'k': case 'l':
case 'q': case 't': case 'u':
case 'v': case 'w': case 'x': case 'y': case 'z':
case 'A': case 'B': case 'C': case 'D':
case 'Q': case 'R': case 'S': case 'T': case 'U':
case 'W': case 'Y': case 'Z':
{
enum reg_class cl;
cl = (c == 'r'
? GENERAL_REGS : REG_CLASS_FROM_CONSTRAINT (c, str));
if (cl != NO_REGS)
return -1;
#ifdef EXTRA_CONSTRAINT_STR
else if (EXTRA_CONSTRAINT_STR (op, c, str))
return -1;
#endif
break;
}
case '0': case '1': case '2': case '3': case '4':
case '5': case '6': case '7': case '8': case '9':
if (original != -1 && original != c)
return -1;
original = c;
break;
}
str += CONSTRAINT_LEN (c, str);
}
if (original == -1)
return -1;
dup = original - '0';
if (use_commut_op_p)
{
if (commutative_constraint_p (recog_data.constraints [dup]))
dup++;
else if (dup > 0
&& commutative_constraint_p (recog_data.constraints [dup -1]))
dup--;
else if (! commut_op_used_p)
return -1;
}
return dup;
}
/* The function returns operand which should be in any case the same
as operand with number OP_NUM. If USE_COMMUT_OP_P is non-zero, the
function makes temporarily commutative operand exchange before
this. */
static rtx
get_dup (int op_num, int use_commut_op_p)
{
int n = get_dup_num (op_num, use_commut_op_p);
if (n < 0)
return NULL_RTX;
else
return recog_data.operand [n];
}
/* The function attaches copy CP to pseudos involved into the copy. */
static void
add_pseudo_copy_to_list (copy_t cp)
{
pseudo_t first = cp->first, second = cp->second;
cp->prev_first_pseudo_copy = NULL;
cp->prev_second_pseudo_copy = NULL;
cp->next_first_pseudo_copy = PSEUDO_COPIES (first);
if (cp->next_first_pseudo_copy != NULL)
{
if (cp->next_first_pseudo_copy->first == first)
cp->next_first_pseudo_copy->prev_first_pseudo_copy = cp;
else
cp->next_first_pseudo_copy->prev_second_pseudo_copy = cp;
}
cp->next_second_pseudo_copy = PSEUDO_COPIES (second);
if (cp->next_second_pseudo_copy != NULL)
{
if (cp->next_second_pseudo_copy->second == second)
cp->next_second_pseudo_copy->prev_second_pseudo_copy = cp;
else
cp->next_second_pseudo_copy->prev_first_pseudo_copy = cp;
}
PSEUDO_COPIES (first) = cp;
PSEUDO_COPIES (second) = cp;
}
/* The function detaches copy CP from pseudos involved into the copy. */
static void
remove_pseudo_copy_from_list (copy_t cp)
{
pseudo_t first = cp->first, second = cp->second;
copy_t prev, next;
next = cp->next_first_pseudo_copy;
prev = cp->prev_first_pseudo_copy;
if (prev == NULL)
PSEUDO_COPIES (first) = next;
else if (prev->first == first)
prev->next_first_pseudo_copy = next;
else
prev->next_second_pseudo_copy = next;
if (next != NULL)
{
if (next->first == first)
next->prev_first_pseudo_copy = prev;
else
next->prev_second_pseudo_copy = prev;
}
cp->prev_first_pseudo_copy = cp->next_first_pseudo_copy = NULL;
next = cp->next_second_pseudo_copy;
prev = cp->prev_second_pseudo_copy;
if (prev == NULL)
PSEUDO_COPIES (second) = next;
else if (prev->second == second)
prev->next_second_pseudo_copy = next;
else
prev->next_first_pseudo_copy = next;
if (next != NULL)
{
if (next->second == second)
next->prev_second_pseudo_copy = prev;
else
next->prev_first_pseudo_copy = prev;
}
cp->prev_second_pseudo_copy = cp->next_second_pseudo_copy = NULL;
}
/* The function makes copy CP a canonical copy where number of the
first pseudo is less than the second one. */
static void
swap_pseudo_copy_ends_if_necessary (copy_t cp)
{
pseudo_t temp;
copy_t temp_cp;
if (PSEUDO_NUM (cp->first) <= PSEUDO_NUM (cp->second))
return;
temp = cp->first;
cp->first = cp->second;
cp->second = temp;
temp_cp = cp->prev_first_pseudo_copy;
cp->prev_first_pseudo_copy = cp->prev_second_pseudo_copy;
cp->prev_second_pseudo_copy = temp_cp;
temp_cp = cp->next_first_pseudo_copy;
cp->next_first_pseudo_copy = cp->next_second_pseudo_copy;
cp->next_second_pseudo_copy = temp_cp;
}
/* The function creates and returns new copy of pseudos FIRST and
SECOND with frequency FREQ corresponding to move insn INSN (if
any). */
copy_t
add_pseudo_copy (pseudo_t first, pseudo_t second, int freq, rtx insn)
{
copy_t cp;
cp = create_copy (first, second, freq, insn);
ira_assert (first != NULL && second != NULL);
add_pseudo_copy_to_list (cp);
swap_pseudo_copy_ends_if_necessary (cp);
return cp;
}
/* The function processes INSN and create pseudo copies if
necessary. For example, it might be because INSN is a
pseudo-register move or INSN is two operand insn. */
static void
add_pseudo_copies (rtx insn)
{
rtx set, operand, dup;
const char *str;
int commut_p;
int i, j, freq, hard_regno, cost, index;
copy_t cp;
pseudo_t p;
enum reg_class class, cover_class;
enum machine_mode mode;
freq = REG_FREQ_FROM_BB (BLOCK_FOR_INSN (insn));
if (freq == 0)
freq = 1;
if ((set = single_set (insn)) != NULL_RTX
&& REG_P (SET_DEST (set)) && REG_P (SET_SRC (set))
&& ! side_effects_p (set)
&& find_reg_note (insn, REG_DEAD, SET_SRC (set)) != NULL_RTX
&& find_reg_note (insn, REG_RETVAL, NULL_RTX) == NULL_RTX)
{
if (HARD_REGISTER_P (SET_SRC (set)) || HARD_REGISTER_P (SET_DEST (set)))
{
if (HARD_REGISTER_P (SET_DEST (set)))
{
if (HARD_REGISTER_P (SET_SRC (set)))
return;
hard_regno = REGNO (SET_DEST (set));
p = curr_regno_pseudo_map [REGNO (SET_SRC (set))];
}
else
{
hard_regno = REGNO (SET_SRC (set));
p = curr_regno_pseudo_map [REGNO (SET_DEST (set))];
}
class = REGNO_REG_CLASS (hard_regno);
mode = PSEUDO_MODE (p);
cover_class = PSEUDO_COVER_CLASS (p);
if (! class_subset_p [class] [cover_class])
return;
if (reg_class_size [class]
<= (unsigned) CLASS_MAX_NREGS (class, mode))
/* It is already taken into account in ira-costs.c. */
return;
index = class_hard_reg_index [cover_class] [hard_regno];
if (index < 0)
return;
if (HARD_REGISTER_P (SET_DEST (set)))
cost = register_move_cost [mode] [cover_class] [class] * freq;
else
cost = register_move_cost [mode] [class] [cover_class] * freq;
PSEUDO_HARD_REG_COSTS (p) [index] -= cost;
PSEUDO_CONFLICT_HARD_REG_COSTS (p) [index] -= cost;
}
else
{
cp = add_pseudo_copy (curr_regno_pseudo_map [REGNO (SET_DEST (set))],
curr_regno_pseudo_map [REGNO (SET_SRC (set))],
freq, insn);
bitmap_set_bit (ira_curr_loop_tree_node->local_copies, cp->num);
}
}
else
{
extract_insn (insn);
for (i = 0; i < recog_data.n_operands; i++)
{
operand = recog_data.operand [i];
if (REG_P (operand)
&& find_reg_note (insn, REG_DEAD, operand) != NULL_RTX)
{
str = recog_data.constraints [i];
while (*str == ' ' && *str == '\t')
str++;
for (j = 0, commut_p = FALSE; j < 2; j++, commut_p = TRUE)
if ((dup = get_dup (i, commut_p)) != NULL_RTX
&& REG_P (dup) && GET_MODE (operand) == GET_MODE (dup))
{
if (HARD_REGISTER_NUM_P (REGNO (operand))
|| HARD_REGISTER_NUM_P (REGNO (dup)))
{
if (HARD_REGISTER_P (operand))
{
if (HARD_REGISTER_P (dup))
continue;
hard_regno = REGNO (operand);
p = curr_regno_pseudo_map [REGNO (dup)];
}
else
{
hard_regno = REGNO (dup);
p = curr_regno_pseudo_map [REGNO (operand)];
}
class = REGNO_REG_CLASS (hard_regno);
mode = PSEUDO_MODE (p);
cover_class = PSEUDO_COVER_CLASS (p);
if (! class_subset_p [class] [cover_class])
continue;
index
= class_hard_reg_index [cover_class] [hard_regno];
if (index < 0)
continue;
if (HARD_REGISTER_P (operand))
cost = (register_move_cost
[mode] [cover_class] [class] * freq);
else
cost = (register_move_cost
[mode] [class] [cover_class] * freq);
PSEUDO_HARD_REG_COSTS (p) [index] -= cost;
PSEUDO_CONFLICT_HARD_REG_COSTS (p) [index] -= cost;
}
else
{
cp = add_pseudo_copy
(curr_regno_pseudo_map [REGNO (dup)],
curr_regno_pseudo_map [REGNO (operand)],
freq, NULL_RTX);
bitmap_set_bit
(ira_curr_loop_tree_node->local_copies, cp->num);
}
}
}
}
}
}
/* The function checks that CONSTRAINTS permits to use only one hard
register. If it is so, the function returns the class of the hard
register. Otherwise it returns NO_REGS.
EQUIV_COSNT ??? */
static enum reg_class
single_reg_class (const char *constraints, rtx op, rtx equiv_const)
{
int ignore_p;
enum reg_class cl, next_cl;
int c;
cl = NO_REGS;
for (ignore_p = FALSE;
(c = *constraints);
constraints += CONSTRAINT_LEN (c, constraints))
if (c == '#')
ignore_p = TRUE;
else if (c == ',')
ignore_p = FALSE;
else if (! ignore_p)
switch (c)
{
case ' ':
case '\t':
case '=':
case '+':
case '*':
case '&':
case '%':
case '!':
case '?':
break;
case 'i':
if (CONSTANT_P (op)
|| (equiv_const != NULL_RTX && CONSTANT_P (equiv_const)))
return NO_REGS;
break;
case 'n':
if (GET_CODE (op) == CONST_INT
|| (GET_CODE (op) == CONST_DOUBLE && GET_MODE (op) == VOIDmode)
|| (equiv_const != NULL_RTX
&& (GET_CODE (equiv_const) == CONST_INT
|| (GET_CODE (equiv_const) == CONST_DOUBLE
&& GET_MODE (equiv_const) == VOIDmode))))
return NO_REGS;
break;
case 's':
if ((CONSTANT_P (op) && GET_CODE (op) != CONST_INT
&& (GET_CODE (op) != CONST_DOUBLE || GET_MODE (op) != VOIDmode))
|| (equiv_const != NULL_RTX
&& CONSTANT_P (equiv_const)
&& GET_CODE (equiv_const) != CONST_INT
&& (GET_CODE (equiv_const) != CONST_DOUBLE
|| GET_MODE (equiv_const) != VOIDmode)))
return NO_REGS;
break;
case 'I':
case 'J':
case 'K':
case 'L':
case 'M':
case 'N':
case 'O':
case 'P':
if ((GET_CODE (op) == CONST_INT
&& CONST_OK_FOR_CONSTRAINT_P (INTVAL (op), c, constraints))
|| (equiv_const != NULL_RTX
&& GET_CODE (equiv_const) == CONST_INT
&& CONST_OK_FOR_CONSTRAINT_P (INTVAL (equiv_const),
c, constraints)))
return NO_REGS;
break;
case 'E':
case 'F':
if (GET_CODE (op) == CONST_DOUBLE
|| (GET_CODE (op) == CONST_VECTOR
&& GET_MODE_CLASS (GET_MODE (op)) == MODE_VECTOR_FLOAT)
|| (equiv_const != NULL_RTX
&& (GET_CODE (equiv_const) == CONST_DOUBLE
|| (GET_CODE (equiv_const) == CONST_VECTOR
&& (GET_MODE_CLASS (GET_MODE (equiv_const))
== MODE_VECTOR_FLOAT)))))
return NO_REGS;
break;
case 'G':
case 'H':
if ((GET_CODE (op) == CONST_DOUBLE
&& CONST_DOUBLE_OK_FOR_CONSTRAINT_P (op, c, constraints))
|| (equiv_const != NULL_RTX
&& GET_CODE (equiv_const) == CONST_DOUBLE
&& CONST_DOUBLE_OK_FOR_CONSTRAINT_P (equiv_const,
c, constraints)))
return NO_REGS;
/* ??? what about memory */
case 'r':
case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
case 'h': case 'j': case 'k': case 'l':
case 'q': case 't': case 'u':
case 'v': case 'w': case 'x': case 'y': case 'z':
case 'A': case 'B': case 'C': case 'D':
case 'Q': case 'R': case 'S': case 'T': case 'U':
case 'W': case 'Y': case 'Z':
next_cl = (c == 'r'
? GENERAL_REGS
: REG_CLASS_FROM_CONSTRAINT (c, constraints));
if ((cl != NO_REGS && next_cl != cl)
|| available_class_regs [next_cl] > 1)
return NO_REGS;
cl = next_cl;
break;
case '0': case '1': case '2': case '3': case '4':
case '5': case '6': case '7': case '8': case '9':
next_cl
= single_reg_class (recog_data.constraints [c - '0'],
recog_data.operand [c - '0'], NULL_RTX);
if ((cl != NO_REGS && next_cl != cl) || next_cl == NO_REGS
|| available_class_regs [next_cl] > 1)
return NO_REGS;
cl = next_cl;
break;
default:
return NO_REGS;
}
return cl;
}
/* The function checks that operand OP_NUM of the current insn can use
only one hard register. If it is so, the function returns the
class of the hard register. Otherwise it returns NO_REGS. */
static enum reg_class
single_reg_operand_class (int op_num)
{
if (op_num < 0 || recog_data.n_alternatives == 0)
return NO_REGS;
return single_reg_class (recog_data.constraints [op_num],
recog_data.operand [op_num], NULL_RTX);
}
/* The function processes input (if IN_P) or output operands to find
pseudo which can use only one hard register and makes other
currently living pseudos conflicting with the hard register. */
static void
process_single_reg_class_operands (int in_p)
{
int i, regno, px;
enum reg_class cl, cover_class;
rtx operand;
pseudo_t operand_p, p;
for (i = 0; i < recog_data.n_operands; i++)
{
operand = recog_data.operand [i];
if (in_p && recog_data.operand_type [i] != OP_IN
&& recog_data.operand_type [i] != OP_INOUT)
continue;
if (! in_p && recog_data.operand_type [i] != OP_OUT
&& recog_data.operand_type [i] != OP_INOUT)
continue;
cl = single_reg_operand_class (i);
if (cl == NO_REGS)
continue;
operand_p = NULL;
if (GET_CODE (operand) == SUBREG)
operand = SUBREG_REG (operand);
if (REG_P (operand)
&& (regno = REGNO (operand)) >= FIRST_PSEUDO_REGISTER)
{
enum machine_mode mode;
enum reg_class cover_class;
operand_p = curr_regno_pseudo_map [regno];
mode = PSEUDO_MODE (operand_p);
cover_class = PSEUDO_MODE (operand_p);
if (class_subset_p [cl] [cover_class]
&& (reg_class_size [cl]
<= (unsigned) CLASS_MAX_NREGS (cl, mode)))
PSEUDO_CONFLICT_HARD_REG_COSTS (operand_p)
[class_hard_reg_index [cover_class] [class_hard_regs [cl] [0]]]
-= (in_p
? register_move_cost [mode] [cover_class] [cl]
: register_move_cost [mode] [cl] [cover_class]);
}
EXECUTE_IF_SET_IN_PSEUDO_SET (pseudos_live, px,
{
p = pseudos [px];
cover_class = PSEUDO_COVER_CLASS (p);
if (p != operand_p)
/* We could increase costs of P instead of making it
conflicting with the hard register. But it works worse
because it will be spilled in reload in anyway. */
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p),
reg_class_contents [cl]);
});
}
}
/* The function processes insns of the basic block given by its
LOOP_TREE_NODE to update pseudo conflict table. */
static void
process_bb_node_for_conflicts (struct ira_loop_tree_node *loop_tree_node)
{
int i;
unsigned int j;
basic_block bb;
rtx insn;
edge e;
edge_iterator ei;
bitmap_iterator bi;
bitmap reg_live_in, reg_live_out;
int px = 0;
bb = loop_tree_node->bb;
if (bb == NULL)
return;
curr_bb_node = loop_tree_node;
curr_regno_pseudo_map = ira_curr_loop_tree_node->regno_pseudo_map;
reg_live_in = DF_UPWARD_LIVE_IN (build_df, bb);
reg_live_out = DF_UPWARD_LIVE_OUT (build_df, bb);
memset (pseudos_live, 0, pseudo_row_words * sizeof (INT_TYPE));
REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_in);
AND_COMPL_HARD_REG_SET (hard_regs_live, eliminable_regset);
bitmap_clear (pseudos_live_bitmap);
EXECUTE_IF_SET_IN_BITMAP (reg_live_in, FIRST_PSEUDO_REGISTER, j, bi)
{
pseudo_t p = curr_regno_pseudo_map [j];
ira_assert (p != NULL || REG_N_REFS (j) == 0);
if (p == NULL)
continue;
set_pseudo_live (p);
record_regno_conflict (j);
}
#ifdef EH_RETURN_DATA_REGNO
if (bb_has_eh_pred (bb))
{
for (j = 0; ; ++j)
{
unsigned int regno = EH_RETURN_DATA_REGNO (j);
if (regno == INVALID_REGNUM)
break;
record_regno_conflict (regno);
}
}
#endif
/* Pseudos can't go in stack regs at the start of a basic block that
is reached by an abnormal edge. Likewise for call clobbered regs,
because caller-save, fixup_abnormal_edges and possibly the table
driven EH machinery are not quite ready to handle such pseudos
live across such edges. */
FOR_EACH_EDGE (e, ei, bb->preds)
if (e->flags & EDGE_ABNORMAL)
break;
if (e != NULL)
{
#ifdef STACK_REGS
EXECUTE_IF_SET_IN_PSEUDO_SET (pseudos_live, px,
{
PSEUDO_NO_STACK_REG_P (pseudos [px]) = TRUE;
});
for (px = FIRST_STACK_REG; px <= LAST_STACK_REG; px++)
record_regno_conflict (px);
#endif
/* No need to record conflicts for call clobbered regs if we
have nonlocal labels around, as we don't ever try to allocate
such regs in this case. */
if (! current_function_has_nonlocal_label)
for (px = 0; px < FIRST_PSEUDO_REGISTER; px++)
if (call_used_regs [px])
record_regno_conflict (px);
}
/* Scan the code of this basic block, noting which pseudos and hard
regs are born or die. When one is born, record a conflict with
all others currently live. */
FOR_BB_INSNS (bb, insn)
{
rtx link;
if (! INSN_P (insn))
continue;
/* Make regs_set an empty set. */
n_regs_set = 0;
/* Mark any pseudos clobbered by INSN as live, so they
conflict with the inputs. */
note_stores (PATTERN (insn), mark_reg_clobber, NULL);
extract_insn (insn);
process_single_reg_class_operands (TRUE);
/* Mark any pseudos dead after INSN as dead now. */
for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
if (REG_NOTE_KIND (link) == REG_DEAD)
mark_reg_death (XEXP (link, 0));
if (CALL_P (insn))
{
EXECUTE_IF_SET_IN_PSEUDO_SET (pseudos_live, i,
{
int freq;
pseudo_t p = pseudos [i];
freq = REG_FREQ_FROM_BB (BLOCK_FOR_INSN (insn));
if (freq == 0)
freq = 1;
PSEUDO_CALL_FREQ (p) += freq;
PSEUDO_CALLS_CROSSED (p) [PSEUDO_CALLS_CROSSED_NUM (p)++] = insn;
ira_assert (PSEUDO_CALLS_CROSSED_NUM (p)
<= REG_N_CALLS_CROSSED (PSEUDO_REGNO (p)));
/* Don't allocate pseudos that cross calls, if this
function receives a nonlocal goto. */
if (current_function_has_nonlocal_label)
SET_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p));
});
}
/* Mark any pseudos set in INSN as live, and mark them as
conflicting with all other live pseudos. Clobbers are
processed again, so they conflict with the pseudos that are
set. */
note_stores (PATTERN (insn), mark_reg_store, NULL);
#ifdef AUTO_INC_DEC
for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
if (REG_NOTE_KIND (link) == REG_INC)
mark_reg_store (XEXP (link, 0), NULL_RTX, NULL);
#endif
/* If INSN has multiple outputs, then any pseudo that dies here
and is used inside of an output must conflict with the other
outputs.
It is unsafe to use !single_set here since it will ignore an
unused output. Just because an output is unused does not
mean the compiler can assume the side effect will not occur.
Consider if PSEUDO appears in the address of an output and we
reload the output. If we allocate PSEUDO to the same hard
register as an unused output we could set the hard register
before the output reload insn. */
if (GET_CODE (PATTERN (insn)) == PARALLEL && multiple_sets (insn))
for (link = REG_NOTES (insn); link; link = XEXP (link, 1))
if (REG_NOTE_KIND (link) == REG_DEAD)
{
int i;
int used_in_output = 0;
rtx reg = XEXP (link, 0);
for (i = XVECLEN (PATTERN (insn), 0) - 1; i >= 0; i--)
{
rtx set = XVECEXP (PATTERN (insn), 0, i);
if (GET_CODE (set) == SET
&& ! REG_P (SET_DEST (set))
&& ! rtx_equal_p (reg, SET_DEST (set))
&& reg_overlap_mentioned_p (reg, SET_DEST (set)))
used_in_output = 1;
}
if (used_in_output)
mark_reg_conflicts (reg);
}
process_single_reg_class_operands (FALSE);
/* Mark any pseudos set in INSN and then never used. */
while (n_regs_set-- > 0)
{
rtx note = find_regno_note (insn, REG_UNUSED,
REGNO (regs_set [n_regs_set]));
if (note)
mark_reg_death (XEXP (note, 0));
}
add_pseudo_copies (insn);
}
}
/* The function builds pseudo conflict table by traversing all basic
blocks and their insns. */
static void
build_conflict_bit_table (void)
{
pseudo_row_words = (pseudos_num + INT_BITS - 1) / INT_BITS;
conflicts = ira_allocate (sizeof (INT_TYPE)
* pseudos_num * pseudo_row_words);
memset (conflicts, 0, sizeof (INT_TYPE) * pseudos_num * pseudo_row_words);
pseudos_live = ira_allocate (sizeof (INT_TYPE) * pseudo_row_words);
pseudos_live_bitmap = ira_allocate_bitmap ();
/* Make a vector that mark_reg_{store,clobber} will store in. */
regs_set = ira_allocate (sizeof (rtx) * max_parallel * 2);
traverse_loop_tree (ira_loop_tree_root, process_bb_node_for_conflicts, NULL);
/* Clean up. */
ira_free (regs_set);
ira_free_bitmap (pseudos_live_bitmap);
ira_free (pseudos_live);
}
/* The function propagates info about pseudo P to the corresponding
pseudo on upper loop tree level. So pseudos on upper levels
accumulate information about the corresponding pseudos in nested
loops. */
static void
propagate_pseudo_info (pseudo_t p)
{
int regno, j, n, pn, father_pn, another_father_pn;
pseudo_t father_p, another_p, another_father_p;
struct ira_loop_tree_node *father;
struct pseudo_copy *cp, *next_cp;
regno = PSEUDO_REGNO (p);
if ((father = PSEUDO_LOOP_TREE_NODE (p)->father) != NULL
&& (father_p = father->regno_pseudo_map [regno]) != NULL)
{
PSEUDO_CALL_FREQ (father_p) += PSEUDO_CALL_FREQ (p);
#ifdef STACK_REGS
if (PSEUDO_NO_STACK_REG_P (p))
PSEUDO_NO_STACK_REG_P (father_p) = TRUE;
#endif
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (father_p),
PSEUDO_CONFLICT_HARD_REGS (p));
pn = PSEUDO_NUM (p);
EXECUTE_IF_SET_IN_PSEUDO_SET (conflicts + pn * pseudo_row_words, j,
{
another_p = pseudos [j];
if ((another_father_p = (father->regno_pseudo_map
[PSEUDO_REGNO (another_p)])) == NULL)
continue;
father_pn = PSEUDO_NUM (father_p);
another_father_pn = PSEUDO_NUM (another_father_p);
SET_PSEUDO_CONFLICT_ROW
(conflicts + father_pn * pseudo_row_words, another_father_pn);
SET_PSEUDO_CONFLICT_ROW
(conflicts + another_father_pn * pseudo_row_words, father_pn);
});
if ((n = PSEUDO_CALLS_CROSSED_NUM (p)) != 0)
{
memcpy (PSEUDO_CALLS_CROSSED (father_p)
+ PSEUDO_CALLS_CROSSED_NUM (father_p),
PSEUDO_CALLS_CROSSED (p), sizeof (rtx) * n);
PSEUDO_CALLS_CROSSED_NUM (father_p) += n;
ira_assert (PSEUDO_CALLS_CROSSED_NUM (father_p)
<= REG_N_CALLS_CROSSED (regno));
}
for (cp = PSEUDO_COPIES (p); cp != NULL; cp = next_cp)
{
if (cp->first == p)
{
next_cp = cp->next_first_pseudo_copy;
another_p = cp->second;
}
else if (cp->second == p)
{
next_cp = cp->next_second_pseudo_copy;
another_p = cp->first;
}
else
gcc_unreachable ();
if ((another_father_p = (father->regno_pseudo_map
[PSEUDO_REGNO (another_p)])) != NULL)
add_pseudo_copy
(father_p, another_father_p, cp->freq, cp->move_insn);
}
}
}
/* The function propagates info about pseudos to the corresponding
pseudos on upper loop tree level. */
static void
propagate_info (void)
{
int i;
pseudo_t p;
for (i = max_reg_num () - 1; i >= FIRST_PSEUDO_REGISTER; i--)
for (p = regno_pseudo_map [i]; p != NULL; p = PSEUDO_NEXT_REGNO_PSEUDO (p))
propagate_pseudo_info (p);
}
/* If CONFLICTP (i, j) is TRUE, make sure CONFLICTP (j, i) is also TRUE. */
static void
mirror_conflicts (void)
{
int i, j;
unsigned INT_TYPE mask;
int rw = pseudo_row_words;
int rwb = rw * INT_BITS;
INT_TYPE *p = conflicts;
INT_TYPE *q0 = conflicts;
INT_TYPE *q1, *q2;
for (i = pseudos_num - 1, mask = 1; i >= 0; i--, mask <<= 1)
{
if (! mask)
{
mask = 1;
q0++;
}
for (j = pseudo_row_words - 1, q1 = q0; j >= 0; j--, q1 += rwb)
{
unsigned INT_TYPE word;
for (word = (unsigned INT_TYPE) *p++, q2 = q1;
word;
word >>= 1, q2 += rw)
{
if (word & 1)
*q2 |= mask;
}
}
}
}
/* The function returns TRUE if pseudo-registers REGNO1 and REGNO2
conflict. The function is called from reload. */
int
pseudo_reg_conflict_p (int regno1, int regno2)
{
int p_no1, p_no2;
ira_assert (regno1 >= FIRST_PSEUDO_REGISTER
&& regno2 >= FIRST_PSEUDO_REGISTER);
/* Reg info caclulated by dataflow infrastructure can be different
from one calculated by regclass. */
if (curr_regno_pseudo_map [regno1] == NULL
|| curr_regno_pseudo_map [regno2] == NULL)
return FALSE;
p_no1 = PSEUDO_NUM (curr_regno_pseudo_map [regno1]);
p_no2 = PSEUDO_NUM (curr_regno_pseudo_map [regno2]);
ira_assert (p_no1 >= 0 && p_no1 < pseudos_num
&& p_no2 >= 0 && p_no2 < pseudos_num);
return CONFLICTP (p_no1, p_no2) != 0;
}
/* Remove copies involving conflicting pseudos. */
static void
remove_conflict_pseudo_copies (void)
{
int i;
pseudo_t p;
copy_t cp, next_cp;
varray_type conflict_pseudo_copy_varray;
VARRAY_GENERIC_PTR_NOGC_INIT (conflict_pseudo_copy_varray, get_max_uid (),
"copies of conflicting pseudos");
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
for (cp = PSEUDO_COPIES (p); cp != NULL; cp = next_cp)
if (cp->first == p)
next_cp = cp->next_first_pseudo_copy;
else
{
next_cp = cp->next_second_pseudo_copy;
VARRAY_PUSH_GENERIC_PTR (conflict_pseudo_copy_varray, cp);
}
}
for (i = VARRAY_ACTIVE_SIZE (conflict_pseudo_copy_varray) - 1; i >= 0; i--)
{
cp = VARRAY_GENERIC_PTR (conflict_pseudo_copy_varray, i);
if (CONFLICTP (PSEUDO_NUM (cp->first), PSEUDO_NUM (cp->second)))
remove_pseudo_copy_from_list (cp);
}
VARRAY_FREE (conflict_pseudo_copy_varray);
}
/* The function builds conflict vectors of all pseudos from the
conflict table. */
static void
build_pseudo_conflict_vects (void)
{
int i, j, px;
pseudo_t p, *conflict_pseudos, *vec;
conflict_pseudos = ira_allocate (sizeof (pseudo_t) * pseudos_num);
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
ira_assert (i == PSEUDO_NUM (p));
px = 0;
EXECUTE_IF_SET_IN_PSEUDO_SET (conflicts + i * pseudo_row_words, j,
{
conflict_pseudos [px++] = pseudos [j];
});
allocate_pseudo_conflicts (p, px);
vec = PSEUDO_CONFLICT_PSEUDO_VEC (p);
memcpy (vec, conflict_pseudos, sizeof (pseudo_t) * px);
vec [px] = NULL;
PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE (p) = px;
}
ira_free (conflict_pseudos);
}
/* The function propagates information about pseudos modified inside
the loops. */
static void
propagate_modified_regnos (struct ira_loop_tree_node *loop_tree_node)
{
if (loop_tree_node->bb != NULL || loop_tree_node == ira_loop_tree_root)
return;
bitmap_ior_into (loop_tree_node->father->modified_regnos,
loop_tree_node->modified_regnos);
}
/* The function outputs information about pseudo conflicts to FILE. */
static void
print_conflicts (FILE *file)
{
int i;
for (i = 0; i < pseudos_num; i++)
{
int j;
pseudo_t p;
basic_block bb;
p = pseudos [i];
fprintf (file, ";; p%d(r%d,", PSEUDO_NUM (p), PSEUDO_REGNO (p));
if ((bb = PSEUDO_LOOP_TREE_NODE (p)->bb) != NULL)
fprintf (file, "b%d", bb->index);
else
fprintf (file, "l%d", PSEUDO_LOOP_TREE_NODE (p)->loop->num);
fprintf (file, ") conflicts:");
for (j = 0; j < pseudos_num; j++)
if (CONFLICTP (j, i))
{
fprintf (file, " p%d(r%d,",
PSEUDO_NUM (pseudos [j]), PSEUDO_REGNO (pseudos [j]));
if ((bb = PSEUDO_LOOP_TREE_NODE (pseudos [j])->bb) != NULL)
fprintf (file, "b%d)", bb->index);
else
fprintf (file, "l%d)",
PSEUDO_LOOP_TREE_NODE (pseudos [j])->loop->num);
}
fprintf (file, "\n;; conflict hard regs:");
for (j = 0; j < FIRST_PSEUDO_REGISTER; j++)
if (TEST_HARD_REG_BIT (PSEUDO_CONFLICT_HARD_REGS (p), j))
fprintf (file, " %d", j);
fprintf (file, "\n");
}
fprintf (file, "\n");
}
/* The function outputs information about pseudo conflicts to
stderr. */
void
debug_conflicts (void)
{
print_conflicts (stderr);
}
/* Entry function which builds pseudo conflicts. */
void
ira_build_conflicts (void)
{
int i;
pseudo_t p;
build_conflict_bit_table ();
mirror_conflicts ();
if (flag_ira_algorithm == IRA_ALGORITHM_REGIONAL)
propagate_info ();
/* We need finished conflict table for the subsequent call. */
remove_conflict_pseudo_copies ();
build_pseudo_conflict_vects ();
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
if (PSEUDO_CALLS_CROSSED_NUM (p) != 0)
{
if (! flag_caller_saves)
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p),
call_used_reg_set);
else
IOR_HARD_REG_SET (PSEUDO_CONFLICT_HARD_REGS (p),
no_caller_save_reg_set);
}
}
traverse_loop_tree (ira_loop_tree_root, NULL, propagate_modified_regnos);
if (ira_dump_file != NULL)
print_conflicts (ira_dump_file);
}
/* Compute cover class of the pseudos and their hard register costs.
Copyright (C) 2006
Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "hard-reg-set.h"
#include "rtl.h"
#include "expr.h"
#include "tm_p.h"
#include "flags.h"
#include "basic-block.h"
#include "regs.h"
#include "addresses.h"
#include "insn-config.h"
#include "recog.h"
#include "toplev.h"
#include "target.h"
#include "params.h"
#include "ira-int.h"
/* The file contains code is analogous to one in regclass but the code
works on the pseudo basis. */
struct costs;
static void record_reg_classes (int, int, rtx *, enum machine_mode *,
const char **, rtx, struct costs *,
enum reg_class *);
static inline int ok_for_index_p_nonstrict (rtx);
static inline int ok_for_base_p_nonstrict (rtx, enum machine_mode,
enum rtx_code, enum rtx_code);
static void record_address_regs (enum machine_mode, rtx x, int,
enum rtx_code, enum rtx_code, int scale);
static void record_operand_costs (rtx, struct costs *, enum reg_class *);
static rtx scan_one_insn (rtx);
static void print_costs (FILE *);
static void process_bb_node_for_costs (struct ira_loop_tree_node *);
static void find_pseudo_class_costs (void);
static void process_bb_node_for_hard_reg_moves (struct ira_loop_tree_node *);
static void setup_pseudo_cover_class_and_costs (void);
#ifdef FORBIDDEN_INC_DEC_CLASSES
/* Indexed by n, is nonzero if (REG n) is used in an auto-inc or
auto-dec context. */
static char *in_inc_dec;
#endif
/* The `costs' struct records the cost of using a hard register of
each class and of using memory for each pseudo. We use this data
to set up register and costs. */
struct costs
{
int cost [N_REG_CLASSES];
int mem_cost;
};
/* Record the cost of each class for each pseudo. */
static struct costs *costs;
/* Initialized once, and used to initialize cost values for each
insn. */
static struct costs init_cost;
/* Record register class preferences of each pseudo. */
static enum reg_class *reg_pref;
/* Allocated buffers for reg_pref. */
static enum reg_class *reg_pref_buffer;
/* Frequency of executions of the current insn. */
static int frequency;
/* Map regno->pseudo for the current loop. */
static pseudo_t *curr_regno_pseudo_map;
/* Compute the cost of loading X into (if TO_P is nonzero) or from (if
TO_P is zero) a register of class CLASS in mode MODE. X must not
be a pseudo register. */
static int
copy_cost (rtx x, enum machine_mode mode, enum reg_class class, int to_p,
secondary_reload_info *prev_sri)
{
secondary_reload_info sri;
enum reg_class secondary_class = NO_REGS;
/* If X is a SCRATCH, there is actually nothing to move since we are
assuming optimal allocation. */
if (GET_CODE (x) == SCRATCH)
return 0;
/* Get the class we will actually use for a reload. */
class = PREFERRED_RELOAD_CLASS (x, class);
/* If we need a secondary reload for an intermediate, the cost is
that to load the input into the intermediate register, then to
copy it. */
sri.prev_sri = prev_sri;
sri.extra_cost = 0;
secondary_class = targetm.secondary_reload (to_p, x, class, mode, &sri);
if (secondary_class != NO_REGS)
return (move_cost [mode] [secondary_class] [class]
+ sri.extra_cost
+ copy_cost (x, mode, secondary_class, to_p, &sri));
/* For memory, use the memory move cost, for (hard) registers, use
the cost to move between the register classes, and use 2 for
everything else (constants). */
if (MEM_P (x) || class == NO_REGS)
return sri.extra_cost + memory_move_cost [mode] [class] [to_p != 0];
else if (REG_P (x))
return
(sri.extra_cost
+ move_cost [mode] [REGNO_REG_CLASS (REGNO (x))] [class]);
else
/* If this is a constant, we may eventually want to call rtx_cost
here. */
return sri.extra_cost + COSTS_N_INSNS (1);
}
/* Record the cost of using memory or registers of various classes for
the operands in INSN.
N_ALTS is the number of alternatives.
N_OPS is the number of operands.
OPS is an array of the operands.
MODES are the modes of the operands, in case any are VOIDmode.
CONSTRAINTS are the constraints to use for the operands. This array
is modified by this procedure.
This procedure works alternative by alternative. For each
alternative we assume that we will be able to allocate all pseudos
to their ideal register class and calculate the cost of using that
alternative. Then we compute for each operand that is a
pseudo-register, the cost of having the pseudo allocated to each
register class and using it in that alternative. To this cost is
added the cost of the alternative.
The cost of each class for this insn is its lowest cost among all
the alternatives. */
static void
record_reg_classes (int n_alts, int n_ops, rtx *ops,
enum machine_mode *modes, const char **constraints,
rtx insn, struct costs *op_costs,
enum reg_class *reg_pref)
{
int alt;
int i, j;
rtx set;
/* Process each alternative, each time minimizing an operand's cost
with the cost for each operand in that alternative. */
for (alt = 0; alt < n_alts; alt++)
{
struct costs this_op_costs [MAX_RECOG_OPERANDS];
enum reg_class classes [MAX_RECOG_OPERANDS];
int allows_mem [MAX_RECOG_OPERANDS];
int class;
int alt_fail = 0;
int alt_cost = 0;
for (i = 0; i < n_ops; i++)
{
unsigned char c;
const char *p = constraints [i];
rtx op = ops [i];
enum machine_mode mode = modes [i];
int allows_addr = 0;
int win = 0;
/* Initially show we know nothing about the register class. */
classes [i] = NO_REGS;
allows_mem [i] = 0;
/* If this operand has no constraints at all, we can
conclude nothing about it since anything is valid. */
if (*p == 0)
{
if (REG_P (op) && REGNO (op) >= FIRST_PSEUDO_REGISTER)
memset (&this_op_costs [i], 0, sizeof this_op_costs [i]);
continue;
}
/* If this alternative is only relevant when this operand
matches a previous operand, we do different things
depending on whether this operand is a pseudo-reg or not.
We must process any modifiers for the operand before we
can make this test. */
while (*p == '%' || *p == '=' || *p == '+' || *p == '&')
p++;
if (p [0] >= '0' && p [0] <= '0' + i && (p [1] == ',' || p [1] == 0))
{
/* Copy class and whether memory is allowed from the
matching alternative. Then perform any needed cost
computations and/or adjustments. */
j = p [0] - '0';
classes [i] = classes [j];
allows_mem [i] = allows_mem [j];
if (! REG_P (op) || REGNO (op) < FIRST_PSEUDO_REGISTER)
{
/* If this matches the other operand, we have no
added cost and we win. */
if (rtx_equal_p (ops [j], op))
win = 1;
/* If we can put the other operand into a register,
add to the cost of this alternative the cost to
copy this operand to the register used for the
other operand. */
else if (classes [j] != NO_REGS)
{
alt_cost += copy_cost (op, mode, classes [j], 1, NULL);
win = 1;
}
}
else if (! REG_P (ops [j])
|| REGNO (ops [j]) < FIRST_PSEUDO_REGISTER)
{
/* This op is a pseudo but the one it matches is
not. */
/* If we can't put the other operand into a
register, this alternative can't be used. */
if (classes [j] == NO_REGS)
alt_fail = 1;
/* Otherwise, add to the cost of this alternative
the cost to copy the other operand to the
register used for this operand. */
else
alt_cost
+= copy_cost (ops [j], mode, classes [j], 1, NULL);
}
else
{
/* The costs of this operand are not the same as the
other operand since move costs are not symmetric.
Moreover, if we cannot tie them, this alternative
needs to do a copy, which is one instruction. */
struct costs *pp = &this_op_costs [i];
for (class = 0; class < N_REG_CLASSES; class++)
pp->cost [class]
= ((recog_data.operand_type [i] != OP_OUT
? may_move_in_cost [mode] [class] [classes [i]]
: 0)
+ (recog_data.operand_type [i] != OP_IN
? may_move_out_cost [mode] [classes [i]] [class]
: 0));
/* If the alternative actually allows memory, make
things a bit cheaper since we won't need an extra
insn to load it. */
pp->mem_cost
= ((recog_data.operand_type [i] != OP_IN
? memory_move_cost [mode] [classes [i]] [0]
: 0)
+ (recog_data.operand_type [i] != OP_OUT
? memory_move_cost [mode] [classes [i]] [1]
: 0) - allows_mem [i]);
/* If we have assigned a class to this pseudo in our
first pass, add a cost to this alternative
corresponding to what we would add if this pseudo
were not in the appropriate class. We could use
cover class here but it is less accurate
approximation. */
if (reg_pref)
alt_cost
+= (may_move_in_cost [mode]
[reg_pref [PSEUDO_NUM
(curr_regno_pseudo_map [REGNO (op)])]]
[classes [i]]);
if (REGNO (ops [i]) != REGNO (ops [j])
&& ! find_reg_note (insn, REG_DEAD, op))
alt_cost += 2;
/* This is in place of ordinary cost computation for
this operand, so skip to the end of the
alternative (should be just one character). */
while (*p && *p++ != ',')
;
constraints [i] = p;
continue;
}
}
/* Scan all the constraint letters. See if the operand
matches any of the constraints. Collect the valid
register classes and see if this operand accepts
memory. */
while ((c = *p))
{
switch (c)
{
case ',':
break;
case '*':
/* Ignore the next letter for this pass. */
c = *++p;
break;
case '?':
alt_cost += 2;
case '!': case '#': case '&':
case '0': case '1': case '2': case '3': case '4':
case '5': case '6': case '7': case '8': case '9':
break;
case 'p':
allows_addr = 1;
win = address_operand (op, GET_MODE (op));
/* We know this operand is an address, so we want it
to be allocated to a register that can be the
base of an address, i.e. BASE_REG_CLASS. */
classes [i]
= reg_class_subunion [classes [i]]
[base_reg_class (VOIDmode, ADDRESS, SCRATCH)];
break;
case 'm': case 'o': case 'V':
/* It doesn't seem worth distinguishing between
offsettable and non-offsettable addresses
here. */
allows_mem [i] = 1;
if (MEM_P (op))
win = 1;
break;
case '<':
if (MEM_P (op)
&& (GET_CODE (XEXP (op, 0)) == PRE_DEC
|| GET_CODE (XEXP (op, 0)) == POST_DEC))
win = 1;
break;
case '>':
if (MEM_P (op)
&& (GET_CODE (XEXP (op, 0)) == PRE_INC
|| GET_CODE (XEXP (op, 0)) == POST_INC))
win = 1;
break;
case 'E':
case 'F':
if (GET_CODE (op) == CONST_DOUBLE
|| (GET_CODE (op) == CONST_VECTOR
&& (GET_MODE_CLASS (GET_MODE (op))
== MODE_VECTOR_FLOAT)))
win = 1;
break;
case 'G':
case 'H':
if (GET_CODE (op) == CONST_DOUBLE
&& CONST_DOUBLE_OK_FOR_CONSTRAINT_P (op, c, p))
win = 1;
break;
case 's':
if (GET_CODE (op) == CONST_INT
|| (GET_CODE (op) == CONST_DOUBLE
&& GET_MODE (op) == VOIDmode))
break;
case 'i':
if (CONSTANT_P (op)
&& (! flag_pic || LEGITIMATE_PIC_OPERAND_P (op)))
win = 1;
break;
case 'n':
if (GET_CODE (op) == CONST_INT
|| (GET_CODE (op) == CONST_DOUBLE
&& GET_MODE (op) == VOIDmode))
win = 1;
break;
case 'I':
case 'J':
case 'K':
case 'L':
case 'M':
case 'N':
case 'O':
case 'P':
if (GET_CODE (op) == CONST_INT
&& CONST_OK_FOR_CONSTRAINT_P (INTVAL (op), c, p))
win = 1;
break;
case 'X':
win = 1;
break;
case 'g':
if (MEM_P (op)
|| (CONSTANT_P (op)
&& (! flag_pic || LEGITIMATE_PIC_OPERAND_P (op))))
win = 1;
allows_mem [i] = 1;
case 'r':
classes [i]
= reg_class_subunion [classes [i]] [GENERAL_REGS];
break;
default:
if (REG_CLASS_FROM_CONSTRAINT (c, p) != NO_REGS)
classes [i]
= reg_class_subunion [classes [i]]
[REG_CLASS_FROM_CONSTRAINT (c, p)];
#ifdef EXTRA_CONSTRAINT_STR
else if (EXTRA_CONSTRAINT_STR (op, c, p))
win = 1;
if (EXTRA_MEMORY_CONSTRAINT (c, p))
{
/* Every MEM can be reloaded to fit. */
allows_mem [i] = 1;
if (MEM_P (op))
win = 1;
}
if (EXTRA_ADDRESS_CONSTRAINT (c, p))
{
/* Every address can be reloaded to fit. */
allows_addr = 1;
if (address_operand (op, GET_MODE (op)))
win = 1;
/* We know this operand is an address, so we
want it to be allocated to a register that
can be the base of an address,
i.e. BASE_REG_CLASS. */
classes [i]
= reg_class_subunion [classes [i]]
[base_reg_class (VOIDmode, ADDRESS, SCRATCH)];
}
#endif
break;
}
p += CONSTRAINT_LEN (c, p);
if (c == ',')
break;
}
constraints [i] = p;
/* How we account for this operand now depends on whether it
is a pseudo register or not. If it is, we first check if
any register classes are valid. If not, we ignore this
alternative, since we want to assume that all pseudos get
allocated for register preferencing. If some register
class is valid, compute the costs of moving the pseudo
into that class. */
if (REG_P (op) && REGNO (op) >= FIRST_PSEUDO_REGISTER)
{
if (classes [i] == NO_REGS)
{
/* We must always fail if the operand is a REG, but
we did not find a suitable class.
Otherwise we may perform an uninitialized read
from this_op_costs after the `continue' statement
below. */
alt_fail = 1;
}
else
{
struct costs *pp = &this_op_costs [i];
for (class = 0; class < N_REG_CLASSES; class++)
pp->cost [class]
= ((recog_data.operand_type [i] != OP_OUT
? may_move_in_cost [mode] [class] [classes [i]]
: 0)
+ (recog_data.operand_type [i] != OP_IN
? may_move_out_cost [mode] [classes [i]] [class]
: 0));
/* If the alternative actually allows memory, make
things a bit cheaper since we won't need an extra
insn to load it. */
pp->mem_cost
= ((recog_data.operand_type [i] != OP_IN
? memory_move_cost [mode] [classes [i]] [0]
: 0)
+ (recog_data.operand_type [i] != OP_OUT
? memory_move_cost [mode] [classes [i]] [1]
: 0) - allows_mem [i]);
/* If we have assigned a class to this pseudo in our
first pass, add a cost to this alternative
corresponding to what we would add if this pseudo
were not in the appropriate class. We could use
cover class here but it is less accurate
approximation. */
if (reg_pref)
alt_cost
+= (may_move_in_cost [mode]
[reg_pref [PSEUDO_NUM
(curr_regno_pseudo_map [REGNO (op)])]]
[classes [i]]);
}
}
/* Otherwise, if this alternative wins, either because we
have already determined that or if we have a hard
register of the proper class, there is no cost for this
alternative. */
else if (win || (REG_P (op)
&& reg_fits_class_p (op, classes [i],
0, GET_MODE (op))))
;
/* If registers are valid, the cost of this alternative
includes copying the object to and/or from a
register. */
else if (classes [i] != NO_REGS)
{
if (recog_data.operand_type [i] != OP_OUT)
alt_cost += copy_cost (op, mode, classes [i], 1, NULL);
if (recog_data.operand_type [i] != OP_IN)
alt_cost += copy_cost (op, mode, classes [i], 0, NULL);
}
/* The only other way this alternative can be used is if
this is a constant that could be placed into memory. */
else if (CONSTANT_P (op) && (allows_addr || allows_mem [i]))
alt_cost += memory_move_cost [mode] [classes [i]] [1];
else
alt_fail = 1;
}
if (alt_fail)
continue;
/* Finally, update the costs with the information we've
calculated about this alternative. */
for (i = 0; i < n_ops; i++)
if (REG_P (ops [i])
&& REGNO (ops [i]) >= FIRST_PSEUDO_REGISTER)
{
struct costs *pp = &op_costs [i], *qq = &this_op_costs [i];
int scale = 1 + (recog_data.operand_type [i] == OP_INOUT);
pp->mem_cost = MIN (pp->mem_cost,
(qq->mem_cost + alt_cost) * scale);
for (class = 0; class < N_REG_CLASSES; class++)
pp->cost [class] = MIN (pp->cost [class],
(qq->cost [class] + alt_cost) * scale);
}
}
/* If this insn is a single set copying operand 1 to operand 0 and
one operand is a pseudo with the other a hard reg or a pseudo
that prefers a register that is in its own register class then we
may want to adjust the cost of that register class to -1.
Avoid the adjustment if the source does not die to avoid
stressing of register allocator by preferrencing two colliding
registers into single class.
Also avoid the adjustment if a copy between registers of the
class is expensive (ten times the cost of a default copy is
considered arbitrarily expensive). This avoids losing when the
preferred class is very expensive as the source of a copy
instruction. */
if ((set = single_set (insn)) != 0
&& ops [0] == SET_DEST (set) && ops [1] == SET_SRC (set)
&& REG_P (ops [0]) && REG_P (ops [1])
&& find_regno_note (insn, REG_DEAD, REGNO (ops [1])))
for (i = 0; i <= 1; i++)
if (REGNO (ops [i]) >= FIRST_PSEUDO_REGISTER)
{
unsigned int regno = REGNO (ops [!i]);
enum machine_mode mode = GET_MODE (ops [!i]);
int class;
unsigned int nr;
if (regno >= FIRST_PSEUDO_REGISTER && reg_pref != 0)
{
enum reg_class pref;
/* We could use cover class here but it is less accurate
approximation. */
pref = reg_pref [PSEUDO_NUM (curr_regno_pseudo_map [regno])];
if ((reg_class_size [pref]
== (unsigned) CLASS_MAX_NREGS (pref, mode))
&& register_move_cost [mode] [pref] [pref] < 10 * 2)
op_costs [i].cost [pref] = -1;
}
else if (regno < FIRST_PSEUDO_REGISTER)
for (class = 0; class < N_REG_CLASSES; class++)
if (TEST_HARD_REG_BIT (reg_class_contents [class], regno)
&& (reg_class_size [class]
== (unsigned) CLASS_MAX_NREGS (class, mode)))
{
if (reg_class_size [class] == 1)
op_costs [i].cost [class] = -1;
else
{
for (nr = 0;
nr < (unsigned) hard_regno_nregs [regno] [mode];
nr++)
if (! TEST_HARD_REG_BIT (reg_class_contents [class],
regno + nr))
break;
if (nr == (unsigned) hard_regno_nregs [regno] [mode])
op_costs [i].cost [class] = -1;
}
}
}
}
/* Wrapper around REGNO_OK_FOR_INDEX_P, to allow pseudo registers. */
static inline int
ok_for_index_p_nonstrict (rtx reg)
{
unsigned regno = REGNO (reg);
return regno >= FIRST_PSEUDO_REGISTER || REGNO_OK_FOR_INDEX_P (regno);
}
/* A version of regno_ok_for_base_p for use during regclass, when all
pseudos should count as OK. Arguments as for
regno_ok_for_base_p. */
static inline int
ok_for_base_p_nonstrict (rtx reg, enum machine_mode mode,
enum rtx_code outer_code, enum rtx_code index_code)
{
unsigned regno = REGNO (reg);
if (regno >= FIRST_PSEUDO_REGISTER)
return TRUE;
return ok_for_base_p_1 (regno, mode, outer_code, index_code);
}
/* Record the pseudo registers we must reload into hard registers in a
subexpression of a memory address, X.
If CONTEXT is 0, we are looking at the base part of an address,
otherwise we are looking at the index part.
MODE is the mode of the memory reference; OUTER_CODE and INDEX_CODE
give the context that the rtx appears in. These three arguments
are passed down to base_reg_class.
SCALE is twice the amount to multiply the cost by (it is twice so
we can represent half-cost adjustments). */
static void
record_address_regs (enum machine_mode mode, rtx x, int context,
enum rtx_code outer_code, enum rtx_code index_code,
int scale)
{
enum rtx_code code = GET_CODE (x);
enum reg_class class;
if (context == 1)
class = INDEX_REG_CLASS;
else
class = base_reg_class (mode, outer_code, index_code);
switch (code)
{
case CONST_INT:
case CONST:
case CC0:
case PC:
case SYMBOL_REF:
case LABEL_REF:
return;
case PLUS:
/* When we have an address that is a sum, we must determine
whether registers are "base" or "index" regs. If there is a
sum of two registers, we must choose one to be the "base".
Luckily, we can use the REG_POINTER to make a good choice
most of the time. We only need to do this on machines that
can have two registers in an address and where the base and
index register classes are different.
??? This code used to set REGNO_POINTER_FLAG in some cases,
but that seems bogus since it should only be set when we are
sure the register is being used as a pointer. */
{
rtx arg0 = XEXP (x, 0);
rtx arg1 = XEXP (x, 1);
enum rtx_code code0 = GET_CODE (arg0);
enum rtx_code code1 = GET_CODE (arg1);
/* Look inside subregs. */
if (code0 == SUBREG)
arg0 = SUBREG_REG (arg0), code0 = GET_CODE (arg0);
if (code1 == SUBREG)
arg1 = SUBREG_REG (arg1), code1 = GET_CODE (arg1);
/* If this machine only allows one register per address, it
must be in the first operand. */
if (MAX_REGS_PER_ADDRESS == 1)
record_address_regs (mode, arg0, 0, PLUS, code1, scale);
/* If index and base registers are the same on this machine,
just record registers in any non-constant operands. We
assume here, as well as in the tests below, that all
addresses are in canonical form. */
else if (INDEX_REG_CLASS == base_reg_class (VOIDmode, PLUS, SCRATCH))
{
record_address_regs (mode, arg0, context, PLUS, code1, scale);
if (! CONSTANT_P (arg1))
record_address_regs (mode, arg1, context, PLUS, code0, scale);
}
/* If the second operand is a constant integer, it doesn't
change what class the first operand must be. */
else if (code1 == CONST_INT || code1 == CONST_DOUBLE)
record_address_regs (mode, arg0, context, PLUS, code1, scale);
/* If the second operand is a symbolic constant, the first
operand must be an index register. */
else if (code1 == SYMBOL_REF || code1 == CONST || code1 == LABEL_REF)
record_address_regs (mode, arg0, 1, PLUS, code1, scale);
/* If both operands are registers but one is already a hard
register of index or reg-base class, give the other the
class that the hard register is not. */
else if (code0 == REG && code1 == REG
&& REGNO (arg0) < FIRST_PSEUDO_REGISTER
&& (ok_for_base_p_nonstrict (arg0, mode, PLUS, REG)
|| ok_for_index_p_nonstrict (arg0)))
record_address_regs (mode, arg1,
ok_for_base_p_nonstrict (arg0, mode, PLUS, REG)
? 1 : 0,
PLUS, REG, scale);
else if (code0 == REG && code1 == REG
&& REGNO (arg1) < FIRST_PSEUDO_REGISTER
&& (ok_for_base_p_nonstrict (arg1, mode, PLUS, REG)
|| ok_for_index_p_nonstrict (arg1)))
record_address_regs (mode, arg0,
ok_for_base_p_nonstrict (arg1, mode, PLUS, REG)
? 1 : 0,
PLUS, REG, scale);
/* If one operand is known to be a pointer, it must be the
base with the other operand the index. Likewise if the
other operand is a MULT. */
else if ((code0 == REG && REG_POINTER (arg0)) || code1 == MULT)
{
record_address_regs (mode, arg0, 0, PLUS, code1, scale);
record_address_regs (mode, arg1, 1, PLUS, code0, scale);
}
else if ((code1 == REG && REG_POINTER (arg1)) || code0 == MULT)
{
record_address_regs (mode, arg0, 1, PLUS, code1, scale);
record_address_regs (mode, arg1, 0, PLUS, code0, scale);
}
/* Otherwise, count equal chances that each might be a base or
index register. This case should be rare. */
else
{
record_address_regs (mode, arg0, 0, PLUS, code1, scale / 2);
record_address_regs (mode, arg0, 1, PLUS, code1, scale / 2);
record_address_regs (mode, arg1, 0, PLUS, code0, scale / 2);
record_address_regs (mode, arg1, 1, PLUS, code0, scale / 2);
}
}
break;
/* Double the importance of a pseudo that is incremented or
decremented, since it would take two extra insns if it ends
up in the wrong place. */
case POST_MODIFY:
case PRE_MODIFY:
record_address_regs (mode, XEXP (x, 0), 0, code,
GET_CODE (XEXP (XEXP (x, 1), 1)), 2 * scale);
if (REG_P (XEXP (XEXP (x, 1), 1)))
record_address_regs (mode, XEXP (XEXP (x, 1), 1), 1, code, REG,
2 * scale);
break;
case POST_INC:
case PRE_INC:
case POST_DEC:
case PRE_DEC:
/* Double the importance of a pseudo that is incremented or
decremented, since it would take two extra insns if it ends
up in the wrong place. If the operand is a pseudo-register,
show it is being used in an INC_DEC context. */
#ifdef FORBIDDEN_INC_DEC_CLASSES
if (REG_P (XEXP (x, 0))
&& REGNO (XEXP (x, 0)) >= FIRST_PSEUDO_REGISTER)
in_inc_dec [PSEUDO_NUM (curr_regno_pseudo_map [REGNO (XEXP (x, 0))])]
= 1;
#endif
record_address_regs (mode, XEXP (x, 0), 0, code, SCRATCH, 2 * scale);
break;
case REG:
{
struct costs *pp;
int i;
if (REGNO (x) < FIRST_PSEUDO_REGISTER)
break;
pp = &costs [PSEUDO_NUM (curr_regno_pseudo_map [REGNO (x)])];
pp->mem_cost += (memory_move_cost [Pmode] [class] [1] * scale) / 2;
for (i = 0; i < N_REG_CLASSES; i++)
pp->cost [i] += (may_move_in_cost [Pmode] [i] [class] * scale) / 2;
}
break;
default:
{
const char *fmt = GET_RTX_FORMAT (code);
int i;
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
if (fmt [i] == 'e')
record_address_regs (mode, XEXP (x, i), context, code, SCRATCH,
scale);
}
}
}
/* Calculate the costs of insn operands. */
static void
record_operand_costs (rtx insn, struct costs *op_costs,
enum reg_class *reg_pref)
{
const char *constraints [MAX_RECOG_OPERANDS];
enum machine_mode modes [MAX_RECOG_OPERANDS];
int i;
for (i = 0; i < recog_data.n_operands; i++)
{
constraints [i] = recog_data.constraints [i];
modes [i] = recog_data.operand_mode [i];
}
/* If we get here, we are set up to record the costs of all the
operands for this insn. Start by initializing the costs. Then
handle any address registers. Finally record the desired classes
for any pseudos, doing it twice if some pair of operands are
commutative. */
for (i = 0; i < recog_data.n_operands; i++)
{
op_costs [i] = init_cost;
if (GET_CODE (recog_data.operand [i]) == SUBREG)
recog_data.operand [i] = SUBREG_REG (recog_data.operand [i]);
if (MEM_P (recog_data.operand [i]))
record_address_regs (GET_MODE (recog_data.operand [i]),
XEXP (recog_data.operand [i], 0),
0, MEM, SCRATCH, frequency * 2);
else if (constraints [i] [0] == 'p'
|| EXTRA_ADDRESS_CONSTRAINT (constraints [i] [0],
constraints [i]))
record_address_regs (VOIDmode, recog_data.operand [i], 0, ADDRESS,
SCRATCH, frequency * 2);
}
/* Check for commutative in a separate loop so everything will have
been initialized. We must do this even if one operand is a
constant--see addsi3 in m68k.md. */
for (i = 0; i < (int) recog_data.n_operands - 1; i++)
if (constraints [i] [0] == '%')
{
const char *xconstraints [MAX_RECOG_OPERANDS];
int j;
/* Handle commutative operands by swapping the constraints.
We assume the modes are the same. */
for (j = 0; j < recog_data.n_operands; j++)
xconstraints [j] = constraints [j];
xconstraints [i] = constraints [i+1];
xconstraints [i+1] = constraints [i];
record_reg_classes (recog_data.n_alternatives, recog_data.n_operands,
recog_data.operand, modes,
xconstraints, insn, op_costs, reg_pref);
}
record_reg_classes (recog_data.n_alternatives, recog_data.n_operands,
recog_data.operand, modes,
constraints, insn, op_costs, reg_pref);
}
/* Process one insn INSN. Scan it and record each time it would save
code to put a certain pseudos in a certain class. Return the last
insn processed, so that the scan can be continued from there. */
static rtx
scan_one_insn (rtx insn)
{
enum rtx_code pat_code;
rtx set, note;
int i, j;
struct costs op_costs [MAX_RECOG_OPERANDS];
if (!INSN_P (insn))
return insn;
pat_code = GET_CODE (PATTERN (insn));
if (pat_code == USE || pat_code == CLOBBER || pat_code == ASM_INPUT
|| pat_code == ADDR_VEC || pat_code == ADDR_DIFF_VEC)
return insn;
set = single_set (insn);
extract_insn (insn);
/* If this insn loads a parameter from its stack slot, then it
represents a savings, rather than a cost, if the parameter is
stored in memory. Record this fact. */
if (set != 0 && REG_P (SET_DEST (set)) && MEM_P (SET_SRC (set))
&& (note = find_reg_note (insn, REG_EQUIV, NULL_RTX)) != NULL_RTX
&& MEM_P (XEXP (note, 0)))
{
costs [PSEUDO_NUM (curr_regno_pseudo_map
[REGNO (SET_DEST (set))])].mem_cost
-= (memory_move_cost [GET_MODE (SET_DEST (set))] [GENERAL_REGS] [1]
* frequency);
record_address_regs (GET_MODE (SET_SRC (set)), XEXP (SET_SRC (set), 0),
0, MEM, SCRATCH, frequency * 2);
return insn;
}
record_operand_costs (insn, op_costs, reg_pref);
/* Now add the cost for each operand to the total costs for its
pseudo. */
for (i = 0; i < recog_data.n_operands; i++)
if (REG_P (recog_data.operand [i])
&& REGNO (recog_data.operand [i]) >= FIRST_PSEUDO_REGISTER)
{
int regno = REGNO (recog_data.operand [i]);
struct costs *p = &costs [PSEUDO_NUM (curr_regno_pseudo_map [regno])];
struct costs *q = &op_costs [i];
p->mem_cost += q->mem_cost * frequency;
for (j = 0; j < N_REG_CLASSES; j++)
p->cost [j] += q->cost [j] * frequency;
}
return insn;
}
/* Dump pseudos costs. */
static void
print_costs (FILE *f)
{
int i;
fprintf (f, "\n");
for (i = 0; i < pseudos_num; i++)
{
int class;
basic_block bb;
pseudo_t p = pseudos [i];
int regno = PSEUDO_REGNO (p);
fprintf (f, " p%d(r%d,", i, regno);
if ((bb = PSEUDO_LOOP_TREE_NODE (p)->bb) != NULL)
fprintf (f, "b%d", bb->index);
else
fprintf (f, "l%d", PSEUDO_LOOP_TREE_NODE (p)->loop->num);
fprintf (f, ") costs:");
for (class = 0; class < (int) N_REG_CLASSES; class++)
if (contains_reg_of_mode [class] [PSEUDO_REGNO_MODE (regno)]
#ifdef FORBIDDEN_INC_DEC_CLASSES
&& (! in_inc_dec [i] || ! forbidden_inc_dec_class [class])
#endif
#ifdef CANNOT_CHANGE_MODE_CLASS
&& ! invalid_mode_change_p (i, (enum reg_class) class,
PSEUDO_REGNO_MODE (regno))
#endif
)
fprintf (f, " %s:%d", reg_class_names [class],
costs [i].cost [class]);
fprintf (f, " MEM:%i\n", costs [i].mem_cost);
}
}
/* The function traverses basic blocks represented by LOOP_TREE_NODE
to find the costs of the pseudos. */
static void
process_bb_node_for_costs (struct ira_loop_tree_node *loop_tree_node)
{
basic_block bb;
rtx insn;
bb = loop_tree_node->bb;
if (bb == NULL)
return;
frequency = REG_FREQ_FROM_BB (bb);
if (frequency == 0)
frequency = 1;
curr_regno_pseudo_map = ira_curr_loop_tree_node->regno_pseudo_map;
FOR_BB_INSNS (bb, insn)
insn = scan_one_insn (insn);
}
/* Entry function to find costs of each class cost for pesudos and
their best classes. */
static void
find_pseudo_class_costs (void)
{
int i;
int pass;
basic_block bb;
init_recog ();
#ifdef FORBIDDEN_INC_DEC_CLASSES
in_inc_dec = ira_allocate (sizeof (char) * pseudos_num);
#endif /* FORBIDDEN_INC_DEC_CLASSES */
reg_pref = NULL;
/* Normally we scan the insns once and determine the best class to
use for each pseudo. However, if -fexpensive_optimizations are
on, we do so twice, the second time using the tentative best
classes to guide the selection. */
for (pass = 0; pass <= flag_expensive_optimizations; pass++)
{
if (ira_dump_file)
fprintf (ira_dump_file, "\nPass %i\n\n",pass);
/* Zero out our accumulation of the cost of each class for each
pseudo. */
memset (costs, 0, pseudos_num * sizeof (struct costs));
#ifdef FORBIDDEN_INC_DEC_CLASSES
memset (in_inc_dec, 0, pseudos_num * sizeof (char));
#endif
/* Scan the instructions and record each time it would save code
to put a certain pseudo in a certain class. */
traverse_loop_tree (ira_loop_tree_root, process_bb_node_for_costs, NULL);
/* Now for each pseudo look at how desirable each class is and
find which class is preferred. Store that in `prefclass'.
Record in `altclass' the largest register class any of whose
registers is better than memory. */
if (pass == 0)
reg_pref = reg_pref_buffer;
for (i = max_reg_num () - 1; i >= FIRST_PSEUDO_REGISTER; i--)
{
pseudo_t p, father_p;
int class, p_num, father_p_num;
struct ira_loop_tree_node *father;
int best_cost = (1 << (HOST_BITS_PER_INT - 2)) - 1;
enum reg_class best = ALL_REGS;
#ifdef FORBIDDEN_INC_DEC_CLASSES
int inc_dec_p = FALSE;
#endif
struct costs reg_costs;
if (regno_pseudo_map [i] == NULL)
continue;
memset (®_costs, 0, sizeof (struct costs));
for (p = regno_pseudo_map [i];
p != NULL;
p = PSEUDO_NEXT_REGNO_PSEUDO (p))
{
p_num = PSEUDO_NUM (p);
if (bitmap_bit_p (PSEUDO_LOOP_TREE_NODE (p)->mentioned_pseudos,
PSEUDO_NUM (p)))
{
if (flag_ira_algorithm == IRA_ALGORITHM_REGIONAL
&& (father = PSEUDO_LOOP_TREE_NODE (p)->father) != NULL
&& (father_p = father->regno_pseudo_map [i]) != NULL)
{
father_p_num = PSEUDO_NUM (father_p);
for (class = (int) ALL_REGS - 1; class > 0; class--)
costs [father_p_num].cost [class]
+= costs [p_num].cost [class];
costs [father_p_num].mem_cost += costs [p_num].mem_cost;
}
for (class = (int) ALL_REGS - 1; class > 0; class--)
reg_costs.cost [class] += costs [p_num].cost [class];
reg_costs.mem_cost += costs [p_num].mem_cost;
#ifdef FORBIDDEN_INC_DEC_CLASSES
if (in_inc_dec [p_num])
inc_dec_p = TRUE;
#endif
}
}
for (class = (int) ALL_REGS - 1; class > 0; class--)
{
/* Ignore classes that are too small for this operand or
invalid for an operand that was auto-incremented. */
if (! contains_reg_of_mode [class] [PSEUDO_REGNO_MODE (i)]
#ifdef FORBIDDEN_INC_DEC_CLASSES
|| (inc_dec_p && forbidden_inc_dec_class [class])
#endif
#ifdef CANNOT_CHANGE_MODE_CLASS
|| invalid_mode_change_p (i, (enum reg_class) class,
PSEUDO_REGNO_MODE (i))
#endif
)
;
else if (reg_costs.cost [class] < best_cost)
{
best_cost = reg_costs.cost [class];
best = (enum reg_class) class;
}
else if (reg_costs.cost [class] == best_cost)
best = reg_class_subunion [best] [class];
}
for (p = regno_pseudo_map [i];
p != NULL;
p = PSEUDO_NEXT_REGNO_PSEUDO (p))
{
p_num = PSEUDO_NUM (p);
if (! bitmap_bit_p (PSEUDO_LOOP_TREE_NODE (p)->mentioned_pseudos,
p_num))
memset (&costs [p_num], 0, sizeof (struct costs));
if (ira_dump_file && (pass == 0 || reg_pref [p_num] != best))
{
fprintf (ira_dump_file, " p%d (r%d,", p_num, i);
if ((bb = PSEUDO_LOOP_TREE_NODE (p)->bb) != NULL)
fprintf (ira_dump_file, "b%d", bb->index);
else
fprintf (ira_dump_file, "l%d",
PSEUDO_LOOP_TREE_NODE (p)->loop->num);
fprintf (ira_dump_file, ") best %s, cover %s\n",
reg_class_names [best],
reg_class_names [class_translate [best]]);
}
reg_pref [p_num] = best;
}
}
if (ira_dump_file)
{
print_costs (ira_dump_file);
fprintf (ira_dump_file,"\n");
}
}
#ifdef FORBIDDEN_INC_DEC_CLASSES
ira_free (in_inc_dec);
#endif
}
/* Process moves involving hard regs to modify pseudo hard register
costs. We can do this only after determining pseudo cover class.
If a hard register forms a register class, than moves with the hard
register are already taken into account slightly in class costs for
the pseudo. */
static void
process_bb_node_for_hard_reg_moves (struct ira_loop_tree_node *loop_tree_node)
{
int i, freq, cost, src_regno, dst_regno, hard_regno, to_p;
pseudo_t p;
enum reg_class class, hard_reg_class;
enum machine_mode mode;
basic_block bb;
rtx insn, set, src, dst;
bb = loop_tree_node->bb;
if (bb == NULL)
return;
freq = REG_FREQ_FROM_BB (bb);
if (freq == 0)
freq = 1;
curr_regno_pseudo_map = ira_curr_loop_tree_node->regno_pseudo_map;
FOR_BB_INSNS (bb, insn)
{
if (! INSN_P (insn))
continue;
set = single_set (insn);
if (set == NULL_RTX)
continue;
dst = SET_DEST (set);
src = SET_SRC (set);
if (! REG_P (dst) || ! REG_P (src))
continue;
dst_regno = REGNO (dst);
src_regno = REGNO (src);
if (dst_regno >= FIRST_PSEUDO_REGISTER
&& src_regno < FIRST_PSEUDO_REGISTER)
{
hard_regno = src_regno;
to_p = TRUE;
p = curr_regno_pseudo_map [dst_regno];
}
else if (src_regno >= FIRST_PSEUDO_REGISTER
&& dst_regno < FIRST_PSEUDO_REGISTER)
{
hard_regno = dst_regno;
to_p = FALSE;
p = curr_regno_pseudo_map [src_regno];
}
else
continue;
class = PSEUDO_COVER_CLASS (p);
if (! TEST_HARD_REG_BIT (reg_class_contents [class], hard_regno))
continue;
i = class_hard_reg_index [class] [hard_regno];
if (i < 0)
continue;
mode = PSEUDO_MODE (p);
hard_reg_class = REGNO_REG_CLASS (hard_regno);
cost = (to_p ? register_move_cost [mode] [hard_reg_class] [class]
: register_move_cost [mode] [class] [hard_reg_class]) * freq;
PSEUDO_HARD_REG_COSTS (p) [i] -= cost;
PSEUDO_CONFLICT_HARD_REG_COSTS (p) [i] -= cost;
PSEUDO_COVER_CLASS_COST (p) = MIN (PSEUDO_COVER_CLASS_COST (p),
PSEUDO_HARD_REG_COSTS (p) [i]);
if (flag_ira_algorithm == IRA_ALGORITHM_REGIONAL)
{
struct ira_loop_tree_node *father;
int regno = PSEUDO_REGNO (p);
for (;;)
{
if ((father = PSEUDO_LOOP_TREE_NODE (p)->father) == NULL)
break;
if ((p = father->regno_pseudo_map [regno]) == NULL)
break;
PSEUDO_HARD_REG_COSTS (p) [i] -= cost;
PSEUDO_CONFLICT_HARD_REG_COSTS (p) [i] -= cost;
PSEUDO_COVER_CLASS_COST (p)
= MIN (PSEUDO_COVER_CLASS_COST (p),
PSEUDO_HARD_REG_COSTS (p) [i]);
}
}
}
}
/* After we find hard register and memory costs for pseudos, define
its cover class and modify hard register cost because insns moving
pseudo to/from hard registers. */
static void
setup_pseudo_cover_class_and_costs (void)
{
int i, j, n, regno, cost, *reg_costs, *reg_conflict_costs;
enum reg_class cover_class, class;
enum machine_mode mode;
pseudo_t p;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
mode = PSEUDO_MODE (p);
cover_class = class_translate [reg_pref [i]];
ira_assert (reg_pref [i] == NO_REGS || cover_class != NO_REGS);
PSEUDO_ORIGINAL_MEMORY_COST (p)
= PSEUDO_MEMORY_COST (p) = costs [i].mem_cost;
#if 0
/* Should we ??? */
if (costs [i].mem_cost < costs [i].cost [cover_class])
cover_class = NO_REGS;
#endif
PSEUDO_COVER_CLASS (p) = cover_class;
if (cover_class == NO_REGS)
continue;
PSEUDO_AVAILABLE_REGS_NUM (p) = available_class_regs [cover_class];
PSEUDO_COVER_CLASS_COST (p) = costs [i].cost [reg_pref [i]];
n = class_hard_regs_num [cover_class];
PSEUDO_HARD_REG_COSTS (p) = reg_costs = ira_allocate (n * sizeof (int));
PSEUDO_CONFLICT_HARD_REG_COSTS (p)
= reg_conflict_costs = ira_allocate (n * sizeof (int));
PSEUDO_CURR_HARD_REG_COSTS (p) = ira_allocate (n * sizeof (int));
PSEUDO_CURR_CONFLICT_HARD_REG_COSTS (p)
= ira_allocate (n * sizeof (int));
memset (reg_conflict_costs, 0, n * sizeof (int));
for (j = n - 1; j >= 0; j--)
{
regno = class_hard_regs [cover_class] [j];
class = REGNO_REG_CLASS (regno);
/* ??? what is cost AREG when DImode. Should be ok. */
cost = costs [i].cost [class];
reg_costs [j] = cost;
}
}
traverse_loop_tree (ira_loop_tree_root,
process_bb_node_for_hard_reg_moves, NULL);
}
/* Function called once during compiler work. It sets up init_cost
whose values don't depend on the compiled function. */
void
init_ira_costs_once (void)
{
int i;
init_cost.mem_cost = 10000;
for (i = 0; i < N_REG_CLASSES; i++)
init_cost.cost [i] = 10000;
}
/* Entry function which defines cover class, memory and hard register
costs for each pseudo. */
void
ira_costs (void)
{
costs = ira_allocate (sizeof (struct costs) * pseudos_num);
reg_pref_buffer = ira_allocate (sizeof (enum reg_class) * pseudos_num);
find_pseudo_class_costs ();
setup_pseudo_cover_class_and_costs ();
ira_free (reg_pref_buffer);
ira_free (costs);
}
/* This function changes hard register costs for pseudos which lives
trough function calls. The function is called only when we found
all intersected calls during building pseudo conflicts. */
void
tune_pseudo_costs_and_cover_classes (void)
{
int i, j, n, regno, cost, min_cost, *reg_costs;
enum reg_class cover_class, class;
enum machine_mode mode;
pseudo_t p;
for (i = 0; i < pseudos_num; i++)
{
p = pseudos [i];
cover_class = PSEUDO_COVER_CLASS (p);
if (cover_class == NO_REGS)
continue;
mode = PSEUDO_MODE (p);
n = class_hard_regs_num [cover_class];
reg_costs = PSEUDO_HARD_REG_COSTS (p);
min_cost = INT_MAX;
if (PSEUDO_CALLS_CROSSED_NUM (p) != 0)
for (j = n - 1; j >= 0; j--)
{
regno = class_hard_regs [cover_class] [j];
class = REGNO_REG_CLASS (regno);
cost = 0;
/* ??? If only part is call clobbered. */
if (! hard_reg_not_in_set_p (regno, mode, call_used_reg_set))
cost += (PSEUDO_CALL_FREQ (p)
* (memory_move_cost [mode] [class] [0]
+ memory_move_cost [mode] [class] [1]));
reg_costs [j] += cost;
if (min_cost > reg_costs [j])
min_cost = reg_costs [j];
}
if (min_cost == INT_MAX)
continue;
PSEUDO_COVER_CLASS_COST (p) = min_cost;
}
}
/* Integrated Register Allocator. Channging code and generating moves.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "regs.h"
#include "rtl.h"
#include "tm_p.h"
#include "target.h"
#include "flags.h"
#include "obstack.h"
#include "bitmap.h"
#include "hard-reg-set.h"
#include "basic-block.h"
#include "expr.h"
#include "recog.h"
#include "params.h"
#include "timevar.h"
#include "tree-pass.h"
#include "output.h"
#include "reload.h"
#include "errors.h"
#include "df.h"
#include "ira-int.h"
struct move;
static struct move *create_move (pseudo_t, pseudo_t);
static void free_move (struct move *);
static void free_move_list (struct move *);
static int eq_move_lists_p (struct move *, struct move *);
static void change_regs (rtx *);
static void add_to_edge_list (edge, struct move *, int);
static rtx create_new_reg (rtx);
static int subloop_tree_node_p (struct ira_loop_tree_node *,
struct ira_loop_tree_node *);
static void set_pseudo_reg (pseudo_t, rtx);
static int not_modified_p (pseudo_t, pseudo_t);
static void generate_edge_moves (edge);
static void change_loop (struct ira_loop_tree_node *);
static int eq_edge_move_lists_p (VEC(edge,gc) *);
static int can_move_through_p (rtx *, struct move *, int);
static void unify_moves (basic_block, int);
static void traverse_moves (struct move *);
static struct move *modify_move_list (struct move *);
static rtx emit_move_list (struct move *, int);
static void emit_moves (void);
/* The structure represents pseudo shuffling. */
struct move
{
/* The shuffled pseudos. */
pseudo_t from, to;
/* The next move in the sequence. */
struct move *next;
/* Use for finding dependencies. */
int visited_p;
/* The size of the following array. */
int deps_num;
/* Moves on which given move depends on. Dependency can be cyclic.
It means we need a temporary to generates the moves. */
struct move **deps;
};
/* Array of moves (indexed by BB index) which should be put at the
start/end of the corresponding blocks. */
static struct move **at_bb_start, **at_bb_end;
/* Max regno before renaming some pseudo-registers. For example, the
same pseudo-register can be renamed in loop if its allocation is
different outside the loop. */
static int max_regno_before_changing;
/* The function returns new move of pseudos TO and FROM. */
static struct move *
create_move (pseudo_t to, pseudo_t from)
{
struct move *move;
move = ira_allocate (sizeof (struct move));
move->deps = NULL;
move->deps_num = 0;
move->to = to;
move->from = from;
move->next = NULL;
move->visited_p = FALSE;
return move;
}
/* The function frees memory for MOVE and its dependencies. */
static void
free_move (struct move *move)
{
if (move->deps != NULL)
ira_free (move->deps);
ira_free (move);
}
/* The function frees memory for list of the moves given by its
HEAD. */
static void
free_move_list (struct move *head)
{
struct move *next;
for (; head != NULL; head = next)
{
next = head->next;
free_move (head);
}
}
/* The function returns nonzero if the the move list LIST1 and LIST2
are equal (two moves are equal if they shuffles the same
pseudos). */
static int
eq_move_lists_p (struct move *list1, struct move *list2)
{
for (; list1 != NULL && list2 != NULL;
list1 = list1->next, list2 = list2->next)
if (list1->from != list2->from || list1->to != list2->to)
return FALSE;
return list1 == list2;
}
/* This recursive function changes pseudo-registers in *LOC if it is
necessary. */
static void
change_regs (rtx *loc)
{
int i, regno;
const char *fmt;
enum rtx_code code;
if (*loc == NULL_RTX)
return;
code = GET_CODE (*loc);
if (code == REG)
{
regno = REGNO (*loc);
if (regno < FIRST_PSEUDO_REGISTER)
return;
if (regno >= max_regno_before_changing)
/* It is a shared register which was changed already. */
return;
/* ??? That is for reg_equal. */
if (ira_curr_loop_tree_node->regno_pseudo_map [regno] != NULL)
*loc = PSEUDO_REG (ira_curr_loop_tree_node->regno_pseudo_map [regno]);
return;
}
fmt = GET_RTX_FORMAT (code);
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
{
if (fmt[i] == 'e')
change_regs (&XEXP (*loc, i));
else if (fmt[i] == 'E')
{
int j;
for (j = XVECLEN (*loc, i) - 1; j >= 0; j--)
change_regs (&XVECEXP (*loc, i, j));
}
}
}
/* The function attaches MOVE to the edge E. The move is attached to
the head of the list if HEAD_P is nonzero. */
static void
add_to_edge_list (edge e, struct move *move, int head_p)
{
struct move *last;
if (head_p || e->aux == NULL)
{
move->next = e->aux;
e->aux = move;
}
else
{
for (last = e->aux; last->next != NULL; last = last->next)
;
last->next = move;
move->next = NULL;
}
}
/* The function creates and returns new pseudo-register with the same
attributes as ORIGINAL_REG. */
static rtx
create_new_reg (rtx original_reg)
{
rtx new_reg;
new_reg = gen_reg_rtx (GET_MODE (original_reg));
ORIGINAL_REGNO (new_reg) = ORIGINAL_REGNO (original_reg);
REG_USERVAR_P (new_reg) = REG_USERVAR_P (original_reg);
REG_POINTER (new_reg) = REG_POINTER (original_reg);
REG_ATTRS (new_reg) = REG_ATTRS (original_reg);
if (ira_dump_file)
fprintf (ira_dump_file, " Creating newreg=%i from oldreg=%i\n",
REGNO (new_reg), REGNO (original_reg));
return new_reg;
}
/* The function returns non-zero if loop given by SUBNODE inside the
loop given by NODE. */
static int
subloop_tree_node_p (struct ira_loop_tree_node *subnode,
struct ira_loop_tree_node *node)
{
for (; subnode != NULL; subnode = subnode->father)
if (subnode == node)
return TRUE;
return FALSE;
}
/* The function sets up field `reg' to REG for pseudos which has the
same regno as PSEUDO and which are inside the loop corresponding to
PSEUDO. */
static void
set_pseudo_reg (pseudo_t pseudo, rtx reg)
{
pseudo_t p;
struct ira_loop_tree_node *node;
node = PSEUDO_LOOP_TREE_NODE (pseudo);
for (p = regno_pseudo_map [PSEUDO_REGNO (pseudo)];
p != NULL;
p = PSEUDO_NEXT_REGNO_PSEUDO (p))
if (subloop_tree_node_p (PSEUDO_LOOP_TREE_NODE (p), node))
PSEUDO_REG (p) = reg;
}
/* The following function returns nonzero if move insn of SRC_PSEUDO
to DEST_PSEUDO does not change value of the destination. */
static int
not_modified_p (pseudo_t src_pseudo, pseudo_t dest_pseudo)
{
int regno, orig_regno;
pseudo_t p;
struct ira_loop_tree_node *node;
orig_regno = PSEUDO_REGNO (src_pseudo);
regno = REGNO (PSEUDO_REG (dest_pseudo));
for (node = PSEUDO_LOOP_TREE_NODE (src_pseudo);
node != NULL;
node = node->father)
if ((p = node->regno_pseudo_map [orig_regno]) == NULL)
break;
else if (REGNO (PSEUDO_REG (p)) == (unsigned) regno)
return TRUE;
else if (bitmap_bit_p (node->modified_regnos, orig_regno))
return FALSE;
return node != NULL;
}
/* The function generates and attaches moves to the edge E. It looks
at the final regnos of pseudos living on the edge with the same
original regno to find what moves should be generated. */
static void
generate_edge_moves (edge e)
{
struct ira_loop_tree_node *src_loop_node, *dest_loop_node;
unsigned int regno;
bitmap_iterator bi;
pseudo_t src_pseudo, dest_pseudo, *src_map, *dest_map;
struct move *move;
src_loop_node = IRA_BB_NODE (e->src)->father;
dest_loop_node = IRA_BB_NODE (e->dest)->father;
e->aux = NULL;
if (src_loop_node == dest_loop_node)
return;
src_map = src_loop_node->regno_pseudo_map;
dest_map = dest_loop_node->regno_pseudo_map;
EXECUTE_IF_SET_IN_REG_SET (DF_UPWARD_LIVE_IN (build_df, e->dest),
FIRST_PSEUDO_REGISTER, regno, bi)
if (bitmap_bit_p (DF_UPWARD_LIVE_OUT (build_df, e->src), regno))
{
src_pseudo = src_map [regno];
dest_pseudo = dest_map [regno];
if (REGNO (PSEUDO_REG (src_pseudo))
== REGNO (PSEUDO_REG (dest_pseudo)))
continue;
/* Actually it is not a optimization we need this code because
the memory (remember about equivalent memory) might be ROM
(or placed in read only section). */
if (PSEUDO_HARD_REGNO (dest_pseudo) < 0
&& PSEUDO_HARD_REGNO (src_pseudo) >= 0
&& not_modified_p (src_pseudo, dest_pseudo))
{
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "Remove r%d:%d->%d\n", regno,
PSEUDO_NUM (src_pseudo), PSEUDO_NUM (dest_pseudo));
continue;
}
move = create_move (dest_pseudo, src_pseudo);
add_to_edge_list (e, move, TRUE);
}
}
/* Bitmap of pseudos local for the current loop. */
static bitmap local_pseudo_bitmap;
/* This bitmap is used to find that we need to generate and use a new
pseudo-register when processing pseudos with the same original
regno. */
static bitmap used_regno_bitmap;
/* The following function changes (if necessary) pseudo-registers
inside loop given by loop tree node NODE. */
static void
change_loop (struct ira_loop_tree_node *node)
{
bitmap_iterator bi;
unsigned int i;
int regno, used_p;
pseudo_t pseudo, father_pseudo, *map;
rtx insn, original_reg;
if (node != ira_loop_tree_root)
{
if (node->bb != NULL)
{
FOR_BB_INSNS (node->bb, insn)
if (INSN_P (insn))
change_regs (&insn);
return;
}
if (ira_dump_file != NULL)
fprintf (ira_dump_file, "Changing RTL for loop %d (header bb%d)\n",
node->loop->num, node->loop->header->index);
map = ira_curr_loop_tree_node->father->regno_pseudo_map;
EXECUTE_IF_SET_IN_REG_SET (ira_curr_loop_tree_node->border_pseudos,
0, i, bi)
{
pseudo = pseudos [i];
regno = PSEUDO_REGNO (pseudo);
father_pseudo = map [regno];
/* We generate the same register move because the reload can
put a pseudo into memory in this case we will have live
range splitting. If it does not happen such the same
hard register moves will be removed. The worst case when
the both pseudos are put into memory by the reload is
very rare. */
if (father_pseudo != NULL
&& (PSEUDO_HARD_REGNO (pseudo)
== PSEUDO_HARD_REGNO (father_pseudo))
&& (PSEUDO_HARD_REGNO (pseudo) < 0
/* don't create copies because reload can spill a
pseudo set by copy although pseudo will not get
memory slot. */
|| reg_equiv_invariant_p [regno]
|| reg_equiv_const [regno] != NULL_RTX))
continue;
original_reg = PSEUDO_REG (pseudo);
if (father_pseudo == NULL
|| REGNO (PSEUDO_REG (father_pseudo)) == REGNO (original_reg))
set_pseudo_reg (pseudo, create_new_reg (original_reg));
}
}
/* Rename locals: Local pseudos with same regno in different loops
might get the different hard register. So we need to change
PSEUDO_REG. */
bitmap_and_compl (local_pseudo_bitmap,
ira_curr_loop_tree_node->mentioned_pseudos,
ira_curr_loop_tree_node->border_pseudos);
EXECUTE_IF_SET_IN_REG_SET (local_pseudo_bitmap, 0, i, bi)
{
pseudo = pseudos [i];
regno = PSEUDO_REGNO (pseudo);
if (regno < 0)
continue;
used_p = bitmap_bit_p (used_regno_bitmap, regno);
bitmap_set_bit (used_regno_bitmap, regno);
if (! used_p)
continue;
set_pseudo_reg (pseudo, create_new_reg (PSEUDO_REG (pseudo)));
}
}
/* The function returns nonzero if move lists on all edges in vector
VEC are equal. */
static int
eq_edge_move_lists_p (VEC(edge,gc) *vec)
{
struct move *list;
int i;
list = EDGE_I (vec, 0)->aux;
for (i = EDGE_COUNT (vec) - 1; i > 0; i--)
if (! eq_move_lists_p (list, EDGE_I (vec, i)->aux))
return FALSE;
return TRUE;
}
/* Current regno pseudo map used to check moves in function
`can_move_through_p'. */
static pseudo_t *curr_jump_map;
/* This recursive function returns nonzero if list of moves LIST can
be moved through setting (if OUTPUT_P is nonzero) or reading
LOC. */
static int
can_move_through_p (rtx *loc, struct move *list, int output_p)
{
int i;
const char *fmt;
enum rtx_code code = GET_CODE (*loc);
code = GET_CODE (*loc);
if (code == REG)
{
int hard_regno, regno;
HARD_REG_SET regs, move_regs;
regno = ORIGINAL_REGNO (*loc);
hard_regno = REGNO (*loc);
if (hard_regno >= FIRST_PSEUDO_REGISTER)
hard_regno = PSEUDO_HARD_REGNO (curr_jump_map [regno]);
if (hard_regno < 0)
CLEAR_HARD_REG_SET (regs);
else
COPY_HARD_REG_SET
(regs, reg_mode_hard_regset [hard_regno] [GET_MODE (*loc)]);
for (;list != NULL; list = list->next)
if (output_p
&& regno == (int) ORIGINAL_REGNO (PSEUDO_REG (list->from)))
return FALSE;
else
{
hard_regno = PSEUDO_HARD_REGNO (list->to);
if (hard_regno < 0)
CLEAR_HARD_REG_SET (move_regs);
else
COPY_HARD_REG_SET
(move_regs,
reg_mode_hard_regset [hard_regno] [PSEUDO_MODE (list->to)]);
if (output_p)
{
hard_regno = PSEUDO_HARD_REGNO (list->from);
if (hard_regno >= 0)
IOR_HARD_REG_SET (move_regs,
reg_mode_hard_regset
[hard_regno] [PSEUDO_MODE (list->from)]);
}
AND_HARD_REG_SET (move_regs, regs);
GO_IF_HARD_REG_EQUAL (move_regs, zero_hard_reg_set, cont);
return FALSE;
cont:
;
}
return TRUE;
}
else if (code == SET)
{
if (! can_move_through_p (&SET_DEST (*loc), list, TRUE))
return FALSE;
if (! can_move_through_p (&SET_SRC (*loc), list, FALSE))
return FALSE;
return TRUE;
}
else if (code == CLOBBER)
{
if (! can_move_through_p (&XEXP (*loc, 0), list, TRUE))
return FALSE;
return TRUE;
}
else if (code == MEM)
{
if (! can_move_through_p (&XEXP (*loc, 0), list, FALSE))
return FALSE;
return TRUE;
}
else if (code == PRE_DEC || code == POST_DEC || code == PRE_INC ||
code == POST_INC || code == POST_MODIFY || code == PRE_MODIFY)
{
if (! can_move_through_p (&XEXP (*loc, 0), list, TRUE))
return FALSE;
if (! can_move_through_p (&XEXP (*loc, 0), list, FALSE))
return FALSE;
return TRUE;
}
fmt = GET_RTX_FORMAT (code);
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
{
if (fmt[i] == 'e')
{
if (! can_move_through_p (&XEXP (*loc, i), list, output_p))
return FALSE;
}
else if (fmt[i] == 'E')
{
int j;
for (j = XVECLEN (*loc, i) - 1; j >= 0; j--)
{
if (! can_move_through_p (&XVECEXP (*loc, i, j), list, output_p))
return FALSE;
}
}
}
return TRUE;
}
/* The function looks at all enter edges (if START_P) or exit edges of
basic block BB and puts move lists at the BB start or end if it is
possible. In other words, it decreases code duplication of
shuffling pseudos. */
static void
unify_moves (basic_block bb, int start_p)
{
int i;
edge e;
struct move *list;
VEC(edge,gc) *vec;
vec = (start_p ? bb->preds : bb->succs);
if (EDGE_COUNT (vec) == 0 || ! eq_edge_move_lists_p (vec))
return;
e = EDGE_I (vec, 0);
list = e->aux;
curr_jump_map = IRA_BB_NODE (bb)->father->regno_pseudo_map;
if (! start_p
&& (control_flow_insn_p (BB_END (bb))
&& ! can_move_through_p (&PATTERN (BB_END (bb)), list, FALSE)))
return;
e->aux = NULL;
for (i = EDGE_COUNT (vec) - 1; i > 0; i--)
{
e = EDGE_I (vec, i);
free_move_list (e->aux);
e->aux = NULL;
}
if (start_p)
at_bb_start [bb->index] = list;
else
at_bb_end [bb->index] = list;
}
/* Last move (in move sequence being processed) setting up the
corresponding hard register. */
static struct move *hard_regno_last_set [FIRST_PSEUDO_REGISTER];
/* If the element value is equal to CURR_TICK then the corresponding
element in `hard_regno_last_set' is defined and correct. */
static int hard_regno_last_set_check [FIRST_PSEUDO_REGISTER];
/* Last move (in move sequence being processed) setting up the
corresponding pseudo. */
static struct move **pseudo_last_set;
/* If the element value is equal to CURR_TICK then the corresponding
element in . `pseudo_last_set' is defined and correct. */
static int *pseudo_last_set_check;
/* This array contains moves sorted topologically (depth-first) on
their dependency graph. */
static varray_type move_varray;
/* The variable value is used to check correctness of values of
elements of arrays `hard_regno_last_set' and
`pseudo_last_set_check'. */
static int curr_tick;
/* This recursive function traverses dependecies of MOVE and do
toplogical sorting (in depth-first order). */
static void
traverse_moves (struct move *move)
{
int i;
if (move->visited_p)
return;
move->visited_p = TRUE;
for (i = move->deps_num - 1; i >= 0; i--)
traverse_moves (move->deps [i]);
VARRAY_PUSH_GENERIC_PTR (move_varray, move);
}
/* The function removes unnecessary moves in the LIST, makes
topological sorting, and removes cycles on hard reg dependencies by
introducing new pseudos assigned to memory and additional moves.
It returns the result move list. */
static struct move *
modify_move_list (struct move *list)
{
int i, n, nregs, hard_regno;
pseudo_t to, from, new_pseudo;
struct move *move, *new_move, *set_move, *first, *last;
if (list == NULL)
return NULL;
/* Creat move deps. */
curr_tick++;
for (move = list; move != NULL; move = move->next)
{
to = move->to;
if ((hard_regno = PSEUDO_HARD_REGNO (to)) < 0)
continue;
nregs = hard_regno_nregs [hard_regno] [PSEUDO_MODE (to)];
for (i = 0; i < nregs; i++)
{
hard_regno_last_set [hard_regno + i] = move;
hard_regno_last_set_check [hard_regno + i] = curr_tick;
}
}
for (move = list; move != NULL; move = move->next)
{
from = move->from;
to = move->to;
if ((hard_regno = PSEUDO_HARD_REGNO (from)) >= 0)
{
nregs = hard_regno_nregs [hard_regno] [PSEUDO_MODE (from)];
for (n = i = 0; i < nregs; i++)
if (hard_regno_last_set_check [hard_regno + i] == curr_tick
&& (PSEUDO_REGNO (hard_regno_last_set [hard_regno + i]->to)
!= PSEUDO_REGNO (from)))
n++;
move->deps = ira_allocate (n * sizeof (struct move *));
for (n = i = 0; i < nregs; i++)
if (hard_regno_last_set_check [hard_regno + i] == curr_tick
&& (PSEUDO_REGNO (hard_regno_last_set [hard_regno + i]->to)
!= PSEUDO_REGNO (from)))
move->deps [n++] = hard_regno_last_set [hard_regno + i];
move->deps_num = n;
}
}
/* Toplogical sorting: */
VARRAY_POP_ALL (move_varray);
for (move = list; move != NULL; move = move->next)
traverse_moves (move);
last = NULL;
for (i = VARRAY_ACTIVE_SIZE (move_varray) - 1; i >= 0; i--)
{
move = VARRAY_GENERIC_PTR (move_varray, i);
move->next = NULL;
if (last != NULL)
last->next = move;
last = move;
}
first = VARRAY_TOP_GENERIC_PTR (move_varray);
/* Removing cycles: */
curr_tick++;
VARRAY_POP_ALL (move_varray);
for (move = first; move != NULL; move = move->next)
{
from = move->from;
to = move->to;
if ((hard_regno = PSEUDO_HARD_REGNO (from)) >= 0)
{
nregs = hard_regno_nregs [hard_regno] [PSEUDO_MODE (from)];
for (i = 0; i < nregs; i++)
if (hard_regno_last_set_check [hard_regno + i] == curr_tick
&& PSEUDO_HARD_REGNO (hard_regno_last_set
[hard_regno + i]->to) >= 0)
{
set_move = hard_regno_last_set [hard_regno + i];
new_pseudo
= create_pseudo (PSEUDO_REGNO (set_move->to),
PSEUDO_LOOP_TREE_NODE (set_move->to));
/* That is a minimum to emit pseudos correctly and for
setting reg_renumber. */
PSEUDO_HARD_REGNO (new_pseudo) = -1;
PSEUDO_REG (new_pseudo)
= create_new_reg (PSEUDO_REG (set_move->to));
new_move = create_move (set_move->to, new_pseudo);
set_move->to = new_pseudo;
VARRAY_PUSH_GENERIC_PTR (move_varray, new_move);
move_loops_num++;
}
}
if ((hard_regno = PSEUDO_HARD_REGNO (to)) < 0)
continue;
nregs = hard_regno_nregs [hard_regno] [PSEUDO_MODE (to)];
for (i = 0; i < nregs; i++)
{
hard_regno_last_set [hard_regno + i] = move;
hard_regno_last_set_check [hard_regno + i] = curr_tick;
}
}
for (i = VARRAY_ACTIVE_SIZE (move_varray) - 1; i >= 0; i--)
{
move = VARRAY_GENERIC_PTR (move_varray, i);
move->next = NULL;
last->next = move;
last = move;
}
return first;
}
/* The function generates rtx move insns from the move list LIST. It
updates allocation cost using move execution frequency FERQ. */
static rtx
emit_move_list (struct move *list, int freq)
{
int cost;
rtx result;
enum machine_mode mode;
enum reg_class cover_class;
list = modify_move_list (list);
start_sequence ();
for (; list != NULL; list = list->next)
{
emit_move_insn (PSEUDO_REG (list->to), PSEUDO_REG (list->from));
mode = PSEUDO_MODE (list->to);
cover_class = PSEUDO_COVER_CLASS (list->to);
cost = 0;
if (PSEUDO_HARD_REGNO (list->to) < 0)
{
if (PSEUDO_HARD_REGNO (list->from) >= 0)
{
cost = memory_move_cost [mode] [cover_class] [0] * freq;
store_cost += cost;
}
}
else if (PSEUDO_HARD_REGNO (list->from) < 0)
{
if (PSEUDO_HARD_REGNO (list->to) >= 0)
{
cost = memory_move_cost [mode] [cover_class] [0] * freq;
load_cost += cost;
}
}
else
{
cost = register_move_cost [mode] [cover_class] [cover_class] * freq;
shuffle_cost += cost;
}
overall_cost += cost;
}
result = get_insns ();
end_sequence ();
return result;
}
/* The function generates rtx move insns from move lists attached to
basic blocks and edges. */
static void
emit_moves (void)
{
basic_block bb;
edge_iterator ei;
edge e;
rtx insns, tmp;
FOR_EACH_BB (bb)
{
if (at_bb_start [bb->index] != NULL)
{
insns = emit_move_list (at_bb_start [bb->index],
REG_FREQ_FROM_BB (bb));
tmp = BB_HEAD (bb);
if (LABEL_P (tmp))
tmp = NEXT_INSN (tmp);
if (NOTE_INSN_BASIC_BLOCK_P (tmp))
tmp = NEXT_INSN (tmp);
if (tmp == BB_HEAD (bb))
emit_insn_before (insns, tmp);
else if (tmp != NULL_RTX)
emit_insn_after (insns, PREV_INSN (tmp));
else
emit_insn_after (insns, get_last_insn ());
}
if (at_bb_end [bb->index] != NULL)
{
insns = emit_move_list (at_bb_end [bb->index],
REG_FREQ_FROM_BB (bb));
if (! control_flow_insn_p (BB_END (bb)))
emit_insn_after (insns, BB_END (bb));
else
emit_insn_before (insns, BB_END (bb));
}
FOR_EACH_EDGE (e, ei, bb->succs)
{
if (e->aux == NULL)
continue;
ira_assert ((e->flags & EDGE_ABNORMAL) == 0
|| ! EDGE_CRITICAL_P (e));
insert_insn_on_edge
(emit_move_list (e->aux,
REG_FREQ_FROM_EDGE_FREQ (EDGE_FREQUENCY (e))),
e);
if (e->src->next_bb != e->dest)
additional_jumps_num++;
}
}
}
/* Entry function changing code and generating pseudo shuffling for
the regional register allocation. */
void
ira_emit (void)
{
int i;
basic_block bb;
edge_iterator ei;
edge e;
at_bb_start = ira_allocate (sizeof (struct move *) * last_basic_block);
memset (at_bb_start, 0, sizeof (struct move *) * last_basic_block);
at_bb_end = ira_allocate (sizeof (struct move *) * last_basic_block);
memset (at_bb_end, 0, sizeof (struct move *) * last_basic_block);
for (i = 0; i < pseudos_num; i++)
if (PSEUDO_REGNO (pseudos [i]) >= 0)
PSEUDO_REG (pseudos [i]) = regno_reg_rtx [PSEUDO_REGNO (pseudos [i])];
local_pseudo_bitmap = ira_allocate_bitmap ();
used_regno_bitmap = ira_allocate_bitmap ();
max_regno_before_changing = max_reg_num ();
traverse_loop_tree (ira_loop_tree_root, change_loop, NULL);
ira_free_bitmap (used_regno_bitmap);
ira_free_bitmap (local_pseudo_bitmap);
FOR_EACH_BB (bb)
{
at_bb_start [bb->index] = NULL;
at_bb_end [bb->index] = NULL;
FOR_EACH_EDGE (e, ei, bb->succs)
if (e->dest != EXIT_BLOCK_PTR)
generate_edge_moves (e);
}
pseudo_last_set = ira_allocate (sizeof (struct move *) * max_reg_num ());
pseudo_last_set_check = ira_allocate (sizeof (int) * max_reg_num ());
memset (pseudo_last_set_check, 0, sizeof (int) * max_reg_num ());
memset (hard_regno_last_set_check, 0, sizeof (hard_regno_last_set_check));
curr_tick = 0;
FOR_EACH_BB (bb)
unify_moves (bb, TRUE);
FOR_EACH_BB (bb)
unify_moves (bb, FALSE);
VARRAY_GENERIC_PTR_NOGC_INIT (move_varray, pseudos_num, "ordered moves");
emit_moves ();
/* Clean up: */
FOR_EACH_BB (bb)
{
free_move_list (at_bb_start [bb->index]);
free_move_list (at_bb_end [bb->index]);
FOR_EACH_EDGE (e, ei, bb->succs)
{
free_move_list (e->aux);
e->aux = NULL;
}
}
VARRAY_FREE (move_varray);
ira_free (pseudo_last_set_check);
ira_free (pseudo_last_set);
commit_edge_insertions ();
ira_free (at_bb_end);
ira_free (at_bb_start);
}
/* Integrated Register Allocator intercommunication header file.
Contributed by Vladimir Makarov.
Copyright (C) 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "cfgloop.h"
#include "ira.h"
#ifdef ENABLE_CHECKING
#define ENABLE_IRA_CHECKING
#endif
#ifdef ENABLE_IRA_CHECKING
#define ira_assert(c) gcc_assert (c)
#else
#define ira_assert(c)
#endif
/* Compute register frequency from edge frequency FREQ. It is
analogous to REG_FREQ_FROM_BB. When optimizing for size, or
profile driven feedback is available and the function is never
executed, frequency is always equivalent. Otherwise rescale edge
frequency. */
#define REG_FREQ_FROM_EDGE_FREQ(freq) \
(optimize_size || (flag_branch_probabilities && !ENTRY_BLOCK_PTR->count) \
? REG_FREQ_MAX : (freq * REG_FREQ_MAX / BB_FREQ_MAX) \
? (freq * REG_FREQ_MAX / BB_FREQ_MAX) : 1)
/* All natural loops. */
extern struct loops ira_loops;
/* Dump file of the allocator if it is not NULL. */
extern FILE *ira_dump_file;
/* Pseudo and copy of pseudos. */
typedef struct pseudo *pseudo_t;
typedef struct pseudo_copy *copy_t;
/* The following structure describes loop tree node (block or loop).
We need such tree because the loop tree from cfgloop.h is not
convenient for the optimization (because basic blocks are not a
part of the tree from cfgloop.h). We also use the nodes for
storing additional information about basic blocks/loops for the
register allocation. */
struct ira_loop_tree_node
{
/* The node represents basic block if inner == NULL. */
basic_block bb; /* NULL for loop. */
struct loop *loop; /* NULL for BB. */
/* The next node on the same tree level. */
struct ira_loop_tree_node *next;
/* The first node immediately inside the node. */
struct ira_loop_tree_node *inner;
/* The node containing given node. */
struct ira_loop_tree_node *father;
/* Pseudos in loop corresponding to regnos. If it is NULL the loop
is not included in the loop tree (e.g. it has abnormal enter/exit
edges). */
pseudo_t *regno_pseudo_map;
/* Pseudos referred in the loop node. */
bitmap mentioned_pseudos;
/* Regnos modified in the loop node (including its subloops). */
bitmap modified_regnos;
/* Pseudos living at the loop borders. */
bitmap border_pseudos;
/* Copies referred in the loop node. */
bitmap local_copies;
};
/* The root of the loop tree corresponding to the all function. */
extern struct ira_loop_tree_node *ira_loop_tree_root;
/* All basic block data are referred through the following array. We
can not use member `aux' for this because it is used for insertion
of insns on edges. */
extern struct ira_loop_tree_node *ira_bb_nodes;
/* Two access macros to the basic block data. */
#if defined ENABLE_IRA_CHECKING && (GCC_VERSION >= 2007)
#define IRA_BB_NODE_BY_INDEX(index) __extension__ \
(({ struct ira_loop_tree_node *const _node = (&ira_bb_nodes [index]); \
if (_node->inner != NULL || _node->loop != NULL || _node->bb == NULL)\
{ \
fprintf (stderr, \
"\n%s: %d: error in %s: it is not a block node\n", \
__FILE__, __LINE__, __FUNCTION__); \
exit (1); \
} \
_node; }))
#else
#define IRA_BB_NODE_BY_INDEX(index) (&ira_bb_nodes [index])
#endif
#define IRA_BB_NODE(bb) IRA_BB_NODE_BY_INDEX ((bb)->index)
/* All loop data are referred through the following array. */
extern struct ira_loop_tree_node *ira_loop_nodes;
/* Two access macros to the loop data. */
#if defined ENABLE_IRA_CHECKING && (GCC_VERSION >= 2007)
#define IRA_LOOP_NODE_BY_INDEX(index) __extension__ \
(({ struct ira_loop_tree_node *const _node = (&ira_loop_nodes [index]);\
if (_node->inner == NULL || _node->bb != NULL || _node->loop == NULL)\
{ \
fprintf (stderr, \
"\n%s: %d: error in %s: it is not a loop node\n", \
__FILE__, __LINE__, __FUNCTION__); \
exit (1); \
} \
_node; }))
#else
#define IRA_LOOP_NODE_BY_INDEX(index) (&ira_loop_nodes [index])
#endif
#define IRA_LOOP_NODE(loop) IRA_LOOP_NODE_BY_INDEX ((loop)->num)
/* Node representing pseudos (register allocation entity). */
struct pseudo
{
/* The pseudo order number starting with 0. */
int num;
/* If it is negative the pseudo represents a set of pseudos (a
cap). */
int regno;
/* Final rtx representation of the pseudo. */
rtx reg;
/* Pseudos with the same regno are linked by the following member.
pseudos corresponding to inner loops are first in the list. */
pseudo_t next_regno_pseudo;
/* There may be different pseudos with the same regno. They are
bound to a loop tree node. */
struct ira_loop_tree_node *loop_tree_node;
/* It is a pseudo (cap) representing given pseudo on upper loop tree
level. */
pseudo_t cap;
/* It is a link to pseudo (cap) on lower loop level represented by
given cap. */
pseudo_t cap_member;
/* Vector of conflicting pseudos with NULL end marker. */
pseudo_t *conflict_pseudo_vec;
/* Allocated and current size (both without NULL marker) of the
previous array. */
int conflict_pseudo_vec_size, conflict_pseudo_vec_active_size;
/* Hard registers conflicting with this pseudo and as a consequences
can not be assigned to the pseudo. */
HARD_REG_SET conflict_hard_regs;
/* Frequency of usage of the pseudo. */
int freq;
/* Hard register assigned to given pseudo. Negative value means
that memory was allocated to the pseudo. */
int hard_regno;
/* Frequency of calls which given pseudo intersects. */
int call_freq;
/* Calls which given pseudo intersects. It can be NULL if there is
no one. */
rtx *calls_crossed;
/* Length of the previous array (number of the intersected
calls). */
int calls_crossed_num;
#ifdef STACK_REGS
/* Set to TRUE if pseudo can't be allocated in the stack
register. */
int no_stack_reg_p;
#endif
/* TRUE value means than the pseudo was not removed from the
conflicting graph during colouring. */
int in_graph_p;
/* TRUE if a hard register or memory has been assigned to the
pseudo. */
int assigned_p;
/* TRUE if it is put on the stack to make other pseudos
colorable. */
int may_be_spilled_p;
/* Mode of the pseudo. */
enum machine_mode mode;
/* Copies to other non-conflicting pseudos. The copies can
represent move insn or potential move insn usually because of two
operand constraints. */
copy_t pseudo_copies;
/* Array of additional costs (initial and current during coloring)
for hard regno of pseudo cover class. If given pseudo represents
a set of pseudos the current costs represents costs of the all
set. */
int *hard_reg_costs, *curr_hard_reg_costs;
/* Array of decreasing costs (initial and current during coloring)
for conflicting pseudos for hard regno of the pseudo cover class.
The member values can be NULL if all costs are the same. If
given pseudo represents a set of pseudos the current costs
represents costs of the all set. */
int *conflict_hard_reg_costs, *curr_conflict_hard_reg_costs;
/* Number of pseudos with TRUE in_graph_p value and conflicting with
given pseudo. */
int left_conflicts_num;
/* Register class which should be used for allocation for given
pseudo. NO_REGS means that we should use memory. */
enum reg_class cover_class;
/* Minimal cost of usage register of the cover class and memory for
the pseudo. */
int cover_class_cost, memory_cost, original_memory_cost;
/* Number of hard register of the pseudo cover class really
availiable for the pseudo allocation. */
int available_regs_num;
/* Pseudos in a bucket (used in coloring) chained by the following
two members. */
pseudo_t next_bucket_pseudo;
pseudo_t prev_bucket_pseudo;
/* Used for temporary purposes. */
int temp;
};
/* All members of the pseudo node should be accessed only through the
following macros. */
#define PSEUDO_NUM(P) ((P)->num)
#define PSEUDO_REGNO(P) ((P)->regno)
#define PSEUDO_REG(P) ((P)->reg)
#define PSEUDO_NEXT_REGNO_PSEUDO(P) ((P)->next_regno_pseudo)
#define PSEUDO_LOOP_TREE_NODE(P) ((P)->loop_tree_node)
#define PSEUDO_CAP(P) ((P)->cap)
#define PSEUDO_CAP_MEMBER(P) ((P)->cap_member)
#define PSEUDO_CONFLICT_PSEUDO_VEC(P) ((P)->conflict_pseudo_vec)
#define PSEUDO_CONFLICT_PSEUDO_VEC_SIZE(P) ((P)->conflict_pseudo_vec_size)
#define PSEUDO_CONFLICT_PSEUDO_VEC_ACTIVE_SIZE(P) \
((P)->conflict_pseudo_vec_active_size)
#define PSEUDO_CONFLICT_HARD_REGS(P) ((P)->conflict_hard_regs)
#define PSEUDO_FREQ(P) ((P)->freq)
#define PSEUDO_HARD_REGNO(P) ((P)->hard_regno)
#define PSEUDO_CALL_FREQ(P) ((P)->call_freq)
#define PSEUDO_CALLS_CROSSED(P) ((P)->calls_crossed)
#define PSEUDO_CALLS_CROSSED_NUM(P) ((P)->calls_crossed_num)
#ifdef STACK_REGS
#define PSEUDO_NO_STACK_REG_P(P) ((P)->no_stack_reg_p)
#endif
#define PSEUDO_IN_GRAPH_P(P) ((P)->in_graph_p)
#define PSEUDO_ASSIGNED_P(P) ((P)->assigned_p)
#define PSEUDO_MAY_BE_SPILLED_P(P) ((P)->may_be_spilled_p)
#define PSEUDO_MODE(P) ((P)->mode)
#define PSEUDO_COPIES(P) ((P)->pseudo_copies)
#define PSEUDO_HARD_REG_COSTS(P) ((P)->hard_reg_costs)
#define PSEUDO_CURR_HARD_REG_COSTS(P) ((P)->curr_hard_reg_costs)
#define PSEUDO_CONFLICT_HARD_REG_COSTS(P) ((P)->conflict_hard_reg_costs)
#define PSEUDO_CURR_CONFLICT_HARD_REG_COSTS(P) \
((P)->curr_conflict_hard_reg_costs)
#define PSEUDO_LEFT_CONFLICTS_NUM(P) ((P)->left_conflicts_num)
#define PSEUDO_COVER_CLASS(P) ((P)->cover_class)
#define PSEUDO_COVER_CLASS_COST(P) ((P)->cover_class_cost)
#define PSEUDO_MEMORY_COST(P) ((P)->memory_cost)
#define PSEUDO_ORIGINAL_MEMORY_COST(P) ((P)->original_memory_cost)
#define PSEUDO_AVAILABLE_REGS_NUM(P) ((P)->available_regs_num)
#define PSEUDO_NEXT_BUCKET_PSEUDO(P) ((P)->next_bucket_pseudo)
#define PSEUDO_PREV_BUCKET_PSEUDO(P) ((P)->prev_bucket_pseudo)
#define PSEUDO_TEMP(P) ((P)->temp)
/* Map regno -> pseudo for the current loop tree node. */
extern pseudo_t *regno_pseudo_map;
/* Array of references to all pseudos. The order number of the pseudo
corresponds to the index in the array. */
extern pseudo_t *pseudos;
/* Size of the previous array. */
extern int pseudos_num;
/* The following structure represents a copy of given pseudo to
another pseudo. The copies represent move insns or potential move
insns usually because of two operand constraints. */
struct pseudo_copy
{
/* The order number of the copy node starting with 0. */
int num;
/* Pseudo connected by the copy. The first one should have smaller
order number than the second one. */
pseudo_t first, second;
/* Execution frequency of the copy. */
int freq;
/* It is a move insn if the copy represents it, potential move insn
is represented by NULL. */
rtx move_insn;
/* Copies with the same pseudo as FIRST are linked by the two
following members. */
copy_t prev_first_pseudo_copy, next_first_pseudo_copy;
/* Copies with the same pseudo as SECOND are linked by the two
following members. */
copy_t prev_second_pseudo_copy, next_second_pseudo_copy;
};
/* Array of references to copies. The order number of the copy
corresponds to the index in the array. */
extern copy_t *copies;
/* Size of the previous array. */
extern int copies_num;
/* The following structure describes a stack slot used for spilled
registers. */
struct spilled_reg_stack_slot
{
/* Pseudo-registers have used the stack slot. */
regset_head spilled_regs;
/* RTL representation of the stack slot. */
rtx mem;
/* Size of the stack slot. */
unsigned int width;
};
/* The number of elements in the following array. */
extern int spilled_reg_stack_slots_num;
/* The following array contains description of spilled registers stack
slots have been used in current function so far. */
extern struct spilled_reg_stack_slot *spilled_reg_stack_slots;
/* Data flow data used for IRA data flow analysis. */
extern struct df *build_df;
/* Correspondingly overall cost of the allocation, cost of hard
register usage for the pseudos, cost of memory usage for the
pseudos, cost of loads, stores and register move insns generated
for register live range splitting. */
extern int overall_cost;
extern int reg_cost, mem_cost;
extern int load_cost, store_cost, shuffle_cost;
extern int move_loops_num, additional_jumps_num;
/* Map: register class x machine mode -> number of hard registers of
given class needed to store value of given mode. If the number is
different, the size will be negative. */
extern int reg_class_nregs [N_REG_CLASSES] [MAX_MACHINE_MODE];
/* Maximal value of the previous array elements. */
extern int max_nregs;
/* ira.c: */
/* Hard regsets whose all bits are correspondingly zero or one. */
extern HARD_REG_SET zero_hard_reg_set;
extern HARD_REG_SET one_hard_reg_set;
/* A mode whose value is immediately contained in given mode
value. */
extern unsigned char mode_inner_mode [NUM_MACHINE_MODES];
/* Map hard regs X modes -> number registers for store value of given
mode starting with given hard register. */
extern HARD_REG_SET reg_mode_hard_regset
[FIRST_PSEUDO_REGISTER] [NUM_MACHINE_MODES];
/* Array analog of macros MEMORY_MOVE_COST and REGISTER_MOVE_COST. */
extern int memory_move_cost [MAX_MACHINE_MODE] [N_REG_CLASSES] [2];
extern int register_move_cost [MAX_MACHINE_MODE] [N_REG_CLASSES]
[N_REG_CLASSES];
/* Register class subset relation. */
extern int class_subset_p [N_REG_CLASSES] [N_REG_CLASSES];
/* Hard registers which can be used for the allocation of given
register class. */
extern short class_hard_regs [N_REG_CLASSES] [FIRST_PSEUDO_REGISTER];
/* The size of the above array for given register class. */
extern int class_hard_regs_num [N_REG_CLASSES];
/* Index (in class_hard_regs) for given register class and hard
register. */
extern short class_hard_reg_index [N_REG_CLASSES] [FIRST_PSEUDO_REGISTER];
/* Hard registers can not be used for the register allocator. */
extern HARD_REG_SET no_alloc_regs;
/* Number of class hard registers available for the register
allocation for given classes. */
extern int available_class_regs [N_REG_CLASSES];
/* Array whose values are hard regset of hard registers of given
register class whose HARD_REGNO_MODE_OK values are zero. */
extern HARD_REG_SET prohibited_class_mode_regs
[N_REG_CLASSES] [NUM_MACHINE_MODES];
/* Number of cover classes. */
extern int reg_class_cover_size;
/* The array containing cover classes whose hard registers are used
for the allocation. */
extern enum reg_class reg_class_cover [N_REG_CLASSES];
/* Map of register classes to corresponding cover class containing the
given class. */
extern enum reg_class class_translate [N_REG_CLASSES];
extern void set_non_alloc_regs (int);
extern void *ira_allocate (size_t);
extern void ira_free (void *addr);
extern bitmap ira_allocate_bitmap (void);
extern void ira_free_bitmap (bitmap);
extern regset ira_allocate_regset (void);
extern void ira_free_regset (regset);
extern int hard_reg_in_set_p (int, enum machine_mode, HARD_REG_SET);
extern int hard_reg_not_in_set_p (int, enum machine_mode, HARD_REG_SET);
extern void print_disposition (FILE *);
extern void debug_disposition (void);
extern void debug_class_cover (void);
/* Regno invariant flags. */
extern int *reg_equiv_invariant_p;
/* Regno equivalent constants. */
extern rtx *reg_equiv_const;
/* ira-build.c */
/* The current loop tree node. */
extern struct ira_loop_tree_node *ira_curr_loop_tree_node;
extern void traverse_loop_tree (struct ira_loop_tree_node *,
void (*) (struct ira_loop_tree_node *),
void (*) (struct ira_loop_tree_node *));
extern pseudo_t create_pseudo (int, struct ira_loop_tree_node *);
extern void allocate_pseudo_conflicts (pseudo_t, int);
extern void print_expanded_pseudo (pseudo_t);
extern copy_t create_copy (pseudo_t, pseudo_t, int, rtx);
extern void ira_build (int);
extern void ira_destroy (void);
/* ira-costs.c */
extern void init_ira_costs_once (void);
extern void ira_costs (void);
extern void tune_pseudo_costs_and_cover_classes (void);
/* ira-conflicts.c */
extern copy_t add_pseudo_copy (pseudo_t, pseudo_t, int, rtx);
extern int pseudo_reg_conflict_p (int, int);
extern void debug_conflicts (void);
extern void ira_build_conflicts (void);
/* ira-color.c */
extern void ira_color (void);
/* ira-emit.c */
extern void ira_emit (void);
/* Decompose multiword subregs.
Contributed by Richard Henderson.
Copyright (C) 2005, 2006 Free Software Foundation, Inc.
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "machmode.h"
#include "tm.h"
#include "rtl.h"
#include "function.h"
#include "expr.h"
#include "obstack.h"
#include "bitmap.h"
#include "tree-pass.h"
#include "timevar.h"
#include "tm_p.h"
#ifdef STACK_GROWS_DOWNWARD
# undef STACK_GROWS_DOWNWARD
# define STACK_GROWS_DOWNWARD 1
#else
# define STACK_GROWS_DOWNWARD 0
#endif
DEF_VEC_P(bitmap);
DEF_VEC_ALLOC_P(bitmap,heap);
/* Bit N set if regno N is used in a context in which we can decompose it. */
static bitmap decomposable_context;
/* Bit N set if regno N is used in a context in which it cannot
be decomposed. */
static bitmap non_decomposable_context;
/* Bit N in element M set if there exists a copy from reg M to reg N. */
static VEC(bitmap,heap) *reg_copy_graph;
/* Return true if INSN is a single set between two objects. Such insns
can always be decomposed. */
static rtx
simple_move (rtx insn)
{
rtx x, set = single_set (insn);
if (!set)
return NULL;
x = SET_DEST (set);
if (!OBJECT_P (x) && GET_CODE (x) != SUBREG)
return NULL;
if (MEM_P (x) && MEM_VOLATILE_P (x))
return NULL;
x = SET_SRC (set);
if (!OBJECT_P (x) && GET_CODE (x) != SUBREG)
return NULL;
if (MEM_P (x) && MEM_VOLATILE_P (x))
return NULL;
return set;
}
/* */
static void
find_pseudo_copy (rtx set)
{
rtx dst = SET_DEST (set);
rtx src = SET_SRC (set);
unsigned int rd, rs;
bitmap b;
if (!REG_P (dst) || !REG_P (src))
return;
rd = REGNO (dst);
rs = REGNO (src);
if (HARD_REGISTER_NUM_P (rd) || HARD_REGISTER_NUM_P (rs))
return;
if (GET_MODE_SIZE (GET_MODE (dst)) < UNITS_PER_WORD)
return;
b = VEC_index (bitmap, reg_copy_graph, rs);
if (b == NULL)
{
b = BITMAP_ALLOC (NULL);
VEC_replace (bitmap, reg_copy_graph, rs, b);
}
bitmap_set_bit (b, rd);
}
/* */
static void
propagate_pseudo_copies (void)
{
bitmap queue, propagate;
queue = BITMAP_ALLOC (NULL);
propagate = BITMAP_ALLOC (NULL);
bitmap_copy (queue, decomposable_context);
do
{
bitmap_iterator iter;
unsigned int i;
bitmap_clear (propagate);
EXECUTE_IF_SET_IN_BITMAP (queue, 0, i, iter)
{
bitmap b = VEC_index (bitmap, reg_copy_graph, i);
if (b)
bitmap_ior_and_compl_into (propagate, b, non_decomposable_context);
}
bitmap_and_compl (queue, propagate, decomposable_context);
bitmap_ior_into (decomposable_context, propagate);
}
while (!bitmap_empty_p (queue));
BITMAP_FREE (queue);
BITMAP_FREE (propagate);
}
/* Called via for_each_rtx. Examine the given expression and set bits as
appropriate in decomposable_context and non_decomposable_context. SM
is the result of simple_move for the complete insn. */
static int
find_decomposable_subregs (rtx *px, void *sm)
{
rtx x = *px, inner;
unsigned int inner_size, outer_size;
unsigned int inner_words, outer_words;
unsigned int regno;
switch (GET_CODE (x))
{
case SUBREG:
/* Ensure we're not looking at something other than a subreg of a
pseudo register. One might hope these tests never fail, since
that would indicate someone not using simplify_gen_subreg or some
related interface, but that no doubt happens all too often. */
inner = SUBREG_REG (x);
if (!REG_P (inner))
break;
regno = REGNO (inner);
if (HARD_REGISTER_NUM_P (regno))
return -1;
/* Compute the number of words covered by the subreg and the reg. */
outer_size = GET_MODE_SIZE (GET_MODE (x));
inner_size = GET_MODE_SIZE (GET_MODE (inner));
outer_words = (outer_size + UNITS_PER_WORD - 1) / UNITS_PER_WORD;
inner_words = (inner_size + UNITS_PER_WORD - 1) / UNITS_PER_WORD;
/* If we've got a single-word subreg of a multi-word reg, then this
should be a candidate for decomposition. Return -1 so that we
don't iterate over the inner register and decide it is used in a
context we can't decompose. */
/* ??? This doesn't allow e.g. DImode subregs of TImode values on
32-bit targets. We'd need to record the way in which the pseudo
is used, and only decompose if all uses were with the same number
of pieces. Hopefully this doesn't happen with any frequency. */
/* ??? This is a bald-faced assumption that the subreg is actually
inside an operand, and is thus replacable. This might be false
if the target plays games with subregs in the patterns. Perhaps
a better approach is to mirror what regrename does wrt recognizing
the insn, iterating over the operands, smashing the operands out
and iterating over the resulting pattern. */
if (outer_words == 1 && inner_words > 1)
{
bitmap_set_bit (decomposable_context, regno);
return -1;
}
break;
case REG:
/* Since we see outer subregs and avoid iterating over inner registers
when we can handle the decomposition, that means that anywhere else
we come across the register must be a place we can't decompose it.
Avoid setting the bit for single-word pseudos to keep down the size
of the bitmap. */
regno = REGNO (x);
if (!HARD_REGISTER_NUM_P (regno) && !sm
&& GET_MODE_SIZE (GET_MODE (x)) > UNITS_PER_WORD)
bitmap_set_bit (non_decomposable_context, regno);
break;
default:
break;
}
return 0;
}
/* Decompose psuedo REGNO into word-sized components. We smash the REG
node in place. This ensures that (1) something goes wrong quickly if
we fail to find a place in which we ought to be performing some
replacement, and (2) the debug information inside the symbol table is
automatically kept up to date. */
static void
decompose_register (unsigned int regno)
{
unsigned int words;
rtx reg;
reg = regno_reg_rtx[regno];
regno_reg_rtx[regno] = NULL;
words = GET_MODE_SIZE (GET_MODE (reg));
words = (words + UNITS_PER_WORD - 1) / UNITS_PER_WORD;
if (0 && words == 2)
{
PUT_CODE (reg, CONCAT);
XEXP (reg, 0) = gen_reg_rtx_offset (reg, word_mode, 0);
XEXP (reg, 1) = gen_reg_rtx_offset (reg, word_mode, UNITS_PER_WORD);
}
else
{
unsigned int i;
rtvec v;
if (dump_file)
fprintf (dump_file, "; Splitting reg %u ->", REGNO (reg));
PUT_CODE (reg, CONCATN);
XVEC (reg, 0) = v = rtvec_alloc (words);
for (i = 0; i < words; ++i)
RTVEC_ELT (v, i)
= gen_reg_rtx_offset (reg, word_mode, i * UNITS_PER_WORD);
if (dump_file)
{
for (i = 0; i < words; ++i)
fprintf (dump_file, " %u", REGNO (XVECEXP (reg, 0, i)));
fputc ('\n', dump_file);
}
}
}
static inline bool
resolve_reg_p (rtx x)
{
return GET_CODE (x) == CONCAT || GET_CODE (x) == CONCATN;
}
static bool
resolve_subreg_p (rtx x)
{
if (GET_CODE (x) == SUBREG)
return resolve_reg_p (SUBREG_REG (x));
return false;
}
/* */
static int
resolve_subreg_use (rtx *px, void *data ATTRIBUTE_UNUSED)
{
rtx x = *px;
if (x == NULL)
return 0;
/* If this is a (subreg (concat)) pattern, then it must be something that
we created via decompose_register. */
if (resolve_subreg_p (x))
{
/* This must be resolvable. */
*px = simplify_subreg (GET_MODE (x), SUBREG_REG (x),
GET_MODE (SUBREG_REG (x)), SUBREG_BYTE (x));
gcc_assert (*px != NULL);
return -1;
}
if (resolve_reg_p (x))
return 1;
return 0;
}
/* */
static void
move_libcall_note (rtx old_start, rtx new_start)
{
rtx note0, note1, end;
note0 = find_reg_note (old_start, REG_LIBCALL, NULL);
if (note0 == NULL)
return;
remove_note (old_start, note0);
end = XEXP (note0, 0);
note1 = find_reg_note (end, REG_RETVAL, NULL);
XEXP (note0, 1) = REG_NOTES (new_start);
REG_NOTES (new_start) = note0;
XEXP (note1, 0) = new_start;
}
/* */
static void
remove_retval_note (rtx insn1)
{
rtx note, note0, insn0, note1, insn;
note1 = find_reg_note (insn1, REG_RETVAL, NULL);
if (note1 == NULL)
return;
insn0 = XEXP (note1, 0);
note0 = find_reg_note (insn0, REG_LIBCALL, NULL);
remove_note (insn0, note0);
remove_note (insn1, note1);
for (insn = insn0; insn != insn1; insn = NEXT_INSN (insn))
while ((note = find_reg_note (insn, REG_NO_CONFLICT, NULL)))
remove_note (insn, note);
}
/* */
static void
resolve_reg_notes (rtx insn)
{
rtx *pnote, note;
note = find_reg_equal_equiv_note (insn);
if (note && for_each_rtx (&XEXP (note, 0), resolve_subreg_use, NULL))
{
remove_note (insn, note);
remove_retval_note (insn);
}
pnote = ®_NOTES (insn);
while ((note = *pnote))
{
bool delete = false;
switch (REG_NOTE_KIND (note))
{
case REG_NO_CONFLICT:
if (resolve_reg_p (XEXP (note, 0)))
delete = true;
break;
default:
break;
}
if (delete)
*pnote = XEXP (note, 1);
else
pnote = &XEXP (note, 1);
}
}
/* */
static bool
cannot_decompose_p (rtx x)
{
if (REG_P (x))
{
unsigned int regno = REGNO (x);
if (HARD_REGISTER_NUM_P (regno))
return !validate_subreg (word_mode, GET_MODE (x), x, UNITS_PER_WORD);
else
return bitmap_bit_p (non_decomposable_context, regno);
}
return false;
}
/* */
static rtx
resolve_simple_move (rtx set, rtx insn)
{
rtx dst, src, tmp;
bool rdp, rsp, sdp, ssp, delete;
unsigned int i, words;
enum machine_mode orig_mode;
dst = SET_DEST (set);
src = SET_SRC (set);
orig_mode = GET_MODE (dst);
sdp = ssp = false;
if (GET_CODE (dst) == SUBREG && GET_MODE_SIZE (orig_mode) > UNITS_PER_WORD)
sdp = rdp = resolve_reg_p (SUBREG_REG (dst));
else
rdp = resolve_reg_p (dst);
if (GET_CODE (src) == SUBREG && GET_MODE_SIZE (orig_mode) > UNITS_PER_WORD)
ssp = rsp = resolve_reg_p (SUBREG_REG (src));
else
rsp = resolve_reg_p (src);
if (!rdp && !rsp)
return insn;
start_sequence ();
delete = true;
if (ssp)
{
tmp = SUBREG_REG (src);
orig_mode = GET_MODE (tmp);
dst = gen_reg_rtx (orig_mode);
SUBREG_REG (src) = dst;
src = tmp;
delete = false;
}
if (!rsp && cannot_decompose_p (src))
{
tmp = gen_reg_rtx (orig_mode);
emit_move_insn (tmp, src);
src = tmp;
}
if (sdp)
{
tmp = SUBREG_REG (dst);
orig_mode = GET_MODE (tmp);
SUBREG_REG (dst) = gen_reg_rtx (orig_mode);
emit_move_insn (dst, src);
src = SUBREG_REG (dst);
dst = tmp;
}
if (!rdp && cannot_decompose_p (dst))
{
dst = gen_reg_rtx (orig_mode);
SET_SRC (set) = dst;
delete = false;
}
words = (GET_MODE_SIZE (orig_mode) + UNITS_PER_WORD - 1) / UNITS_PER_WORD;
if (push_operand (dst, orig_mode))
{
unsigned int j, jinc;
gcc_assert (GET_MODE_SIZE (orig_mode) % UNITS_PER_WORD == 0);
gcc_assert (GET_CODE (XEXP (dst, 0)) != PRE_MODIFY);
gcc_assert (GET_CODE (XEXP (dst, 0)) != POST_MODIFY);
if (WORDS_BIG_ENDIAN == STACK_GROWS_DOWNWARD)
j = 0, jinc = 1;
else
j = words - 1, jinc = -1;
for (i = 0; i < words; i++, j += jinc)
{
tmp = copy_rtx (XEXP (dst, 0));
tmp = adjust_automodify_address_nv (dst, word_mode, tmp,
j * UNITS_PER_WORD);
emit_move_insn (tmp, simplify_subreg (word_mode, src, orig_mode,
j * UNITS_PER_WORD));
}
}
else
{
gcc_assert (!MEM_P (dst)
|| GET_RTX_CLASS (GET_CODE (XEXP (dst, 0))) != RTX_AUTOINC);
gcc_assert (!MEM_P (src)
|| GET_RTX_CLASS (GET_CODE (XEXP (src, 0))) != RTX_AUTOINC);
if (REG_P (dst) && !HARD_REGISTER_NUM_P (REGNO (dst)))
emit_insn (gen_rtx_CLOBBER (VOIDmode, dst));
for (i = 0; i < words; ++i)
emit_move_insn (simplify_gen_subreg (word_mode, dst, orig_mode,
UNITS_PER_WORD * i),
simplify_gen_subreg (word_mode, src, orig_mode,
UNITS_PER_WORD * i));
}
tmp = get_insns ();
end_sequence ();
emit_insn_before (tmp, insn);
if (delete)
{
move_libcall_note (insn, tmp);
remove_retval_note (insn);
delete_insn (insn);
}
return tmp;
}
static void
resolve_clobber (rtx pat, rtx insn)
{
rtx reg = XEXP (pat, 0);
unsigned int words, i;
enum machine_mode orig_mode;
if (!resolve_reg_p (reg))
return;
orig_mode = GET_MODE (reg);
words = GET_MODE_SIZE (orig_mode);
words = (words + UNITS_PER_WORD - 1) / UNITS_PER_WORD;
XEXP (pat, 0) = simplify_subreg (word_mode, reg, orig_mode, 0);
for (i = words - 1; i > 0; --i)
{
pat = simplify_subreg (word_mode, reg, orig_mode, i * UNITS_PER_WORD);
pat = gen_rtx_CLOBBER (VOIDmode, pat);
emit_insn_after (pat, insn);
}
}
static void
resolve_use (rtx pat, rtx insn)
{
if (resolve_subreg_p (XEXP (pat, 0)))
delete_insn (insn);
}
/* */
static void
decompose_multiword_subregs (void)
{
rtx insn, set;
decomposable_context = BITMAP_ALLOC (NULL);
non_decomposable_context = BITMAP_ALLOC (NULL);
{
unsigned int max = max_reg_num ();
reg_copy_graph = VEC_alloc (bitmap, heap, max);
VEC_safe_grow (bitmap, heap, reg_copy_graph, max);
memset (VEC_address (bitmap, reg_copy_graph), 0, sizeof (bitmap) * max);
}
for (insn = get_insns (); insn ; insn = NEXT_INSN (insn))
if (INSN_P (insn))
{
if (GET_CODE (PATTERN (insn)) == CLOBBER
|| GET_CODE (PATTERN (insn)) == USE)
continue;
set = simple_move (insn);
if (set)
{
/* (set cc0 reg) is a comparison instruction and cannot be
decomposed. Clear SET so that we recognize this fact when
we see it in find_decomposable_subregs. */
if (CC0_P (SET_DEST (set)))
set = NULL;
else
find_pseudo_copy (set);
}
for_each_rtx (&PATTERN (insn), find_decomposable_subregs, set);
}
bitmap_and_compl_into (decomposable_context, non_decomposable_context);
if (!bitmap_empty_p (decomposable_context))
{
bitmap_iterator iter;
unsigned int regno;
propagate_pseudo_copies ();
EXECUTE_IF_SET_IN_BITMAP (decomposable_context, 0, regno, iter)
decompose_register (regno);
for (insn = get_insns (); insn ; insn = NEXT_INSN (insn))
{
rtx pat;
if (!INSN_P (insn))
continue;
pat = PATTERN (insn);
if (GET_CODE (pat) == CLOBBER)
resolve_clobber (pat, insn);
else if (GET_CODE (pat) == USE)
resolve_use (pat, insn);
else
{
set = simple_move (insn);
if (set)
insn = resolve_simple_move (set, insn);
for_each_rtx (&PATTERN (insn), resolve_subreg_use, NULL);
resolve_reg_notes (insn);
}
}
}
BITMAP_FREE (decomposable_context);
BITMAP_FREE (non_decomposable_context);
{
unsigned int i;
bitmap b;
for (i = 0; VEC_iterate (bitmap, reg_copy_graph, i, b); ++i)
if (b)
BITMAP_FREE (b);
}
VEC_free (bitmap, heap, reg_copy_graph);
}
static bool
gate_lower_subreg (void)
{
return flag_lower_subreg;
}
/* Run yet another register allocator. */
static unsigned int
rest_of_handle_lower_subreg (void)
{
int max_reg_num_before = max_reg_num ();
decompose_multiword_subregs ();
if (max_reg_num_before != max_reg_num ())
reg_scan (get_insns (), max_reg_num ());
return 0;
}
struct tree_opt_pass pass_lower_subreg =
{
"lower_subreg", /* name */
gate_lower_subreg, /* gate */
rest_of_handle_lower_subreg, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_LOWER_SUBREG, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
0, /* todo_flags_start */
TODO_dump_func |
TODO_ggc_collect, /* todo_flags_finish */
'Y' /* letter */
};