]>
Commit | Line | Data |
---|---|---|
8c660648 | 1 | /* Instruction scheduling pass. |
d050d723 | 2 | Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, |
1d088dee | 3 | 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. |
8c660648 JL |
4 | Contributed by Michael Tiemann (tiemann@cygnus.com) Enhanced by, |
5 | and currently maintained by, Jim Wilson (wilson@cygnus.com) | |
6 | ||
1322177d | 7 | This file is part of GCC. |
5d14e356 | 8 | |
1322177d LB |
9 | GCC is free software; you can redistribute it and/or modify it under |
10 | the terms of the GNU General Public License as published by the Free | |
11 | Software Foundation; either version 2, or (at your option) any later | |
12 | version. | |
5d14e356 | 13 | |
1322177d LB |
14 | GCC is distributed in the hope that it will be useful, but WITHOUT ANY |
15 | WARRANTY; without even the implied warranty of MERCHANTABILITY or | |
5d14e356 RK |
16 | FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License |
17 | for more details. | |
18 | ||
19 | You should have received a copy of the GNU General Public License | |
47a1bd82 NC |
20 | along with GCC; see the file COPYING. If not, write to the Free |
21 | Software Foundation, 59 Temple Place - Suite 330, Boston, MA | |
5d14e356 | 22 | 02111-1307, USA. */ |
8c660648 | 23 | |
b4ead7d4 BS |
24 | /* Instruction scheduling pass. This file, along with sched-deps.c, |
25 | contains the generic parts. The actual entry point is found for | |
26 | the normal instruction scheduling pass is found in sched-rgn.c. | |
8c660648 JL |
27 | |
28 | We compute insn priorities based on data dependencies. Flow | |
29 | analysis only creates a fraction of the data-dependencies we must | |
30 | observe: namely, only those dependencies which the combiner can be | |
31 | expected to use. For this pass, we must therefore create the | |
32 | remaining dependencies we need to observe: register dependencies, | |
33 | memory dependencies, dependencies to keep function calls in order, | |
34 | and the dependence between a conditional branch and the setting of | |
35 | condition codes are all dealt with here. | |
36 | ||
37 | The scheduler first traverses the data flow graph, starting with | |
38 | the last instruction, and proceeding to the first, assigning values | |
39 | to insn_priority as it goes. This sorts the instructions | |
40 | topologically by data dependence. | |
41 | ||
42 | Once priorities have been established, we order the insns using | |
43 | list scheduling. This works as follows: starting with a list of | |
44 | all the ready insns, and sorted according to priority number, we | |
45 | schedule the insn from the end of the list by placing its | |
46 | predecessors in the list according to their priority order. We | |
47 | consider this insn scheduled by setting the pointer to the "end" of | |
48 | the list to point to the previous insn. When an insn has no | |
49 | predecessors, we either queue it until sufficient time has elapsed | |
50 | or add it to the ready list. As the instructions are scheduled or | |
51 | when stalls are introduced, the queue advances and dumps insns into | |
52 | the ready list. When all insns down to the lowest priority have | |
53 | been scheduled, the critical path of the basic block has been made | |
54 | as short as possible. The remaining insns are then scheduled in | |
55 | remaining slots. | |
56 | ||
57 | Function unit conflicts are resolved during forward list scheduling | |
58 | by tracking the time when each insn is committed to the schedule | |
59 | and from that, the time the function units it uses must be free. | |
60 | As insns on the ready list are considered for scheduling, those | |
61 | that would result in a blockage of the already committed insns are | |
62 | queued until no blockage will result. | |
63 | ||
64 | The following list shows the order in which we want to break ties | |
65 | among insns in the ready list: | |
66 | ||
67 | 1. choose insn with the longest path to end of bb, ties | |
68 | broken by | |
69 | 2. choose insn with least contribution to register pressure, | |
70 | ties broken by | |
71 | 3. prefer in-block upon interblock motion, ties broken by | |
72 | 4. prefer useful upon speculative motion, ties broken by | |
73 | 5. choose insn with largest control flow probability, ties | |
74 | broken by | |
75 | 6. choose insn with the least dependences upon the previously | |
76 | scheduled insn, or finally | |
2db45993 JL |
77 | 7 choose the insn which has the most insns dependent on it. |
78 | 8. choose insn with lowest UID. | |
8c660648 JL |
79 | |
80 | Memory references complicate matters. Only if we can be certain | |
81 | that memory references are not part of the data dependency graph | |
82 | (via true, anti, or output dependence), can we move operations past | |
83 | memory references. To first approximation, reads can be done | |
84 | independently, while writes introduce dependencies. Better | |
85 | approximations will yield fewer dependencies. | |
86 | ||
87 | Before reload, an extended analysis of interblock data dependences | |
88 | is required for interblock scheduling. This is performed in | |
89 | compute_block_backward_dependences (). | |
90 | ||
91 | Dependencies set up by memory references are treated in exactly the | |
92 | same way as other dependencies, by using LOG_LINKS backward | |
93 | dependences. LOG_LINKS are translated into INSN_DEPEND forward | |
94 | dependences for the purpose of forward list scheduling. | |
95 | ||
96 | Having optimized the critical path, we may have also unduly | |
97 | extended the lifetimes of some registers. If an operation requires | |
98 | that constants be loaded into registers, it is certainly desirable | |
99 | to load those constants as early as necessary, but no earlier. | |
100 | I.e., it will not do to load up a bunch of registers at the | |
101 | beginning of a basic block only to use them at the end, if they | |
102 | could be loaded later, since this may result in excessive register | |
103 | utilization. | |
104 | ||
105 | Note that since branches are never in basic blocks, but only end | |
106 | basic blocks, this pass will not move branches. But that is ok, | |
107 | since we can use GNU's delayed branch scheduling pass to take care | |
108 | of this case. | |
109 | ||
110 | Also note that no further optimizations based on algebraic | |
111 | identities are performed, so this pass would be a good one to | |
112 | perform instruction splitting, such as breaking up a multiply | |
113 | instruction into shifts and adds where that is profitable. | |
114 | ||
115 | Given the memory aliasing analysis that this pass should perform, | |
116 | it should be possible to remove redundant stores to memory, and to | |
117 | load values from registers instead of hitting memory. | |
118 | ||
119 | Before reload, speculative insns are moved only if a 'proof' exists | |
120 | that no exception will be caused by this, and if no live registers | |
121 | exist that inhibit the motion (live registers constraints are not | |
122 | represented by data dependence edges). | |
123 | ||
124 | This pass must update information that subsequent passes expect to | |
125 | be correct. Namely: reg_n_refs, reg_n_sets, reg_n_deaths, | |
3b413743 RH |
126 | reg_n_calls_crossed, and reg_live_length. Also, BLOCK_HEAD, |
127 | BLOCK_END. | |
8c660648 JL |
128 | |
129 | The information in the line number notes is carefully retained by | |
130 | this pass. Notes that refer to the starting and ending of | |
131 | exception regions are also carefully retained by this pass. All | |
132 | other NOTE insns are grouped in their same relative order at the | |
b4ead7d4 | 133 | beginning of basic blocks and regions that have been scheduled. */ |
8c660648 | 134 | \f |
8c660648 | 135 | #include "config.h" |
5835e573 | 136 | #include "system.h" |
4977bab6 ZW |
137 | #include "coretypes.h" |
138 | #include "tm.h" | |
01198c2f | 139 | #include "toplev.h" |
8c660648 | 140 | #include "rtl.h" |
6baf1cc8 | 141 | #include "tm_p.h" |
efc9bd41 | 142 | #include "hard-reg-set.h" |
8c660648 JL |
143 | #include "basic-block.h" |
144 | #include "regs.h" | |
49ad7cfa | 145 | #include "function.h" |
8c660648 JL |
146 | #include "flags.h" |
147 | #include "insn-config.h" | |
148 | #include "insn-attr.h" | |
149 | #include "except.h" | |
487a6e06 | 150 | #include "toplev.h" |
79c9824e | 151 | #include "recog.h" |
1708fd40 | 152 | #include "sched-int.h" |
c237e94a | 153 | #include "target.h" |
8c660648 | 154 | |
8c660648 JL |
155 | #ifdef INSN_SCHEDULING |
156 | ||
8c660648 JL |
157 | /* issue_rate is the number of insns that can be scheduled in the same |
158 | machine cycle. It can be defined in the config/mach/mach.h file, | |
159 | otherwise we set it to 1. */ | |
160 | ||
161 | static int issue_rate; | |
162 | ||
88cad84b | 163 | /* If the following variable value is nonzero, the scheduler inserts |
fae15c93 VM |
164 | bubbles (nop insns). The value of variable affects on scheduler |
165 | behavior only if automaton pipeline interface with multipass | |
166 | scheduling is used and hook dfa_bubble is defined. */ | |
167 | int insert_schedule_bubbles_p = 0; | |
168 | ||
cc132865 | 169 | /* sched-verbose controls the amount of debugging output the |
409f8483 | 170 | scheduler prints. It is controlled by -fsched-verbose=N: |
8c660648 JL |
171 | N>0 and no -DSR : the output is directed to stderr. |
172 | N>=10 will direct the printouts to stderr (regardless of -dSR). | |
173 | N=1: same as -dSR. | |
174 | N=2: bb's probabilities, detailed ready list info, unit/insn info. | |
175 | N=3: rtl at abort point, control-flow, regions info. | |
cc132865 | 176 | N=5: dependences info. */ |
8c660648 | 177 | |
8c660648 | 178 | static int sched_verbose_param = 0; |
b4ead7d4 | 179 | int sched_verbose = 0; |
8c660648 | 180 | |
63de6c74 | 181 | /* Debugging file. All printouts are sent to dump, which is always set, |
8c660648 | 182 | either to stderr, or to the dump listing file (-dRS). */ |
c62c2659 | 183 | FILE *sched_dump = 0; |
a88f02e7 BS |
184 | |
185 | /* Highest uid before scheduling. */ | |
186 | static int old_max_uid; | |
8c660648 JL |
187 | |
188 | /* fix_sched_param() is called from toplev.c upon detection | |
409f8483 | 189 | of the -fsched-verbose=N option. */ |
8c660648 JL |
190 | |
191 | void | |
1d088dee | 192 | fix_sched_param (const char *param, const char *val) |
8c660648 | 193 | { |
cc132865 | 194 | if (!strcmp (param, "verbose")) |
8c660648 | 195 | sched_verbose_param = atoi (val); |
8c660648 JL |
196 | else |
197 | warning ("fix_sched_param: unknown param: %s", param); | |
198 | } | |
199 | ||
16f6ece6 | 200 | struct haifa_insn_data *h_i_d; |
f66d83e1 | 201 | |
f66d83e1 RH |
202 | #define LINE_NOTE(INSN) (h_i_d[INSN_UID (INSN)].line_note) |
203 | #define INSN_TICK(INSN) (h_i_d[INSN_UID (INSN)].tick) | |
8c660648 JL |
204 | |
205 | /* Vector indexed by basic block number giving the starting line-number | |
206 | for each basic block. */ | |
207 | static rtx *line_note_head; | |
208 | ||
209 | /* List of important notes we must keep around. This is a pointer to the | |
210 | last element in the list. */ | |
211 | static rtx note_list; | |
212 | ||
8c660648 JL |
213 | /* Queues, etc. */ |
214 | ||
215 | /* An instruction is ready to be scheduled when all insns preceding it | |
216 | have already been scheduled. It is important to ensure that all | |
217 | insns which use its result will not be executed until its result | |
218 | has been computed. An insn is maintained in one of four structures: | |
219 | ||
220 | (P) the "Pending" set of insns which cannot be scheduled until | |
221 | their dependencies have been satisfied. | |
222 | (Q) the "Queued" set of insns that can be scheduled when sufficient | |
223 | time has passed. | |
224 | (R) the "Ready" list of unscheduled, uncommitted insns. | |
225 | (S) the "Scheduled" list of insns. | |
226 | ||
227 | Initially, all insns are either "Pending" or "Ready" depending on | |
228 | whether their dependencies are satisfied. | |
229 | ||
230 | Insns move from the "Ready" list to the "Scheduled" list as they | |
231 | are committed to the schedule. As this occurs, the insns in the | |
232 | "Pending" list have their dependencies satisfied and move to either | |
233 | the "Ready" list or the "Queued" set depending on whether | |
234 | sufficient time has passed to make them ready. As time passes, | |
235 | insns move from the "Queued" set to the "Ready" list. Insns may | |
236 | move from the "Ready" list to the "Queued" set if they are blocked | |
237 | due to a function unit conflict. | |
238 | ||
239 | The "Pending" list (P) are the insns in the INSN_DEPEND of the unscheduled | |
240 | insns, i.e., those that are ready, queued, and pending. | |
241 | The "Queued" set (Q) is implemented by the variable `insn_queue'. | |
242 | The "Ready" list (R) is implemented by the variables `ready' and | |
243 | `n_ready'. | |
244 | The "Scheduled" list (S) is the new insn chain built by this pass. | |
245 | ||
246 | The transition (R->S) is implemented in the scheduling loop in | |
247 | `schedule_block' when the best insn to schedule is chosen. | |
248 | The transition (R->Q) is implemented in `queue_insn' when an | |
38e01259 | 249 | insn is found to have a function unit conflict with the already |
8c660648 JL |
250 | committed insns. |
251 | The transitions (P->R and P->Q) are implemented in `schedule_insn' as | |
252 | insns move from the ready list to the scheduled list. | |
253 | The transition (Q->R) is implemented in 'queue_to_insn' as time | |
254 | passes or stalls are introduced. */ | |
255 | ||
256 | /* Implement a circular buffer to delay instructions until sufficient | |
fae15c93 VM |
257 | time has passed. For the old pipeline description interface, |
258 | INSN_QUEUE_SIZE is a power of two larger than MAX_BLOCKAGE and | |
259 | MAX_READY_COST computed by genattr.c. For the new pipeline | |
260 | description interface, MAX_INSN_QUEUE_INDEX is a power of two minus | |
261 | one which is larger than maximal time of instruction execution | |
262 | computed by genattr.c on the base maximal time of functional unit | |
263 | reservations and geting a result. This is the longest time an | |
264 | insn may be queued. */ | |
265 | ||
266 | #define MAX_INSN_QUEUE_INDEX max_insn_queue_index_macro_value | |
267 | ||
268 | static rtx *insn_queue; | |
8c660648 JL |
269 | static int q_ptr = 0; |
270 | static int q_size = 0; | |
fae15c93 VM |
271 | #define NEXT_Q(X) (((X)+1) & MAX_INSN_QUEUE_INDEX) |
272 | #define NEXT_Q_AFTER(X, C) (((X)+C) & MAX_INSN_QUEUE_INDEX) | |
273 | ||
274 | /* The following variable defines value for macro | |
275 | MAX_INSN_QUEUE_INDEX. */ | |
276 | static int max_insn_queue_index_macro_value; | |
277 | ||
278 | /* The following variable value refers for all current and future | |
279 | reservations of the processor units. */ | |
280 | state_t curr_state; | |
281 | ||
282 | /* The following variable value is size of memory representing all | |
283 | current and future reservations of the processor units. It is used | |
284 | only by DFA based scheduler. */ | |
285 | static size_t dfa_state_size; | |
286 | ||
287 | /* The following array is used to find the best insn from ready when | |
288 | the automaton pipeline interface is used. */ | |
289 | static char *ready_try; | |
8c660648 | 290 | |
176f9a7b BS |
291 | /* Describe the ready list of the scheduler. |
292 | VEC holds space enough for all insns in the current region. VECLEN | |
293 | says how many exactly. | |
294 | FIRST is the index of the element with the highest priority; i.e. the | |
295 | last one in the ready list, since elements are ordered by ascending | |
296 | priority. | |
297 | N_READY determines how many insns are on the ready list. */ | |
298 | ||
299 | struct ready_list | |
300 | { | |
301 | rtx *vec; | |
302 | int veclen; | |
303 | int first; | |
304 | int n_ready; | |
305 | }; | |
306 | ||
1d088dee | 307 | static int may_trap_exp (rtx, int); |
15aab9c0 VM |
308 | |
309 | /* Nonzero iff the address is comprised from at most 1 register. */ | |
310 | #define CONST_BASED_ADDRESS_P(x) \ | |
311 | (GET_CODE (x) == REG \ | |
312 | || ((GET_CODE (x) == PLUS || GET_CODE (x) == MINUS \ | |
313 | || (GET_CODE (x) == LO_SUM)) \ | |
314 | && (CONSTANT_P (XEXP (x, 0)) \ | |
315 | || CONSTANT_P (XEXP (x, 1))))) | |
316 | ||
317 | /* Returns a class that insn with GET_DEST(insn)=x may belong to, | |
318 | as found by analyzing insn's expression. */ | |
319 | ||
320 | static int | |
1d088dee | 321 | may_trap_exp (rtx x, int is_store) |
15aab9c0 VM |
322 | { |
323 | enum rtx_code code; | |
324 | ||
325 | if (x == 0) | |
326 | return TRAP_FREE; | |
327 | code = GET_CODE (x); | |
328 | if (is_store) | |
329 | { | |
330 | if (code == MEM && may_trap_p (x)) | |
331 | return TRAP_RISKY; | |
332 | else | |
333 | return TRAP_FREE; | |
334 | } | |
335 | if (code == MEM) | |
336 | { | |
337 | /* The insn uses memory: a volatile load. */ | |
338 | if (MEM_VOLATILE_P (x)) | |
339 | return IRISKY; | |
340 | /* An exception-free load. */ | |
341 | if (!may_trap_p (x)) | |
342 | return IFREE; | |
343 | /* A load with 1 base register, to be further checked. */ | |
344 | if (CONST_BASED_ADDRESS_P (XEXP (x, 0))) | |
345 | return PFREE_CANDIDATE; | |
346 | /* No info on the load, to be further checked. */ | |
347 | return PRISKY_CANDIDATE; | |
348 | } | |
349 | else | |
350 | { | |
351 | const char *fmt; | |
352 | int i, insn_class = TRAP_FREE; | |
353 | ||
354 | /* Neither store nor load, check if it may cause a trap. */ | |
355 | if (may_trap_p (x)) | |
356 | return TRAP_RISKY; | |
357 | /* Recursive step: walk the insn... */ | |
358 | fmt = GET_RTX_FORMAT (code); | |
359 | for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--) | |
360 | { | |
361 | if (fmt[i] == 'e') | |
362 | { | |
363 | int tmp_class = may_trap_exp (XEXP (x, i), is_store); | |
364 | insn_class = WORST_CLASS (insn_class, tmp_class); | |
365 | } | |
366 | else if (fmt[i] == 'E') | |
367 | { | |
368 | int j; | |
369 | for (j = 0; j < XVECLEN (x, i); j++) | |
370 | { | |
371 | int tmp_class = may_trap_exp (XVECEXP (x, i, j), is_store); | |
372 | insn_class = WORST_CLASS (insn_class, tmp_class); | |
373 | if (insn_class == TRAP_RISKY || insn_class == IRISKY) | |
374 | break; | |
375 | } | |
376 | } | |
377 | if (insn_class == TRAP_RISKY || insn_class == IRISKY) | |
378 | break; | |
379 | } | |
380 | return insn_class; | |
381 | } | |
382 | } | |
383 | ||
384 | /* Classifies insn for the purpose of verifying that it can be | |
385 | moved speculatively, by examining it's patterns, returning: | |
386 | TRAP_RISKY: store, or risky non-load insn (e.g. division by variable). | |
387 | TRAP_FREE: non-load insn. | |
388 | IFREE: load from a globaly safe location. | |
389 | IRISKY: volatile load. | |
390 | PFREE_CANDIDATE, PRISKY_CANDIDATE: load that need to be checked for | |
391 | being either PFREE or PRISKY. */ | |
392 | ||
393 | int | |
1d088dee | 394 | haifa_classify_insn (rtx insn) |
15aab9c0 VM |
395 | { |
396 | rtx pat = PATTERN (insn); | |
397 | int tmp_class = TRAP_FREE; | |
398 | int insn_class = TRAP_FREE; | |
399 | enum rtx_code code; | |
400 | ||
401 | if (GET_CODE (pat) == PARALLEL) | |
402 | { | |
403 | int i, len = XVECLEN (pat, 0); | |
404 | ||
405 | for (i = len - 1; i >= 0; i--) | |
406 | { | |
407 | code = GET_CODE (XVECEXP (pat, 0, i)); | |
408 | switch (code) | |
409 | { | |
410 | case CLOBBER: | |
411 | /* Test if it is a 'store'. */ | |
412 | tmp_class = may_trap_exp (XEXP (XVECEXP (pat, 0, i), 0), 1); | |
413 | break; | |
414 | case SET: | |
415 | /* Test if it is a store. */ | |
416 | tmp_class = may_trap_exp (SET_DEST (XVECEXP (pat, 0, i)), 1); | |
417 | if (tmp_class == TRAP_RISKY) | |
418 | break; | |
419 | /* Test if it is a load. */ | |
420 | tmp_class | |
421 | = WORST_CLASS (tmp_class, | |
422 | may_trap_exp (SET_SRC (XVECEXP (pat, 0, i)), | |
423 | 0)); | |
424 | break; | |
425 | case COND_EXEC: | |
426 | case TRAP_IF: | |
427 | tmp_class = TRAP_RISKY; | |
428 | break; | |
429 | default: | |
430 | ; | |
431 | } | |
432 | insn_class = WORST_CLASS (insn_class, tmp_class); | |
433 | if (insn_class == TRAP_RISKY || insn_class == IRISKY) | |
434 | break; | |
435 | } | |
436 | } | |
437 | else | |
438 | { | |
439 | code = GET_CODE (pat); | |
440 | switch (code) | |
441 | { | |
442 | case CLOBBER: | |
443 | /* Test if it is a 'store'. */ | |
444 | tmp_class = may_trap_exp (XEXP (pat, 0), 1); | |
445 | break; | |
446 | case SET: | |
447 | /* Test if it is a store. */ | |
448 | tmp_class = may_trap_exp (SET_DEST (pat), 1); | |
449 | if (tmp_class == TRAP_RISKY) | |
450 | break; | |
451 | /* Test if it is a load. */ | |
452 | tmp_class = | |
453 | WORST_CLASS (tmp_class, | |
454 | may_trap_exp (SET_SRC (pat), 0)); | |
455 | break; | |
456 | case COND_EXEC: | |
457 | case TRAP_IF: | |
458 | tmp_class = TRAP_RISKY; | |
459 | break; | |
460 | default:; | |
461 | } | |
462 | insn_class = tmp_class; | |
463 | } | |
464 | ||
465 | return insn_class; | |
466 | } | |
467 | ||
8c660648 | 468 | /* Forward declarations. */ |
fae15c93 VM |
469 | |
470 | /* The scheduler using only DFA description should never use the | |
471 | following five functions: */ | |
1d088dee AJ |
472 | static unsigned int blockage_range (int, rtx); |
473 | static void clear_units (void); | |
474 | static void schedule_unit (int, rtx, int); | |
475 | static int actual_hazard (int, rtx, int, int); | |
476 | static int potential_hazard (int, rtx, int); | |
477 | ||
478 | static int priority (rtx); | |
479 | static int rank_for_schedule (const void *, const void *); | |
480 | static void swap_sort (rtx *, int); | |
481 | static void queue_insn (rtx, int); | |
482 | static int schedule_insn (rtx, struct ready_list *, int); | |
483 | static int find_set_reg_weight (rtx); | |
484 | static void find_insn_reg_weight (int); | |
485 | static void adjust_priority (rtx); | |
486 | static void advance_one_cycle (void); | |
8c660648 | 487 | |
8c660648 JL |
488 | /* Notes handling mechanism: |
489 | ========================= | |
490 | Generally, NOTES are saved before scheduling and restored after scheduling. | |
491 | The scheduler distinguishes between three types of notes: | |
492 | ||
493 | (1) LINE_NUMBER notes, generated and used for debugging. Here, | |
494 | before scheduling a region, a pointer to the LINE_NUMBER note is | |
495 | added to the insn following it (in save_line_notes()), and the note | |
496 | is removed (in rm_line_notes() and unlink_line_notes()). After | |
497 | scheduling the region, this pointer is used for regeneration of | |
498 | the LINE_NUMBER note (in restore_line_notes()). | |
499 | ||
500 | (2) LOOP_BEGIN, LOOP_END, SETJMP, EHREGION_BEG, EHREGION_END notes: | |
501 | Before scheduling a region, a pointer to the note is added to the insn | |
502 | that follows or precedes it. (This happens as part of the data dependence | |
503 | computation). After scheduling an insn, the pointer contained in it is | |
504 | used for regenerating the corresponding note (in reemit_notes). | |
505 | ||
506 | (3) All other notes (e.g. INSN_DELETED): Before scheduling a block, | |
507 | these notes are put in a list (in rm_other_notes() and | |
508 | unlink_other_notes ()). After scheduling the block, these notes are | |
509 | inserted at the beginning of the block (in schedule_block()). */ | |
510 | ||
1d088dee AJ |
511 | static rtx unlink_other_notes (rtx, rtx); |
512 | static rtx unlink_line_notes (rtx, rtx); | |
513 | static rtx reemit_notes (rtx, rtx); | |
3fe41456 | 514 | |
1d088dee AJ |
515 | static rtx *ready_lastpos (struct ready_list *); |
516 | static void ready_sort (struct ready_list *); | |
517 | static rtx ready_remove_first (struct ready_list *); | |
3fe41456 | 518 | |
1d088dee | 519 | static void queue_to_ready (struct ready_list *); |
176f9a7b | 520 | |
1d088dee | 521 | static void debug_ready_list (struct ready_list *); |
3fe41456 | 522 | |
1d088dee AJ |
523 | static rtx move_insn1 (rtx, rtx); |
524 | static rtx move_insn (rtx, rtx); | |
8c660648 | 525 | |
fae15c93 VM |
526 | /* The following functions are used to implement multi-pass scheduling |
527 | on the first cycle. It is used only for DFA based scheduler. */ | |
1d088dee AJ |
528 | static rtx ready_element (struct ready_list *, int); |
529 | static rtx ready_remove (struct ready_list *, int); | |
530 | static int max_issue (struct ready_list *, int *); | |
fae15c93 | 531 | |
1d088dee | 532 | static rtx choose_ready (struct ready_list *); |
fae15c93 | 533 | |
8c660648 JL |
534 | #endif /* INSN_SCHEDULING */ |
535 | \f | |
1708fd40 BS |
536 | /* Point to state used for the current scheduling pass. */ |
537 | struct sched_info *current_sched_info; | |
8c660648 JL |
538 | \f |
539 | #ifndef INSN_SCHEDULING | |
540 | void | |
1d088dee | 541 | schedule_insns (FILE *dump_file ATTRIBUTE_UNUSED) |
8c660648 JL |
542 | { |
543 | } | |
544 | #else | |
cbb13457 | 545 | |
8c660648 JL |
546 | /* Pointer to the last instruction scheduled. Used by rank_for_schedule, |
547 | so that insns independent of the last scheduled insn will be preferred | |
548 | over dependent instructions. */ | |
549 | ||
550 | static rtx last_scheduled_insn; | |
551 | ||
b4ead7d4 BS |
552 | /* Compute the function units used by INSN. This caches the value |
553 | returned by function_units_used. A function unit is encoded as the | |
43aa4e05 | 554 | unit number if the value is non-negative and the complement of a |
b4ead7d4 | 555 | mask if the value is negative. A function unit index is the |
fae15c93 VM |
556 | non-negative encoding. The scheduler using only DFA description |
557 | should never use the following function. */ | |
168cbdf9 | 558 | |
b4ead7d4 | 559 | HAIFA_INLINE int |
1d088dee | 560 | insn_unit (rtx insn) |
8c660648 | 561 | { |
b3694847 | 562 | int unit = INSN_UNIT (insn); |
168cbdf9 | 563 | |
b4ead7d4 | 564 | if (unit == 0) |
6b8cf0c5 | 565 | { |
b4ead7d4 | 566 | recog_memoized (insn); |
8c660648 | 567 | |
b4ead7d4 BS |
568 | /* A USE insn, or something else we don't need to understand. |
569 | We can't pass these directly to function_units_used because it will | |
570 | trigger a fatal error for unrecognizable insns. */ | |
571 | if (INSN_CODE (insn) < 0) | |
572 | unit = -1; | |
573 | else | |
8c660648 | 574 | { |
b4ead7d4 BS |
575 | unit = function_units_used (insn); |
576 | /* Increment non-negative values so we can cache zero. */ | |
577 | if (unit >= 0) | |
578 | unit++; | |
8c660648 | 579 | } |
b4ead7d4 BS |
580 | /* We only cache 16 bits of the result, so if the value is out of |
581 | range, don't cache it. */ | |
582 | if (FUNCTION_UNITS_SIZE < HOST_BITS_PER_SHORT | |
583 | || unit >= 0 | |
584 | || (unit & ~((1 << (HOST_BITS_PER_SHORT - 1)) - 1)) == 0) | |
585 | INSN_UNIT (insn) = unit; | |
8c660648 | 586 | } |
b4ead7d4 BS |
587 | return (unit > 0 ? unit - 1 : unit); |
588 | } | |
8c660648 | 589 | |
b4ead7d4 BS |
590 | /* Compute the blockage range for executing INSN on UNIT. This caches |
591 | the value returned by the blockage_range_function for the unit. | |
592 | These values are encoded in an int where the upper half gives the | |
fae15c93 VM |
593 | minimum value and the lower half gives the maximum value. The |
594 | scheduler using only DFA description should never use the following | |
595 | function. */ | |
8c660648 | 596 | |
b4ead7d4 | 597 | HAIFA_INLINE static unsigned int |
1d088dee | 598 | blockage_range (int unit, rtx insn) |
b4ead7d4 BS |
599 | { |
600 | unsigned int blockage = INSN_BLOCKAGE (insn); | |
601 | unsigned int range; | |
8c660648 | 602 | |
b4ead7d4 | 603 | if ((int) UNIT_BLOCKED (blockage) != unit + 1) |
8c660648 | 604 | { |
b4ead7d4 BS |
605 | range = function_units[unit].blockage_range_function (insn); |
606 | /* We only cache the blockage range for one unit and then only if | |
607 | the values fit. */ | |
608 | if (HOST_BITS_PER_INT >= UNIT_BITS + 2 * BLOCKAGE_BITS) | |
609 | INSN_BLOCKAGE (insn) = ENCODE_BLOCKAGE (unit + 1, range); | |
8c660648 JL |
610 | } |
611 | else | |
b4ead7d4 | 612 | range = BLOCKAGE_RANGE (blockage); |
8c660648 | 613 | |
b4ead7d4 | 614 | return range; |
8c660648 JL |
615 | } |
616 | ||
fae15c93 VM |
617 | /* A vector indexed by function unit instance giving the last insn to |
618 | use the unit. The value of the function unit instance index for | |
619 | unit U instance I is (U + I * FUNCTION_UNITS_SIZE). The scheduler | |
620 | using only DFA description should never use the following variable. */ | |
621 | #if FUNCTION_UNITS_SIZE | |
b4ead7d4 | 622 | static rtx unit_last_insn[FUNCTION_UNITS_SIZE * MAX_MULTIPLICITY]; |
fae15c93 VM |
623 | #else |
624 | static rtx unit_last_insn[1]; | |
625 | #endif | |
8c660648 | 626 | |
fae15c93 VM |
627 | /* A vector indexed by function unit instance giving the minimum time |
628 | when the unit will unblock based on the maximum blockage cost. The | |
629 | scheduler using only DFA description should never use the following | |
630 | variable. */ | |
631 | #if FUNCTION_UNITS_SIZE | |
b4ead7d4 | 632 | static int unit_tick[FUNCTION_UNITS_SIZE * MAX_MULTIPLICITY]; |
fae15c93 VM |
633 | #else |
634 | static int unit_tick[1]; | |
635 | #endif | |
b4ead7d4 BS |
636 | |
637 | /* A vector indexed by function unit number giving the number of insns | |
fae15c93 VM |
638 | that remain to use the unit. The scheduler using only DFA |
639 | description should never use the following variable. */ | |
640 | #if FUNCTION_UNITS_SIZE | |
b4ead7d4 | 641 | static int unit_n_insns[FUNCTION_UNITS_SIZE]; |
fae15c93 VM |
642 | #else |
643 | static int unit_n_insns[1]; | |
644 | #endif | |
8c660648 | 645 | |
fae15c93 VM |
646 | /* Access the unit_last_insn array. Used by the visualization code. |
647 | The scheduler using only DFA description should never use the | |
648 | following function. */ | |
8c660648 | 649 | |
b4ead7d4 | 650 | rtx |
1d088dee | 651 | get_unit_last_insn (int instance) |
8c660648 | 652 | { |
b4ead7d4 | 653 | return unit_last_insn[instance]; |
8c660648 JL |
654 | } |
655 | ||
b4ead7d4 | 656 | /* Reset the function unit state to the null state. */ |
8c660648 JL |
657 | |
658 | static void | |
1d088dee | 659 | clear_units (void) |
8c660648 | 660 | { |
703ad42b KG |
661 | memset (unit_last_insn, 0, sizeof (unit_last_insn)); |
662 | memset (unit_tick, 0, sizeof (unit_tick)); | |
663 | memset (unit_n_insns, 0, sizeof (unit_n_insns)); | |
b4ead7d4 | 664 | } |
8c660648 | 665 | |
fae15c93 VM |
666 | /* Return the issue-delay of an insn. The scheduler using only DFA |
667 | description should never use the following function. */ | |
8c660648 | 668 | |
b4ead7d4 | 669 | HAIFA_INLINE int |
1d088dee | 670 | insn_issue_delay (rtx insn) |
b4ead7d4 BS |
671 | { |
672 | int i, delay = 0; | |
673 | int unit = insn_unit (insn); | |
8c660648 | 674 | |
b4ead7d4 BS |
675 | /* Efficiency note: in fact, we are working 'hard' to compute a |
676 | value that was available in md file, and is not available in | |
677 | function_units[] structure. It would be nice to have this | |
678 | value there, too. */ | |
679 | if (unit >= 0) | |
8c660648 | 680 | { |
b4ead7d4 BS |
681 | if (function_units[unit].blockage_range_function && |
682 | function_units[unit].blockage_function) | |
683 | delay = function_units[unit].blockage_function (insn, insn); | |
8c660648 | 684 | } |
b4ead7d4 BS |
685 | else |
686 | for (i = 0, unit = ~unit; unit; i++, unit >>= 1) | |
687 | if ((unit & 1) != 0 && function_units[i].blockage_range_function | |
688 | && function_units[i].blockage_function) | |
689 | delay = MAX (delay, function_units[i].blockage_function (insn, insn)); | |
8c660648 | 690 | |
b4ead7d4 | 691 | return delay; |
8c660648 JL |
692 | } |
693 | ||
b4ead7d4 BS |
694 | /* Return the actual hazard cost of executing INSN on the unit UNIT, |
695 | instance INSTANCE at time CLOCK if the previous actual hazard cost | |
fae15c93 VM |
696 | was COST. The scheduler using only DFA description should never |
697 | use the following function. */ | |
8c660648 | 698 | |
b4ead7d4 | 699 | HAIFA_INLINE int |
1d088dee | 700 | actual_hazard_this_instance (int unit, int instance, rtx insn, int clock, int cost) |
8c660648 | 701 | { |
b4ead7d4 | 702 | int tick = unit_tick[instance]; /* Issue time of the last issued insn. */ |
8c660648 | 703 | |
b4ead7d4 | 704 | if (tick - clock > cost) |
8c660648 | 705 | { |
b4ead7d4 BS |
706 | /* The scheduler is operating forward, so unit's last insn is the |
707 | executing insn and INSN is the candidate insn. We want a | |
708 | more exact measure of the blockage if we execute INSN at CLOCK | |
709 | given when we committed the execution of the unit's last insn. | |
8c660648 | 710 | |
b4ead7d4 BS |
711 | The blockage value is given by either the unit's max blockage |
712 | constant, blockage range function, or blockage function. Use | |
713 | the most exact form for the given unit. */ | |
8c660648 | 714 | |
b4ead7d4 BS |
715 | if (function_units[unit].blockage_range_function) |
716 | { | |
717 | if (function_units[unit].blockage_function) | |
718 | tick += (function_units[unit].blockage_function | |
719 | (unit_last_insn[instance], insn) | |
720 | - function_units[unit].max_blockage); | |
721 | else | |
722 | tick += ((int) MAX_BLOCKAGE_COST (blockage_range (unit, insn)) | |
723 | - function_units[unit].max_blockage); | |
8c660648 | 724 | } |
b4ead7d4 BS |
725 | if (tick - clock > cost) |
726 | cost = tick - clock; | |
8c660648 | 727 | } |
b4ead7d4 | 728 | return cost; |
8c660648 JL |
729 | } |
730 | ||
fae15c93 VM |
731 | /* Record INSN as having begun execution on the units encoded by UNIT |
732 | at time CLOCK. The scheduler using only DFA description should | |
733 | never use the following function. */ | |
8c660648 | 734 | |
cbb13457 | 735 | HAIFA_INLINE static void |
1d088dee | 736 | schedule_unit (int unit, rtx insn, int clock) |
8c660648 JL |
737 | { |
738 | int i; | |
739 | ||
740 | if (unit >= 0) | |
741 | { | |
742 | int instance = unit; | |
743 | #if MAX_MULTIPLICITY > 1 | |
744 | /* Find the first free instance of the function unit and use that | |
745 | one. We assume that one is free. */ | |
746 | for (i = function_units[unit].multiplicity - 1; i > 0; i--) | |
747 | { | |
748 | if (!actual_hazard_this_instance (unit, instance, insn, clock, 0)) | |
749 | break; | |
750 | instance += FUNCTION_UNITS_SIZE; | |
751 | } | |
752 | #endif | |
753 | unit_last_insn[instance] = insn; | |
754 | unit_tick[instance] = (clock + function_units[unit].max_blockage); | |
755 | } | |
756 | else | |
757 | for (i = 0, unit = ~unit; unit; i++, unit >>= 1) | |
758 | if ((unit & 1) != 0) | |
759 | schedule_unit (i, insn, clock); | |
760 | } | |
761 | ||
fae15c93 VM |
762 | /* Return the actual hazard cost of executing INSN on the units |
763 | encoded by UNIT at time CLOCK if the previous actual hazard cost | |
764 | was COST. The scheduler using only DFA description should never | |
765 | use the following function. */ | |
8c660648 | 766 | |
cbb13457 | 767 | HAIFA_INLINE static int |
1d088dee | 768 | actual_hazard (int unit, rtx insn, int clock, int cost) |
8c660648 JL |
769 | { |
770 | int i; | |
771 | ||
772 | if (unit >= 0) | |
773 | { | |
774 | /* Find the instance of the function unit with the minimum hazard. */ | |
775 | int instance = unit; | |
776 | int best_cost = actual_hazard_this_instance (unit, instance, insn, | |
777 | clock, cost); | |
1eda7a81 | 778 | #if MAX_MULTIPLICITY > 1 |
8c660648 JL |
779 | int this_cost; |
780 | ||
8c660648 JL |
781 | if (best_cost > cost) |
782 | { | |
783 | for (i = function_units[unit].multiplicity - 1; i > 0; i--) | |
784 | { | |
785 | instance += FUNCTION_UNITS_SIZE; | |
786 | this_cost = actual_hazard_this_instance (unit, instance, insn, | |
787 | clock, cost); | |
788 | if (this_cost < best_cost) | |
789 | { | |
790 | best_cost = this_cost; | |
791 | if (this_cost <= cost) | |
792 | break; | |
793 | } | |
794 | } | |
795 | } | |
796 | #endif | |
797 | cost = MAX (cost, best_cost); | |
798 | } | |
799 | else | |
800 | for (i = 0, unit = ~unit; unit; i++, unit >>= 1) | |
801 | if ((unit & 1) != 0) | |
802 | cost = actual_hazard (i, insn, clock, cost); | |
803 | ||
804 | return cost; | |
805 | } | |
806 | ||
807 | /* Return the potential hazard cost of executing an instruction on the | |
fae15c93 VM |
808 | units encoded by UNIT if the previous potential hazard cost was |
809 | COST. An insn with a large blockage time is chosen in preference | |
810 | to one with a smaller time; an insn that uses a unit that is more | |
811 | likely to be used is chosen in preference to one with a unit that | |
812 | is less used. We are trying to minimize a subsequent actual | |
813 | hazard. The scheduler using only DFA description should never use | |
814 | the following function. */ | |
8c660648 | 815 | |
cbb13457 | 816 | HAIFA_INLINE static int |
1d088dee | 817 | potential_hazard (int unit, rtx insn, int cost) |
8c660648 JL |
818 | { |
819 | int i, ncost; | |
820 | unsigned int minb, maxb; | |
821 | ||
822 | if (unit >= 0) | |
823 | { | |
824 | minb = maxb = function_units[unit].max_blockage; | |
825 | if (maxb > 1) | |
826 | { | |
827 | if (function_units[unit].blockage_range_function) | |
828 | { | |
829 | maxb = minb = blockage_range (unit, insn); | |
830 | maxb = MAX_BLOCKAGE_COST (maxb); | |
831 | minb = MIN_BLOCKAGE_COST (minb); | |
832 | } | |
833 | ||
834 | if (maxb > 1) | |
835 | { | |
836 | /* Make the number of instructions left dominate. Make the | |
837 | minimum delay dominate the maximum delay. If all these | |
838 | are the same, use the unit number to add an arbitrary | |
839 | ordering. Other terms can be added. */ | |
840 | ncost = minb * 0x40 + maxb; | |
841 | ncost *= (unit_n_insns[unit] - 1) * 0x1000 + unit; | |
842 | if (ncost > cost) | |
843 | cost = ncost; | |
844 | } | |
845 | } | |
846 | } | |
847 | else | |
848 | for (i = 0, unit = ~unit; unit; i++, unit >>= 1) | |
849 | if ((unit & 1) != 0) | |
850 | cost = potential_hazard (i, insn, cost); | |
851 | ||
852 | return cost; | |
853 | } | |
854 | ||
855 | /* Compute cost of executing INSN given the dependence LINK on the insn USED. | |
856 | This is the number of cycles between instruction issue and | |
857 | instruction results. */ | |
858 | ||
b4ead7d4 | 859 | HAIFA_INLINE int |
1d088dee | 860 | insn_cost (rtx insn, rtx link, rtx used) |
8c660648 | 861 | { |
b3694847 | 862 | int cost = INSN_COST (insn); |
8c660648 | 863 | |
fae15c93 | 864 | if (cost < 0) |
8c660648 | 865 | { |
fae15c93 VM |
866 | /* A USE insn, or something else we don't need to |
867 | understand. We can't pass these directly to | |
868 | result_ready_cost or insn_default_latency because it will | |
869 | trigger a fatal error for unrecognizable insns. */ | |
870 | if (recog_memoized (insn) < 0) | |
8c660648 | 871 | { |
fae15c93 VM |
872 | INSN_COST (insn) = 0; |
873 | return 0; | |
8c660648 JL |
874 | } |
875 | else | |
876 | { | |
fae15c93 VM |
877 | if (targetm.sched.use_dfa_pipeline_interface |
878 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
879 | cost = insn_default_latency (insn); | |
880 | else | |
881 | cost = result_ready_cost (insn); | |
1d088dee | 882 | |
fae15c93 VM |
883 | if (cost < 0) |
884 | cost = 0; | |
1d088dee | 885 | |
8c660648 JL |
886 | INSN_COST (insn) = cost; |
887 | } | |
888 | } | |
889 | ||
63de6c74 | 890 | /* In this case estimate cost without caring how insn is used. */ |
fae15c93 | 891 | if (link == 0 || used == 0) |
8c660648 JL |
892 | return cost; |
893 | ||
fae15c93 VM |
894 | /* A USE insn should never require the value used to be computed. |
895 | This allows the computation of a function's result and parameter | |
896 | values to overlap the return and call. */ | |
897 | if (recog_memoized (used) < 0) | |
197043f5 | 898 | cost = 0; |
fae15c93 | 899 | else |
8c660648 | 900 | { |
fae15c93 VM |
901 | if (targetm.sched.use_dfa_pipeline_interface |
902 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
197043f5 | 903 | { |
fae15c93 VM |
904 | if (INSN_CODE (insn) >= 0) |
905 | { | |
906 | if (REG_NOTE_KIND (link) == REG_DEP_ANTI) | |
907 | cost = 0; | |
908 | else if (REG_NOTE_KIND (link) == REG_DEP_OUTPUT) | |
909 | { | |
910 | cost = (insn_default_latency (insn) | |
911 | - insn_default_latency (used)); | |
912 | if (cost <= 0) | |
913 | cost = 1; | |
914 | } | |
915 | else if (bypass_p (insn)) | |
916 | cost = insn_latency (insn, used); | |
917 | } | |
197043f5 | 918 | } |
b8ec5764 | 919 | |
fae15c93 VM |
920 | if (targetm.sched.adjust_cost) |
921 | cost = (*targetm.sched.adjust_cost) (used, link, insn, cost); | |
922 | ||
923 | if (cost < 0) | |
924 | cost = 0; | |
925 | } | |
1d088dee | 926 | |
8c660648 JL |
927 | return cost; |
928 | } | |
929 | ||
930 | /* Compute the priority number for INSN. */ | |
931 | ||
932 | static int | |
1d088dee | 933 | priority (rtx insn) |
8c660648 | 934 | { |
8c660648 JL |
935 | rtx link; |
936 | ||
2c3c49de | 937 | if (! INSN_P (insn)) |
8c660648 JL |
938 | return 0; |
939 | ||
21e4c9a8 | 940 | if (! INSN_PRIORITY_KNOWN (insn)) |
8c660648 | 941 | { |
21e4c9a8 BS |
942 | int this_priority = 0; |
943 | ||
8c660648 JL |
944 | if (INSN_DEPEND (insn) == 0) |
945 | this_priority = insn_cost (insn, 0, 0); | |
946 | else | |
21e4c9a8 BS |
947 | { |
948 | for (link = INSN_DEPEND (insn); link; link = XEXP (link, 1)) | |
949 | { | |
950 | rtx next; | |
951 | int next_priority; | |
8c660648 | 952 | |
21e4c9a8 BS |
953 | if (RTX_INTEGRATED_P (link)) |
954 | continue; | |
6d8ccdbb | 955 | |
21e4c9a8 | 956 | next = XEXP (link, 0); |
8c660648 | 957 | |
21e4c9a8 BS |
958 | /* Critical path is meaningful in block boundaries only. */ |
959 | if (! (*current_sched_info->contributes_to_priority) (next, insn)) | |
960 | continue; | |
8c660648 | 961 | |
21e4c9a8 BS |
962 | next_priority = insn_cost (insn, link, next) + priority (next); |
963 | if (next_priority > this_priority) | |
964 | this_priority = next_priority; | |
965 | } | |
966 | } | |
8c660648 | 967 | INSN_PRIORITY (insn) = this_priority; |
21e4c9a8 | 968 | INSN_PRIORITY_KNOWN (insn) = 1; |
8c660648 | 969 | } |
21e4c9a8 BS |
970 | |
971 | return INSN_PRIORITY (insn); | |
8c660648 JL |
972 | } |
973 | \f | |
8c660648 JL |
974 | /* Macros and functions for keeping the priority queue sorted, and |
975 | dealing with queueing and dequeueing of instructions. */ | |
976 | ||
977 | #define SCHED_SORT(READY, N_READY) \ | |
978 | do { if ((N_READY) == 2) \ | |
979 | swap_sort (READY, N_READY); \ | |
980 | else if ((N_READY) > 2) \ | |
981 | qsort (READY, N_READY, sizeof (rtx), rank_for_schedule); } \ | |
982 | while (0) | |
983 | ||
984 | /* Returns a positive value if x is preferred; returns a negative value if | |
985 | y is preferred. Should never return 0, since that will make the sort | |
986 | unstable. */ | |
987 | ||
988 | static int | |
1d088dee | 989 | rank_for_schedule (const void *x, const void *y) |
8c660648 | 990 | { |
7a403706 KH |
991 | rtx tmp = *(const rtx *) y; |
992 | rtx tmp2 = *(const rtx *) x; | |
8f39865a | 993 | rtx link; |
2db45993 | 994 | int tmp_class, tmp2_class, depend_count1, depend_count2; |
1708fd40 | 995 | int val, priority_val, weight_val, info_val; |
8c660648 | 996 | |
58fb7809 VM |
997 | /* The insn in a schedule group should be issued the first. */ |
998 | if (SCHED_GROUP_P (tmp) != SCHED_GROUP_P (tmp2)) | |
999 | return SCHED_GROUP_P (tmp2) ? 1 : -1; | |
1000 | ||
63de6c74 | 1001 | /* Prefer insn with higher priority. */ |
8c660648 | 1002 | priority_val = INSN_PRIORITY (tmp2) - INSN_PRIORITY (tmp); |
30028c85 | 1003 | |
8c660648 JL |
1004 | if (priority_val) |
1005 | return priority_val; | |
1006 | ||
63de6c74 | 1007 | /* Prefer an insn with smaller contribution to registers-pressure. */ |
8c660648 JL |
1008 | if (!reload_completed && |
1009 | (weight_val = INSN_REG_WEIGHT (tmp) - INSN_REG_WEIGHT (tmp2))) | |
4977bab6 | 1010 | return weight_val; |
8c660648 | 1011 | |
1708fd40 BS |
1012 | info_val = (*current_sched_info->rank) (tmp, tmp2); |
1013 | if (info_val) | |
1014 | return info_val; | |
8c660648 | 1015 | |
63de6c74 | 1016 | /* Compare insns based on their relation to the last-scheduled-insn. */ |
8f39865a | 1017 | if (last_scheduled_insn) |
8c660648 JL |
1018 | { |
1019 | /* Classify the instructions into three classes: | |
1020 | 1) Data dependent on last schedule insn. | |
1021 | 2) Anti/Output dependent on last scheduled insn. | |
1022 | 3) Independent of last scheduled insn, or has latency of one. | |
1023 | Choose the insn from the highest numbered class if different. */ | |
8f39865a DM |
1024 | link = find_insn_list (tmp, INSN_DEPEND (last_scheduled_insn)); |
1025 | if (link == 0 || insn_cost (last_scheduled_insn, link, tmp) == 1) | |
8c660648 JL |
1026 | tmp_class = 3; |
1027 | else if (REG_NOTE_KIND (link) == 0) /* Data dependence. */ | |
1028 | tmp_class = 1; | |
1029 | else | |
1030 | tmp_class = 2; | |
1031 | ||
8f39865a DM |
1032 | link = find_insn_list (tmp2, INSN_DEPEND (last_scheduled_insn)); |
1033 | if (link == 0 || insn_cost (last_scheduled_insn, link, tmp2) == 1) | |
8c660648 JL |
1034 | tmp2_class = 3; |
1035 | else if (REG_NOTE_KIND (link) == 0) /* Data dependence. */ | |
1036 | tmp2_class = 1; | |
1037 | else | |
1038 | tmp2_class = 2; | |
1039 | ||
1040 | if ((val = tmp2_class - tmp_class)) | |
1041 | return val; | |
1042 | } | |
1043 | ||
7a403706 | 1044 | /* Prefer the insn which has more later insns that depend on it. |
2db45993 JL |
1045 | This gives the scheduler more freedom when scheduling later |
1046 | instructions at the expense of added register pressure. */ | |
1047 | depend_count1 = 0; | |
1048 | for (link = INSN_DEPEND (tmp); link; link = XEXP (link, 1)) | |
1049 | depend_count1++; | |
1050 | ||
1051 | depend_count2 = 0; | |
1052 | for (link = INSN_DEPEND (tmp2); link; link = XEXP (link, 1)) | |
1053 | depend_count2++; | |
1054 | ||
1055 | val = depend_count2 - depend_count1; | |
1056 | if (val) | |
1057 | return val; | |
7a403706 | 1058 | |
8c660648 JL |
1059 | /* If insns are equally good, sort by INSN_LUID (original insn order), |
1060 | so that we make the sort stable. This minimizes instruction movement, | |
1061 | thus minimizing sched's effect on debugging and cross-jumping. */ | |
1062 | return INSN_LUID (tmp) - INSN_LUID (tmp2); | |
1063 | } | |
1064 | ||
1065 | /* Resort the array A in which only element at index N may be out of order. */ | |
1066 | ||
cbb13457 | 1067 | HAIFA_INLINE static void |
1d088dee | 1068 | swap_sort (rtx *a, int n) |
8c660648 JL |
1069 | { |
1070 | rtx insn = a[n - 1]; | |
1071 | int i = n - 2; | |
1072 | ||
1073 | while (i >= 0 && rank_for_schedule (a + i, &insn) >= 0) | |
1074 | { | |
1075 | a[i + 1] = a[i]; | |
1076 | i -= 1; | |
1077 | } | |
1078 | a[i + 1] = insn; | |
1079 | } | |
1080 | ||
8c660648 JL |
1081 | /* Add INSN to the insn queue so that it can be executed at least |
1082 | N_CYCLES after the currently executing insn. Preserve insns | |
1083 | chain for debugging purposes. */ | |
1084 | ||
cbb13457 | 1085 | HAIFA_INLINE static void |
1d088dee | 1086 | queue_insn (rtx insn, int n_cycles) |
8c660648 JL |
1087 | { |
1088 | int next_q = NEXT_Q_AFTER (q_ptr, n_cycles); | |
ebb7b10b | 1089 | rtx link = alloc_INSN_LIST (insn, insn_queue[next_q]); |
8c660648 JL |
1090 | insn_queue[next_q] = link; |
1091 | q_size += 1; | |
1092 | ||
1093 | if (sched_verbose >= 2) | |
1094 | { | |
1708fd40 BS |
1095 | fprintf (sched_dump, ";;\t\tReady-->Q: insn %s: ", |
1096 | (*current_sched_info->print_insn) (insn, 0)); | |
8c660648 | 1097 | |
a88f02e7 | 1098 | fprintf (sched_dump, "queued for %d cycles.\n", n_cycles); |
8c660648 | 1099 | } |
176f9a7b BS |
1100 | } |
1101 | ||
1102 | /* Return a pointer to the bottom of the ready list, i.e. the insn | |
1103 | with the lowest priority. */ | |
1104 | ||
1105 | HAIFA_INLINE static rtx * | |
1d088dee | 1106 | ready_lastpos (struct ready_list *ready) |
176f9a7b BS |
1107 | { |
1108 | if (ready->n_ready == 0) | |
1109 | abort (); | |
1110 | return ready->vec + ready->first - ready->n_ready + 1; | |
1111 | } | |
1112 | ||
1113 | /* Add an element INSN to the ready list so that it ends up with the lowest | |
1114 | priority. */ | |
1115 | ||
b4ead7d4 | 1116 | HAIFA_INLINE void |
1d088dee | 1117 | ready_add (struct ready_list *ready, rtx insn) |
176f9a7b BS |
1118 | { |
1119 | if (ready->first == ready->n_ready) | |
1120 | { | |
1121 | memmove (ready->vec + ready->veclen - ready->n_ready, | |
1122 | ready_lastpos (ready), | |
1123 | ready->n_ready * sizeof (rtx)); | |
1124 | ready->first = ready->veclen - 1; | |
1125 | } | |
1126 | ready->vec[ready->first - ready->n_ready] = insn; | |
1127 | ready->n_ready++; | |
1128 | } | |
8c660648 | 1129 | |
176f9a7b BS |
1130 | /* Remove the element with the highest priority from the ready list and |
1131 | return it. */ | |
1132 | ||
1133 | HAIFA_INLINE static rtx | |
1d088dee | 1134 | ready_remove_first (struct ready_list *ready) |
176f9a7b BS |
1135 | { |
1136 | rtx t; | |
1137 | if (ready->n_ready == 0) | |
1138 | abort (); | |
1139 | t = ready->vec[ready->first--]; | |
1140 | ready->n_ready--; | |
1141 | /* If the queue becomes empty, reset it. */ | |
1142 | if (ready->n_ready == 0) | |
1143 | ready->first = ready->veclen - 1; | |
1144 | return t; | |
1145 | } | |
1146 | ||
fae15c93 VM |
1147 | /* The following code implements multi-pass scheduling for the first |
1148 | cycle. In other words, we will try to choose ready insn which | |
1149 | permits to start maximum number of insns on the same cycle. */ | |
1150 | ||
1151 | /* Return a pointer to the element INDEX from the ready. INDEX for | |
1152 | insn with the highest priority is 0, and the lowest priority has | |
1153 | N_READY - 1. */ | |
1154 | ||
1155 | HAIFA_INLINE static rtx | |
1d088dee | 1156 | ready_element (struct ready_list *ready, int index) |
fae15c93 | 1157 | { |
30028c85 | 1158 | #ifdef ENABLE_CHECKING |
fae15c93 VM |
1159 | if (ready->n_ready == 0 || index >= ready->n_ready) |
1160 | abort (); | |
30028c85 | 1161 | #endif |
fae15c93 VM |
1162 | return ready->vec[ready->first - index]; |
1163 | } | |
1164 | ||
1165 | /* Remove the element INDEX from the ready list and return it. INDEX | |
1166 | for insn with the highest priority is 0, and the lowest priority | |
1167 | has N_READY - 1. */ | |
1168 | ||
1169 | HAIFA_INLINE static rtx | |
1d088dee | 1170 | ready_remove (struct ready_list *ready, int index) |
fae15c93 VM |
1171 | { |
1172 | rtx t; | |
1173 | int i; | |
1174 | ||
1175 | if (index == 0) | |
1176 | return ready_remove_first (ready); | |
1177 | if (ready->n_ready == 0 || index >= ready->n_ready) | |
1178 | abort (); | |
1179 | t = ready->vec[ready->first - index]; | |
1180 | ready->n_ready--; | |
1181 | for (i = index; i < ready->n_ready; i++) | |
1182 | ready->vec[ready->first - i] = ready->vec[ready->first - i - 1]; | |
1183 | return t; | |
1184 | } | |
1185 | ||
1186 | ||
176f9a7b BS |
1187 | /* Sort the ready list READY by ascending priority, using the SCHED_SORT |
1188 | macro. */ | |
1189 | ||
1190 | HAIFA_INLINE static void | |
1d088dee | 1191 | ready_sort (struct ready_list *ready) |
176f9a7b BS |
1192 | { |
1193 | rtx *first = ready_lastpos (ready); | |
1194 | SCHED_SORT (first, ready->n_ready); | |
8c660648 JL |
1195 | } |
1196 | ||
8c660648 | 1197 | /* PREV is an insn that is ready to execute. Adjust its priority if that |
c46a37c4 RH |
1198 | will help shorten or lengthen register lifetimes as appropriate. Also |
1199 | provide a hook for the target to tweek itself. */ | |
8c660648 | 1200 | |
cbb13457 | 1201 | HAIFA_INLINE static void |
1d088dee | 1202 | adjust_priority (rtx prev) |
8c660648 | 1203 | { |
c46a37c4 RH |
1204 | /* ??? There used to be code here to try and estimate how an insn |
1205 | affected register lifetimes, but it did it by looking at REG_DEAD | |
7a403706 | 1206 | notes, which we removed in schedule_region. Nor did it try to |
c46a37c4 | 1207 | take into account register pressure or anything useful like that. |
8c660648 | 1208 | |
c46a37c4 | 1209 | Revisit when we have a machine model to work with and not before. */ |
197043f5 | 1210 | |
c237e94a ZW |
1211 | if (targetm.sched.adjust_priority) |
1212 | INSN_PRIORITY (prev) = | |
1213 | (*targetm.sched.adjust_priority) (prev, INSN_PRIORITY (prev)); | |
8c660648 JL |
1214 | } |
1215 | ||
fae15c93 VM |
1216 | /* Advance time on one cycle. */ |
1217 | HAIFA_INLINE static void | |
1d088dee | 1218 | advance_one_cycle (void) |
fae15c93 VM |
1219 | { |
1220 | if (targetm.sched.use_dfa_pipeline_interface | |
1221 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
1222 | { | |
1223 | if (targetm.sched.dfa_pre_cycle_insn) | |
1224 | state_transition (curr_state, | |
1225 | (*targetm.sched.dfa_pre_cycle_insn) ()); | |
1226 | ||
1227 | state_transition (curr_state, NULL); | |
1228 | ||
1229 | if (targetm.sched.dfa_post_cycle_insn) | |
1230 | state_transition (curr_state, | |
1231 | (*targetm.sched.dfa_post_cycle_insn) ()); | |
1232 | } | |
1233 | } | |
1234 | ||
4bdc8810 RH |
1235 | /* Clock at which the previous instruction was issued. */ |
1236 | static int last_clock_var; | |
1237 | ||
8c660648 | 1238 | /* INSN is the "currently executing insn". Launch each insn which was |
176f9a7b | 1239 | waiting on INSN. READY is the ready list which contains the insns |
58fb7809 VM |
1240 | that are ready to fire. CLOCK is the current cycle. The function |
1241 | returns necessary cycle advance after issuing the insn (it is not | |
1242 | zero for insns in a schedule group). */ | |
8c660648 | 1243 | |
58fb7809 | 1244 | static int |
1d088dee | 1245 | schedule_insn (rtx insn, struct ready_list *ready, int clock) |
8c660648 JL |
1246 | { |
1247 | rtx link; | |
58fb7809 | 1248 | int advance = 0; |
fae15c93 | 1249 | int unit = 0; |
8c660648 | 1250 | |
fae15c93 VM |
1251 | if (!targetm.sched.use_dfa_pipeline_interface |
1252 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
1253 | unit = insn_unit (insn); | |
8c660648 | 1254 | |
6d0de005 JH |
1255 | if (targetm.sched.use_dfa_pipeline_interface |
1256 | && (*targetm.sched.use_dfa_pipeline_interface) () | |
1257 | && sched_verbose >= 1) | |
8c660648 | 1258 | { |
6d0de005 | 1259 | char buf[2048]; |
fae15c93 | 1260 | |
6d0de005 | 1261 | print_insn (buf, insn, 0); |
6a87d634 | 1262 | buf[40] = 0; |
6d0de005 | 1263 | fprintf (sched_dump, ";;\t%3i--> %-40s:", clock, buf); |
fae15c93 | 1264 | |
6d0de005 JH |
1265 | if (recog_memoized (insn) < 0) |
1266 | fprintf (sched_dump, "nothing"); | |
1267 | else | |
1268 | print_reservation (sched_dump, insn); | |
1269 | fputc ('\n', sched_dump); | |
1270 | } | |
1271 | else if (sched_verbose >= 2) | |
1272 | { | |
1273 | fprintf (sched_dump, ";;\t\t--> scheduling insn <<<%d>>> on unit ", | |
1274 | INSN_UID (insn)); | |
1275 | insn_print_units (insn); | |
1276 | fputc ('\n', sched_dump); | |
8c660648 JL |
1277 | } |
1278 | ||
fae15c93 VM |
1279 | if (!targetm.sched.use_dfa_pipeline_interface |
1280 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
1281 | { | |
1282 | if (sched_verbose && unit == -1) | |
1283 | visualize_no_unit (insn); | |
8c660648 | 1284 | |
8c660648 | 1285 | |
fae15c93 VM |
1286 | if (MAX_BLOCKAGE > 1 || issue_rate > 1 || sched_verbose) |
1287 | schedule_unit (unit, insn, clock); | |
1d088dee | 1288 | |
fae15c93 | 1289 | if (INSN_DEPEND (insn) == 0) |
58fb7809 | 1290 | return 0; |
fae15c93 | 1291 | } |
8c660648 JL |
1292 | |
1293 | for (link = INSN_DEPEND (insn); link != 0; link = XEXP (link, 1)) | |
1294 | { | |
1295 | rtx next = XEXP (link, 0); | |
1296 | int cost = insn_cost (insn, link, next); | |
1297 | ||
1298 | INSN_TICK (next) = MAX (INSN_TICK (next), clock + cost); | |
1299 | ||
1300 | if ((INSN_DEP_COUNT (next) -= 1) == 0) | |
1301 | { | |
1302 | int effective_cost = INSN_TICK (next) - clock; | |
1303 | ||
1708fd40 | 1304 | if (! (*current_sched_info->new_ready) (next)) |
8c660648 JL |
1305 | continue; |
1306 | ||
1307 | if (sched_verbose >= 2) | |
1308 | { | |
1708fd40 BS |
1309 | fprintf (sched_dump, ";;\t\tdependences resolved: insn %s ", |
1310 | (*current_sched_info->print_insn) (next, 0)); | |
8c660648 | 1311 | |
197043f5 | 1312 | if (effective_cost < 1) |
a88f02e7 | 1313 | fprintf (sched_dump, "into ready\n"); |
8c660648 | 1314 | else |
58fb7809 VM |
1315 | fprintf (sched_dump, "into queue with cost=%d\n", |
1316 | effective_cost); | |
8c660648 JL |
1317 | } |
1318 | ||
1319 | /* Adjust the priority of NEXT and either put it on the ready | |
1320 | list or queue it. */ | |
1321 | adjust_priority (next); | |
197043f5 | 1322 | if (effective_cost < 1) |
176f9a7b | 1323 | ready_add (ready, next); |
8c660648 | 1324 | else |
58fb7809 VM |
1325 | { |
1326 | queue_insn (next, effective_cost); | |
1327 | ||
1328 | if (SCHED_GROUP_P (next) && advance < effective_cost) | |
1329 | advance = effective_cost; | |
1330 | } | |
8c660648 JL |
1331 | } |
1332 | } | |
1333 | ||
7a403706 | 1334 | /* Annotate the instruction with issue information -- TImode |
4bdc8810 RH |
1335 | indicates that the instruction is expected not to be able |
1336 | to issue on the same cycle as the previous insn. A machine | |
1337 | may use this information to decide how the instruction should | |
1338 | be aligned. */ | |
30028c85 | 1339 | if (issue_rate > 1 |
fae15c93 VM |
1340 | && GET_CODE (PATTERN (insn)) != USE |
1341 | && GET_CODE (PATTERN (insn)) != CLOBBER) | |
4bdc8810 | 1342 | { |
30028c85 VM |
1343 | if (reload_completed) |
1344 | PUT_MODE (insn, clock > last_clock_var ? TImode : VOIDmode); | |
4bdc8810 RH |
1345 | last_clock_var = clock; |
1346 | } | |
58fb7809 | 1347 | return advance; |
8c660648 JL |
1348 | } |
1349 | ||
63de6c74 | 1350 | /* Functions for handling of notes. */ |
8c660648 JL |
1351 | |
1352 | /* Delete notes beginning with INSN and put them in the chain | |
1353 | of notes ended by NOTE_LIST. | |
1354 | Returns the insn following the notes. */ | |
1355 | ||
1356 | static rtx | |
1d088dee | 1357 | unlink_other_notes (rtx insn, rtx tail) |
8c660648 JL |
1358 | { |
1359 | rtx prev = PREV_INSN (insn); | |
1360 | ||
1361 | while (insn != tail && GET_CODE (insn) == NOTE) | |
1362 | { | |
1363 | rtx next = NEXT_INSN (insn); | |
1364 | /* Delete the note from its current position. */ | |
1365 | if (prev) | |
1366 | NEXT_INSN (prev) = next; | |
1367 | if (next) | |
1368 | PREV_INSN (next) = prev; | |
1369 | ||
c46a37c4 | 1370 | /* See sched_analyze to see how these are handled. */ |
570a98eb | 1371 | if (NOTE_LINE_NUMBER (insn) != NOTE_INSN_LOOP_BEG |
8c660648 | 1372 | && NOTE_LINE_NUMBER (insn) != NOTE_INSN_LOOP_END |
8f62128d | 1373 | && NOTE_LINE_NUMBER (insn) != NOTE_INSN_BASIC_BLOCK |
8c660648 JL |
1374 | && NOTE_LINE_NUMBER (insn) != NOTE_INSN_EH_REGION_BEG |
1375 | && NOTE_LINE_NUMBER (insn) != NOTE_INSN_EH_REGION_END) | |
1376 | { | |
1377 | /* Insert the note at the end of the notes list. */ | |
1378 | PREV_INSN (insn) = note_list; | |
1379 | if (note_list) | |
1380 | NEXT_INSN (note_list) = insn; | |
1381 | note_list = insn; | |
1382 | } | |
1383 | ||
1384 | insn = next; | |
1385 | } | |
1386 | return insn; | |
1387 | } | |
1388 | ||
1389 | /* Delete line notes beginning with INSN. Record line-number notes so | |
1390 | they can be reused. Returns the insn following the notes. */ | |
1391 | ||
1392 | static rtx | |
1d088dee | 1393 | unlink_line_notes (rtx insn, rtx tail) |
8c660648 JL |
1394 | { |
1395 | rtx prev = PREV_INSN (insn); | |
1396 | ||
1397 | while (insn != tail && GET_CODE (insn) == NOTE) | |
1398 | { | |
1399 | rtx next = NEXT_INSN (insn); | |
1400 | ||
1401 | if (write_symbols != NO_DEBUG && NOTE_LINE_NUMBER (insn) > 0) | |
1402 | { | |
1403 | /* Delete the note from its current position. */ | |
1404 | if (prev) | |
1405 | NEXT_INSN (prev) = next; | |
1406 | if (next) | |
1407 | PREV_INSN (next) = prev; | |
1408 | ||
1409 | /* Record line-number notes so they can be reused. */ | |
1410 | LINE_NOTE (insn) = insn; | |
1411 | } | |
1412 | else | |
1413 | prev = insn; | |
1414 | ||
1415 | insn = next; | |
1416 | } | |
1417 | return insn; | |
1418 | } | |
1419 | ||
1420 | /* Return the head and tail pointers of BB. */ | |
1421 | ||
b4ead7d4 | 1422 | void |
1d088dee | 1423 | get_block_head_tail (int b, rtx *headp, rtx *tailp) |
8c660648 | 1424 | { |
8c660648 | 1425 | /* HEAD and TAIL delimit the basic block being scheduled. */ |
1708fd40 BS |
1426 | rtx head = BLOCK_HEAD (b); |
1427 | rtx tail = BLOCK_END (b); | |
8c660648 JL |
1428 | |
1429 | /* Don't include any notes or labels at the beginning of the | |
1430 | basic block, or notes at the ends of basic blocks. */ | |
1431 | while (head != tail) | |
1432 | { | |
1433 | if (GET_CODE (head) == NOTE) | |
1434 | head = NEXT_INSN (head); | |
1435 | else if (GET_CODE (tail) == NOTE) | |
1436 | tail = PREV_INSN (tail); | |
1437 | else if (GET_CODE (head) == CODE_LABEL) | |
1438 | head = NEXT_INSN (head); | |
1439 | else | |
1440 | break; | |
1441 | } | |
1442 | ||
1443 | *headp = head; | |
1444 | *tailp = tail; | |
1445 | } | |
1446 | ||
1708fd40 BS |
1447 | /* Return nonzero if there are no real insns in the range [ HEAD, TAIL ]. */ |
1448 | ||
b4ead7d4 | 1449 | int |
1d088dee | 1450 | no_real_insns_p (rtx head, rtx tail) |
1708fd40 BS |
1451 | { |
1452 | while (head != NEXT_INSN (tail)) | |
1453 | { | |
1454 | if (GET_CODE (head) != NOTE && GET_CODE (head) != CODE_LABEL) | |
1455 | return 0; | |
1456 | head = NEXT_INSN (head); | |
1457 | } | |
1458 | return 1; | |
1459 | } | |
1460 | ||
79c2ffde BS |
1461 | /* Delete line notes from one block. Save them so they can be later restored |
1462 | (in restore_line_notes). HEAD and TAIL are the boundaries of the | |
1463 | block in which notes should be processed. */ | |
8c660648 | 1464 | |
b4ead7d4 | 1465 | void |
1d088dee | 1466 | rm_line_notes (rtx head, rtx tail) |
8c660648 JL |
1467 | { |
1468 | rtx next_tail; | |
8c660648 JL |
1469 | rtx insn; |
1470 | ||
8c660648 JL |
1471 | next_tail = NEXT_INSN (tail); |
1472 | for (insn = head; insn != next_tail; insn = NEXT_INSN (insn)) | |
1473 | { | |
1474 | rtx prev; | |
1475 | ||
1476 | /* Farm out notes, and maybe save them in NOTE_LIST. | |
1477 | This is needed to keep the debugger from | |
1478 | getting completely deranged. */ | |
1479 | if (GET_CODE (insn) == NOTE) | |
1480 | { | |
1481 | prev = insn; | |
1482 | insn = unlink_line_notes (insn, next_tail); | |
1483 | ||
1484 | if (prev == tail) | |
1485 | abort (); | |
1486 | if (prev == head) | |
1487 | abort (); | |
1488 | if (insn == next_tail) | |
1489 | abort (); | |
1490 | } | |
1491 | } | |
1492 | } | |
1493 | ||
79c2ffde | 1494 | /* Save line number notes for each insn in block B. HEAD and TAIL are |
3ef42a0c | 1495 | the boundaries of the block in which notes should be processed. */ |
8c660648 | 1496 | |
b4ead7d4 | 1497 | void |
1d088dee | 1498 | save_line_notes (int b, rtx head, rtx tail) |
8c660648 | 1499 | { |
8c660648 JL |
1500 | rtx next_tail; |
1501 | ||
1502 | /* We must use the true line number for the first insn in the block | |
1503 | that was computed and saved at the start of this pass. We can't | |
1504 | use the current line number, because scheduling of the previous | |
1505 | block may have changed the current line number. */ | |
1506 | ||
b4ead7d4 | 1507 | rtx line = line_note_head[b]; |
8c660648 JL |
1508 | rtx insn; |
1509 | ||
8c660648 JL |
1510 | next_tail = NEXT_INSN (tail); |
1511 | ||
79c2ffde | 1512 | for (insn = head; insn != next_tail; insn = NEXT_INSN (insn)) |
8c660648 JL |
1513 | if (GET_CODE (insn) == NOTE && NOTE_LINE_NUMBER (insn) > 0) |
1514 | line = insn; | |
1515 | else | |
1516 | LINE_NOTE (insn) = line; | |
1517 | } | |
1518 | ||
14052b68 | 1519 | /* After a block was scheduled, insert line notes into the insns list. |
79c2ffde | 1520 | HEAD and TAIL are the boundaries of the block in which notes should |
3ef42a0c | 1521 | be processed. */ |
8c660648 | 1522 | |
b4ead7d4 | 1523 | void |
1d088dee | 1524 | restore_line_notes (rtx head, rtx tail) |
8c660648 JL |
1525 | { |
1526 | rtx line, note, prev, new; | |
1527 | int added_notes = 0; | |
79c2ffde | 1528 | rtx next_tail, insn; |
8c660648 | 1529 | |
79c2ffde BS |
1530 | head = head; |
1531 | next_tail = NEXT_INSN (tail); | |
8c660648 JL |
1532 | |
1533 | /* Determine the current line-number. We want to know the current | |
1534 | line number of the first insn of the block here, in case it is | |
1535 | different from the true line number that was saved earlier. If | |
1536 | different, then we need a line number note before the first insn | |
1537 | of this block. If it happens to be the same, then we don't want to | |
1538 | emit another line number note here. */ | |
1539 | for (line = head; line; line = PREV_INSN (line)) | |
1540 | if (GET_CODE (line) == NOTE && NOTE_LINE_NUMBER (line) > 0) | |
1541 | break; | |
1542 | ||
1543 | /* Walk the insns keeping track of the current line-number and inserting | |
1544 | the line-number notes as needed. */ | |
1545 | for (insn = head; insn != next_tail; insn = NEXT_INSN (insn)) | |
1546 | if (GET_CODE (insn) == NOTE && NOTE_LINE_NUMBER (insn) > 0) | |
1547 | line = insn; | |
1548 | /* This used to emit line number notes before every non-deleted note. | |
1549 | However, this confuses a debugger, because line notes not separated | |
1550 | by real instructions all end up at the same address. I can find no | |
1551 | use for line number notes before other notes, so none are emitted. */ | |
1552 | else if (GET_CODE (insn) != NOTE | |
79c2ffde | 1553 | && INSN_UID (insn) < old_max_uid |
8c660648 JL |
1554 | && (note = LINE_NOTE (insn)) != 0 |
1555 | && note != line | |
1556 | && (line == 0 | |
1557 | || NOTE_LINE_NUMBER (note) != NOTE_LINE_NUMBER (line) | |
1558 | || NOTE_SOURCE_FILE (note) != NOTE_SOURCE_FILE (line))) | |
1559 | { | |
1560 | line = note; | |
1561 | prev = PREV_INSN (insn); | |
1562 | if (LINE_NOTE (note)) | |
1563 | { | |
1564 | /* Re-use the original line-number note. */ | |
1565 | LINE_NOTE (note) = 0; | |
1566 | PREV_INSN (note) = prev; | |
1567 | NEXT_INSN (prev) = note; | |
1568 | PREV_INSN (insn) = note; | |
1569 | NEXT_INSN (note) = insn; | |
1570 | } | |
1571 | else | |
1572 | { | |
1573 | added_notes++; | |
1574 | new = emit_note_after (NOTE_LINE_NUMBER (note), prev); | |
1575 | NOTE_SOURCE_FILE (new) = NOTE_SOURCE_FILE (note); | |
1576 | RTX_INTEGRATED_P (new) = RTX_INTEGRATED_P (note); | |
1577 | } | |
1578 | } | |
1579 | if (sched_verbose && added_notes) | |
a88f02e7 | 1580 | fprintf (sched_dump, ";; added %d line-number notes\n", added_notes); |
8c660648 JL |
1581 | } |
1582 | ||
1583 | /* After scheduling the function, delete redundant line notes from the | |
1584 | insns list. */ | |
1585 | ||
b4ead7d4 | 1586 | void |
1d088dee | 1587 | rm_redundant_line_notes (void) |
8c660648 JL |
1588 | { |
1589 | rtx line = 0; | |
1590 | rtx insn = get_insns (); | |
1591 | int active_insn = 0; | |
1592 | int notes = 0; | |
1593 | ||
1594 | /* Walk the insns deleting redundant line-number notes. Many of these | |
1595 | are already present. The remainder tend to occur at basic | |
1596 | block boundaries. */ | |
1597 | for (insn = get_last_insn (); insn; insn = PREV_INSN (insn)) | |
1598 | if (GET_CODE (insn) == NOTE && NOTE_LINE_NUMBER (insn) > 0) | |
1599 | { | |
1600 | /* If there are no active insns following, INSN is redundant. */ | |
1601 | if (active_insn == 0) | |
1602 | { | |
1603 | notes++; | |
1604 | NOTE_SOURCE_FILE (insn) = 0; | |
1605 | NOTE_LINE_NUMBER (insn) = NOTE_INSN_DELETED; | |
1606 | } | |
1607 | /* If the line number is unchanged, LINE is redundant. */ | |
1608 | else if (line | |
1609 | && NOTE_LINE_NUMBER (line) == NOTE_LINE_NUMBER (insn) | |
1610 | && NOTE_SOURCE_FILE (line) == NOTE_SOURCE_FILE (insn)) | |
1611 | { | |
1612 | notes++; | |
1613 | NOTE_SOURCE_FILE (line) = 0; | |
1614 | NOTE_LINE_NUMBER (line) = NOTE_INSN_DELETED; | |
1615 | line = insn; | |
1616 | } | |
1617 | else | |
1618 | line = insn; | |
1619 | active_insn = 0; | |
1620 | } | |
1621 | else if (!((GET_CODE (insn) == NOTE | |
1622 | && NOTE_LINE_NUMBER (insn) == NOTE_INSN_DELETED) | |
1623 | || (GET_CODE (insn) == INSN | |
1624 | && (GET_CODE (PATTERN (insn)) == USE | |
1625 | || GET_CODE (PATTERN (insn)) == CLOBBER)))) | |
1626 | active_insn++; | |
1627 | ||
1628 | if (sched_verbose && notes) | |
a88f02e7 | 1629 | fprintf (sched_dump, ";; deleted %d line-number notes\n", notes); |
8c660648 JL |
1630 | } |
1631 | ||
79c2ffde | 1632 | /* Delete notes between HEAD and TAIL and put them in the chain |
8c660648 JL |
1633 | of notes ended by NOTE_LIST. */ |
1634 | ||
b4ead7d4 | 1635 | void |
1d088dee | 1636 | rm_other_notes (rtx head, rtx tail) |
8c660648 JL |
1637 | { |
1638 | rtx next_tail; | |
1639 | rtx insn; | |
1640 | ||
b4ead7d4 | 1641 | note_list = 0; |
2c3c49de | 1642 | if (head == tail && (! INSN_P (head))) |
8c660648 JL |
1643 | return; |
1644 | ||
1645 | next_tail = NEXT_INSN (tail); | |
1646 | for (insn = head; insn != next_tail; insn = NEXT_INSN (insn)) | |
1647 | { | |
1648 | rtx prev; | |
1649 | ||
1650 | /* Farm out notes, and maybe save them in NOTE_LIST. | |
1651 | This is needed to keep the debugger from | |
1652 | getting completely deranged. */ | |
1653 | if (GET_CODE (insn) == NOTE) | |
1654 | { | |
1655 | prev = insn; | |
1656 | ||
1657 | insn = unlink_other_notes (insn, next_tail); | |
1658 | ||
1659 | if (prev == tail) | |
1660 | abort (); | |
1661 | if (prev == head) | |
1662 | abort (); | |
1663 | if (insn == next_tail) | |
1664 | abort (); | |
1665 | } | |
1666 | } | |
1667 | } | |
1668 | ||
63de6c74 | 1669 | /* Functions for computation of registers live/usage info. */ |
8c660648 | 1670 | |
30028c85 VM |
1671 | /* This function looks for a new register being defined. |
1672 | If the destination register is already used by the source, | |
32dd366d | 1673 | a new register is not needed. */ |
30028c85 VM |
1674 | |
1675 | static int | |
1d088dee | 1676 | find_set_reg_weight (rtx x) |
30028c85 VM |
1677 | { |
1678 | if (GET_CODE (x) == CLOBBER | |
1679 | && register_operand (SET_DEST (x), VOIDmode)) | |
1680 | return 1; | |
1681 | if (GET_CODE (x) == SET | |
1682 | && register_operand (SET_DEST (x), VOIDmode)) | |
1683 | { | |
1684 | if (GET_CODE (SET_DEST (x)) == REG) | |
1685 | { | |
1686 | if (!reg_mentioned_p (SET_DEST (x), SET_SRC (x))) | |
1687 | return 1; | |
1688 | else | |
1689 | return 0; | |
1690 | } | |
1691 | return 1; | |
1692 | } | |
1693 | return 0; | |
1694 | } | |
1695 | ||
c46a37c4 | 1696 | /* Calculate INSN_REG_WEIGHT for all insns of a block. */ |
8c660648 JL |
1697 | |
1698 | static void | |
1d088dee | 1699 | find_insn_reg_weight (int b) |
8c660648 JL |
1700 | { |
1701 | rtx insn, next_tail, head, tail; | |
8c660648 | 1702 | |
49c3bb12 | 1703 | get_block_head_tail (b, &head, &tail); |
8c660648 JL |
1704 | next_tail = NEXT_INSN (tail); |
1705 | ||
1706 | for (insn = head; insn != next_tail; insn = NEXT_INSN (insn)) | |
1707 | { | |
8c660648 | 1708 | int reg_weight = 0; |
c46a37c4 | 1709 | rtx x; |
8c660648 JL |
1710 | |
1711 | /* Handle register life information. */ | |
2c3c49de | 1712 | if (! INSN_P (insn)) |
8c660648 JL |
1713 | continue; |
1714 | ||
c46a37c4 RH |
1715 | /* Increment weight for each register born here. */ |
1716 | x = PATTERN (insn); | |
30028c85 VM |
1717 | reg_weight += find_set_reg_weight (x); |
1718 | if (GET_CODE (x) == PARALLEL) | |
1d088dee AJ |
1719 | { |
1720 | int j; | |
1721 | for (j = XVECLEN (x, 0) - 1; j >= 0; j--) | |
1722 | { | |
1723 | x = XVECEXP (PATTERN (insn), 0, j); | |
30028c85 | 1724 | reg_weight += find_set_reg_weight (x); |
1d088dee AJ |
1725 | } |
1726 | } | |
c46a37c4 RH |
1727 | /* Decrement weight for each register that dies here. */ |
1728 | for (x = REG_NOTES (insn); x; x = XEXP (x, 1)) | |
8c660648 | 1729 | { |
c46a37c4 RH |
1730 | if (REG_NOTE_KIND (x) == REG_DEAD |
1731 | || REG_NOTE_KIND (x) == REG_UNUSED) | |
1732 | reg_weight--; | |
8c660648 JL |
1733 | } |
1734 | ||
c46a37c4 | 1735 | INSN_REG_WEIGHT (insn) = reg_weight; |
8c660648 | 1736 | } |
8c660648 JL |
1737 | } |
1738 | ||
63de6c74 | 1739 | /* Scheduling clock, modified in schedule_block() and queue_to_ready (). */ |
8c660648 JL |
1740 | static int clock_var; |
1741 | ||
1742 | /* Move insns that became ready to fire from queue to ready list. */ | |
1743 | ||
176f9a7b | 1744 | static void |
1d088dee | 1745 | queue_to_ready (struct ready_list *ready) |
8c660648 JL |
1746 | { |
1747 | rtx insn; | |
1748 | rtx link; | |
1749 | ||
1750 | q_ptr = NEXT_Q (q_ptr); | |
1751 | ||
1752 | /* Add all pending insns that can be scheduled without stalls to the | |
b4ead7d4 BS |
1753 | ready list. */ |
1754 | for (link = insn_queue[q_ptr]; link; link = XEXP (link, 1)) | |
1755 | { | |
1756 | insn = XEXP (link, 0); | |
1757 | q_size -= 1; | |
1708fd40 | 1758 | |
b4ead7d4 BS |
1759 | if (sched_verbose >= 2) |
1760 | fprintf (sched_dump, ";;\t\tQ-->Ready: insn %s: ", | |
1761 | (*current_sched_info->print_insn) (insn, 0)); | |
1708fd40 | 1762 | |
b4ead7d4 BS |
1763 | ready_add (ready, insn); |
1764 | if (sched_verbose >= 2) | |
1765 | fprintf (sched_dump, "moving to ready without stalls\n"); | |
1708fd40 | 1766 | } |
b4ead7d4 BS |
1767 | insn_queue[q_ptr] = 0; |
1768 | ||
1769 | /* If there are no ready insns, stall until one is ready and add all | |
1770 | of the pending insns at that point to the ready list. */ | |
1771 | if (ready->n_ready == 0) | |
1708fd40 | 1772 | { |
b3694847 | 1773 | int stalls; |
1708fd40 | 1774 | |
fae15c93 | 1775 | for (stalls = 1; stalls <= MAX_INSN_QUEUE_INDEX; stalls++) |
b4ead7d4 BS |
1776 | { |
1777 | if ((link = insn_queue[NEXT_Q_AFTER (q_ptr, stalls)])) | |
1778 | { | |
1779 | for (; link; link = XEXP (link, 1)) | |
1780 | { | |
1781 | insn = XEXP (link, 0); | |
1782 | q_size -= 1; | |
1708fd40 | 1783 | |
b4ead7d4 BS |
1784 | if (sched_verbose >= 2) |
1785 | fprintf (sched_dump, ";;\t\tQ-->Ready: insn %s: ", | |
1786 | (*current_sched_info->print_insn) (insn, 0)); | |
1708fd40 | 1787 | |
b4ead7d4 BS |
1788 | ready_add (ready, insn); |
1789 | if (sched_verbose >= 2) | |
1790 | fprintf (sched_dump, "moving to ready with %d stalls\n", stalls); | |
1791 | } | |
1792 | insn_queue[NEXT_Q_AFTER (q_ptr, stalls)] = 0; | |
1708fd40 | 1793 | |
fae15c93 VM |
1794 | advance_one_cycle (); |
1795 | ||
1796 | break; | |
b4ead7d4 | 1797 | } |
fae15c93 VM |
1798 | |
1799 | advance_one_cycle (); | |
b4ead7d4 | 1800 | } |
1708fd40 | 1801 | |
fae15c93 VM |
1802 | if ((!targetm.sched.use_dfa_pipeline_interface |
1803 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
1804 | && sched_verbose && stalls) | |
b4ead7d4 | 1805 | visualize_stall_cycles (stalls); |
fae15c93 | 1806 | |
b4ead7d4 BS |
1807 | q_ptr = NEXT_Q_AFTER (q_ptr, stalls); |
1808 | clock_var += stalls; | |
1708fd40 | 1809 | } |
1708fd40 BS |
1810 | } |
1811 | ||
b4ead7d4 | 1812 | /* Print the ready list for debugging purposes. Callable from debugger. */ |
1708fd40 | 1813 | |
b4ead7d4 | 1814 | static void |
1d088dee | 1815 | debug_ready_list (struct ready_list *ready) |
1708fd40 | 1816 | { |
b4ead7d4 BS |
1817 | rtx *p; |
1818 | int i; | |
1708fd40 | 1819 | |
b4ead7d4 | 1820 | if (ready->n_ready == 0) |
fae15c93 VM |
1821 | { |
1822 | fprintf (sched_dump, "\n"); | |
1823 | return; | |
1824 | } | |
1708fd40 | 1825 | |
b4ead7d4 BS |
1826 | p = ready_lastpos (ready); |
1827 | for (i = 0; i < ready->n_ready; i++) | |
1828 | fprintf (sched_dump, " %s", (*current_sched_info->print_insn) (p[i], 0)); | |
1829 | fprintf (sched_dump, "\n"); | |
1830 | } | |
1708fd40 | 1831 | |
63de6c74 | 1832 | /* move_insn1: Remove INSN from insn chain, and link it after LAST insn. */ |
8c660648 JL |
1833 | |
1834 | static rtx | |
1d088dee | 1835 | move_insn1 (rtx insn, rtx last) |
8c660648 JL |
1836 | { |
1837 | NEXT_INSN (PREV_INSN (insn)) = NEXT_INSN (insn); | |
1838 | PREV_INSN (NEXT_INSN (insn)) = PREV_INSN (insn); | |
1839 | ||
1840 | NEXT_INSN (insn) = NEXT_INSN (last); | |
1841 | PREV_INSN (NEXT_INSN (last)) = insn; | |
1842 | ||
1843 | NEXT_INSN (last) = insn; | |
1844 | PREV_INSN (insn) = last; | |
1845 | ||
1846 | return insn; | |
1847 | } | |
1848 | ||
570a98eb | 1849 | /* Search INSN for REG_SAVE_NOTE note pairs for |
8c660648 | 1850 | NOTE_INSN_{LOOP,EHREGION}_{BEG,END}; and convert them back into |
c46a37c4 RH |
1851 | NOTEs. The REG_SAVE_NOTE note following first one is contains the |
1852 | saved value for NOTE_BLOCK_NUMBER which is useful for | |
8c660648 JL |
1853 | NOTE_INSN_EH_REGION_{BEG,END} NOTEs. LAST is the last instruction |
1854 | output by the instruction scheduler. Return the new value of LAST. */ | |
1855 | ||
1856 | static rtx | |
1d088dee | 1857 | reemit_notes (rtx insn, rtx last) |
8c660648 JL |
1858 | { |
1859 | rtx note, retval; | |
1860 | ||
1861 | retval = last; | |
1862 | for (note = REG_NOTES (insn); note; note = XEXP (note, 1)) | |
1863 | { | |
c46a37c4 | 1864 | if (REG_NOTE_KIND (note) == REG_SAVE_NOTE) |
8c660648 | 1865 | { |
b3b42a4d RK |
1866 | enum insn_note note_type = INTVAL (XEXP (note, 0)); |
1867 | ||
63f4a88e JH |
1868 | last = emit_note_before (note_type, last); |
1869 | remove_note (insn, note); | |
1870 | note = XEXP (note, 1); | |
1871 | if (note_type == NOTE_INSN_EH_REGION_BEG | |
1872 | || note_type == NOTE_INSN_EH_REGION_END) | |
1873 | NOTE_EH_HANDLER (last) = INTVAL (XEXP (note, 0)); | |
8c660648 JL |
1874 | remove_note (insn, note); |
1875 | } | |
1876 | } | |
1877 | return retval; | |
1878 | } | |
1879 | ||
58fb7809 | 1880 | /* Move INSN. Reemit notes if needed. |
c9e03727 JL |
1881 | |
1882 | Return the last insn emitted by the scheduler, which is the | |
1883 | return value from the first call to reemit_notes. */ | |
8c660648 JL |
1884 | |
1885 | static rtx | |
1d088dee | 1886 | move_insn (rtx insn, rtx last) |
8c660648 | 1887 | { |
c9e03727 | 1888 | rtx retval = NULL; |
8c660648 | 1889 | |
8c660648 | 1890 | move_insn1 (insn, last); |
c9e03727 JL |
1891 | |
1892 | /* If this is the first call to reemit_notes, then record | |
1893 | its return value. */ | |
1894 | if (retval == NULL_RTX) | |
1895 | retval = reemit_notes (insn, insn); | |
1896 | else | |
1897 | reemit_notes (insn, insn); | |
1898 | ||
58fb7809 VM |
1899 | SCHED_GROUP_P (insn) = 0; |
1900 | ||
c9e03727 | 1901 | return retval; |
8c660648 JL |
1902 | } |
1903 | ||
30028c85 VM |
1904 | /* The following structure describe an entry of the stack of choices. */ |
1905 | struct choice_entry | |
1906 | { | |
1907 | /* Ordinal number of the issued insn in the ready queue. */ | |
1908 | int index; | |
1909 | /* The number of the rest insns whose issues we should try. */ | |
1910 | int rest; | |
1911 | /* The number of issued essential insns. */ | |
1912 | int n; | |
1913 | /* State after issuing the insn. */ | |
1914 | state_t state; | |
1915 | }; | |
1916 | ||
1917 | /* The following array is used to implement a stack of choices used in | |
1918 | function max_issue. */ | |
1919 | static struct choice_entry *choice_stack; | |
1920 | ||
1921 | /* The following variable value is number of essential insns issued on | |
1922 | the current cycle. An insn is essential one if it changes the | |
1923 | processors state. */ | |
1924 | static int cycle_issued_insns; | |
1925 | ||
880efc46 VM |
1926 | /* The following variable value is maximal number of tries of issuing |
1927 | insns for the first cycle multipass insn scheduling. We define | |
1928 | this value as constant*(DFA_LOOKAHEAD**ISSUE_RATE). We would not | |
1929 | need this constraint if all real insns (with non-negative codes) | |
1930 | had reservations because in this case the algorithm complexity is | |
1931 | O(DFA_LOOKAHEAD**ISSUE_RATE). Unfortunately, the dfa descriptions | |
1932 | might be incomplete and such insn might occur. For such | |
1933 | descriptions, the complexity of algorithm (without the constraint) | |
1934 | could achieve DFA_LOOKAHEAD ** N , where N is the queue length. */ | |
1935 | static int max_lookahead_tries; | |
1936 | ||
1937 | /* The following value is value of hook | |
1938 | `first_cycle_multipass_dfa_lookahead' at the last call of | |
1939 | `max_issue'. */ | |
1940 | static int cached_first_cycle_multipass_dfa_lookahead = 0; | |
1941 | ||
1942 | /* The following value is value of `issue_rate' at the last call of | |
1943 | `sched_init'. */ | |
1944 | static int cached_issue_rate = 0; | |
1945 | ||
fae15c93 VM |
1946 | /* The following function returns maximal (or close to maximal) number |
1947 | of insns which can be issued on the same cycle and one of which | |
30028c85 | 1948 | insns is insns with the best rank (the first insn in READY). To |
fae15c93 VM |
1949 | make this function tries different samples of ready insns. READY |
1950 | is current queue `ready'. Global array READY_TRY reflects what | |
30028c85 | 1951 | insns are already issued in this try. INDEX will contain index |
fae15c93 VM |
1952 | of the best insn in READY. The following function is used only for |
1953 | first cycle multipass scheduling. */ | |
fae15c93 | 1954 | static int |
1d088dee | 1955 | max_issue (struct ready_list *ready, int *index) |
fae15c93 | 1956 | { |
880efc46 | 1957 | int n, i, all, n_ready, best, delay, tries_num; |
30028c85 | 1958 | struct choice_entry *top; |
fae15c93 | 1959 | rtx insn; |
fae15c93 | 1960 | |
fae15c93 | 1961 | best = 0; |
30028c85 VM |
1962 | memcpy (choice_stack->state, curr_state, dfa_state_size); |
1963 | top = choice_stack; | |
880efc46 | 1964 | top->rest = cached_first_cycle_multipass_dfa_lookahead; |
30028c85 VM |
1965 | top->n = 0; |
1966 | n_ready = ready->n_ready; | |
1967 | for (all = i = 0; i < n_ready; i++) | |
fae15c93 | 1968 | if (!ready_try [i]) |
30028c85 VM |
1969 | all++; |
1970 | i = 0; | |
880efc46 | 1971 | tries_num = 0; |
30028c85 VM |
1972 | for (;;) |
1973 | { | |
1974 | if (top->rest == 0 || i >= n_ready) | |
1975 | { | |
1976 | if (top == choice_stack) | |
1977 | break; | |
1978 | if (best < top - choice_stack && ready_try [0]) | |
1979 | { | |
1980 | best = top - choice_stack; | |
1981 | *index = choice_stack [1].index; | |
1982 | if (top->n == issue_rate - cycle_issued_insns || best == all) | |
1983 | break; | |
1984 | } | |
1985 | i = top->index; | |
1986 | ready_try [i] = 0; | |
1987 | top--; | |
1988 | memcpy (curr_state, top->state, dfa_state_size); | |
1989 | } | |
1990 | else if (!ready_try [i]) | |
1991 | { | |
880efc46 VM |
1992 | tries_num++; |
1993 | if (tries_num > max_lookahead_tries) | |
1994 | break; | |
30028c85 VM |
1995 | insn = ready_element (ready, i); |
1996 | delay = state_transition (curr_state, insn); | |
1997 | if (delay < 0) | |
1998 | { | |
1999 | if (state_dead_lock_p (curr_state)) | |
2000 | top->rest = 0; | |
2001 | else | |
2002 | top->rest--; | |
2003 | n = top->n; | |
2004 | if (memcmp (top->state, curr_state, dfa_state_size) != 0) | |
2005 | n++; | |
2006 | top++; | |
880efc46 | 2007 | top->rest = cached_first_cycle_multipass_dfa_lookahead; |
30028c85 VM |
2008 | top->index = i; |
2009 | top->n = n; | |
2010 | memcpy (top->state, curr_state, dfa_state_size); | |
2011 | ready_try [i] = 1; | |
2012 | i = -1; | |
2013 | } | |
2014 | } | |
2015 | i++; | |
2016 | } | |
2017 | while (top != choice_stack) | |
2018 | { | |
2019 | ready_try [top->index] = 0; | |
2020 | top--; | |
2021 | } | |
2022 | memcpy (curr_state, choice_stack->state, dfa_state_size); | |
fae15c93 VM |
2023 | return best; |
2024 | } | |
2025 | ||
2026 | /* The following function chooses insn from READY and modifies | |
2027 | *N_READY and READY. The following function is used only for first | |
2028 | cycle multipass scheduling. */ | |
2029 | ||
2030 | static rtx | |
1d088dee | 2031 | choose_ready (struct ready_list *ready) |
fae15c93 | 2032 | { |
880efc46 VM |
2033 | int lookahead = 0; |
2034 | ||
2035 | if (targetm.sched.first_cycle_multipass_dfa_lookahead) | |
2036 | lookahead = (*targetm.sched.first_cycle_multipass_dfa_lookahead) (); | |
2037 | if (lookahead <= 0 || SCHED_GROUP_P (ready_element (ready, 0))) | |
fae15c93 VM |
2038 | return ready_remove_first (ready); |
2039 | else | |
2040 | { | |
2041 | /* Try to choose the better insn. */ | |
30028c85 VM |
2042 | int index, i; |
2043 | rtx insn; | |
fae15c93 | 2044 | |
880efc46 VM |
2045 | if (cached_first_cycle_multipass_dfa_lookahead != lookahead) |
2046 | { | |
2047 | cached_first_cycle_multipass_dfa_lookahead = lookahead; | |
2048 | max_lookahead_tries = 100; | |
2049 | for (i = 0; i < issue_rate; i++) | |
2050 | max_lookahead_tries *= lookahead; | |
2051 | } | |
30028c85 VM |
2052 | insn = ready_element (ready, 0); |
2053 | if (INSN_CODE (insn) < 0) | |
2054 | return ready_remove_first (ready); | |
2055 | for (i = 1; i < ready->n_ready; i++) | |
2056 | { | |
2057 | insn = ready_element (ready, i); | |
2058 | ready_try [i] | |
2059 | = (INSN_CODE (insn) < 0 | |
2060 | || (targetm.sched.first_cycle_multipass_dfa_lookahead_guard | |
2061 | && !(*targetm.sched.first_cycle_multipass_dfa_lookahead_guard) (insn))); | |
2062 | } | |
2063 | if (max_issue (ready, &index) == 0) | |
fae15c93 VM |
2064 | return ready_remove_first (ready); |
2065 | else | |
2066 | return ready_remove (ready, index); | |
2067 | } | |
2068 | } | |
2069 | ||
89076bb3 RH |
2070 | /* Called from backends from targetm.sched.reorder to emit stuff into |
2071 | the instruction stream. */ | |
2072 | ||
2073 | rtx | |
1d088dee | 2074 | sched_emit_insn (rtx pat) |
89076bb3 RH |
2075 | { |
2076 | rtx insn = emit_insn_after (pat, last_scheduled_insn); | |
2077 | last_scheduled_insn = insn; | |
2078 | return insn; | |
2079 | } | |
2080 | ||
b4ead7d4 | 2081 | /* Use forward list scheduling to rearrange insns of block B in region RGN, |
1708fd40 | 2082 | possibly bringing insns from subsequent blocks in the same region. */ |
8c660648 | 2083 | |
b4ead7d4 | 2084 | void |
1d088dee | 2085 | schedule_block (int b, int rgn_n_insns) |
8c660648 | 2086 | { |
176f9a7b | 2087 | struct ready_list ready; |
30028c85 | 2088 | int i, first_cycle_insn_p; |
8c660648 | 2089 | int can_issue_more; |
fae15c93 | 2090 | state_t temp_state = NULL; /* It is used for multipass scheduling. */ |
58fb7809 | 2091 | int sort_p, advance, start_clock_var; |
8c660648 | 2092 | |
63de6c74 | 2093 | /* Head/tail info for this block. */ |
1708fd40 BS |
2094 | rtx prev_head = current_sched_info->prev_head; |
2095 | rtx next_tail = current_sched_info->next_tail; | |
2096 | rtx head = NEXT_INSN (prev_head); | |
2097 | rtx tail = PREV_INSN (next_tail); | |
8c660648 | 2098 | |
484df988 JL |
2099 | /* We used to have code to avoid getting parameters moved from hard |
2100 | argument registers into pseudos. | |
8c660648 | 2101 | |
484df988 JL |
2102 | However, it was removed when it proved to be of marginal benefit |
2103 | and caused problems because schedule_block and compute_forward_dependences | |
2104 | had different notions of what the "head" insn was. */ | |
8c660648 | 2105 | |
2c3c49de | 2106 | if (head == tail && (! INSN_P (head))) |
1708fd40 | 2107 | abort (); |
8c660648 | 2108 | |
63de6c74 | 2109 | /* Debug info. */ |
8c660648 JL |
2110 | if (sched_verbose) |
2111 | { | |
a88f02e7 BS |
2112 | fprintf (sched_dump, ";; ======================================================\n"); |
2113 | fprintf (sched_dump, | |
8c660648 | 2114 | ";; -- basic block %d from %d to %d -- %s reload\n", |
79c2ffde | 2115 | b, INSN_UID (head), INSN_UID (tail), |
8c660648 | 2116 | (reload_completed ? "after" : "before")); |
a88f02e7 BS |
2117 | fprintf (sched_dump, ";; ======================================================\n"); |
2118 | fprintf (sched_dump, "\n"); | |
8c660648 | 2119 | |
c62c2659 | 2120 | visualize_alloc (); |
8c660648 JL |
2121 | init_block_visualization (); |
2122 | } | |
2123 | ||
fae15c93 VM |
2124 | if (targetm.sched.use_dfa_pipeline_interface |
2125 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
2126 | state_reset (curr_state); | |
2127 | else | |
2128 | clear_units (); | |
8c660648 | 2129 | |
63de6c74 | 2130 | /* Allocate the ready list. */ |
c237e94a | 2131 | ready.veclen = rgn_n_insns + 1 + issue_rate; |
176f9a7b | 2132 | ready.first = ready.veclen - 1; |
703ad42b | 2133 | ready.vec = xmalloc (ready.veclen * sizeof (rtx)); |
176f9a7b | 2134 | ready.n_ready = 0; |
8c660648 | 2135 | |
fae15c93 VM |
2136 | if (targetm.sched.use_dfa_pipeline_interface |
2137 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
2138 | { | |
2139 | /* It is used for first cycle multipass scheduling. */ | |
2140 | temp_state = alloca (dfa_state_size); | |
29da5c92 | 2141 | ready_try = xcalloc ((rgn_n_insns + 1), sizeof (char)); |
703ad42b KG |
2142 | choice_stack = xmalloc ((rgn_n_insns + 1) |
2143 | * sizeof (struct choice_entry)); | |
30028c85 | 2144 | for (i = 0; i <= rgn_n_insns; i++) |
703ad42b | 2145 | choice_stack[i].state = xmalloc (dfa_state_size); |
fae15c93 VM |
2146 | } |
2147 | ||
1708fd40 | 2148 | (*current_sched_info->init_ready_list) (&ready); |
8c660648 | 2149 | |
c237e94a ZW |
2150 | if (targetm.sched.md_init) |
2151 | (*targetm.sched.md_init) (sched_dump, sched_verbose, ready.veclen); | |
e4da5f6d | 2152 | |
89076bb3 RH |
2153 | /* We start inserting insns after PREV_HEAD. */ |
2154 | last_scheduled_insn = prev_head; | |
8c660648 | 2155 | |
1708fd40 BS |
2156 | /* Initialize INSN_QUEUE. Q_SIZE is the total number of insns in the |
2157 | queue. */ | |
8c660648 JL |
2158 | q_ptr = 0; |
2159 | q_size = 0; | |
fae15c93 VM |
2160 | |
2161 | if (!targetm.sched.use_dfa_pipeline_interface | |
2162 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
2163 | max_insn_queue_index_macro_value = INSN_QUEUE_SIZE - 1; | |
2164 | else | |
2165 | max_insn_queue_index_macro_value = max_insn_queue_index; | |
2166 | ||
703ad42b KG |
2167 | insn_queue = alloca ((MAX_INSN_QUEUE_INDEX + 1) * sizeof (rtx)); |
2168 | memset (insn_queue, 0, (MAX_INSN_QUEUE_INDEX + 1) * sizeof (rtx)); | |
fae15c93 | 2169 | last_clock_var = -1; |
8c660648 | 2170 | |
197043f5 RH |
2171 | /* Start just before the beginning of time. */ |
2172 | clock_var = -1; | |
58fb7809 | 2173 | advance = 0; |
197043f5 | 2174 | |
30028c85 | 2175 | sort_p = TRUE; |
63de6c74 | 2176 | /* Loop until all the insns in BB are scheduled. */ |
1708fd40 | 2177 | while ((*current_sched_info->schedule_more_p) ()) |
8c660648 | 2178 | { |
58fb7809 | 2179 | do |
8c660648 | 2180 | { |
58fb7809 VM |
2181 | start_clock_var = clock_var; |
2182 | ||
2183 | clock_var++; | |
1d088dee | 2184 | |
58fb7809 | 2185 | advance_one_cycle (); |
1d088dee | 2186 | |
58fb7809 VM |
2187 | /* Add to the ready list all pending insns that can be issued now. |
2188 | If there are no ready insns, increment clock until one | |
2189 | is ready and add all pending insns at that point to the ready | |
2190 | list. */ | |
2191 | queue_to_ready (&ready); | |
1d088dee | 2192 | |
58fb7809 VM |
2193 | if (ready.n_ready == 0) |
2194 | abort (); | |
1d088dee | 2195 | |
58fb7809 VM |
2196 | if (sched_verbose >= 2) |
2197 | { | |
2198 | fprintf (sched_dump, ";;\t\tReady list after queue_to_ready: "); | |
2199 | debug_ready_list (&ready); | |
2200 | } | |
2201 | advance -= clock_var - start_clock_var; | |
8c660648 | 2202 | } |
58fb7809 | 2203 | while (advance > 0); |
8c660648 | 2204 | |
30028c85 VM |
2205 | if (sort_p) |
2206 | { | |
2207 | /* Sort the ready list based on priority. */ | |
2208 | ready_sort (&ready); | |
1d088dee | 2209 | |
30028c85 VM |
2210 | if (sched_verbose >= 2) |
2211 | { | |
2212 | fprintf (sched_dump, ";;\t\tReady list after ready_sort: "); | |
2213 | debug_ready_list (&ready); | |
2214 | } | |
2215 | } | |
197043f5 | 2216 | |
b4ead7d4 BS |
2217 | /* Allow the target to reorder the list, typically for |
2218 | better instruction bundling. */ | |
14484a78 | 2219 | if (sort_p && targetm.sched.reorder |
58fb7809 VM |
2220 | && (ready.n_ready == 0 |
2221 | || !SCHED_GROUP_P (ready_element (&ready, 0)))) | |
c237e94a ZW |
2222 | can_issue_more = |
2223 | (*targetm.sched.reorder) (sched_dump, sched_verbose, | |
2224 | ready_lastpos (&ready), | |
2225 | &ready.n_ready, clock_var); | |
2226 | else | |
2227 | can_issue_more = issue_rate; | |
e1306f49 | 2228 | |
fae15c93 | 2229 | first_cycle_insn_p = 1; |
30028c85 | 2230 | cycle_issued_insns = 0; |
fae15c93 | 2231 | for (;;) |
e1306f49 | 2232 | { |
fae15c93 VM |
2233 | rtx insn; |
2234 | int cost; | |
2235 | ||
6d0de005 | 2236 | if (sched_verbose >= 2) |
fae15c93 VM |
2237 | { |
2238 | fprintf (sched_dump, ";;\tReady list (t =%3d): ", | |
2239 | clock_var); | |
2240 | debug_ready_list (&ready); | |
2241 | } | |
2242 | ||
2243 | if (!targetm.sched.use_dfa_pipeline_interface | |
2244 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
2245 | { | |
2246 | if (ready.n_ready == 0 || !can_issue_more | |
2247 | || !(*current_sched_info->schedule_more_p) ()) | |
2248 | break; | |
2249 | insn = choose_ready (&ready); | |
2250 | cost = actual_hazard (insn_unit (insn), insn, clock_var, 0); | |
2251 | } | |
2252 | else | |
2253 | { | |
2254 | if (ready.n_ready == 0 || !can_issue_more | |
2255 | || state_dead_lock_p (curr_state) | |
2256 | || !(*current_sched_info->schedule_more_p) ()) | |
2257 | break; | |
1d088dee | 2258 | |
fae15c93 | 2259 | /* Select and remove the insn from the ready list. */ |
30028c85 VM |
2260 | if (sort_p) |
2261 | insn = choose_ready (&ready); | |
2262 | else | |
2263 | insn = ready_remove_first (&ready); | |
1d088dee | 2264 | |
30028c85 VM |
2265 | if (targetm.sched.dfa_new_cycle |
2266 | && (*targetm.sched.dfa_new_cycle) (sched_dump, sched_verbose, | |
2267 | insn, last_clock_var, | |
2268 | clock_var, &sort_p)) | |
2269 | { | |
2270 | ready_add (&ready, insn); | |
2271 | break; | |
2272 | } | |
1d088dee | 2273 | |
30028c85 | 2274 | sort_p = TRUE; |
fae15c93 VM |
2275 | memcpy (temp_state, curr_state, dfa_state_size); |
2276 | if (recog_memoized (insn) < 0) | |
2277 | { | |
2278 | if (!first_cycle_insn_p | |
2279 | && (GET_CODE (PATTERN (insn)) == ASM_INPUT | |
2280 | || asm_noperands (PATTERN (insn)) >= 0)) | |
2281 | /* This is asm insn which is tryed to be issued on the | |
2282 | cycle not first. Issue it on the next cycle. */ | |
2283 | cost = 1; | |
2284 | else | |
2285 | /* A USE insn, or something else we don't need to | |
2286 | understand. We can't pass these directly to | |
2287 | state_transition because it will trigger a | |
2288 | fatal error for unrecognizable insns. */ | |
2289 | cost = 0; | |
2290 | } | |
2291 | else | |
2292 | { | |
2293 | cost = state_transition (temp_state, insn); | |
2294 | ||
2295 | if (targetm.sched.first_cycle_multipass_dfa_lookahead | |
2296 | && targetm.sched.dfa_bubble) | |
2297 | { | |
2298 | if (cost == 0) | |
2299 | { | |
2300 | int j; | |
2301 | rtx bubble; | |
1d088dee | 2302 | |
fae15c93 VM |
2303 | for (j = 0; |
2304 | (bubble = (*targetm.sched.dfa_bubble) (j)) | |
2305 | != NULL_RTX; | |
2306 | j++) | |
2307 | { | |
2308 | memcpy (temp_state, curr_state, dfa_state_size); | |
1d088dee | 2309 | |
fae15c93 VM |
2310 | if (state_transition (temp_state, bubble) < 0 |
2311 | && state_transition (temp_state, insn) < 0) | |
2312 | break; | |
2313 | } | |
1d088dee | 2314 | |
fae15c93 VM |
2315 | if (bubble != NULL_RTX) |
2316 | { | |
2317 | if (insert_schedule_bubbles_p) | |
2318 | { | |
2319 | rtx copy; | |
1d088dee | 2320 | |
fae15c93 VM |
2321 | copy = copy_rtx (PATTERN (bubble)); |
2322 | emit_insn_after (copy, last_scheduled_insn); | |
2323 | last_scheduled_insn | |
2324 | = NEXT_INSN (last_scheduled_insn); | |
2325 | INSN_CODE (last_scheduled_insn) | |
2326 | = INSN_CODE (bubble); | |
1d088dee | 2327 | |
fae15c93 VM |
2328 | /* Annotate the same for the first insns |
2329 | scheduling by using mode. */ | |
2330 | PUT_MODE (last_scheduled_insn, | |
2331 | (clock_var > last_clock_var | |
2332 | ? clock_var - last_clock_var | |
2333 | : VOIDmode)); | |
2334 | last_clock_var = clock_var; | |
1d088dee | 2335 | |
fae15c93 VM |
2336 | if (sched_verbose >= 2) |
2337 | { | |
2338 | fprintf (sched_dump, | |
2339 | ";;\t\t--> scheduling bubble insn <<<%d>>>:reservation ", | |
2340 | INSN_UID (last_scheduled_insn)); | |
1d088dee | 2341 | |
fae15c93 VM |
2342 | if (recog_memoized (last_scheduled_insn) |
2343 | < 0) | |
2344 | fprintf (sched_dump, "nothing"); | |
2345 | else | |
2346 | print_reservation | |
2347 | (sched_dump, last_scheduled_insn); | |
1d088dee | 2348 | |
fae15c93 VM |
2349 | fprintf (sched_dump, "\n"); |
2350 | } | |
2351 | } | |
2352 | cost = -1; | |
2353 | } | |
2354 | } | |
2355 | } | |
2356 | ||
2357 | if (cost < 0) | |
2358 | cost = 0; | |
2359 | else if (cost == 0) | |
2360 | cost = 1; | |
2361 | } | |
2362 | } | |
e1306f49 | 2363 | |
e1306f49 | 2364 | |
b4ead7d4 BS |
2365 | if (cost >= 1) |
2366 | { | |
2367 | queue_insn (insn, cost); | |
2368 | continue; | |
2369 | } | |
e1306f49 | 2370 | |
b4ead7d4 BS |
2371 | if (! (*current_sched_info->can_schedule_ready_p) (insn)) |
2372 | goto next; | |
8c660648 | 2373 | |
89076bb3 | 2374 | last_scheduled_insn = move_insn (insn, last_scheduled_insn); |
8c660648 | 2375 | |
fae15c93 VM |
2376 | if (targetm.sched.use_dfa_pipeline_interface |
2377 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
30028c85 VM |
2378 | { |
2379 | if (memcmp (curr_state, temp_state, dfa_state_size) != 0) | |
2380 | cycle_issued_insns++; | |
2381 | memcpy (curr_state, temp_state, dfa_state_size); | |
2382 | } | |
1d088dee | 2383 | |
c237e94a ZW |
2384 | if (targetm.sched.variable_issue) |
2385 | can_issue_more = | |
2386 | (*targetm.sched.variable_issue) (sched_dump, sched_verbose, | |
2387 | insn, can_issue_more); | |
85d69216 JL |
2388 | /* A naked CLOBBER or USE generates no instruction, so do |
2389 | not count them against the issue rate. */ | |
2390 | else if (GET_CODE (PATTERN (insn)) != USE | |
2391 | && GET_CODE (PATTERN (insn)) != CLOBBER) | |
c237e94a | 2392 | can_issue_more--; |
8c660648 | 2393 | |
58fb7809 VM |
2394 | advance = schedule_insn (insn, &ready, clock_var); |
2395 | if (advance != 0) | |
2396 | break; | |
8c660648 | 2397 | |
b4ead7d4 | 2398 | next: |
fae15c93 VM |
2399 | first_cycle_insn_p = 0; |
2400 | ||
30028c85 VM |
2401 | /* Sort the ready list based on priority. This must be |
2402 | redone here, as schedule_insn may have readied additional | |
32dd366d | 2403 | insns that will not be sorted correctly. */ |
30028c85 VM |
2404 | if (ready.n_ready > 0) |
2405 | ready_sort (&ready); | |
2406 | ||
58fb7809 VM |
2407 | if (targetm.sched.reorder2 |
2408 | && (ready.n_ready == 0 | |
2409 | || !SCHED_GROUP_P (ready_element (&ready, 0)))) | |
c237e94a | 2410 | { |
c237e94a | 2411 | can_issue_more = |
30028c85 | 2412 | (*targetm.sched.reorder2) (sched_dump, sched_verbose, |
c237e94a ZW |
2413 | ready.n_ready |
2414 | ? ready_lastpos (&ready) : NULL, | |
2415 | &ready.n_ready, clock_var); | |
2416 | } | |
b4ead7d4 | 2417 | } |
8c660648 | 2418 | |
fae15c93 VM |
2419 | if ((!targetm.sched.use_dfa_pipeline_interface |
2420 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
2421 | && sched_verbose) | |
2422 | /* Debug info. */ | |
b4ead7d4 BS |
2423 | visualize_scheduled_insns (clock_var); |
2424 | } | |
8c660648 | 2425 | |
c237e94a ZW |
2426 | if (targetm.sched.md_finish) |
2427 | (*targetm.sched.md_finish) (sched_dump, sched_verbose); | |
79c2ffde | 2428 | |
b4ead7d4 BS |
2429 | /* Debug info. */ |
2430 | if (sched_verbose) | |
2431 | { | |
2432 | fprintf (sched_dump, ";;\tReady list (final): "); | |
2433 | debug_ready_list (&ready); | |
fae15c93 VM |
2434 | if (!targetm.sched.use_dfa_pipeline_interface |
2435 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
2436 | print_block_visualization (""); | |
b4ead7d4 | 2437 | } |
8c660648 | 2438 | |
b4ead7d4 BS |
2439 | /* Sanity check -- queue must be empty now. Meaningless if region has |
2440 | multiple bbs. */ | |
2441 | if (current_sched_info->queue_must_finish_empty && q_size != 0) | |
2442 | abort (); | |
ebb7b10b | 2443 | |
b4ead7d4 BS |
2444 | /* Update head/tail boundaries. */ |
2445 | head = NEXT_INSN (prev_head); | |
89076bb3 | 2446 | tail = last_scheduled_insn; |
ebb7b10b | 2447 | |
30028c85 VM |
2448 | if (!reload_completed) |
2449 | { | |
2450 | rtx insn, link, next; | |
1d088dee | 2451 | |
30028c85 VM |
2452 | /* INSN_TICK (minimum clock tick at which the insn becomes |
2453 | ready) may be not correct for the insn in the subsequent | |
2454 | blocks of the region. We should use a correct value of | |
2455 | `clock_var' or modify INSN_TICK. It is better to keep | |
2456 | clock_var value equal to 0 at the start of a basic block. | |
2457 | Therefore we modify INSN_TICK here. */ | |
2458 | for (insn = head; insn != tail; insn = NEXT_INSN (insn)) | |
2459 | if (INSN_P (insn)) | |
2460 | { | |
2461 | for (link = INSN_DEPEND (insn); link != 0; link = XEXP (link, 1)) | |
2462 | { | |
2463 | next = XEXP (link, 0); | |
2464 | INSN_TICK (next) -= clock_var; | |
2465 | } | |
2466 | } | |
2467 | } | |
2468 | ||
d73b1f07 RH |
2469 | /* Restore-other-notes: NOTE_LIST is the end of a chain of notes |
2470 | previously found among the insns. Insert them at the beginning | |
2471 | of the insns. */ | |
2472 | if (note_list != 0) | |
2473 | { | |
2474 | rtx note_head = note_list; | |
2475 | ||
2476 | while (PREV_INSN (note_head)) | |
2477 | { | |
2478 | note_head = PREV_INSN (note_head); | |
2479 | } | |
2480 | ||
2481 | PREV_INSN (note_head) = PREV_INSN (head); | |
2482 | NEXT_INSN (PREV_INSN (head)) = note_head; | |
2483 | PREV_INSN (head) = note_list; | |
2484 | NEXT_INSN (note_list) = head; | |
2485 | head = note_head; | |
2486 | } | |
8c660648 | 2487 | |
b4ead7d4 BS |
2488 | /* Debugging. */ |
2489 | if (sched_verbose) | |
8c660648 | 2490 | { |
b4ead7d4 BS |
2491 | fprintf (sched_dump, ";; total time = %d\n;; new head = %d\n", |
2492 | clock_var, INSN_UID (head)); | |
2493 | fprintf (sched_dump, ";; new tail = %d\n\n", | |
2494 | INSN_UID (tail)); | |
2495 | visualize_free (); | |
2496 | } | |
8c660648 | 2497 | |
b4ead7d4 BS |
2498 | current_sched_info->head = head; |
2499 | current_sched_info->tail = tail; | |
8c660648 | 2500 | |
b4ead7d4 | 2501 | free (ready.vec); |
fae15c93 VM |
2502 | |
2503 | if (targetm.sched.use_dfa_pipeline_interface | |
2504 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
30028c85 VM |
2505 | { |
2506 | free (ready_try); | |
2507 | for (i = 0; i <= rgn_n_insns; i++) | |
2508 | free (choice_stack [i].state); | |
2509 | free (choice_stack); | |
2510 | } | |
8c660648 | 2511 | } |
b4ead7d4 | 2512 | \f |
63de6c74 | 2513 | /* Set_priorities: compute priority of each insn in the block. */ |
8c660648 | 2514 | |
b4ead7d4 | 2515 | int |
1d088dee | 2516 | set_priorities (rtx head, rtx tail) |
8c660648 JL |
2517 | { |
2518 | rtx insn; | |
2519 | int n_insn; | |
2520 | ||
8c660648 | 2521 | rtx prev_head; |
8c660648 | 2522 | |
8c660648 JL |
2523 | prev_head = PREV_INSN (head); |
2524 | ||
2c3c49de | 2525 | if (head == tail && (! INSN_P (head))) |
8c660648 JL |
2526 | return 0; |
2527 | ||
2528 | n_insn = 0; | |
2529 | for (insn = tail; insn != prev_head; insn = PREV_INSN (insn)) | |
2530 | { | |
8c660648 JL |
2531 | if (GET_CODE (insn) == NOTE) |
2532 | continue; | |
2533 | ||
58fb7809 | 2534 | n_insn++; |
8c660648 JL |
2535 | (void) priority (insn); |
2536 | } | |
2537 | ||
2538 | return n_insn; | |
2539 | } | |
2540 | ||
a88f02e7 BS |
2541 | /* Initialize some global state for the scheduler. DUMP_FILE is to be used |
2542 | for debugging output. */ | |
8c660648 | 2543 | |
b4ead7d4 | 2544 | void |
1d088dee | 2545 | sched_init (FILE *dump_file) |
8c660648 | 2546 | { |
e0082a72 ZD |
2547 | int luid; |
2548 | basic_block b; | |
8c660648 | 2549 | rtx insn; |
fae15c93 | 2550 | int i; |
8c660648 | 2551 | |
63de6c74 | 2552 | /* Disable speculative loads in their presence if cc0 defined. */ |
8c660648 JL |
2553 | #ifdef HAVE_cc0 |
2554 | flag_schedule_speculative_load = 0; | |
2555 | #endif | |
2556 | ||
63de6c74 | 2557 | /* Set dump and sched_verbose for the desired debugging output. If no |
409f8483 DE |
2558 | dump-file was specified, but -fsched-verbose=N (any N), print to stderr. |
2559 | For -fsched-verbose=N, N>=10, print everything to stderr. */ | |
8c660648 JL |
2560 | sched_verbose = sched_verbose_param; |
2561 | if (sched_verbose_param == 0 && dump_file) | |
2562 | sched_verbose = 1; | |
a88f02e7 BS |
2563 | sched_dump = ((sched_verbose_param >= 10 || !dump_file) |
2564 | ? stderr : dump_file); | |
8c660648 | 2565 | |
63de6c74 | 2566 | /* Initialize issue_rate. */ |
c237e94a ZW |
2567 | if (targetm.sched.issue_rate) |
2568 | issue_rate = (*targetm.sched.issue_rate) (); | |
2569 | else | |
2570 | issue_rate = 1; | |
8c660648 | 2571 | |
880efc46 VM |
2572 | if (cached_issue_rate != issue_rate) |
2573 | { | |
2574 | cached_issue_rate = issue_rate; | |
2575 | /* To invalidate max_lookahead_tries: */ | |
2576 | cached_first_cycle_multipass_dfa_lookahead = 0; | |
2577 | } | |
2578 | ||
c88e8206 RH |
2579 | /* We use LUID 0 for the fake insn (UID 0) which holds dependencies for |
2580 | pseudos which do not cross calls. */ | |
a88f02e7 | 2581 | old_max_uid = get_max_uid () + 1; |
8c660648 | 2582 | |
703ad42b | 2583 | h_i_d = xcalloc (old_max_uid, sizeof (*h_i_d)); |
8c660648 | 2584 | |
fae15c93 VM |
2585 | for (i = 0; i < old_max_uid; i++) |
2586 | h_i_d [i].cost = -1; | |
2587 | ||
2588 | if (targetm.sched.use_dfa_pipeline_interface | |
2589 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
2590 | { | |
2591 | if (targetm.sched.init_dfa_pre_cycle_insn) | |
2592 | (*targetm.sched.init_dfa_pre_cycle_insn) (); | |
1d088dee | 2593 | |
fae15c93 VM |
2594 | if (targetm.sched.init_dfa_post_cycle_insn) |
2595 | (*targetm.sched.init_dfa_post_cycle_insn) (); | |
1d088dee | 2596 | |
fae15c93 VM |
2597 | if (targetm.sched.first_cycle_multipass_dfa_lookahead |
2598 | && targetm.sched.init_dfa_bubbles) | |
2599 | (*targetm.sched.init_dfa_bubbles) (); | |
1d088dee | 2600 | |
fae15c93 VM |
2601 | dfa_start (); |
2602 | dfa_state_size = state_size (); | |
2603 | curr_state = xmalloc (dfa_state_size); | |
2604 | } | |
2605 | ||
f66d83e1 | 2606 | h_i_d[0].luid = 0; |
356edbd7 | 2607 | luid = 1; |
e0082a72 ZD |
2608 | FOR_EACH_BB (b) |
2609 | for (insn = b->head;; insn = NEXT_INSN (insn)) | |
8c660648 | 2610 | { |
f77e39fc MM |
2611 | INSN_LUID (insn) = luid; |
2612 | ||
2613 | /* Increment the next luid, unless this is a note. We don't | |
2614 | really need separate IDs for notes and we don't want to | |
2615 | schedule differently depending on whether or not there are | |
2616 | line-number notes, i.e., depending on whether or not we're | |
2617 | generating debugging information. */ | |
2618 | if (GET_CODE (insn) != NOTE) | |
2619 | ++luid; | |
2620 | ||
e0082a72 | 2621 | if (insn == b->end) |
8c660648 JL |
2622 | break; |
2623 | } | |
7a403706 | 2624 | |
a88f02e7 BS |
2625 | init_dependency_caches (luid); |
2626 | ||
a88f02e7 BS |
2627 | init_alias_analysis (); |
2628 | ||
2629 | if (write_symbols != NO_DEBUG) | |
aae0390e | 2630 | { |
a88f02e7 BS |
2631 | rtx line; |
2632 | ||
703ad42b | 2633 | line_note_head = xcalloc (last_basic_block, sizeof (rtx)); |
a88f02e7 BS |
2634 | |
2635 | /* Save-line-note-head: | |
2636 | Determine the line-number at the start of each basic block. | |
2637 | This must be computed and saved now, because after a basic block's | |
2638 | predecessor has been scheduled, it is impossible to accurately | |
2639 | determine the correct line number for the first insn of the block. */ | |
2640 | ||
e0082a72 | 2641 | FOR_EACH_BB (b) |
79c2ffde | 2642 | { |
e0082a72 | 2643 | for (line = b->head; line; line = PREV_INSN (line)) |
79c2ffde BS |
2644 | if (GET_CODE (line) == NOTE && NOTE_LINE_NUMBER (line) > 0) |
2645 | { | |
e0082a72 | 2646 | line_note_head[b->index] = line; |
79c2ffde BS |
2647 | break; |
2648 | } | |
2649 | /* Do a forward search as well, since we won't get to see the first | |
2650 | notes in a basic block. */ | |
e0082a72 | 2651 | for (line = b->head; line; line = NEXT_INSN (line)) |
a88f02e7 | 2652 | { |
79c2ffde BS |
2653 | if (INSN_P (line)) |
2654 | break; | |
2655 | if (GET_CODE (line) == NOTE && NOTE_LINE_NUMBER (line) > 0) | |
e0082a72 | 2656 | line_note_head[b->index] = line; |
a88f02e7 | 2657 | } |
79c2ffde | 2658 | } |
aae0390e | 2659 | } |
8c660648 | 2660 | |
fae15c93 VM |
2661 | if ((!targetm.sched.use_dfa_pipeline_interface |
2662 | || !(*targetm.sched.use_dfa_pipeline_interface) ()) | |
2663 | && sched_verbose) | |
2664 | /* Find units used in this function, for visualization. */ | |
a88f02e7 BS |
2665 | init_target_units (); |
2666 | ||
2667 | /* ??? Add a NOTE after the last insn of the last basic block. It is not | |
2668 | known why this is done. */ | |
2669 | ||
e0082a72 | 2670 | insn = EXIT_BLOCK_PTR->prev_bb->end; |
a88f02e7 BS |
2671 | if (NEXT_INSN (insn) == 0 |
2672 | || (GET_CODE (insn) != NOTE | |
2673 | && GET_CODE (insn) != CODE_LABEL | |
4cf37b4a R |
2674 | /* Don't emit a NOTE if it would end up before a BARRIER. */ |
2675 | && GET_CODE (NEXT_INSN (insn)) != BARRIER)) | |
3c030e88 | 2676 | { |
e0082a72 | 2677 | emit_note_after (NOTE_INSN_DELETED, EXIT_BLOCK_PTR->prev_bb->end); |
3c030e88 | 2678 | /* Make insn to appear outside BB. */ |
e0082a72 | 2679 | EXIT_BLOCK_PTR->prev_bb->end = PREV_INSN (EXIT_BLOCK_PTR->prev_bb->end); |
3c030e88 | 2680 | } |
a88f02e7 BS |
2681 | |
2682 | /* Compute INSN_REG_WEIGHT for all blocks. We must do this before | |
2683 | removing death notes. */ | |
e0082a72 ZD |
2684 | FOR_EACH_BB_REVERSE (b) |
2685 | find_insn_reg_weight (b->index); | |
a88f02e7 BS |
2686 | } |
2687 | ||
b4ead7d4 | 2688 | /* Free global data used during insn scheduling. */ |
8c660648 | 2689 | |
a88f02e7 | 2690 | void |
1d088dee | 2691 | sched_finish (void) |
a88f02e7 | 2692 | { |
f66d83e1 | 2693 | free (h_i_d); |
fae15c93 VM |
2694 | |
2695 | if (targetm.sched.use_dfa_pipeline_interface | |
2696 | && (*targetm.sched.use_dfa_pipeline_interface) ()) | |
2697 | { | |
2698 | free (curr_state); | |
2699 | dfa_finish (); | |
2700 | } | |
b4ead7d4 BS |
2701 | free_dependency_caches (); |
2702 | end_alias_analysis (); | |
7c74b010 | 2703 | if (write_symbols != NO_DEBUG) |
f66d83e1 | 2704 | free (line_note_head); |
8c660648 JL |
2705 | } |
2706 | #endif /* INSN_SCHEDULING */ |