Convert profile probabilities to new type
Christophe Lyon
christophe.lyon@linaro.org
Thu Jun 29 18:28:00 GMT 2017
Hi,
On 29 June 2017 at 14:27, Jan Hubicka <hubicka@ucw.cz> wrote:
> Hi,
> this is second step of the profile maintenance revamp. It implements
> profile_probability type which is pretty much symmetric to profile_count
> except that it implements fixed point arithmetics for values 0...1.
> It is used to maintain probabilities of edges out of basic blocks.
> In addition it tracks information about quality of the estimate which can
> later be used for optimization.
>
> Large patch is symmetric just updating existing code to new API. Because
> probabilities are not just integers, some changes are needed. In particular
> 1) I have introduced names for common probabilities, which corresponds to
> PROB_* macros. A lot of code however used explicit calcuation of
> REG_BR_PROB_BASE.
> 2) There is now difference between probability of 0 and uninitialized
> probability. There are a lot of places where we forget to set edge
> probabilities after updating CFG which originally went unnoticed.
> Now we can see ICE when such missing probabilities are intorudced between
> profile estimation and inlning. Rest of code now accepts missing
> probabilities and I will implement checker and fix bugs incremetnally.
>
> Before inlining however I need probabilities to exist because inliner
> redoes probability->frequency computation and for that one can not have
> missing info.
>
> This noticed few erorrs which I knew how to fix, and some I added fixmes
> for:
>
> omp-simd-clone.c:
> - in simd_clone_adjust for edge incr_bb->new_exit_bb
> tree-parloops.c:
> - in create_parallel_loop for guard edge
> - in oacc_entry_exit_single_gang for e3 edge.
> tree-ssa-loop-split.c
> - loop profile updating seems just completely wrong here.
>
> I the first two files I would welcome some suggestions what to do.
>
> To make conversion possible, sometimes I still go to old representation of
> counts via to_reg_br_prob_base/from_reg_br_prob_base. I plan to eliminate most
> of uses of this incrementaly (there are 60 of them).
>
> Bootstrapped/regtested x86_64-linux, will commit it later today after bit
> more testing.
>
After this commit (r249800), GCC builds fail for arm and aarch64:
/gccsrc/gcc/except.c: In function ‘void
sjlj_emit_function_enter(rtx_code_label*)’:
/gcc-fsf/gccsrc/gcc/except.c:1183: error: conversion from ‘int’ to
non-scalar type ‘profile_probability’ requested
make[2]: *** [except.o] Error 1
> Honza
>
> 2017-06-28 Jan Hubicka <hubicka@ucw.cz>
>
> * asan.c (asan_emit_stack_protection): Update.
> (create_cond_insert_point): Update.
> * auto-profile.c (afdo_propagate_circuit): Update.
> * basic-block.h (struct edge_def): Turn probability to
> profile_probability.
> (EDGE_FREQUENCY): Update.
> * bb-reorder.c (find_traces_1_round): Update.
> (better_edge_p): Update.
> (sanitize_hot_paths): Update.
> * cfg.c (unchecked_make_edge): Initialize probability to uninitialized.
> (make_single_succ_edge): Update.
> (check_bb_profile): Update.
> (dump_edge_info): Update.
> (update_bb_profile_for_threading): Update.
> * cfganal.c (connect_infinite_loops_to_exit): Initialize new edge
> probabilitycount to 0.
> * cfgbuild.c (compute_outgoing_frequencies): Update.
> * cfgcleanup.c (try_forward_edges): Update.
> (outgoing_edges_match): Update.
> (try_crossjump_to_edge): Update.
> * cfgexpand.c (expand_gimple_cond): Update make_single_succ_edge.
> (expand_gimple_tailcall): Update.
> (construct_init_block): Use make_single_succ_edge.
> (construct_exit_block): Use make_single_succ_edge.
> * cfghooks.c (verify_flow_info): Update.
> (redirect_edge_succ_nodup): Update.
> (split_edge): Update.
> (account_profile_record): Update.
> * cfgloopanal.c (single_likely_exit): Update.
> * cfgloopmanip.c (scale_loop_profile): Update.
> (set_zero_probability): Remove.
> (duplicate_loop_to_header_edge): Update.
> * cfgloopmanip.h (loop_version): Update prototype.
> * cfgrtl.c (try_redirect_by_replacing_jump): Update.
> (force_nonfallthru_and_redirect): Update.
> (update_br_prob_note): Update.
> (rtl_verify_edges): Update.
> (purge_dead_edges): Update.
> (rtl_lv_add_condition_to_bb): Update.
> * cgraph.c: (cgraph_edge::redirect_call_stmt_to_calle): Update.
> * cgraphunit.c (init_lowered_empty_function): Update.
> (cgraph_node::expand_thunk): Update.
> * cilk-common.c: Include profile-count.h
> * dojump.c (inv): Remove.
> (jumpifnot): Update.
> (jumpifnot_1): Update.
> (do_jump_1): Update.
> (do_jump): Update.
> (do_jump_by_parts_greater_rtx): Update.
> (do_compare_rtx_and_jump): Update.
> * dojump.h (jumpifnot, jumpifnot_1, jumpif_1, jumpif, do_jump,
> do_jump_1. do_compare_rtx_and_jump): Update prototype.
> * dwarf2cfi.c: Include profile-count.h
> * except.c (dw2_build_landing_pads): Use make_single_succ_edge.
> (sjlj_emit_dispatch_table): Likewise.
> * explow.c: Include profile-count.h
> * expmed.c (emit_store_flag_force): Update.
> (do_cmp_and_jump): Update.
> * expr.c (compare_by_pieces_d::generate): Update.
> (compare_by_pieces_d::finish_mode): Update.
> (emit_block_move_via_loop): Update.
> (store_expr_with_bounds): Update.
> (store_constructor): Update.
> (expand_expr_real_2): Update.
> (expand_expr_real_1): Update.
> * expr.h (try_casesi, try_tablejump): Update prototypes.
> * gimple-pretty-print.c (dump_probability): Update.
> (dump_profile): New.
> (dump_gimple_label): Update.
> (dump_gimple_bb_header): Update.
> * graph.c (draw_cfg_node_succ_edges): Update.
> * hsa-gen.c (convert_switch_statements): Update.
> * ifcvt.c (cheap_bb_rtx_cost_p): Update.
> (find_if_case_1): Update.
> (find_if_case_2): Update.
> * internal-fn.c (expand_arith_overflow_result_store): Update.
> (expand_addsub_overflow): Update.
> (expand_neg_overflow): Update.
> (expand_mul_overflow): Update.
> (expand_vector_ubsan_overflow): Update.
> * ipa-cp.c (good_cloning_opportunity_p): Update.
> * ipa-split.c (split_function): Use make_single_succ_edge.
> * ipa-utils.c (ipa_merge_profiles): Update.
> * loop-doloop.c (add_test): Update.
> (doloop_modify): Update.
> * loop-unroll.c (compare_and_jump_seq): Update.
> (unroll_loop_runtime_iterations): Update.
> * lra-constraints.c (lra_inheritance): Update.
> * lto-streamer-in.c (input_cfg): Update.
> * lto-streamer-out.c (output_cfg): Update.
> * mcf.c (adjust_cfg_counts): Update.
> * modulo-sched.c (sms_schedule): Update.
> * omp-expand.c (expand_omp_for_init_counts): Update.
> (extract_omp_for_update_vars): Update.
> (expand_omp_ordered_sink): Update.
> (expand_omp_for_ordered_loops): Update.
> (expand_omp_for_generic): Update.
> (expand_omp_for_static_nochunk): Update.
> (expand_omp_for_static_chunk): Update.
> (expand_cilk_for): Update.
> (expand_omp_simd): Update.
> (expand_omp_taskloop_for_outer): Update.
> (expand_omp_taskloop_for_inner): Update.
> * omp-simd-clone.c (simd_clone_adjust): Update.
> * optabs.c (expand_doubleword_shift): Update.
> (expand_abs): Update.
> (emit_cmp_and_jump_insn_1): Update.
> (expand_compare_and_swap_loop): Update.
> * optabs.h (emit_cmp_and_jump_insns): Update prototype.
> * predict.c (predictable_edge_p): Update.
> (edge_probability_reliable_p): Update.
> (set_even_probabilities): Update.
> (combine_predictions_for_insn): Update.
> (combine_predictions_for_bb): Update.
> (propagate_freq): Update.
> (estimate_bb_frequencies): Update.
> (force_edge_cold): Update.
> * profile-count.c (profile_count::dump): Add missing space into dump.
> (profile_count::debug): Add newline.
> (profile_count::differs_from_p): Explicitly convert to unsigned.
> (profile_count::stream_in): Update.
> (profile_probability::dump): New member function.
> (profile_probability::debug): New member function.
> (profile_probability::differs_from_p): New member function.
> (profile_probability::differs_lot_from_p): New member function.
> (profile_probability::stream_in): New member function.
> (profile_probability::stream_out): New member function.
> * profile-count.h (profile_count_quality): Rename to ...
> (profile_quality): ... this one.
> (profile_probability): New.
> (profile_count): Update.
> * profile.c (compute_branch_probabilities): Update.
> * recog.c (peep2_attempt): Update.
> * sched-ebb.c (schedule_ebbs): Update.
> * sched-rgn.c (find_single_block_region): Update.
> (compute_dom_prob_ps): Update.
> (schedule_region): Update.
> * sel-sched-ir.c (compute_succs_info): Update.
> * stmt.c (struct case_node): Update.
> (do_jump_if_equal): Update.
> (get_outgoing_edge_probs): Update.
> (conditional_probability): Update.
> (emit_case_dispatch_table): Update.
> (expand_case): Update.
> (expand_sjlj_dispatch_table): Update.
> (emit_case_nodes): Update.
> * targhooks.c: Update.
> * tracer.c (better_p): Update.
> (find_best_successor): Update.
> * trans-mem.c (expand_transaction): Update.
> * tree-call-cdce.c: Update.
> * tree-cfg.c (gimple_split_edge): Upate.
> (move_sese_region_to_fn): Upate.
> * tree-cfgcleanup.c (cleanup_control_expr_graph): Upate.
> * tree-eh.c (lower_resx): Upate.
> (cleanup_empty_eh_move_lp): Upate.
> * tree-if-conv.c (version_loop_for_if_conversion): Update.
> * tree-inline.c (copy_edges_for_bb): Update.
> (copy_cfg_body): Update.
> * tree-parloops.c (gen_parallel_loop): Update.
> * tree-profile.c (gimple_gen_ic_func_profiler): Update.
> (gimple_gen_time_profiler): Update.
> * tree-ssa-dce.c (remove_dead_stmt): Update.
> * tree-ssa-ifcombine.c (update_profile_after_ifcombine): Update.
> * tree-ssa-loop-im.c (execute_sm_if_changed): Update.
> * tree-ssa-loop-ivcanon.c (remove_exits_and_undefined_stmts): Update.
> (unloop_loops): Update.
> (try_peel_loop): Update.
> * tree-ssa-loop-manip.c (tree_transform_and_unroll_loop): Update.
> * tree-ssa-loop-split.c (connect_loops): Update.
> (split_loop): Update.
> * tree-ssa-loop-unswitch.c (tree_unswitch_loop): Update.
> (hoist_guard): Update.
> * tree-ssa-phionlycprop.c (propagate_rhs_into_lhs): Update.
> * tree-ssa-phiopt.c (replace_phi_edge_with_variable): Update.
> (value_replacement): Update.
> * tree-ssa-reassoc.c (branch_fixup): Update.
> * tree-ssa-tail-merge.c (replace_block_by): Update.
> * tree-ssa-threadupdate.c (remove_ctrl_stmt_and_useless_edges): Update.
> (create_edge_and_update_destination_phis): Update.
> (compute_path_counts): Update.
> (recompute_probabilities): Update.
> (update_joiner_offpath_counts): Update.
> (freqs_to_counts_path): Update.
> (duplicate_thread_path): Update.
> * tree-switch-conversion.c (hoist_edge_and_branch_if_true): Update.
> (struct switch_conv_info): Update.
> (gen_inbound_check): Update.
> * tree-vect-loop-manip.c (slpeel_add_loop_guard): Update.
> (vect_do_peeling): Update.
> (vect_loop_versioning): Update.
> * tree-vect-loop.c (scale_profile_for_vect_loop): Update.
> (optimize_mask_stores): Update.
> * ubsan.c (ubsan_expand_null_ifn): Update.
> * value-prof.c (gimple_divmod_fixed_value): Update.
> (gimple_divmod_fixed_value_transform): Update.
> (gimple_mod_pow2): Update.
> (gimple_mod_pow2_value_transform): Update.
> (gimple_mod_subtract): Update.
> (gimple_mod_subtract_transform): Update.
> (gimple_ic): Update.
> (gimple_stringop_fixed_value): Update.
> (gimple_stringops_transform): Update.
> * value-prof.h: Update.
>
> Index: asan.c
> ===================================================================
> --- asan.c (revision 249769)
> +++ asan.c (working copy)
> @@ -1145,9 +1145,9 @@ asan_emit_stack_protection (rtx base, rt
> emit_move_insn (orig_base, base);
> ret = expand_normal (asan_detect_stack_use_after_return);
> lab = gen_label_rtx ();
> - int very_likely = REG_BR_PROB_BASE - (REG_BR_PROB_BASE / 2000 - 1);
> emit_cmp_and_jump_insns (ret, const0_rtx, EQ, NULL_RTX,
> - VOIDmode, 0, lab, very_likely);
> + VOIDmode, 0, lab,
> + profile_probability::very_likely ());
> snprintf (buf, sizeof buf, "__asan_stack_malloc_%d",
> use_after_return_class);
> ret = init_one_libfunc (buf);
> @@ -1158,9 +1158,9 @@ asan_emit_stack_protection (rtx base, rt
> /* __asan_stack_malloc_[n] returns a pointer to fake stack if succeeded
> and NULL otherwise. Check RET value is NULL here and jump over the
> BASE reassignment in this case. Otherwise, reassign BASE to RET. */
> - int very_unlikely = REG_BR_PROB_BASE / 2000 - 1;
> emit_cmp_and_jump_insns (ret, const0_rtx, EQ, NULL_RTX,
> - VOIDmode, 0, lab, very_unlikely);
> + VOIDmode, 0, lab,
> + profile_probability:: very_unlikely ());
> ret = convert_memory_address (Pmode, ret);
> emit_move_insn (base, ret);
> emit_label (lab);
> @@ -1255,9 +1255,9 @@ asan_emit_stack_protection (rtx base, rt
> {
> rtx_code_label *lab2 = gen_label_rtx ();
> char c = (char) ASAN_STACK_MAGIC_USE_AFTER_RET;
> - int very_likely = REG_BR_PROB_BASE - (REG_BR_PROB_BASE / 2000 - 1);
> emit_cmp_and_jump_insns (orig_base, base, EQ, NULL_RTX,
> - VOIDmode, 0, lab2, very_likely);
> + VOIDmode, 0, lab2,
> + profile_probability::very_likely ());
> shadow_mem = gen_rtx_MEM (BLKmode, shadow_base);
> set_mem_alias_set (shadow_mem, asan_shadow_set);
> mem = gen_rtx_MEM (ptr_mode, base);
> @@ -1588,7 +1588,8 @@ create_cond_insert_point (gimple_stmt_it
> = then_more_likely_p
> ? PROB_VERY_UNLIKELY
> : PROB_ALWAYS - PROB_VERY_UNLIKELY;
> - e->probability = PROB_ALWAYS - fallthrough_probability;
> + e->probability = profile_probability::from_reg_br_prob_base
> + (PROB_ALWAYS - fallthrough_probability);
> if (create_then_fallthru_edge)
> make_single_succ_edge (then_bb, fallthru_bb, EDGE_FALLTHRU);
>
> @@ -1596,7 +1597,8 @@ create_cond_insert_point (gimple_stmt_it
> e = find_edge (cond_bb, fallthru_bb);
> e->flags = EDGE_FALSE_VALUE;
> e->count = cond_bb->count;
> - e->probability = fallthrough_probability;
> + e->probability
> + = profile_probability::from_reg_br_prob_base (fallthrough_probability);
>
> /* Update dominance info for the newly created then_bb; note that
> fallthru_bb's dominance info has already been updated by
> Index: auto-profile.c
> ===================================================================
> --- auto-profile.c (revision 249769)
> +++ auto-profile.c (working copy)
> @@ -1346,9 +1346,10 @@ afdo_propagate_circuit (const bb_set &an
> continue;
> total++;
> only_one = ep;
> - if (e->probability == 0 && !is_edge_annotated (ep, *annotated_edge))
> + if (!e->probability.initialized_p ()
> + && !is_edge_annotated (ep, *annotated_edge))
> {
> - ep->probability = 0;
> + ep->probability = profile_probability::never ();
> ep->count = profile_count::zero ();
> set_edge_annotated (ep, annotated_edge);
> }
> Index: basic-block.h
> ===================================================================
> --- basic-block.h (revision 249769)
> +++ basic-block.h (working copy)
> @@ -45,7 +45,7 @@ struct GTY((user)) edge_def {
> unsigned int dest_idx;
>
> int flags; /* see cfg-flags.def */
> - int probability; /* biased by REG_BR_PROB_BASE */
> + profile_probability probability;
> profile_count count; /* Expected number of executions calculated
> in profile.c */
> };
> @@ -300,8 +300,7 @@ enum cfg_bb_flags
> ? EDGE_SUCC ((bb), 1) : EDGE_SUCC ((bb), 0))
>
> /* Return expected execution frequency of the edge E. */
> -#define EDGE_FREQUENCY(e) RDIV ((e)->src->frequency * (e)->probability, \
> - REG_BR_PROB_BASE)
> +#define EDGE_FREQUENCY(e) e->probability.apply (e->src->frequency)
>
> /* Compute a scale factor (or probability) suitable for scaling of
> gcov_type values via apply_probability() and apply_scale(). */
> Index: bb-reorder.c
> ===================================================================
> --- bb-reorder.c (revision 249769)
> +++ bb-reorder.c (working copy)
> @@ -206,8 +206,8 @@ static void find_traces_1_round (int, in
> int, bb_heap_t **, int);
> static basic_block copy_bb (basic_block, edge, basic_block, int);
> static long bb_to_key (basic_block);
> -static bool better_edge_p (const_basic_block, const_edge, int, int, int, int,
> - const_edge);
> +static bool better_edge_p (const_basic_block, const_edge, profile_probability,
> + int, profile_probability, int, const_edge);
> static bool connect_better_edge_p (const_edge, bool, int, const_edge,
> struct trace *);
> static void connect_traces (int, struct trace *);
> @@ -513,11 +513,12 @@ find_traces_1_round (int branch_th, int
>
> do
> {
> - int prob, freq;
> + profile_probability prob;
> + int freq;
> bool ends_in_call;
>
> /* The probability and frequency of the best edge. */
> - int best_prob = INT_MIN / 2;
> + profile_probability best_prob = profile_probability::uninitialized ();
> int best_freq = INT_MIN / 2;
>
> best_edge = NULL;
> @@ -565,7 +566,9 @@ find_traces_1_round (int branch_th, int
> successor (i.e. it is unsuitable successor). When optimizing
> for size, ignore the probability and frequency. */
> if (!(e->flags & EDGE_CAN_FALLTHRU) || (e->flags & EDGE_COMPLEX)
> - || ((prob < branch_th || EDGE_FREQUENCY (e) < exec_th
> + || !prob.initialized_p ()
> + || ((prob.to_reg_br_prob_base () < branch_th
> + || EDGE_FREQUENCY (e) < exec_th
> || e->count < count_th) && (!for_size)))
> continue;
>
> @@ -648,7 +651,9 @@ find_traces_1_round (int branch_th, int
>
> if (!(e->flags & EDGE_CAN_FALLTHRU)
> || (e->flags & EDGE_COMPLEX)
> - || prob < branch_th || freq < exec_th
> + || !prob.initialized_p ()
> + || prob.to_reg_br_prob_base () < branch_th
> + || freq < exec_th
> || e->count < count_th)
> {
> /* When partitioning hot/cold basic blocks, make sure
> @@ -936,14 +941,15 @@ bb_to_key (basic_block bb)
> BEST_PROB; similarly for frequency. */
>
> static bool
> -better_edge_p (const_basic_block bb, const_edge e, int prob, int freq,
> - int best_prob, int best_freq, const_edge cur_best_edge)
> +better_edge_p (const_basic_block bb, const_edge e, profile_probability prob,
> + int freq, profile_probability best_prob, int best_freq,
> + const_edge cur_best_edge)
> {
> bool is_better_edge;
>
> /* The BEST_* values do not have to be best, but can be a bit smaller than
> maximum values. */
> - int diff_prob = best_prob / 10;
> + profile_probability diff_prob = best_prob.apply_scale (1, 10);
> int diff_freq = best_freq / 10;
>
> /* The smaller one is better to keep the original order. */
> @@ -1494,7 +1500,8 @@ sanitize_hot_paths (bool walk_up, unsign
> vec<edge, va_gc> *edges = walk_up ? bb->preds : bb->succs;
> edge e;
> edge_iterator ei;
> - int highest_probability = 0;
> + profile_probability highest_probability
> + = profile_probability::uninitialized ();
> int highest_freq = 0;
> profile_count highest_count = profile_count::uninitialized ();
> bool found = false;
> @@ -1517,12 +1524,13 @@ sanitize_hot_paths (bool walk_up, unsign
> /* The following loop will look for the hottest edge via
> the edge count, if it is non-zero, then fallback to the edge
> frequency and finally the edge probability. */
> - if (e->count > highest_count)
> + if (!highest_count.initialized_p () || e->count > highest_count)
> highest_count = e->count;
> int edge_freq = EDGE_FREQUENCY (e);
> if (edge_freq > highest_freq)
> highest_freq = edge_freq;
> - if (e->probability > highest_probability)
> + if (!highest_probability.initialized_p ()
> + || e->probability > highest_probability)
> highest_probability = e->probability;
> }
>
> Index: cfg.c
> ===================================================================
> --- cfg.c (revision 249769)
> +++ cfg.c (working copy)
> @@ -264,6 +264,7 @@ unchecked_make_edge (basic_block src, ba
> n_edges_for_fn (cfun)++;
>
> e->count = profile_count::uninitialized ();
> + e->probability = profile_probability::uninitialized ();
> e->src = src;
> e->dest = dst;
> e->flags = flags;
> @@ -332,7 +333,7 @@ make_single_succ_edge (basic_block src,
> {
> edge e = make_edge (src, dest, flags);
>
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = src->count;
> return e;
> }
> @@ -400,7 +401,6 @@ static void
> check_bb_profile (basic_block bb, FILE * file, int indent)
> {
> edge e;
> - int sum = 0;
> edge_iterator ei;
> struct function *fun = DECL_STRUCT_FUNCTION (current_function_decl);
> char *s_indent = (char *) alloca ((size_t) indent + 1);
> @@ -413,21 +413,38 @@ check_bb_profile (basic_block bb, FILE *
> if (bb != EXIT_BLOCK_PTR_FOR_FN (fun))
> {
> bool found = false;
> + profile_probability sum = profile_probability::never ();
> + int isum = 0;
> +
> FOR_EACH_EDGE (e, ei, bb->succs)
> {
> - if (!(e->flags & EDGE_EH))
> + if (!(e->flags & (EDGE_EH | EDGE_FAKE)))
> found = true;
> sum += e->probability;
> + if (e->probability.initialized_p ())
> + isum += e->probability.to_reg_br_prob_base ();
> }
> /* Only report mismatches for non-EH control flow. If there are only EH
> edges it means that the BB ends by noreturn call. Here the control
> flow may just terminate. */
> if (found)
> {
> - if (EDGE_COUNT (bb->succs) && abs (sum - REG_BR_PROB_BASE) > 100)
> - fprintf (file,
> - ";; %sInvalid sum of outgoing probabilities %.1f%%\n",
> - s_indent, sum * 100.0 / REG_BR_PROB_BASE);
> + if (sum.differs_from_p (profile_probability::always ()))
> + {
> + fprintf (file,
> + ";; %sInvalid sum of outgoing probabilities ",
> + s_indent);
> + sum.dump (file);
> + fprintf (file, "\n");
> + }
> + /* Probabilities caps to 100% and thus the previous test will never
> + fire if the sum of probabilities is too large. */
> + else if (isum > REG_BR_PROB_BASE + 100)
> + {
> + fprintf (file,
> + ";; %sInvalid sum of outgoing probabilities %.1f%%\n",
> + s_indent, isum * 100.0 / REG_BR_PROB_BASE);
> + }
> profile_count lsum = profile_count::zero ();
> FOR_EACH_EDGE (e, ei, bb->succs)
> lsum += e->count;
> @@ -442,9 +459,9 @@ check_bb_profile (basic_block bb, FILE *
> }
> }
> }
> - if (bb != ENTRY_BLOCK_PTR_FOR_FN (fun))
> + if (bb != ENTRY_BLOCK_PTR_FOR_FN (fun))
> {
> - sum = 0;
> + int sum = 0;
> FOR_EACH_EDGE (e, ei, bb->preds)
> sum += EDGE_FREQUENCY (e);
> if (abs (sum - bb->frequency) > 100)
> @@ -498,8 +515,12 @@ dump_edge_info (FILE *file, edge e, dump
> else
> fprintf (file, " %d", side->index);
>
> - if (e->probability && do_details)
> - fprintf (file, " [%.1f%%] ", e->probability * 100.0 / REG_BR_PROB_BASE);
> + if (e->probability.initialized_p () && do_details)
> + {
> + fprintf (file, " [");
> + e->probability.dump (file);
> + fprintf (file, "] ");
> + }
>
> if (e->count.initialized_p () && do_details)
> {
> @@ -860,7 +881,7 @@ update_bb_profile_for_threading (basic_b
> profile_count count, edge taken_edge)
> {
> edge c;
> - int prob;
> + profile_probability prob;
> edge_iterator ei;
>
> if (bb->count < count)
> @@ -878,59 +899,51 @@ update_bb_profile_for_threading (basic_b
> /* Compute the probability of TAKEN_EDGE being reached via threaded edge.
> Watch for overflows. */
> if (bb->frequency)
> - prob = GCOV_COMPUTE_SCALE (edge_frequency, bb->frequency);
> + /* FIXME: We should get edge frequency as count. */
> + prob = profile_probability::probability_in_gcov_type
> + (edge_frequency, bb->frequency);
> else
> - prob = 0;
> + prob = profile_probability::never ();
> if (prob > taken_edge->probability)
> {
> if (dump_file)
> - fprintf (dump_file, "Jump threading proved probability of edge "
> - "%i->%i too small (it is %i, should be %i).\n",
> - taken_edge->src->index, taken_edge->dest->index,
> - taken_edge->probability, prob);
> - prob = taken_edge->probability * 6 / 8;
> + {
> + fprintf (dump_file, "Jump threading proved probability of edge "
> + "%i->%i too small (it is ",
> + taken_edge->src->index, taken_edge->dest->index);
> + taken_edge->probability.dump (dump_file);
> + fprintf (dump_file, " should be ");
> + prob.dump (dump_file);
> + fprintf (dump_file, ")\n");
> + }
> + prob = taken_edge->probability.apply_scale (6, 8);
> }
>
> /* Now rescale the probabilities. */
> taken_edge->probability -= prob;
> - prob = REG_BR_PROB_BASE - prob;
> - if (prob <= 0)
> + prob = prob.invert ();
> + if (prob == profile_probability::never ())
> {
> if (dump_file)
> fprintf (dump_file, "Edge frequencies of bb %i has been reset, "
> "frequency of block should end up being 0, it is %i\n",
> bb->index, bb->frequency);
> - EDGE_SUCC (bb, 0)->probability = REG_BR_PROB_BASE;
> + EDGE_SUCC (bb, 0)->probability = profile_probability::guessed_always ();
> ei = ei_start (bb->succs);
> ei_next (&ei);
> for (; (c = ei_safe_edge (ei)); ei_next (&ei))
> - c->probability = 0;
> + c->probability = profile_probability::guessed_never ();
> }
> - else if (prob != REG_BR_PROB_BASE)
> + else if (!(prob == profile_probability::always ()))
> {
> - int scale = RDIV (65536 * REG_BR_PROB_BASE, prob);
> -
> FOR_EACH_EDGE (c, ei, bb->succs)
> - {
> - /* Protect from overflow due to additional scaling. */
> - if (c->probability > prob)
> - c->probability = REG_BR_PROB_BASE;
> - else
> - {
> - c->probability = RDIV (c->probability * scale, 65536);
> - if (c->probability > REG_BR_PROB_BASE)
> - c->probability = REG_BR_PROB_BASE;
> - }
> - }
> + c->probability /= prob;
> }
>
> gcc_assert (bb == taken_edge->src);
> - if (taken_edge->count < count)
> - {
> - if (dump_file)
> - fprintf (dump_file, "edge %i->%i count became negative after threading",
> - taken_edge->src->index, taken_edge->dest->index);
> - }
> + if (dump_file && taken_edge->count < count)
> + fprintf (dump_file, "edge %i->%i count became negative after threading",
> + taken_edge->src->index, taken_edge->dest->index);
> taken_edge->count -= count;
> }
>
> Index: cfganal.c
> ===================================================================
> --- cfganal.c (revision 249769)
> +++ cfganal.c (working copy)
> @@ -610,7 +610,10 @@ connect_infinite_loops_to_exit (void)
> break;
>
> basic_block deadend_block = dfs_find_deadend (unvisited_block);
> - make_edge (deadend_block, EXIT_BLOCK_PTR_FOR_FN (cfun), EDGE_FAKE);
> + edge e = make_edge (deadend_block, EXIT_BLOCK_PTR_FOR_FN (cfun),
> + EDGE_FAKE);
> + e->count = profile_count::zero ();
> + e->probability = profile_probability::never ();
> dfs.add_bb (deadend_block);
> }
> }
> Index: cfgbuild.c
> ===================================================================
> --- cfgbuild.c (revision 249769)
> +++ cfgbuild.c (working copy)
> @@ -545,10 +545,13 @@ compute_outgoing_frequencies (basic_bloc
> {
> probability = XINT (note, 0);
> e = BRANCH_EDGE (b);
> - e->probability = probability;
> + e->probability
> + = profile_probability::from_reg_br_prob_base (probability);
> e->count = b->count.apply_probability (probability);
> f = FALLTHRU_EDGE (b);
> - f->probability = REG_BR_PROB_BASE - probability;
> + f->probability
> + = profile_probability::from_reg_br_prob_base (REG_BR_PROB_BASE
> + - probability);
> f->count = b->count - e->count;
> return;
> }
> @@ -560,7 +563,7 @@ compute_outgoing_frequencies (basic_bloc
> else if (single_succ_p (b))
> {
> e = single_succ_edge (b);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = b->count;
> return;
> }
> @@ -656,7 +659,8 @@ find_many_sub_basic_blocks (sbitmap bloc
> }
> else
> uninitialized_src = true;
> - bb->frequency += EDGE_FREQUENCY (e);
> + if (e->probability.initialized_p ())
> + bb->frequency += EDGE_FREQUENCY (e);
> }
> /* When some edges are missing with read profile, this is
> most likely because RTL expansion introduced loop.
> Index: cfgcleanup.c
> ===================================================================
> --- cfgcleanup.c (revision 249769)
> +++ cfgcleanup.c (working copy)
> @@ -559,7 +559,7 @@ try_forward_edges (int mode, basic_block
> {
> /* Save the values now, as the edge may get removed. */
> profile_count edge_count = e->count;
> - int edge_probability = e->probability;
> + profile_probability edge_probability = e->probability;
> int edge_frequency;
> int n = 0;
>
> @@ -585,7 +585,7 @@ try_forward_edges (int mode, basic_block
> /* We successfully forwarded the edge. Now update profile
> data: for each edge we traversed in the chain, remove
> the original edge's execution count. */
> - edge_frequency = apply_probability (b->frequency, edge_probability);
> + edge_frequency = edge_probability.apply (b->frequency);
>
> do
> {
> @@ -1710,24 +1710,28 @@ outgoing_edges_match (int mode, basic_bl
> && optimize_bb_for_speed_p (bb1)
> && optimize_bb_for_speed_p (bb2))
> {
> - int prob2;
> + profile_probability prob2;
>
> if (b1->dest == b2->dest)
> prob2 = b2->probability;
> else
> /* Do not use f2 probability as f2 may be forwarded. */
> - prob2 = REG_BR_PROB_BASE - b2->probability;
> + prob2 = b2->probability.invert ();
>
> /* Fail if the difference in probabilities is greater than 50%.
> This rules out two well-predicted branches with opposite
> outcomes. */
> - if (abs (b1->probability - prob2) > REG_BR_PROB_BASE / 2)
> + if (b1->probability.differs_lot_from_p (prob2))
> {
> if (dump_file)
> - fprintf (dump_file,
> - "Outcomes of branch in bb %i and %i differ too much (%i %i)\n",
> - bb1->index, bb2->index, b1->probability, prob2);
> -
> + {
> + fprintf (dump_file,
> + "Outcomes of branch in bb %i and %i differ too"
> + " much (", bb1->index, bb2->index);
> + b1->probability.dump (dump_file);
> + prob2.dump (dump_file);
> + fprintf (dump_file, ")\n");
> + }
> return false;
> }
> }
> @@ -2149,12 +2153,9 @@ try_crossjump_to_edge (int mode, edge e1
> }
>
> if (!redirect_edges_to->frequency && !src1->frequency)
> - s->probability = (s->probability + s2->probability) / 2;
> - else
> - s->probability
> - = ((s->probability * redirect_edges_to->frequency +
> - s2->probability * src1->frequency)
> - / (redirect_edges_to->frequency + src1->frequency));
> + s->probability = s->probability.combine_with_freq
> + (redirect_edges_to->frequency,
> + s2->probability, src1->frequency);
> }
>
> /* Adjust count and frequency for the block. An earlier jump
> Index: cfgexpand.c
> ===================================================================
> --- cfgexpand.c (revision 249769)
> +++ cfgexpand.c (working copy)
> @@ -2389,7 +2389,6 @@ static basic_block
> expand_gimple_cond (basic_block bb, gcond *stmt)
> {
> basic_block new_bb, dest;
> - edge new_edge;
> edge true_edge;
> edge false_edge;
> rtx_insn *last2, *last;
> @@ -2508,9 +2507,7 @@ expand_gimple_cond (basic_block bb, gcon
> if (loop->latch == bb
> && loop->header == dest)
> loop->latch = new_bb;
> - new_edge = make_edge (new_bb, dest, 0);
> - new_edge->probability = REG_BR_PROB_BASE;
> - new_edge->count = new_bb->count;
> + make_single_succ_edge (new_bb, dest, 0);
> if (BARRIER_P (BB_END (new_bb)))
> BB_END (new_bb) = PREV_INSN (BB_END (new_bb));
> update_bb_for_insn (new_bb);
> @@ -3782,7 +3779,7 @@ expand_gimple_tailcall (basic_block bb,
> rtx_insn *last2, *last;
> edge e;
> edge_iterator ei;
> - int probability;
> + profile_probability probability;
>
> last2 = last = expand_gimple_stmt (stmt);
>
> @@ -3807,7 +3804,7 @@ expand_gimple_tailcall (basic_block bb,
> all edges here, or redirecting the existing fallthru edge to
> the exit block. */
>
> - probability = 0;
> + probability = profile_probability::never ();
> profile_count count = profile_count::zero ();
>
> for (ei = ei_start (bb->succs); (e = ei_safe_edge (ei)); )
> @@ -5833,12 +5830,11 @@ construct_init_block (void)
> {
> first_block = e->dest;
> redirect_edge_succ (e, init_block);
> - e = make_edge (init_block, first_block, flags);
> + e = make_single_succ_edge (init_block, first_block, flags);
> }
> else
> - e = make_edge (init_block, EXIT_BLOCK_PTR_FOR_FN (cfun), EDGE_FALLTHRU);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = ENTRY_BLOCK_PTR_FOR_FN (cfun)->count;
> + e = make_single_succ_edge (init_block, EXIT_BLOCK_PTR_FOR_FN (cfun),
> + EDGE_FALLTHRU);
>
> update_bb_for_insn (init_block);
> return init_block;
> @@ -5918,9 +5914,8 @@ construct_exit_block (void)
> ix++;
> }
>
> - e = make_edge (exit_block, EXIT_BLOCK_PTR_FOR_FN (cfun), EDGE_FALLTHRU);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = EXIT_BLOCK_PTR_FOR_FN (cfun)->count;
> + e = make_single_succ_edge (exit_block, EXIT_BLOCK_PTR_FOR_FN (cfun),
> + EDGE_FALLTHRU);
> FOR_EACH_EDGE (e2, ei, EXIT_BLOCK_PTR_FOR_FN (cfun)->preds)
> if (e2 != e)
> {
> Index: cfghooks.c
> ===================================================================
> --- cfghooks.c (revision 249769)
> +++ cfghooks.c (working copy)
> @@ -160,10 +160,10 @@ verify_flow_info (void)
> e->src->index, e->dest->index);
> err = 1;
> }
> - if (e->probability < 0 || e->probability > REG_BR_PROB_BASE)
> + if (!e->probability.verify ())
> {
> - error ("verify_flow_info: Wrong probability of edge %i->%i %i",
> - e->src->index, e->dest->index, e->probability);
> + error ("verify_flow_info: Wrong probability of edge %i->%i",
> + e->src->index, e->dest->index);
> err = 1;
> }
> if (!e->count.verify ())
> @@ -443,8 +443,6 @@ redirect_edge_succ_nodup (edge e, basic_
> {
> s->flags |= e->flags;
> s->probability += e->probability;
> - if (s->probability > REG_BR_PROB_BASE)
> - s->probability = REG_BR_PROB_BASE;
> s->count += e->count;
> /* FIXME: This should be called via a hook and only for IR_GIMPLE. */
> redirect_edge_var_map_dup (s, e);
> @@ -640,7 +638,7 @@ split_edge (edge e)
> ret = cfg_hooks->split_edge (e);
> ret->count = count;
> ret->frequency = freq;
> - single_succ_edge (ret)->probability = REG_BR_PROB_BASE;
> + single_succ_edge (ret)->probability = profile_probability::always ();
> single_succ_edge (ret)->count = count;
>
> if (irr)
> @@ -1451,17 +1449,17 @@ account_profile_record (struct profile_r
> basic_block bb;
> edge_iterator ei;
> edge e;
> - int sum;
>
> FOR_ALL_BB_FN (bb, cfun)
> {
> if (bb != EXIT_BLOCK_PTR_FOR_FN (cfun)
> && profile_status_for_fn (cfun) != PROFILE_ABSENT)
> {
> - sum = 0;
> + profile_probability sum = profile_probability::never ();
> FOR_EACH_EDGE (e, ei, bb->succs)
> sum += e->probability;
> - if (EDGE_COUNT (bb->succs) && abs (sum - REG_BR_PROB_BASE) > 100)
> + if (EDGE_COUNT (bb->succs)
> + && sum.differs_from_p (profile_probability::always ()))
> record->num_mismatched_freq_out[after_pass]++;
> profile_count lsum = profile_count::zero ();
> FOR_EACH_EDGE (e, ei, bb->succs)
> @@ -1472,7 +1470,7 @@ account_profile_record (struct profile_r
> if (bb != ENTRY_BLOCK_PTR_FOR_FN (cfun)
> && profile_status_for_fn (cfun) != PROFILE_ABSENT)
> {
> - sum = 0;
> + int sum = 0;
> FOR_EACH_EDGE (e, ei, bb->preds)
> sum += EDGE_FREQUENCY (e);
> if (abs (sum - bb->frequency) > 100
> Index: cfgloopanal.c
> ===================================================================
> --- cfgloopanal.c (revision 249769)
> +++ cfgloopanal.c (working copy)
> @@ -477,7 +477,7 @@ single_likely_exit (struct loop *loop)
> reasons.
> FIXME: Turn to likely_never_executed */
> if ((profile_status_for_fn (cfun) != PROFILE_ABSENT
> - && ex->probability < 5)
> + && ex->probability < profile_probability::from_reg_br_prob_base (5))
> || ex->count == profile_count::zero ())
> continue;
> if (!found)
> Index: cfgloopmanip.c
> ===================================================================
> --- cfgloopmanip.c (revision 249769)
> +++ cfgloopmanip.c (working copy)
> @@ -542,8 +542,9 @@ scale_loop_profile (struct loop *loop, i
>
> /* Probability of exit must be 1/iterations. */
> freq_delta = EDGE_FREQUENCY (e);
> - e->probability = REG_BR_PROB_BASE / iteration_bound;
> - other_e->probability = inverse_probability (e->probability);
> + e->probability = profile_probability::from_reg_br_prob_base
> + (REG_BR_PROB_BASE / iteration_bound);
> + other_e->probability = e->probability.invert ();
> freq_delta -= EDGE_FREQUENCY (e);
>
> /* Adjust counts accordingly. */
> @@ -1101,43 +1102,6 @@ can_duplicate_loop_p (const struct loop
> return ret;
> }
>
> -/* Sets probability and count of edge E to zero. The probability and count
> - is redistributed evenly to the remaining edges coming from E->src. */
> -
> -static void
> -set_zero_probability (edge e)
> -{
> - basic_block bb = e->src;
> - edge_iterator ei;
> - edge ae, last = NULL;
> - unsigned n = EDGE_COUNT (bb->succs);
> - profile_count cnt = e->count, cnt1;
> - unsigned prob = e->probability, prob1;
> -
> - gcc_assert (n > 1);
> - cnt1 = cnt.apply_scale (1, (n - 1));
> - prob1 = prob / (n - 1);
> -
> - FOR_EACH_EDGE (ae, ei, bb->succs)
> - {
> - if (ae == e)
> - continue;
> -
> - ae->probability += prob1;
> - ae->count += cnt1;
> - last = ae;
> - }
> -
> - /* Move the rest to one of the edges. */
> - last->probability += prob % (n - 1);
> - /* TODO: Remove once we have fractional counts. */
> - if (cnt.initialized_p ())
> - last->count += profile_count::from_gcov_type (cnt.to_gcov_type () % (n - 1));
> -
> - e->probability = 0;
> - e->count = profile_count::zero ();
> -}
> -
> /* Duplicates body of LOOP to given edge E NDUPL times. Takes care of updating
> loop structure and dominators. E's destination must be LOOP header for
> this to work, i.e. it must be entry or latch edge of this loop; these are
> @@ -1224,14 +1188,18 @@ duplicate_loop_to_header_edge (struct lo
> prob_pass_wont_exit =
> RDIV (REG_BR_PROB_BASE * (freq_le + freq_out_orig), freq_in);
>
> - if (orig
> - && REG_BR_PROB_BASE - orig->probability != 0)
> + if (orig && orig->probability.initialized_p ()
> + && !(orig->probability == profile_probability::always ()))
> {
> /* The blocks that are dominated by a removed exit edge ORIG have
> frequencies scaled by this. */
> - scale_after_exit
> - = GCOV_COMPUTE_SCALE (REG_BR_PROB_BASE,
> - REG_BR_PROB_BASE - orig->probability);
> + if (orig->probability.initialized_p ())
> + scale_after_exit
> + = GCOV_COMPUTE_SCALE (REG_BR_PROB_BASE,
> + REG_BR_PROB_BASE
> + - orig->probability.to_reg_br_prob_base ());
> + else
> + scale_after_exit = REG_BR_PROB_BASE;
> bbs_to_scale = BITMAP_ALLOC (NULL);
> for (i = 0; i < n; i++)
> {
> @@ -1387,7 +1355,7 @@ duplicate_loop_to_header_edge (struct lo
> {
> if (to_remove)
> to_remove->safe_push (new_spec_edges[SE_ORIG]);
> - set_zero_probability (new_spec_edges[SE_ORIG]);
> + force_edge_cold (new_spec_edges[SE_ORIG], true);
>
> /* Scale the frequencies of the blocks dominated by the exit. */
> if (bbs_to_scale)
> @@ -1423,7 +1391,7 @@ duplicate_loop_to_header_edge (struct lo
> {
> if (to_remove)
> to_remove->safe_push (orig);
> - set_zero_probability (orig);
> + force_edge_cold (orig, true);
>
> /* Scale the frequencies of the blocks dominated by the exit. */
> if (bbs_to_scale)
> @@ -1657,8 +1625,9 @@ force_single_succ_latches (void)
>
> static basic_block
> lv_adjust_loop_entry_edge (basic_block first_head, basic_block second_head,
> - edge e, void *cond_expr, unsigned then_prob,
> - unsigned else_prob)
> + edge e, void *cond_expr,
> + profile_probability then_prob,
> + profile_probability else_prob)
> {
> basic_block new_head = NULL;
> edge e1;
> @@ -1713,7 +1682,7 @@ lv_adjust_loop_entry_edge (basic_block f
> struct loop *
> loop_version (struct loop *loop,
> void *cond_expr, basic_block *condition_bb,
> - unsigned then_prob, unsigned else_prob,
> + profile_probability then_prob, profile_probability else_prob,
> unsigned then_scale, unsigned else_scale,
> bool place_after)
> {
> Index: cfgloopmanip.h
> ===================================================================
> --- cfgloopmanip.h (revision 249769)
> +++ cfgloopmanip.h (working copy)
> @@ -58,7 +58,8 @@ basic_block create_preheader (struct loo
> extern void create_preheaders (int);
> extern void force_single_succ_latches (void);
> struct loop * loop_version (struct loop *, void *,
> - basic_block *, unsigned, unsigned,
> + basic_block *,
> + profile_probability, profile_probability,
> unsigned, unsigned, bool);
>
> #endif /* GCC_CFGLOOPMANIP_H */
> Index: cfgrtl.c
> ===================================================================
> --- cfgrtl.c (revision 249769)
> +++ cfgrtl.c (working copy)
> @@ -1155,7 +1155,7 @@ try_redirect_by_replacing_jump (edge e,
> else
> e->flags = 0;
>
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = src->count;
>
> if (e->dest != target)
> @@ -1504,12 +1504,10 @@ force_nonfallthru_and_redirect (edge e,
> {
> int prob = XINT (note, 0);
>
> - b->probability = prob;
> + b->probability = profile_probability::from_reg_br_prob_base (prob);
> b->count = e->count.apply_probability (prob);
> e->probability -= e->probability;
> e->count -= b->count;
> - if (e->probability < 0)
> - e->probability = 0;
> }
> }
>
> @@ -1618,7 +1616,7 @@ force_nonfallthru_and_redirect (edge e,
> {
> rtx_insn *new_head;
> profile_count count = e->count;
> - int probability = e->probability;
> + profile_probability probability = e->probability;
> /* Create the new structures. */
>
> /* If the old block ended with a tablejump, skip its table
> @@ -1646,7 +1644,7 @@ force_nonfallthru_and_redirect (edge e,
>
> /* Redirect old edge. */
> redirect_edge_pred (e, jump_block);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
>
> /* If e->src was previously region crossing, it no longer is
> and the reg crossing note should be removed. */
> @@ -1656,7 +1654,7 @@ force_nonfallthru_and_redirect (edge e,
> add also edge from asm goto bb to target. */
> if (asm_goto_edge)
> {
> - new_edge->probability /= 2;
> + new_edge->probability = new_edge->probability.apply_scale (1, 2);
> new_edge->count = new_edge->count.apply_scale (1, 2);
> jump_block->count = jump_block->count.apply_scale (1, 2);
> jump_block->frequency /= 2;
> @@ -2251,12 +2249,13 @@ void
> update_br_prob_note (basic_block bb)
> {
> rtx note;
> - if (!JUMP_P (BB_END (bb)))
> + if (!JUMP_P (BB_END (bb)) || !BRANCH_EDGE (bb)->probability.initialized_p ())
> return;
> note = find_reg_note (BB_END (bb), REG_BR_PROB, NULL_RTX);
> - if (!note || XINT (note, 0) == BRANCH_EDGE (bb)->probability)
> + if (!note
> + || XINT (note, 0) == BRANCH_EDGE (bb)->probability.to_reg_br_prob_base ())
> return;
> - XINT (note, 0) = BRANCH_EDGE (bb)->probability;
> + XINT (note, 0) = BRANCH_EDGE (bb)->probability.to_reg_br_prob_base ();
> }
>
> /* Get the last insn associated with block BB (that includes barriers and
> @@ -2447,11 +2446,19 @@ rtl_verify_edges (void)
> && EDGE_COUNT (bb->succs) >= 2
> && any_condjump_p (BB_END (bb)))
> {
> - if (XINT (note, 0) != BRANCH_EDGE (bb)->probability
> - && profile_status_for_fn (cfun) != PROFILE_ABSENT)
> + if (!BRANCH_EDGE (bb)->probability.initialized_p ())
> + {
> + error ("verify_flow_info: "
> + "REG_BR_PROB is set but cfg probability is not");
> + err = 1;
> + }
> + else if (XINT (note, 0)
> + != BRANCH_EDGE (bb)->probability.to_reg_br_prob_base ()
> + && profile_status_for_fn (cfun) != PROFILE_ABSENT)
> {
> error ("verify_flow_info: REG_BR_PROB does not match cfg %i %i",
> - XINT (note, 0), BRANCH_EDGE (bb)->probability);
> + XINT (note, 0),
> + BRANCH_EDGE (bb)->probability.to_reg_br_prob_base ());
> err = 1;
> }
> }
> @@ -3143,7 +3150,7 @@ purge_dead_edges (basic_block bb)
> /* Redistribute probabilities. */
> if (single_succ_p (bb))
> {
> - single_succ_edge (bb)->probability = REG_BR_PROB_BASE;
> + single_succ_edge (bb)->probability = profile_probability::always ();
> single_succ_edge (bb)->count = bb->count;
> }
> else
> @@ -3154,8 +3161,9 @@ purge_dead_edges (basic_block bb)
>
> b = BRANCH_EDGE (bb);
> f = FALLTHRU_EDGE (bb);
> - b->probability = XINT (note, 0);
> - f->probability = REG_BR_PROB_BASE - b->probability;
> + b->probability = profile_probability::from_reg_br_prob_base
> + (XINT (note, 0));
> + f->probability = profile_probability::always () - b->probability;
> b->count = bb->count.apply_probability (b->probability);
> f->count = bb->count.apply_probability (f->probability);
> }
> @@ -3208,7 +3216,7 @@ purge_dead_edges (basic_block bb)
>
> gcc_assert (single_succ_p (bb));
>
> - single_succ_edge (bb)->probability = REG_BR_PROB_BASE;
> + single_succ_edge (bb)->probability = profile_probability::always ();
> single_succ_edge (bb)->count = bb->count;
>
> if (dump_file)
> @@ -4931,7 +4939,8 @@ rtl_lv_add_condition_to_bb (basic_block
> start_sequence ();
> op0 = force_operand (op0, NULL_RTX);
> op1 = force_operand (op1, NULL_RTX);
> - do_compare_rtx_and_jump (op0, op1, comp, 0, mode, NULL_RTX, NULL, label, -1);
> + do_compare_rtx_and_jump (op0, op1, comp, 0, mode, NULL_RTX, NULL, label,
> + profile_probability::uninitialized ());
> jump = get_last_insn ();
> JUMP_LABEL (jump) = label;
> LABEL_NUSES (label)++;
> Index: cgraph.c
> ===================================================================
> --- cgraph.c (revision 249769)
> +++ cgraph.c (working copy)
> @@ -20,8 +20,9 @@ along with GCC; see the file COPYING3.
>
> /* This file contains basic routines manipulating call graph
>
> - The call-graph is a data structure designed for intra-procedural optimization.
> - It represents a multi-graph where nodes are functions and edges are call sites. */
> + The call-graph is a data structure designed for inter-procedural
> + optimization. It represents a multi-graph where nodes are functions
> + (symbols within symbol table) and edges are call sites. */
>
> #include "config.h"
> #include "system.h"
> @@ -1316,13 +1317,16 @@ cgraph_edge::redirect_call_stmt_to_calle
> push_cfun (DECL_STRUCT_FUNCTION (e->caller->decl));
> new_stmt = gimple_ic (e->call_stmt,
> dyn_cast<cgraph_node *> (ref->referred),
> + /* FIXME: cleanup. */
> + profile_probability::from_reg_br_prob_base (
> e->count > profile_count::zero ()
> || e2->count > profile_count::zero ()
> - ? e->count.probability_in (e->count + e2->count)
> + ? e->count.probability_in
> + (e->count + e2->count).to_reg_br_prob_base ()
> : e->frequency || e2->frequency
> ? RDIV (e->frequency * REG_BR_PROB_BASE,
> e->frequency + e2->frequency)
> - : REG_BR_PROB_BASE / 2,
> + : REG_BR_PROB_BASE / 2),
> e->count, e->count + e2->count);
> e->speculative = false;
> e->caller->set_call_stmt_including_clones (e->call_stmt, new_stmt,
> Index: cgraphunit.c
> ===================================================================
> --- cgraphunit.c (revision 249769)
> +++ cgraphunit.c (working copy)
> @@ -1506,18 +1506,18 @@ init_lowered_empty_function (tree decl,
>
> /* Create BB for body of the function and connect it properly. */
> ENTRY_BLOCK_PTR_FOR_FN (cfun)->count = count;
> - ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency = REG_BR_PROB_BASE;
> + ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency = BB_FREQ_MAX;
> EXIT_BLOCK_PTR_FOR_FN (cfun)->count = count;
> - EXIT_BLOCK_PTR_FOR_FN (cfun)->frequency = REG_BR_PROB_BASE;
> + EXIT_BLOCK_PTR_FOR_FN (cfun)->frequency = BB_FREQ_MAX;
> bb = create_basic_block (NULL, ENTRY_BLOCK_PTR_FOR_FN (cfun));
> bb->count = count;
> bb->frequency = BB_FREQ_MAX;
> e = make_edge (ENTRY_BLOCK_PTR_FOR_FN (cfun), bb, EDGE_FALLTHRU);
> e->count = count;
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e = make_edge (bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> e->count = count;
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> add_bb_to_loop (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun)->loop_father);
>
> return bb;
> @@ -1891,19 +1891,18 @@ cgraph_node::expand_thunk (bool output_a
> NULL_TREE, NULL_TREE);
> gsi_insert_after (&bsi, stmt, GSI_NEW_STMT);
> e = make_edge (bb, then_bb, EDGE_TRUE_VALUE);
> - e->probability = REG_BR_PROB_BASE - REG_BR_PROB_BASE / 16;
> + e->probability = profile_probability::guessed_always ()
> + .apply_scale (1, 16);
> e->count = count - count.apply_scale (1, 16);
> e = make_edge (bb, else_bb, EDGE_FALSE_VALUE);
> - e->probability = REG_BR_PROB_BASE / 16;
> + e->probability = profile_probability::guessed_always ()
> + .apply_scale (1, 16);
> e->count = count.apply_scale (1, 16);
> - e = make_edge (return_bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = count;
> - e = make_edge (then_bb, return_bb, EDGE_FALLTHRU);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = count - count.apply_scale (1, 16);
> + make_single_succ_edge (return_bb,
> + EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> + make_single_succ_edge (then_bb, return_bb, EDGE_FALLTHRU);
> e = make_edge (else_bb, return_bb, EDGE_FALLTHRU);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = count.apply_scale (1, 16);
> bsi = gsi_last_bb (then_bb);
> }
> Index: cilk-common.c
> ===================================================================
> --- cilk-common.c (revision 249769)
> +++ cilk-common.c (working copy)
> @@ -38,6 +38,7 @@ along with GCC; see the file COPYING3.
> #include "stor-layout.h"
> #include "langhooks.h"
> #include "explow.h"
> +#include "profile-count.h"
> #include "expr.h"
> #include "tree-iterator.h"
> #include "gimplify.h"
> Index: dojump.c
> ===================================================================
> --- dojump.c (revision 249769)
> +++ dojump.c (working copy)
> @@ -39,19 +39,13 @@ along with GCC; see the file COPYING3.
>
> static bool prefer_and_bit_test (machine_mode, int);
> static void do_jump_by_parts_greater (tree, tree, int,
> - rtx_code_label *, rtx_code_label *, int);
> + rtx_code_label *, rtx_code_label *,
> + profile_probability);
> static void do_jump_by_parts_equality (tree, tree, rtx_code_label *,
> - rtx_code_label *, int);
> + rtx_code_label *, profile_probability);
> static void do_compare_and_jump (tree, tree, enum rtx_code, enum rtx_code,
> - rtx_code_label *, rtx_code_label *, int);
> -
> -/* Invert probability if there is any. -1 stands for unknown. */
> -
> -static inline int
> -inv (int prob)
> -{
> - return prob == -1 ? -1 : REG_BR_PROB_BASE - prob;
> -}
> + rtx_code_label *, rtx_code_label *,
> + profile_probability);
>
> /* At the start of a function, record that we have no previously-pushed
> arguments waiting to be popped. */
> @@ -128,29 +122,29 @@ restore_pending_stack_adjust (saved_pend
> /* Generate code to evaluate EXP and jump to LABEL if the value is zero. */
>
> void
> -jumpifnot (tree exp, rtx_code_label *label, int prob)
> +jumpifnot (tree exp, rtx_code_label *label, profile_probability prob)
> {
> - do_jump (exp, label, NULL, inv (prob));
> + do_jump (exp, label, NULL, prob.invert ());
> }
>
> void
> jumpifnot_1 (enum tree_code code, tree op0, tree op1, rtx_code_label *label,
> - int prob)
> + profile_probability prob)
> {
> - do_jump_1 (code, op0, op1, label, NULL, inv (prob));
> + do_jump_1 (code, op0, op1, label, NULL, prob.invert ());
> }
>
> /* Generate code to evaluate EXP and jump to LABEL if the value is nonzero. */
>
> void
> -jumpif (tree exp, rtx_code_label *label, int prob)
> +jumpif (tree exp, rtx_code_label *label, profile_probability prob)
> {
> do_jump (exp, NULL, label, prob);
> }
>
> void
> jumpif_1 (enum tree_code code, tree op0, tree op1,
> - rtx_code_label *label, int prob)
> + rtx_code_label *label, profile_probability prob)
> {
> do_jump_1 (code, op0, op1, NULL, label, prob);
> }
> @@ -200,12 +194,12 @@ prefer_and_bit_test (machine_mode mode,
>
> /* Subroutine of do_jump, dealing with exploded comparisons of the type
> OP0 CODE OP1 . IF_FALSE_LABEL and IF_TRUE_LABEL like in do_jump.
> - PROB is probability of jump to if_true_label, or -1 if unknown. */
> + PROB is probability of jump to if_true_label. */
>
> void
> do_jump_1 (enum tree_code code, tree op0, tree op1,
> rtx_code_label *if_false_label, rtx_code_label *if_true_label,
> - int prob)
> + profile_probability prob)
> {
> machine_mode mode;
> rtx_code_label *drop_through_label = 0;
> @@ -222,7 +216,8 @@ do_jump_1 (enum tree_code code, tree op0
> != MODE_COMPLEX_INT);
>
> if (integer_zerop (op1))
> - do_jump (op0, if_true_label, if_false_label, inv (prob));
> + do_jump (op0, if_true_label, if_false_label,
> + prob.invert ());
> else if (GET_MODE_CLASS (TYPE_MODE (inner_type)) == MODE_INT
> && !can_compare_p (EQ, TYPE_MODE (inner_type), ccp_jump))
> do_jump_by_parts_equality (op0, op1, if_false_label, if_true_label,
> @@ -247,7 +242,7 @@ do_jump_1 (enum tree_code code, tree op0
> else if (GET_MODE_CLASS (TYPE_MODE (inner_type)) == MODE_INT
> && !can_compare_p (NE, TYPE_MODE (inner_type), ccp_jump))
> do_jump_by_parts_equality (op0, op1, if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> else
> do_compare_and_jump (op0, op1, NE, NE, if_false_label, if_true_label,
> prob);
> @@ -270,7 +265,7 @@ do_jump_1 (enum tree_code code, tree op0
> if (GET_MODE_CLASS (mode) == MODE_INT
> && ! can_compare_p (LE, mode, ccp_jump))
> do_jump_by_parts_greater (op0, op1, 0, if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> else
> do_compare_and_jump (op0, op1, LE, LEU, if_false_label, if_true_label,
> prob);
> @@ -292,7 +287,7 @@ do_jump_1 (enum tree_code code, tree op0
> if (GET_MODE_CLASS (mode) == MODE_INT
> && ! can_compare_p (GE, mode, ccp_jump))
> do_jump_by_parts_greater (op0, op1, 1, if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> else
> do_compare_and_jump (op0, op1, GE, GEU, if_false_label, if_true_label,
> prob);
> @@ -346,17 +341,17 @@ do_jump_1 (enum tree_code code, tree op0
> half of the total probability of being false, so its jump has a false
> probability of half the total, relative to the probability we
> reached it (i.e. the first condition was true). */
> - int op0_prob = -1;
> - int op1_prob = -1;
> - if (prob != -1)
> + profile_probability op0_prob = profile_probability::uninitialized ();
> + profile_probability op1_prob = profile_probability::uninitialized ();
> + if (prob.initialized_p ())
> {
> - int false_prob = inv (prob);
> - int op0_false_prob = false_prob / 2;
> - int op1_false_prob = GCOV_COMPUTE_SCALE ((false_prob / 2),
> - inv (op0_false_prob));
> + profile_probability false_prob = prob.invert ();
> + profile_probability op0_false_prob = false_prob.apply_scale (1, 2);
> + profile_probability op1_false_prob = false_prob.apply_scale (1, 2)
> + / op0_false_prob.invert ();
> /* Get the probability that each jump below is true. */
> - op0_prob = inv (op0_false_prob);
> - op1_prob = inv (op1_false_prob);
> + op0_prob = op0_false_prob.invert ();
> + op1_prob = op1_false_prob.invert ();
> }
> if (if_false_label == NULL)
> {
> @@ -379,12 +374,12 @@ do_jump_1 (enum tree_code code, tree op0
> The second condition has the other half of the total probability,
> so its jump has a probability of half the total, relative to
> the probability we reached it (i.e. the first condition was false). */
> - int op0_prob = -1;
> - int op1_prob = -1;
> - if (prob != -1)
> + profile_probability op0_prob = profile_probability::uninitialized ();
> + profile_probability op1_prob = profile_probability::uninitialized ();
> + if (prob.initialized_p ())
> {
> - op0_prob = prob / 2;
> - op1_prob = GCOV_COMPUTE_SCALE ((prob / 2), inv (op0_prob));
> + op0_prob = prob.apply_scale (1, 2);
> + op1_prob = prob.apply_scale (1, 2) / op0_prob.invert ();
> }
> if (if_true_label == NULL)
> {
> @@ -420,11 +415,11 @@ do_jump_1 (enum tree_code code, tree op0
> actually perform a jump. An example where there is no jump
> is when EXP is `(foo (), 0)' and IF_FALSE_LABEL is null.
>
> - PROB is probability of jump to if_true_label, or -1 if unknown. */
> + PROB is probability of jump to if_true_label. */
>
> void
> do_jump (tree exp, rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label, profile_probability prob)
> {
> enum tree_code code = TREE_CODE (exp);
> rtx temp;
> @@ -481,7 +476,7 @@ do_jump (tree exp, rtx_code_label *if_fa
>
> case TRUTH_NOT_EXPR:
> do_jump (TREE_OPERAND (exp, 0), if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> break;
>
> case COND_EXPR:
> @@ -497,7 +492,8 @@ do_jump (tree exp, rtx_code_label *if_fa
> }
>
> do_pending_stack_adjust ();
> - do_jump (TREE_OPERAND (exp, 0), label1, NULL, -1);
> + do_jump (TREE_OPERAND (exp, 0), label1, NULL,
> + profile_probability::uninitialized ());
> do_jump (TREE_OPERAND (exp, 1), if_false_label, if_true_label, prob);
> emit_label (label1);
> do_jump (TREE_OPERAND (exp, 2), if_false_label, if_true_label, prob);
> @@ -542,7 +538,7 @@ do_jump (tree exp, rtx_code_label *if_fa
> {
> tree exp0 = TREE_OPERAND (exp, 0);
> rtx_code_label *set_label, *clr_label;
> - int setclr_prob = prob;
> + profile_probability setclr_prob = prob;
>
> /* Strip narrowing integral type conversions. */
> while (CONVERT_EXPR_P (exp0)
> @@ -558,7 +554,7 @@ do_jump (tree exp, rtx_code_label *if_fa
> exp0 = TREE_OPERAND (exp0, 0);
> clr_label = if_true_label;
> set_label = if_false_label;
> - setclr_prob = inv (prob);
> + setclr_prob = prob.invert ();
> }
> else
> {
> @@ -673,7 +669,7 @@ static void
> do_jump_by_parts_greater_rtx (machine_mode mode, int unsignedp, rtx op0,
> rtx op1, rtx_code_label *if_false_label,
> rtx_code_label *if_true_label,
> - int prob)
> + profile_probability prob)
> {
> int nwords = (GET_MODE_SIZE (mode) / UNITS_PER_WORD);
> rtx_code_label *drop_through_label = 0;
> @@ -703,7 +699,7 @@ do_jump_by_parts_greater_rtx (machine_mo
> if_false_label = drop_through_label;
> drop_through_if_true = false;
> drop_through_if_false = true;
> - prob = inv (prob);
> + prob = prob.invert ();
> }
>
> /* Compare a word at a time, high order first. */
> @@ -733,7 +729,8 @@ do_jump_by_parts_greater_rtx (machine_mo
>
> /* Consider lower words only if these are equal. */
> do_compare_rtx_and_jump (op0_word, op1_word, NE, unsignedp, word_mode,
> - NULL_RTX, NULL, if_false_label, inv (prob));
> + NULL_RTX, NULL, if_false_label,
> + prob.invert ());
> }
>
> if (!drop_through_if_false)
> @@ -750,7 +747,8 @@ do_jump_by_parts_greater_rtx (machine_mo
> static void
> do_jump_by_parts_greater (tree treeop0, tree treeop1, int swap,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label,
> + profile_probability prob)
> {
> rtx op0 = expand_normal (swap ? treeop1 : treeop0);
> rtx op1 = expand_normal (swap ? treeop0 : treeop1);
> @@ -769,7 +767,8 @@ do_jump_by_parts_greater (tree treeop0,
> static void
> do_jump_by_parts_zero_rtx (machine_mode mode, rtx op0,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label,
> + profile_probability prob)
> {
> int nwords = GET_MODE_SIZE (mode) / UNITS_PER_WORD;
> rtx part;
> @@ -819,7 +818,8 @@ do_jump_by_parts_zero_rtx (machine_mode
> static void
> do_jump_by_parts_equality_rtx (machine_mode mode, rtx op0, rtx op1,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label,
> + profile_probability prob)
> {
> int nwords = (GET_MODE_SIZE (mode) / UNITS_PER_WORD);
> rtx_code_label *drop_through_label = NULL;
> @@ -859,7 +859,8 @@ do_jump_by_parts_equality_rtx (machine_m
> static void
> do_jump_by_parts_equality (tree treeop0, tree treeop1,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label,
> + profile_probability prob)
> {
> rtx op0 = expand_normal (treeop0);
> rtx op1 = expand_normal (treeop1);
> @@ -956,7 +957,8 @@ void
> do_compare_rtx_and_jump (rtx op0, rtx op1, enum rtx_code code, int unsignedp,
> machine_mode mode, rtx size,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label,
> + profile_probability prob)
> {
> rtx tem;
> rtx_code_label *dummy_label = NULL;
> @@ -983,7 +985,7 @@ do_compare_rtx_and_jump (rtx op0, rtx op
> {
> std::swap (if_true_label, if_false_label);
> code = rcode;
> - prob = inv (prob);
> + prob = prob.invert ();
> }
> }
>
> @@ -1035,7 +1037,7 @@ do_compare_rtx_and_jump (rtx op0, rtx op
> case LEU:
> do_jump_by_parts_greater_rtx (mode, 1, op0, op1,
> if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> break;
>
> case GTU:
> @@ -1046,7 +1048,7 @@ do_compare_rtx_and_jump (rtx op0, rtx op
> case GEU:
> do_jump_by_parts_greater_rtx (mode, 1, op1, op0,
> if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> break;
>
> case LT:
> @@ -1057,7 +1059,7 @@ do_compare_rtx_and_jump (rtx op0, rtx op
> case LE:
> do_jump_by_parts_greater_rtx (mode, 0, op0, op1,
> if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> break;
>
> case GT:
> @@ -1068,7 +1070,7 @@ do_compare_rtx_and_jump (rtx op0, rtx op
> case GE:
> do_jump_by_parts_greater_rtx (mode, 0, op1, op0,
> if_true_label, if_false_label,
> - inv (prob));
> + prob.invert ());
> break;
>
> case EQ:
> @@ -1078,7 +1080,8 @@ do_compare_rtx_and_jump (rtx op0, rtx op
>
> case NE:
> do_jump_by_parts_equality_rtx (mode, op0, op1, if_true_label,
> - if_false_label, inv (prob));
> + if_false_label,
> + prob.invert ());
> break;
>
> default:
> @@ -1115,11 +1118,13 @@ do_compare_rtx_and_jump (rtx op0, rtx op
>
> else
> {
> - int first_prob = prob;
> + profile_probability first_prob = prob;
> if (first_code == UNORDERED)
> - first_prob = REG_BR_PROB_BASE / 100;
> + first_prob = profile_probability::guessed_always ().apply_scale
> + (1, 100);
> else if (first_code == ORDERED)
> - first_prob = REG_BR_PROB_BASE - REG_BR_PROB_BASE / 100;
> + first_prob = profile_probability::guessed_always ().apply_scale
> + (99, 100);
> if (and_them)
> {
> rtx_code_label *dest_label;
> @@ -1165,7 +1170,7 @@ static void
> do_compare_and_jump (tree treeop0, tree treeop1, enum rtx_code signed_code,
> enum rtx_code unsigned_code,
> rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob)
> + rtx_code_label *if_true_label, profile_probability prob)
> {
> rtx op0, op1;
> tree type;
> Index: dojump.h
> ===================================================================
> --- dojump.h (revision 249769)
> +++ dojump.h (working copy)
> @@ -57,23 +57,26 @@ extern void save_pending_stack_adjust (s
> extern void restore_pending_stack_adjust (saved_pending_stack_adjust *);
>
> /* Generate code to evaluate EXP and jump to LABEL if the value is zero. */
> -extern void jumpifnot (tree exp, rtx_code_label *label, int prob);
> -extern void jumpifnot_1 (enum tree_code, tree, tree, rtx_code_label *, int);
> +extern void jumpifnot (tree exp, rtx_code_label *label,
> + profile_probability prob);
> +extern void jumpifnot_1 (enum tree_code, tree, tree, rtx_code_label *,
> + profile_probability);
>
> /* Generate code to evaluate EXP and jump to LABEL if the value is nonzero. */
> -extern void jumpif (tree exp, rtx_code_label *label, int prob);
> -extern void jumpif_1 (enum tree_code, tree, tree, rtx_code_label *, int);
> +extern void jumpif (tree exp, rtx_code_label *label, profile_probability prob);
> +extern void jumpif_1 (enum tree_code, tree, tree, rtx_code_label *,
> + profile_probability);
>
> /* Generate code to evaluate EXP and jump to IF_FALSE_LABEL if
> the result is zero, or IF_TRUE_LABEL if the result is one. */
> extern void do_jump (tree exp, rtx_code_label *if_false_label,
> - rtx_code_label *if_true_label, int prob);
> + rtx_code_label *if_true_label, profile_probability prob);
> extern void do_jump_1 (enum tree_code, tree, tree, rtx_code_label *,
> - rtx_code_label *, int);
> + rtx_code_label *, profile_probability);
>
> extern void do_compare_rtx_and_jump (rtx, rtx, enum rtx_code, int,
> machine_mode, rtx, rtx_code_label *,
> - rtx_code_label *, int);
> + rtx_code_label *, profile_probability);
>
> extern bool split_comparison (enum rtx_code, machine_mode,
> enum rtx_code *, enum rtx_code *);
> Index: dwarf2cfi.c
> ===================================================================
> --- dwarf2cfi.c (revision 249769)
> +++ dwarf2cfi.c (working copy)
> @@ -35,6 +35,7 @@ along with GCC; see the file COPYING3.
> #include "common/common-target.h"
>
> #include "except.h" /* expand_builtin_dwarf_sp_column */
> +#include "profile-count.h" /* For expr.h */
> #include "expr.h" /* init_return_column_size */
> #include "output.h" /* asm_out_file */
> #include "debug.h" /* dwarf2out_do_frame, dwarf2out_do_cfi_asm */
> Index: except.c
> ===================================================================
> --- except.c (revision 249769)
> +++ except.c (working copy)
> @@ -974,7 +974,6 @@ dw2_build_landing_pads (void)
> {
> basic_block bb;
> rtx_insn *seq;
> - edge e;
>
> if (lp == NULL || lp->post_landing_pad == NULL)
> continue;
> @@ -991,9 +990,7 @@ dw2_build_landing_pads (void)
> end_sequence ();
>
> bb = emit_to_new_bb_before (seq, label_rtx (lp->post_landing_pad));
> - e = make_edge (bb, bb->next_bb, e_flags);
> - e->count = bb->count;
> - e->probability = REG_BR_PROB_BASE;
> + make_single_succ_edge (bb, bb->next_bb, e_flags);
> if (current_loops)
> {
> struct loop *loop = bb->next_bb->loop_father;
> @@ -1258,7 +1255,6 @@ sjlj_emit_dispatch_table (rtx_code_label
> rtx_insn *seq;
> basic_block bb;
> eh_region r;
> - edge e;
> int i, disp_index;
> vec<tree> dispatch_labels = vNULL;
>
> @@ -1346,9 +1342,7 @@ sjlj_emit_dispatch_table (rtx_code_label
>
> rtx_insn *before = label_rtx (lp->post_landing_pad);
> bb = emit_to_new_bb_before (seq2, before);
> - e = make_edge (bb, bb->next_bb, EDGE_FALLTHRU);
> - e->count = bb->count;
> - e->probability = REG_BR_PROB_BASE;
> + make_single_succ_edge (bb, bb->next_bb, EDGE_FALLTHRU);
> if (current_loops)
> {
> struct loop *loop = bb->next_bb->loop_father;
> @@ -1386,9 +1380,7 @@ sjlj_emit_dispatch_table (rtx_code_label
> bb = emit_to_new_bb_before (seq, first_reachable_label);
> if (num_dispatch == 1)
> {
> - e = make_edge (bb, bb->next_bb, EDGE_FALLTHRU);
> - e->count = bb->count;
> - e->probability = REG_BR_PROB_BASE;
> + make_single_succ_edge (bb, bb->next_bb, EDGE_FALLTHRU);
> if (current_loops)
> {
> struct loop *loop = bb->next_bb->loop_father;
> Index: explow.c
> ===================================================================
> --- explow.c (revision 249769)
> +++ explow.c (working copy)
> @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3.
> #include "memmodel.h"
> #include "tm_p.h"
> #include "expmed.h"
> +#include "profile-count.h"
> #include "optabs.h"
> #include "emit-rtl.h"
> #include "recog.h"
> Index: expmed.c
> ===================================================================
> --- expmed.c (revision 249769)
> +++ expmed.c (working copy)
> @@ -5905,7 +5905,8 @@ emit_store_flag_force (rtx target, enum
> {
> label = gen_label_rtx ();
> do_compare_rtx_and_jump (target, const0_rtx, EQ, unsignedp, mode,
> - NULL_RTX, NULL, label, -1);
> + NULL_RTX, NULL, label,
> + profile_probability::uninitialized ());
> emit_move_insn (target, trueval);
> emit_label (label);
> return target;
> @@ -5943,7 +5944,7 @@ emit_store_flag_force (rtx target, enum
> emit_move_insn (target, trueval);
> label = gen_label_rtx ();
> do_compare_rtx_and_jump (op0, op1, code, unsignedp, mode, NULL_RTX, NULL,
> - label, -1);
> + label, profile_probability::uninitialized ());
>
> emit_move_insn (target, falseval);
> emit_label (label);
> @@ -5961,5 +5962,5 @@ do_cmp_and_jump (rtx arg1, rtx arg2, enu
> {
> int unsignedp = (op == LTU || op == LEU || op == GTU || op == GEU);
> do_compare_rtx_and_jump (arg1, arg2, op, unsignedp, mode, NULL_RTX,
> - NULL, label, -1);
> + NULL, label, profile_probability::uninitialized ());
> }
> Index: expr.c
> ===================================================================
> --- expr.c (revision 249769)
> +++ expr.c (working copy)
> @@ -96,7 +96,8 @@ static rtx do_store_flag (sepops, rtx, m
> #ifdef PUSH_ROUNDING
> static void emit_single_push_insn (machine_mode, rtx, tree);
> #endif
> -static void do_tablejump (rtx, machine_mode, rtx, rtx, rtx, int);
> +static void do_tablejump (rtx, machine_mode, rtx, rtx, rtx,
> + profile_probability);
> static rtx const_vector_from_tree (tree);
> static rtx const_scalar_mask_from_tree (tree);
> static tree tree_expr_size (const_tree);
> @@ -1452,7 +1453,7 @@ compare_by_pieces_d::generate (rtx op0,
> m_accumulator = NULL_RTX;
> }
> do_compare_rtx_and_jump (op0, op1, NE, true, mode, NULL_RTX, NULL,
> - m_fail_label, -1);
> + m_fail_label, profile_probability::uninitialized ());
> }
>
> /* Return true if MODE can be used for a set of moves and comparisons,
> @@ -1484,7 +1485,8 @@ compare_by_pieces_d::finish_mode (machin
> {
> if (m_accumulator != NULL_RTX)
> do_compare_rtx_and_jump (m_accumulator, const0_rtx, NE, true, mode,
> - NULL_RTX, NULL, m_fail_label, -1);
> + NULL_RTX, NULL, m_fail_label,
> + profile_probability::uninitialized ());
> }
>
> /* Generate several move instructions to compare LEN bytes from blocks
> @@ -1845,7 +1847,9 @@ emit_block_move_via_loop (rtx x, rtx y,
> emit_label (cmp_label);
>
> emit_cmp_and_jump_insns (iter, size, LT, NULL_RTX, iter_mode,
> - true, top_label, REG_BR_PROB_BASE * 90 / 100);
> + true, top_label,
> + profile_probability::guessed_always ()
> + .apply_scale (9, 10));
> }
>
> /* Expand a call to memcpy or memmove or memcmp, and return the result.
> @@ -5402,7 +5406,8 @@ store_expr_with_bounds (tree exp, rtx ta
>
> do_pending_stack_adjust ();
> NO_DEFER_POP;
> - jumpifnot (TREE_OPERAND (exp, 0), lab1, -1);
> + jumpifnot (TREE_OPERAND (exp, 0), lab1,
> + profile_probability::uninitialized ());
> store_expr_with_bounds (TREE_OPERAND (exp, 1), target, call_param_p,
> nontemporal, reverse, btarget);
> emit_jump_insn (targetm.gen_jump (lab2));
> @@ -6504,7 +6509,8 @@ store_constructor (tree exp, rtx target,
> /* Generate a conditional jump to exit the loop. */
> exit_cond = build2 (LT_EXPR, integer_type_node,
> index, hi_index);
> - jumpif (exit_cond, loop_end, -1);
> + jumpif (exit_cond, loop_end,
> + profile_probability::uninitialized ());
>
> /* Update the loop counter, and jump to the head of
> the loop. */
> @@ -9043,7 +9049,7 @@ expand_expr_real_2 (sepops ops, rtx targ
> lab = gen_label_rtx ();
> do_compare_rtx_and_jump (target, cmpop1, comparison_code,
> unsignedp, mode, NULL_RTX, NULL, lab,
> - -1);
> + profile_probability::uninitialized ());
> }
> emit_move_insn (target, op1);
> emit_label (lab);
> @@ -9272,7 +9278,8 @@ expand_expr_real_2 (sepops ops, rtx targ
> emit_move_insn (target, const0_rtx);
>
> rtx_code_label *lab1 = gen_label_rtx ();
> - jumpifnot_1 (code, treeop0, treeop1, lab1, -1);
> + jumpifnot_1 (code, treeop0, treeop1, lab1,
> + profile_probability::uninitialized ());
>
> if (TYPE_PRECISION (type) == 1 && !TYPE_UNSIGNED (type))
> emit_move_insn (target, constm1_rtx);
> @@ -9523,7 +9530,8 @@ expand_expr_real_2 (sepops ops, rtx targ
> NO_DEFER_POP;
> rtx_code_label *lab0 = gen_label_rtx ();
> rtx_code_label *lab1 = gen_label_rtx ();
> - jumpifnot (treeop0, lab0, -1);
> + jumpifnot (treeop0, lab0,
> + profile_probability::uninitialized ());
> store_expr (treeop1, temp,
> modifier == EXPAND_STACK_PARM,
> false, false);
> @@ -11042,7 +11050,8 @@ expand_expr_real_1 (tree exp, rtx target
> int value = TREE_CODE (rhs) == BIT_IOR_EXPR;
> do_jump (TREE_OPERAND (rhs, 1),
> value ? label : 0,
> - value ? 0 : label, -1);
> + value ? 0 : label,
> + profile_probability::uninitialized ());
> expand_assignment (lhs, build_int_cst (TREE_TYPE (rhs), value),
> false);
> do_pending_stack_adjust ();
> @@ -11512,7 +11521,7 @@ do_store_flag (sepops ops, rtx target, m
> int
> try_casesi (tree index_type, tree index_expr, tree minval, tree range,
> rtx table_label, rtx default_label, rtx fallback_label,
> - int default_probability)
> + profile_probability default_probability)
> {
> struct expand_operand ops[5];
> machine_mode index_mode = SImode;
> @@ -11582,7 +11591,7 @@ try_casesi (tree index_type, tree index_
>
> static void
> do_tablejump (rtx index, machine_mode mode, rtx range, rtx table_label,
> - rtx default_label, int default_probability)
> + rtx default_label, profile_probability default_probability)
> {
> rtx temp, vector;
>
> @@ -11645,7 +11654,8 @@ do_tablejump (rtx index, machine_mode mo
>
> int
> try_tablejump (tree index_type, tree index_expr, tree minval, tree range,
> - rtx table_label, rtx default_label, int default_probability)
> + rtx table_label, rtx default_label,
> + profile_probability default_probability)
> {
> rtx index;
>
> Index: expr.h
> ===================================================================
> --- expr.h (revision 249769)
> +++ expr.h (working copy)
> @@ -288,8 +288,8 @@ expand_normal (tree exp)
> extern tree string_constant (tree, tree *);
>
> /* Two different ways of generating switch statements. */
> -extern int try_casesi (tree, tree, tree, tree, rtx, rtx, rtx, int);
> -extern int try_tablejump (tree, tree, tree, tree, rtx, rtx, int);
> +extern int try_casesi (tree, tree, tree, tree, rtx, rtx, rtx, profile_probability);
> +extern int try_tablejump (tree, tree, tree, tree, rtx, rtx, profile_probability);
>
> extern int safe_from_p (const_rtx, tree, int);
>
> Index: gimple-pretty-print.c
> ===================================================================
> --- gimple-pretty-print.c (revision 249769)
> +++ gimple-pretty-print.c (working copy)
> @@ -80,7 +80,7 @@ debug_gimple_stmt (gimple *gs)
> by xstrdup_for_dump. */
>
> static const char *
> -dump_probability (int frequency, profile_count &count)
> +dump_profile (int frequency, profile_count &count)
> {
> float minimum = 0.01f;
>
> @@ -102,6 +102,38 @@ dump_probability (int frequency, profile
> return ret;
> }
>
> +/* Return formatted string of a VALUE probability
> + (biased by REG_BR_PROB_BASE). Returned string is allocated
> + by xstrdup_for_dump. */
> +
> +static const char *
> +dump_probability (profile_probability probability, profile_count &count)
> +{
> + float minimum = 0.01f;
> + float fvalue = -1;
> +
> + if (probability.initialized_p ())
> + {
> + fvalue = probability.to_reg_br_prob_base () * 100.0f / REG_BR_PROB_BASE;
> + if (fvalue < minimum && probability.to_reg_br_prob_base ())
> + fvalue = minimum;
> + }
> +
> + char *buf;
> + if (count.initialized_p ())
> + asprintf (&buf, "[%.2f%%] [count: %" PRId64 "]", fvalue,
> + count.to_gcov_type ());
> + else if (probability.initialized_p ())
> + asprintf (&buf, "[%.2f%%] [count: INV]", fvalue);
> + else
> + asprintf (&buf, "[INV] [count: INV]");
> +
> + const char *ret = xstrdup_for_dump (buf);
> + free (buf);
> +
> + return ret;
> +}
> +
> /* Dump E probability to BUFFER. */
>
> static void
> @@ -1090,7 +1122,7 @@ dump_gimple_label (pretty_printer *buffe
> dump_generic_node (buffer, label, spc, flags, false);
> basic_block bb = gimple_bb (gs);
> if (bb && !(flags & TDF_GIMPLE))
> - pp_scalar (buffer, " %s", dump_probability (bb->frequency, bb->count));
> + pp_scalar (buffer, " %s", dump_profile (bb->frequency, bb->count));
> pp_colon (buffer);
> }
> if (flags & TDF_GIMPLE)
> @@ -2670,8 +2702,8 @@ dump_gimple_bb_header (FILE *outf, basic
> fprintf (outf, "%*sbb_%d:\n", indent, "", bb->index);
> else
> fprintf (outf, "%*s<bb %d> %s:\n",
> - indent, "", bb->index, dump_probability (bb->frequency,
> - bb->count));
> + indent, "", bb->index, dump_profile (bb->frequency,
> + bb->count));
> }
> }
> }
> Index: graph.c
> ===================================================================
> --- graph.c (revision 249769)
> +++ graph.c (working copy)
> @@ -136,12 +136,16 @@ draw_cfg_node_succ_edges (pretty_printer
>
> pp_printf (pp,
> "\tfn_%d_basic_block_%d:s -> fn_%d_basic_block_%d:n "
> - "[style=%s,color=%s,weight=%d,constraint=%s, label=\"[%i%%]\"];\n",
> + "[style=%s,color=%s,weight=%d,constraint=%s];\n",
> funcdef_no, e->src->index,
> funcdef_no, e->dest->index,
> style, color, weight,
> - (e->flags & (EDGE_FAKE | EDGE_DFS_BACK)) ? "false" : "true",
> - e->probability * 100 / REG_BR_PROB_BASE);
> + (e->flags & (EDGE_FAKE | EDGE_DFS_BACK)) ? "false" : "true");
> + if (e->probability.initialized_p ())
> + pp_printf (pp, ", label=\"[%i%%]\"",
> + e->probability.to_reg_br_prob_base ()
> + * 100 / REG_BR_PROB_BASE);
> + pp_printf (pp, "];\n");
> }
> pp_flush (pp);
> }
> Index: hsa-gen.c
> ===================================================================
> --- hsa-gen.c (revision 249769)
> +++ hsa-gen.c (working copy)
> @@ -6139,7 +6139,7 @@ convert_switch_statements (void)
> auto_vec <edge> new_edges;
> auto_vec <phi_definition *> phi_todo_list;
> auto_vec <profile_count> edge_counts;
> - auto_vec <int> edge_probabilities;
> + auto_vec <profile_probability> edge_probabilities;
>
> /* Investigate all labels that and PHI nodes in these edges which
> should be fixed after we add new collection of edges. */
> @@ -6231,12 +6231,12 @@ convert_switch_statements (void)
> basic_block label_bb
> = label_to_block_fn (func, CASE_LABEL (label));
> edge new_edge = make_edge (cur_bb, label_bb, EDGE_TRUE_VALUE);
> - int prob_sum = sum_slice <int> (edge_probabilities, i, labels, 0) +
> - edge_probabilities[0];
> + profile_probability prob_sum = sum_slice <profile_probability>
> + (edge_probabilities, i, labels, profile_probability::never ())
> + + edge_probabilities[0];
>
> - if (prob_sum)
> - new_edge->probability
> - = RDIV (REG_BR_PROB_BASE * edge_probabilities[i], prob_sum);
> + if (prob_sum.initialized_p ())
> + new_edge->probability = edge_probabilities[i] / prob_sum;
>
> new_edge->count = edge_counts[i];
> new_edges.safe_push (new_edge);
> @@ -6253,8 +6253,7 @@ convert_switch_statements (void)
> }
>
> edge next_edge = make_edge (cur_bb, next_bb, EDGE_FALSE_VALUE);
> - next_edge->probability
> - = inverse_probability (new_edge->probability);
> + next_edge->probability = new_edge->probability.invert ();
> next_edge->count = edge_counts[0]
> + sum_slice <profile_count> (edge_counts, i, labels,
> profile_count::zero ());
> @@ -6265,7 +6264,7 @@ convert_switch_statements (void)
> of the switch. */
> {
> edge e = make_edge (cur_bb, default_label_bb, EDGE_FALSE_VALUE);
> - e->probability = inverse_probability (new_edge->probability);
> + e->probability = new_edge->probability.invert ();
> e->count = edge_counts[0];
> new_edges.safe_insert (0, e);
> }
> Index: ifcvt.c
> ===================================================================
> --- ifcvt.c (revision 249769)
> +++ ifcvt.c (working copy)
> @@ -78,7 +78,7 @@ static int cond_exec_changed_p;
>
> /* Forward references. */
> static int count_bb_insns (const_basic_block);
> -static bool cheap_bb_rtx_cost_p (const_basic_block, int, int);
> +static bool cheap_bb_rtx_cost_p (const_basic_block, profile_probability, int);
> static rtx_insn *first_active_insn (basic_block);
> static rtx_insn *last_active_insn (basic_block, int);
> static rtx_insn *find_active_insn_before (basic_block, rtx_insn *);
> @@ -132,11 +132,14 @@ count_bb_insns (const_basic_block bb)
> plus a small fudge factor. */
>
> static bool
> -cheap_bb_rtx_cost_p (const_basic_block bb, int scale, int max_cost)
> +cheap_bb_rtx_cost_p (const_basic_block bb,
> + profile_probability prob, int max_cost)
> {
> int count = 0;
> rtx_insn *insn = BB_HEAD (bb);
> bool speed = optimize_bb_for_speed_p (bb);
> + int scale = prob.initialized_p () ? prob.to_reg_br_prob_base ()
> + : REG_BR_PROB_BASE;
>
> /* Set scale to REG_BR_PROB_BASE to void the identical scaling
> applied to insn_rtx_cost when optimizing for size. Only do
> @@ -4807,7 +4810,8 @@ find_if_case_1 (basic_block test_bb, edg
> basic_block then_bb = then_edge->dest;
> basic_block else_bb = else_edge->dest;
> basic_block new_bb;
> - int then_bb_index, then_prob;
> + int then_bb_index;
> + profile_probability then_prob;
> rtx else_target = NULL_RTX;
>
> /* If we are partitioning hot/cold basic blocks, we don't want to
> @@ -4853,10 +4857,7 @@ find_if_case_1 (basic_block test_bb, edg
> "\nIF-CASE-1 found, start %d, then %d\n",
> test_bb->index, then_bb->index);
>
> - if (then_edge->probability)
> - then_prob = REG_BR_PROB_BASE - then_edge->probability;
> - else
> - then_prob = REG_BR_PROB_BASE / 2;
> + then_prob = then_edge->probability.invert ();
>
> /* We're speculating from the THEN path, we want to make sure the cost
> of speculation is within reason. */
> @@ -4927,7 +4928,7 @@ find_if_case_2 (basic_block test_bb, edg
> basic_block then_bb = then_edge->dest;
> basic_block else_bb = else_edge->dest;
> edge else_succ;
> - int then_prob, else_prob;
> + profile_probability then_prob, else_prob;
>
> /* We do not want to speculate (empty) loop latches. */
> if (current_loops
> @@ -4973,16 +4974,8 @@ find_if_case_2 (basic_block test_bb, edg
> if (then_bb->index < NUM_FIXED_BLOCKS)
> return FALSE;
>
> - if (else_edge->probability)
> - {
> - else_prob = else_edge->probability;
> - then_prob = REG_BR_PROB_BASE - else_prob;
> - }
> - else
> - {
> - else_prob = REG_BR_PROB_BASE / 2;
> - then_prob = REG_BR_PROB_BASE / 2;
> - }
> + else_prob = else_edge->probability;
> + then_prob = else_prob.invert ();
>
> /* ELSE is predicted or SUCC(ELSE) postdominates THEN. */
> if (else_prob > then_prob)
> Index: internal-fn.c
> ===================================================================
> --- internal-fn.c (revision 249769)
> +++ internal-fn.c (working copy)
> @@ -569,7 +569,7 @@ expand_arith_overflow_result_store (tree
> gcc_assert (GET_MODE_PRECISION (tgtmode) < GET_MODE_PRECISION (mode));
> do_compare_rtx_and_jump (res, convert_modes (mode, tgtmode, lres, uns),
> EQ, true, mode, NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> emit_label (done_label);
> }
> @@ -597,7 +597,7 @@ expand_arith_overflow_result_store (tree
> }
> do_compare_rtx_and_jump (res, lres,
> EQ, true, tgtmode, NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> emit_label (done_label);
> }
> @@ -767,7 +767,7 @@ expand_addsub_overflow (location_t loc,
> tem = op1;
> do_compare_rtx_and_jump (res, tem, code == PLUS_EXPR ? GEU : LEU,
> true, mode, NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -782,7 +782,7 @@ expand_addsub_overflow (location_t loc,
> code == PLUS_EXPR ? res : op0, sgn,
> NULL_RTX, false, OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, op1, GEU, true, mode, NULL_RTX, NULL,
> - done_label, PROB_VERY_LIKELY);
> + done_label, profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -824,9 +824,9 @@ expand_addsub_overflow (location_t loc,
> else if (pos_neg == 3)
> /* If ARG0 is not known to be always positive, check at runtime. */
> do_compare_rtx_and_jump (op0, const0_rtx, LT, false, mode, NULL_RTX,
> - NULL, do_error, PROB_VERY_UNLIKELY);
> + NULL, do_error, profile_probability::very_unlikely ());
> do_compare_rtx_and_jump (op1, op0, LEU, true, mode, NULL_RTX, NULL,
> - done_label, PROB_VERY_LIKELY);
> + done_label, profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -840,7 +840,7 @@ expand_addsub_overflow (location_t loc,
> rtx tem = expand_binop (mode, add_optab, op1, sgn, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (op0, tem, LTU, true, mode, NULL_RTX, NULL,
> - done_label, PROB_VERY_LIKELY);
> + done_label, profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -852,7 +852,7 @@ expand_addsub_overflow (location_t loc,
> res = expand_binop (mode, add_optab, op0, op1, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (res, const0_rtx, LT, false, mode, NULL_RTX,
> - NULL, do_error, PROB_VERY_UNLIKELY);
> + NULL, do_error, profile_probability::very_unlikely ());
> rtx tem = op1;
> /* The operation is commutative, so we can pick operand to compare
> against. For prec <= BITS_PER_WORD, I think preferring REG operand
> @@ -866,7 +866,7 @@ expand_addsub_overflow (location_t loc,
> : CONST_SCALAR_INT_P (op0))
> tem = op0;
> do_compare_rtx_and_jump (res, tem, GEU, true, mode, NULL_RTX, NULL,
> - done_label, PROB_VERY_LIKELY);
> + done_label, profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -894,7 +894,7 @@ expand_addsub_overflow (location_t loc,
> ? and_optab : ior_optab,
> op0, res, NULL_RTX, false, OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, const0_rtx, GE, false, mode, NULL,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> }
> else
> {
> @@ -902,17 +902,17 @@ expand_addsub_overflow (location_t loc,
> do_compare_rtx_and_jump (op1, const0_rtx,
> code == MINUS_EXPR ? GE : LT, false, mode,
> NULL_RTX, NULL, do_ior_label,
> - PROB_EVEN);
> + profile_probability::even ());
> tem = expand_binop (mode, and_optab, op0, res, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> emit_jump (do_error);
> emit_label (do_ior_label);
> tem = expand_binop (mode, ior_optab, op0, res, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> }
> goto do_error_label;
> }
> @@ -926,13 +926,13 @@ expand_addsub_overflow (location_t loc,
> OPTAB_LIB_WIDEN);
> rtx_code_label *op0_geu_op1 = gen_label_rtx ();
> do_compare_rtx_and_jump (op0, op1, GEU, true, mode, NULL_RTX, NULL,
> - op0_geu_op1, PROB_EVEN);
> + op0_geu_op1, profile_probability::even ());
> do_compare_rtx_and_jump (res, const0_rtx, LT, false, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> emit_jump (do_error);
> emit_label (op0_geu_op1);
> do_compare_rtx_and_jump (res, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> goto do_error_label;
> }
>
> @@ -960,7 +960,7 @@ expand_addsub_overflow (location_t loc,
> && JUMP_P (last)
> && any_condjump_p (last)
> && !find_reg_note (last, REG_BR_PROB, 0))
> - add_int_reg_note (last, REG_BR_PROB, PROB_VERY_UNLIKELY);
> + add_int_reg_note (last, REG_BR_PROB, PROB_UNLIKELY);
> emit_jump (done_label);
> goto do_error_label;
> }
> @@ -1020,7 +1020,7 @@ expand_addsub_overflow (location_t loc,
>
> /* No overflow if the result has bit sign cleared. */
> do_compare_rtx_and_jump (tem, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> }
>
> /* Compare the result of the operation with the first operand.
> @@ -1031,7 +1031,7 @@ expand_addsub_overflow (location_t loc,
> do_compare_rtx_and_jump (res, op0,
> (pos_neg == 1) ^ (code == MINUS_EXPR) ? GE : LE,
> false, mode, NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> }
>
> do_error_label:
> @@ -1128,7 +1128,7 @@ expand_neg_overflow (location_t loc, tre
> /* Compare the operand with the most negative value. */
> rtx minv = expand_normal (TYPE_MIN_VALUE (TREE_TYPE (arg1)));
> do_compare_rtx_and_jump (op1, minv, NE, true, mode, NULL_RTX, NULL,
> - done_label, PROB_VERY_LIKELY);
> + done_label, profile_probability::very_likely ());
> }
>
> emit_label (do_error);
> @@ -1261,15 +1261,15 @@ expand_mul_overflow (location_t loc, tre
> ops.location = loc;
> res = expand_expr_real_2 (&ops, NULL_RTX, mode, EXPAND_NORMAL);
> do_compare_rtx_and_jump (op1, const0_rtx, EQ, true, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> goto do_error_label;
> case 3:
> rtx_code_label *do_main_label;
> do_main_label = gen_label_rtx ();
> do_compare_rtx_and_jump (op0, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, do_main_label, PROB_VERY_LIKELY);
> + NULL, do_main_label, profile_probability::very_likely ());
> do_compare_rtx_and_jump (op1, const0_rtx, EQ, true, mode, NULL_RTX,
> - NULL, do_main_label, PROB_VERY_LIKELY);
> + NULL, do_main_label, profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> emit_label (do_main_label);
> goto do_main;
> @@ -1306,15 +1306,15 @@ expand_mul_overflow (location_t loc, tre
> ops.location = loc;
> res = expand_expr_real_2 (&ops, NULL_RTX, mode, EXPAND_NORMAL);
> do_compare_rtx_and_jump (op0, const0_rtx, EQ, true, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> do_compare_rtx_and_jump (op0, constm1_rtx, NE, true, mode, NULL_RTX,
> - NULL, do_error, PROB_VERY_UNLIKELY);
> + NULL, do_error, profile_probability::very_unlikely ());
> int prec;
> prec = GET_MODE_PRECISION (mode);
> rtx sgn;
> sgn = immed_wide_int_const (wi::min_value (prec, SIGNED), mode);
> do_compare_rtx_and_jump (op1, sgn, EQ, true, mode, NULL_RTX,
> - NULL, done_label, PROB_VERY_LIKELY);
> + NULL, done_label, profile_probability::very_likely ());
> goto do_error_label;
> case 3:
> /* Rest of handling of this case after res is computed. */
> @@ -1361,7 +1361,7 @@ expand_mul_overflow (location_t loc, tre
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, const0_rtx, EQ, true, mode,
> NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> goto do_error_label;
> }
> /* The general case, do all the needed comparisons at runtime. */
> @@ -1378,7 +1378,7 @@ expand_mul_overflow (location_t loc, tre
> tem = expand_binop (mode, and_optab, op0, op1, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, after_negate_label, PROB_VERY_LIKELY);
> + NULL, after_negate_label, profile_probability::very_likely ());
> /* Both arguments negative here, negate them and continue with
> normal unsigned overflow checking multiplication. */
> emit_move_insn (op0, expand_unop (mode, neg_optab, op0,
> @@ -1394,13 +1394,13 @@ expand_mul_overflow (location_t loc, tre
> tem2 = expand_binop (mode, xor_optab, op0, op1, NULL_RTX, false,
> OPTAB_LIB_WIDEN);
> do_compare_rtx_and_jump (tem2, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, do_main_label, PROB_VERY_LIKELY);
> + NULL, do_main_label, profile_probability::very_likely ());
> /* One argument is negative here, the other positive. This
> overflows always, unless one of the arguments is 0. But
> if e.g. s2 is 0, (U) s1 * 0 doesn't overflow, whatever s1
> is, thus we can keep do_main code oring in overflow as is. */
> do_compare_rtx_and_jump (tem, const0_rtx, EQ, true, mode, NULL_RTX,
> - NULL, do_main_label, PROB_VERY_LIKELY);
> + NULL, do_main_label, profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> emit_label (do_main_label);
> goto do_main;
> @@ -1467,7 +1467,7 @@ expand_mul_overflow (location_t loc, tre
> HIPART is non-zero. */
> do_compare_rtx_and_jump (hipart, const0_rtx, EQ, true, mode,
> NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> else
> {
> rtx signbit = expand_shift (RSHIFT_EXPR, mode, res, prec - 1,
> @@ -1477,7 +1477,7 @@ expand_mul_overflow (location_t loc, tre
> HIPART is different from RES < 0 ? -1 : 0. */
> do_compare_rtx_and_jump (signbit, hipart, EQ, true, mode,
> NULL_RTX, NULL, done_label,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> }
> }
> else if (hmode != BLKmode && 2 * GET_MODE_PRECISION (hmode) == prec)
> @@ -1570,12 +1570,12 @@ expand_mul_overflow (location_t loc, tre
> if (!op0_small_p)
> do_compare_rtx_and_jump (signbit0, hipart0, NE, true, hmode,
> NULL_RTX, NULL, large_op0,
> - PROB_UNLIKELY);
> + profile_probability::unlikely ());
>
> if (!op1_small_p)
> do_compare_rtx_and_jump (signbit1, hipart1, NE, true, hmode,
> NULL_RTX, NULL, small_op0_large_op1,
> - PROB_UNLIKELY);
> + profile_probability::unlikely ());
>
> /* If both op0 and op1 are sign (!uns) or zero (uns) extended from
> hmode to mode, the multiplication will never overflow. We can
> @@ -1621,7 +1621,7 @@ expand_mul_overflow (location_t loc, tre
> if (!op1_small_p)
> do_compare_rtx_and_jump (signbit1, hipart1, NE, true, hmode,
> NULL_RTX, NULL, both_ops_large,
> - PROB_UNLIKELY);
> + profile_probability::unlikely ());
>
> /* If op1 is sign (!uns) or zero (uns) extended from hmode to mode,
> but op0 is not, prepare larger, hipart and lopart pseudos and
> @@ -1658,7 +1658,7 @@ expand_mul_overflow (location_t loc, tre
> else if (larger_sign != -1)
> do_compare_rtx_and_jump (hipart, const0_rtx, GE, false, hmode,
> NULL_RTX, NULL, after_hipart_neg,
> - PROB_EVEN);
> + profile_probability::even ());
>
> tem = convert_modes (mode, hmode, lopart, 1);
> tem = expand_shift (LSHIFT_EXPR, mode, tem, hprec, NULL_RTX, 1);
> @@ -1674,7 +1674,7 @@ expand_mul_overflow (location_t loc, tre
> else if (smaller_sign != -1)
> do_compare_rtx_and_jump (lopart, const0_rtx, GE, false, hmode,
> NULL_RTX, NULL, after_lopart_neg,
> - PROB_EVEN);
> + profile_probability::even ());
>
> tem = expand_simple_binop (mode, MINUS, loxhi, larger, NULL_RTX,
> 1, OPTAB_DIRECT);
> @@ -1704,7 +1704,7 @@ expand_mul_overflow (location_t loc, tre
>
> do_compare_rtx_and_jump (signbitloxhi, hipartloxhi, NE, true, hmode,
> NULL_RTX, NULL, do_overflow,
> - PROB_VERY_UNLIKELY);
> + profile_probability::very_unlikely ());
>
> /* res = (loxhi << (bitsize / 2)) | (hmode) lo0xlo1; */
> rtx loxhishifted = expand_shift (LSHIFT_EXPR, mode, loxhi, hprec,
> @@ -1741,7 +1741,7 @@ expand_mul_overflow (location_t loc, tre
> NULL_RTX, 1, OPTAB_DIRECT);
> do_compare_rtx_and_jump (tem, const1_rtx, GTU, true, hmode,
> NULL_RTX, NULL, do_error,
> - PROB_VERY_UNLIKELY);
> + profile_probability::very_unlikely ());
> }
>
> if (!op1_medium_p)
> @@ -1750,7 +1750,7 @@ expand_mul_overflow (location_t loc, tre
> NULL_RTX, 1, OPTAB_DIRECT);
> do_compare_rtx_and_jump (tem, const1_rtx, GTU, true, hmode,
> NULL_RTX, NULL, do_error,
> - PROB_VERY_UNLIKELY);
> + profile_probability::very_unlikely ());
> }
>
> /* At this point hipart{0,1} are both in [-1, 0]. If they are
> @@ -1761,18 +1761,18 @@ expand_mul_overflow (location_t loc, tre
> else if (op0_sign == 1 || op1_sign == 1)
> do_compare_rtx_and_jump (hipart0, hipart1, NE, true, hmode,
> NULL_RTX, NULL, hipart_different,
> - PROB_EVEN);
> + profile_probability::even ());
>
> do_compare_rtx_and_jump (res, const0_rtx, LT, false, mode,
> NULL_RTX, NULL, do_error,
> - PROB_VERY_UNLIKELY);
> + profile_probability::very_unlikely ());
> emit_jump (done_label);
>
> emit_label (hipart_different);
>
> do_compare_rtx_and_jump (res, const0_rtx, GE, false, mode,
> NULL_RTX, NULL, do_error,
> - PROB_VERY_UNLIKELY);
> + profile_probability::very_unlikely ());
> emit_jump (done_label);
> }
>
> @@ -1817,7 +1817,7 @@ expand_mul_overflow (location_t loc, tre
> {
> rtx_code_label *all_done_label = gen_label_rtx ();
> do_compare_rtx_and_jump (res, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, all_done_label, PROB_VERY_LIKELY);
> + NULL, all_done_label, profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> emit_label (all_done_label);
> }
> @@ -1828,14 +1828,14 @@ expand_mul_overflow (location_t loc, tre
> rtx_code_label *all_done_label = gen_label_rtx ();
> rtx_code_label *set_noovf = gen_label_rtx ();
> do_compare_rtx_and_jump (op1, const0_rtx, GE, false, mode, NULL_RTX,
> - NULL, all_done_label, PROB_VERY_LIKELY);
> + NULL, all_done_label, profile_probability::very_likely ());
> expand_arith_set_overflow (lhs, target);
> do_compare_rtx_and_jump (op0, const0_rtx, EQ, true, mode, NULL_RTX,
> - NULL, set_noovf, PROB_VERY_LIKELY);
> + NULL, set_noovf, profile_probability::very_likely ());
> do_compare_rtx_and_jump (op0, constm1_rtx, NE, true, mode, NULL_RTX,
> - NULL, all_done_label, PROB_VERY_UNLIKELY);
> + NULL, all_done_label, profile_probability::very_unlikely ());
> do_compare_rtx_and_jump (op1, res, NE, true, mode, NULL_RTX, NULL,
> - all_done_label, PROB_VERY_UNLIKELY);
> + all_done_label, profile_probability::very_unlikely ());
> emit_label (set_noovf);
> write_complex_part (target, const0_rtx, true);
> emit_label (all_done_label);
> @@ -1977,7 +1977,7 @@ expand_vector_ubsan_overflow (location_t
> emit_move_insn (cntvar, ret);
> do_compare_rtx_and_jump (cntvar, GEN_INT (cnt), NE, false,
> TYPE_MODE (sizetype), NULL_RTX, NULL, loop_lab,
> - PROB_VERY_LIKELY);
> + profile_probability::very_likely ());
> }
> if (lhs && resv == NULL_TREE)
> {
> Index: ipa-cp.c
> ===================================================================
> --- ipa-cp.c (revision 249769)
> +++ ipa-cp.c (working copy)
> @@ -2623,7 +2623,8 @@ good_cloning_opportunity_p (struct cgrap
> struct ipa_node_params *info = IPA_NODE_REF (node);
> if (max_count > profile_count::zero ())
> {
> - int factor = RDIV (count_sum.probability_in (max_count)
> + int factor = RDIV (count_sum.probability_in
> + (max_count).to_reg_br_prob_base ()
> * 1000, REG_BR_PROB_BASE);
> int64_t evaluation = (((int64_t) time_benefit * factor)
> / size_cost);
> Index: ipa-split.c
> ===================================================================
> --- ipa-split.c (revision 249769)
> +++ ipa-split.c (working copy)
> @@ -1292,9 +1292,7 @@ split_function (basic_block return_bb, s
> break;
> }
> }
> - e = make_edge (new_return_bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = new_return_bb->count;
> + e = make_single_succ_edge (new_return_bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> add_bb_to_loop (new_return_bb, current_loops->tree_root);
> bitmap_set_bit (split_point->split_bbs, new_return_bb->index);
> retbnd = find_retbnd (return_bb);
> @@ -1527,11 +1525,9 @@ split_function (basic_block return_bb, s
> }
> else
> {
> - e = make_edge (call_bb, return_bb,
> - return_bb == EXIT_BLOCK_PTR_FOR_FN (cfun)
> - ? 0 : EDGE_FALLTHRU);
> - e->count = call_bb->count;
> - e->probability = REG_BR_PROB_BASE;
> + e = make_single_succ_edge (call_bb, return_bb,
> + return_bb == EXIT_BLOCK_PTR_FOR_FN (cfun)
> + ? 0 : EDGE_FALLTHRU);
>
> /* If there is return basic block, see what value we need to store
> return value into and put call just before it. */
> Index: ipa-utils.c
> ===================================================================
> --- ipa-utils.c (revision 249769)
> +++ ipa-utils.c (working copy)
> @@ -598,7 +598,8 @@ ipa_merge_profiles (struct cgraph_node *
> }
> }
> int prob = direct->count.probability_in (direct->count
> - + indirect->count);
> + + indirect->count).
> + to_reg_br_prob_base ();
> direct->frequency = RDIV (freq * prob, REG_BR_PROB_BASE);
> indirect->frequency = RDIV (freq * (REG_BR_PROB_BASE - prob),
> REG_BR_PROB_BASE);
> @@ -616,7 +617,8 @@ ipa_merge_profiles (struct cgraph_node *
> e2->speculative_call_info (direct, indirect, ref);
> e->count = count;
> e->frequency = freq;
> - int prob = direct->count.probability_in (e->count);
> + int prob = direct->count.probability_in (e->count)
> + .to_reg_br_prob_base ();
> e->make_speculative (direct->callee, direct->count,
> RDIV (freq * prob, REG_BR_PROB_BASE));
> }
> Index: loop-doloop.c
> ===================================================================
> --- loop-doloop.c (revision 249769)
> +++ loop-doloop.c (working copy)
> @@ -356,7 +356,8 @@ add_test (rtx cond, edge *e, basic_block
> op0 = force_operand (op0, NULL_RTX);
> op1 = force_operand (op1, NULL_RTX);
> label = block_label (dest);
> - do_compare_rtx_and_jump (op0, op1, code, 0, mode, NULL_RTX, NULL, label, -1);
> + do_compare_rtx_and_jump (op0, op1, code, 0, mode, NULL_RTX, NULL, label,
> + profile_probability::uninitialized ());
>
> jump = get_last_insn ();
> if (!jump || !JUMP_P (jump))
> @@ -575,10 +576,11 @@ doloop_modify (struct loop *loop, struct
> add_reg_note (jump_insn, REG_NONNEG, NULL_RTX);
>
> /* Update the REG_BR_PROB note. */
> - if (true_prob_val)
> + if (true_prob_val && desc->in_edge->probability.initialized_p ())
> {
> /* Seems safer to use the branch probability. */
> - add_int_reg_note (jump_insn, REG_BR_PROB, desc->in_edge->probability);
> + add_int_reg_note (jump_insn, REG_BR_PROB,
> + desc->in_edge->probability.to_reg_br_prob_base ());
> }
> }
>
> Index: loop-unroll.c
> ===================================================================
> --- loop-unroll.c (revision 249769)
> +++ loop-unroll.c (working copy)
> @@ -774,7 +774,8 @@ split_edge_and_insert (edge e, rtx_insn
>
> static rtx_insn *
> compare_and_jump_seq (rtx op0, rtx op1, enum rtx_code comp,
> - rtx_code_label *label, int prob, rtx_insn *cinsn)
> + rtx_code_label *label, profile_probability prob,
> + rtx_insn *cinsn)
> {
> rtx_insn *seq;
> rtx_jump_insn *jump;
> @@ -808,12 +809,14 @@ compare_and_jump_seq (rtx op0, rtx op1,
> op0 = force_operand (op0, NULL_RTX);
> op1 = force_operand (op1, NULL_RTX);
> do_compare_rtx_and_jump (op0, op1, comp, 0,
> - mode, NULL_RTX, NULL, label, -1);
> + mode, NULL_RTX, NULL, label,
> + profile_probability::uninitialized ());
> jump = as_a <rtx_jump_insn *> (get_last_insn ());
> jump->set_jump_target (label);
> LABEL_NUSES (label)++;
> }
> - add_int_reg_note (jump, REG_BR_PROB, prob);
> + if (prob.initialized_p ())
> + add_int_reg_note (jump, REG_BR_PROB, prob.to_reg_br_prob_base ());
>
> seq = get_insns ();
> end_sequence ();
> @@ -857,7 +860,8 @@ unroll_loop_runtime_iterations (struct l
> {
> rtx old_niter, niter, tmp;
> rtx_insn *init_code, *branch_code;
> - unsigned i, j, p;
> + unsigned i, j;
> + profile_probability p;
> basic_block preheader, *body, swtch, ezc_swtch = NULL;
> int may_exit_copy, iter_freq, new_freq;
> profile_count iter_count, new_count;
> @@ -989,7 +993,7 @@ unroll_loop_runtime_iterations (struct l
>
> /* Create item for switch. */
> j = n_peel - i - (extra_zero_check ? 0 : 1);
> - p = REG_BR_PROB_BASE / (i + 2);
> + p = profile_probability::always ().apply_scale (1, i + 2);
>
> preheader = split_edge (loop_preheader_edge (loop));
> /* Add in frequency/count of edge from switch block. */
> @@ -1006,7 +1010,7 @@ unroll_loop_runtime_iterations (struct l
>
> swtch = split_edge_and_insert (single_pred_edge (swtch), branch_code);
> set_immediate_dominator (CDI_DOMINATORS, preheader, swtch);
> - single_succ_edge (swtch)->probability = REG_BR_PROB_BASE - p;
> + single_succ_edge (swtch)->probability = p.invert ();
> single_succ_edge (swtch)->count = new_count;
> new_freq += iter_freq;
> new_count += iter_count;
> @@ -1021,7 +1025,7 @@ unroll_loop_runtime_iterations (struct l
> if (extra_zero_check)
> {
> /* Add branch for zero iterations. */
> - p = REG_BR_PROB_BASE / (max_unroll + 1);
> + p = profile_probability::always ().apply_scale (1, max_unroll + 1);
> swtch = ezc_swtch;
> preheader = split_edge (loop_preheader_edge (loop));
> /* Recompute frequency/count adjustments since initial peel copy may
> @@ -1039,7 +1043,7 @@ unroll_loop_runtime_iterations (struct l
>
> swtch = split_edge_and_insert (single_succ_edge (swtch), branch_code);
> set_immediate_dominator (CDI_DOMINATORS, preheader, swtch);
> - single_succ_edge (swtch)->probability = REG_BR_PROB_BASE - p;
> + single_succ_edge (swtch)->probability = p.invert ();
> single_succ_edge (swtch)->count -= iter_count;
> e = make_edge (swtch, preheader,
> single_succ_edge (swtch)->flags & EDGE_IRREDUCIBLE_LOOP);
> Index: lra-constraints.c
> ===================================================================
> --- lra-constraints.c (revision 249769)
> +++ lra-constraints.c (working copy)
> @@ -6482,7 +6482,8 @@ lra_inheritance (void)
> e = find_fallthru_edge (bb->succs);
> if (! e)
> break;
> - if (e->probability < EBB_PROBABILITY_CUTOFF)
> + if (e->probability.initialized_p ()
> + && e->probability.to_reg_br_prob_base () < EBB_PROBABILITY_CUTOFF)
> break;
> bb = bb->next_bb;
> }
> Index: lto-streamer-in.c
> ===================================================================
> --- lto-streamer-in.c (revision 249769)
> +++ lto-streamer-in.c (working copy)
> @@ -754,12 +754,12 @@ input_cfg (struct lto_input_block *ib, s
> unsigned int dest_index;
> unsigned int edge_flags;
> basic_block dest;
> - int probability;
> + profile_probability probability;
> profile_count count;
> edge e;
>
> dest_index = streamer_read_uhwi (ib);
> - probability = (int) streamer_read_hwi (ib);
> + probability = profile_probability::stream_in (ib);
> count = profile_count::stream_in (ib).apply_scale
> (count_materialization_scale, REG_BR_PROB_BASE);
> edge_flags = streamer_read_uhwi (ib);
> Index: lto-streamer-out.c
> ===================================================================
> --- lto-streamer-out.c (revision 249769)
> +++ lto-streamer-out.c (working copy)
> @@ -1860,7 +1860,7 @@ output_cfg (struct output_block *ob, str
> FOR_EACH_EDGE (e, ei, bb->succs)
> {
> streamer_write_uhwi (ob, e->dest->index);
> - streamer_write_hwi (ob, e->probability);
> + e->probability.stream_out (ob);
> e->count.stream_out (ob);
> streamer_write_uhwi (ob, e->flags);
> }
> Index: mcf.c
> ===================================================================
> --- mcf.c (revision 249769)
> +++ mcf.c (working copy)
> @@ -1231,12 +1231,15 @@ adjust_cfg_counts (fixup_graph_type *fix
> }
>
> if (bb_gcov_count (bb))
> - e->probability = RDIV (REG_BR_PROB_BASE * edge_gcov_count (e),
> - bb_gcov_count (bb));
> + e->probability = profile_probability::probability_in_gcov_type
> + (edge_gcov_count (e), bb_gcov_count (bb));
> if (dump_file)
> - fprintf (dump_file, " = %" PRId64 "\t(%.1f%%)\n",
> - edge_gcov_count (e),
> - e->probability * 100.0 / REG_BR_PROB_BASE);
> + {
> + fprintf (dump_file, " = %" PRId64 "\t",
> + edge_gcov_count (e));
> + e->probability.dump (dump_file);
> + fprintf (dump_file, "\n");
> + }
> }
> }
>
> @@ -1251,31 +1254,8 @@ adjust_cfg_counts (fixup_graph_type *fix
> if (bb_gcov_count (bb))
> {
> FOR_EACH_EDGE (e, ei, bb->succs)
> - e->probability = RDIV (REG_BR_PROB_BASE * edge_gcov_count (e),
> - bb_gcov_count (bb));
> - }
> - else
> - {
> - int total = 0;
> - FOR_EACH_EDGE (e, ei, bb->succs)
> - if (!(e->flags & (EDGE_COMPLEX | EDGE_FAKE)))
> - total++;
> - if (total)
> - {
> - FOR_EACH_EDGE (e, ei, bb->succs)
> - {
> - if (!(e->flags & (EDGE_COMPLEX | EDGE_FAKE)))
> - e->probability = REG_BR_PROB_BASE / total;
> - else
> - e->probability = 0;
> - }
> - }
> - else
> - {
> - total += EDGE_COUNT (bb->succs);
> - FOR_EACH_EDGE (e, ei, bb->succs)
> - e->probability = REG_BR_PROB_BASE / total;
> - }
> + e->probability = profile_probability::probability_in_gcov_type
> + (edge_gcov_count (e), bb_gcov_count (bb));
> }
> }
>
> Index: modulo-sched.c
> ===================================================================
> --- modulo-sched.c (revision 249769)
> +++ modulo-sched.c (working copy)
> @@ -1713,12 +1713,13 @@ sms_schedule (void)
> rtx comp_rtx = gen_rtx_GT (VOIDmode, count_reg,
> gen_int_mode (stage_count,
> GET_MODE (count_reg)));
> - unsigned prob = (PROB_SMS_ENOUGH_ITERATIONS
> - * REG_BR_PROB_BASE) / 100;
> + profile_probability prob = profile_probability::guessed_always ()
> + .apply_scale (PROB_SMS_ENOUGH_ITERATIONS, 100);
>
> loop_version (loop, comp_rtx, &condition_bb,
> - prob, REG_BR_PROB_BASE - prob,
> - prob, REG_BR_PROB_BASE - prob,
> + prob, prob.invert (),
> + prob.to_reg_br_prob_base (),
> + prob.invert ().to_reg_br_prob_base (),
> true);
> }
>
> Index: omp-expand.c
> ===================================================================
> --- omp-expand.c (revision 249769)
> +++ omp-expand.c (working copy)
> @@ -1741,9 +1741,9 @@ expand_omp_for_init_counts (struct omp_f
> entry_bb);
> }
> ne = make_edge (entry_bb, zero_iter_bb, EDGE_FALSE_VALUE);
> - ne->probability = REG_BR_PROB_BASE / 2000 - 1;
> + ne->probability = profile_probability::very_unlikely ();
> e->flags = EDGE_TRUE_VALUE;
> - e->probability = REG_BR_PROB_BASE - ne->probability;
> + e->probability = ne->probability.invert ();
> if (l2_dom_bb == NULL)
> l2_dom_bb = entry_bb;
> entry_bb = e->dest;
> @@ -1920,7 +1920,7 @@ extract_omp_for_update_vars (struct omp_
> if (i < fd->collapse - 1)
> {
> e = make_edge (last_bb, bb, EDGE_FALSE_VALUE);
> - e->probability = REG_BR_PROB_BASE / 8;
> + e->probability = profile_probability::guessed_always ().apply_scale (1, 8);
>
> t = fd->loops[i + 1].n1;
> t = force_gimple_operand_gsi (&gsi, t,
> @@ -1961,7 +1961,7 @@ extract_omp_for_update_vars (struct omp_
> stmt = gimple_build_cond_empty (t);
> gsi_insert_after (&gsi, stmt, GSI_CONTINUE_LINKING);
> e = make_edge (bb, body_bb, EDGE_TRUE_VALUE);
> - e->probability = REG_BR_PROB_BASE * 7 / 8;
> + e->probability = profile_probability::guessed_always ().apply_scale (7, 8);
> }
> else
> make_edge (bb, body_bb, EDGE_FALLTHRU);
> @@ -2219,8 +2219,8 @@ expand_omp_ordered_sink (gimple_stmt_ite
> GSI_CONTINUE_LINKING);
> gsi_insert_after (gsi, gimple_build_cond_empty (cond), GSI_NEW_STMT);
> edge e3 = make_edge (e1->src, e2->dest, EDGE_FALSE_VALUE);
> - e3->probability = REG_BR_PROB_BASE / 8;
> - e1->probability = REG_BR_PROB_BASE - e3->probability;
> + e3->probability = profile_probability::guessed_always ().apply_scale (1, 8);
> + e1->probability = e3->probability.invert ();
> e1->flags = EDGE_TRUE_VALUE;
> set_immediate_dominator (CDI_DOMINATORS, e2->dest, e1->src);
>
> @@ -2373,9 +2373,9 @@ expand_omp_for_ordered_loops (struct omp
> remove_edge (e1);
> make_edge (body_bb, new_header, EDGE_FALLTHRU);
> e3->flags = EDGE_FALSE_VALUE;
> - e3->probability = REG_BR_PROB_BASE / 8;
> + e3->probability = profile_probability::guessed_always ().apply_scale (1, 8);
> e1 = make_edge (new_header, new_body, EDGE_TRUE_VALUE);
> - e1->probability = REG_BR_PROB_BASE - e3->probability;
> + e1->probability = e3->probability.invert ();
>
> set_immediate_dominator (CDI_DOMINATORS, new_header, body_bb);
> set_immediate_dominator (CDI_DOMINATORS, new_body, new_header);
> @@ -3149,8 +3149,8 @@ expand_omp_for_generic (struct omp_regio
> e->flags = EDGE_TRUE_VALUE;
> if (e)
> {
> - e->probability = REG_BR_PROB_BASE * 7 / 8;
> - find_edge (cont_bb, l2_bb)->probability = REG_BR_PROB_BASE / 8;
> + e->probability = profile_probability::guessed_always ().apply_scale (7, 8);
> + find_edge (cont_bb, l2_bb)->probability = e->probability.invert ();
> }
> else
> {
> @@ -3351,9 +3351,9 @@ expand_omp_for_static_nochunk (struct om
> ep = split_block (entry_bb, cond_stmt);
> ep->flags = EDGE_TRUE_VALUE;
> entry_bb = ep->dest;
> - ep->probability = REG_BR_PROB_BASE - (REG_BR_PROB_BASE / 2000 - 1);
> + ep->probability = profile_probability::very_likely ();
> ep = make_edge (ep->src, fin_bb, EDGE_FALSE_VALUE);
> - ep->probability = REG_BR_PROB_BASE / 2000 - 1;
> + ep->probability = profile_probability::very_unlikely ();
> if (gimple_in_ssa_p (cfun))
> {
> int dest_idx = find_edge (entry_bb, fin_bb)->dest_idx;
> @@ -3634,10 +3634,10 @@ expand_omp_for_static_nochunk (struct om
>
> /* Connect all the blocks. */
> ep = make_edge (entry_bb, third_bb, EDGE_FALSE_VALUE);
> - ep->probability = REG_BR_PROB_BASE / 4 * 3;
> + ep->probability = profile_probability::guessed_always ().apply_scale (3, 4);
> ep = find_edge (entry_bb, second_bb);
> ep->flags = EDGE_TRUE_VALUE;
> - ep->probability = REG_BR_PROB_BASE / 4;
> + ep->probability = profile_probability::guessed_always ().apply_scale (1, 4);
> find_edge (third_bb, seq_start_bb)->flags = EDGE_FALSE_VALUE;
> find_edge (third_bb, fin_bb)->flags = EDGE_TRUE_VALUE;
>
> @@ -3835,9 +3835,9 @@ expand_omp_for_static_chunk (struct omp_
> se = split_block (entry_bb, cond_stmt);
> se->flags = EDGE_TRUE_VALUE;
> entry_bb = se->dest;
> - se->probability = REG_BR_PROB_BASE - (REG_BR_PROB_BASE / 2000 - 1);
> + se->probability = profile_probability::very_likely ();
> se = make_edge (se->src, fin_bb, EDGE_FALSE_VALUE);
> - se->probability = REG_BR_PROB_BASE / 2000 - 1;
> + se->probability = profile_probability::very_unlikely ();
> if (gimple_in_ssa_p (cfun))
> {
> int dest_idx = find_edge (iter_part_bb, fin_bb)->dest_idx;
> @@ -4448,8 +4448,8 @@ expand_cilk_for (struct omp_region *regi
>
> }
> ne->flags = EDGE_FALSE_VALUE;
> - e->probability = REG_BR_PROB_BASE * 7 / 8;
> - ne->probability = REG_BR_PROB_BASE / 8;
> + e->probability = profile_probability::guessed_always ().apply_scale (7, 8);
> + ne->probability = e->probability.invert ();
>
> set_immediate_dominator (CDI_DOMINATORS, l1_bb, entry_bb);
> set_immediate_dominator (CDI_DOMINATORS, l2_bb, l2_dom_bb);
> @@ -4810,8 +4810,8 @@ expand_omp_simd (struct omp_region *regi
>
> }
> ne->flags = EDGE_FALSE_VALUE;
> - e->probability = REG_BR_PROB_BASE * 7 / 8;
> - ne->probability = REG_BR_PROB_BASE / 8;
> + e->probability = profile_probability::guessed_always ().apply_scale (7, 8);
> + ne->probability = e->probability.invert ();
>
> set_immediate_dominator (CDI_DOMINATORS, l1_bb, entry_bb);
> set_immediate_dominator (CDI_DOMINATORS, l0_bb, l1_bb);
> @@ -4824,8 +4824,10 @@ expand_omp_simd (struct omp_region *regi
> gsi_insert_after (&gsi, cond_stmt, GSI_NEW_STMT);
> make_edge (entry_bb, l2_bb, EDGE_FALSE_VALUE);
> FALLTHRU_EDGE (entry_bb)->flags = EDGE_TRUE_VALUE;
> - FALLTHRU_EDGE (entry_bb)->probability = REG_BR_PROB_BASE * 7 / 8;
> - BRANCH_EDGE (entry_bb)->probability = REG_BR_PROB_BASE / 8;
> + FALLTHRU_EDGE (entry_bb)->probability
> + = profile_probability::guessed_always ().apply_scale (7, 8);
> + BRANCH_EDGE (entry_bb)->probability
> + = FALLTHRU_EDGE (entry_bb)->probability.invert ();
> l2_dom_bb = entry_bb;
> }
> set_immediate_dominator (CDI_DOMINATORS, l2_bb, l2_dom_bb);
> @@ -5018,9 +5020,9 @@ expand_omp_taskloop_for_outer (struct om
> gsi = gsi_last_bb (exit_bb);
> gsi_remove (&gsi, true);
>
> - FALLTHRU_EDGE (entry_bb)->probability = REG_BR_PROB_BASE;
> + FALLTHRU_EDGE (entry_bb)->probability = profile_probability::always ();
> remove_edge (BRANCH_EDGE (entry_bb));
> - FALLTHRU_EDGE (cont_bb)->probability = REG_BR_PROB_BASE;
> + FALLTHRU_EDGE (cont_bb)->probability = profile_probability::always ();
> remove_edge (BRANCH_EDGE (cont_bb));
> set_immediate_dominator (CDI_DOMINATORS, exit_bb, cont_bb);
> set_immediate_dominator (CDI_DOMINATORS, region->entry,
> @@ -5208,7 +5210,7 @@ expand_omp_taskloop_for_inner (struct om
> gsi = gsi_last_bb (exit_bb);
> gsi_remove (&gsi, true);
>
> - FALLTHRU_EDGE (entry_bb)->probability = REG_BR_PROB_BASE;
> + FALLTHRU_EDGE (entry_bb)->probability = profile_probability::always ();
> if (!broken_loop)
> remove_edge (BRANCH_EDGE (entry_bb));
> else
> @@ -6604,8 +6606,11 @@ expand_omp_atomic_pipeline (basic_block
> e = single_succ_edge (store_bb);
> e->flags &= ~EDGE_FALLTHRU;
> e->flags |= EDGE_FALSE_VALUE;
> + /* Expect no looping. */
> + e->probability = profile_probability::guessed_always ();
>
> e = make_edge (store_bb, loop_header, EDGE_TRUE_VALUE);
> + e->probability = profile_probability::guessed_never ();
>
> /* Copy the new value to loadedi (we already did that before the condition
> if we are not in SSA). */
> Index: omp-simd-clone.c
> ===================================================================
> --- omp-simd-clone.c (revision 249769)
> +++ omp-simd-clone.c (working copy)
> @@ -1151,8 +1151,7 @@ simd_clone_adjust (struct cgraph_node *n
>
> if (incr_bb)
> {
> - edge e = make_edge (incr_bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> - e->probability = REG_BR_PROB_BASE;
> + make_single_succ_edge (incr_bb, EXIT_BLOCK_PTR_FOR_FN (cfun), 0);
> gsi = gsi_last_bb (incr_bb);
> iter2 = make_ssa_name (iter);
> g = gimple_build_assign (iter2, PLUS_EXPR, iter1,
> @@ -1264,7 +1263,10 @@ simd_clone_adjust (struct cgraph_node *n
>
> redirect_edge_succ (FALLTHRU_EDGE (latch_bb), body_bb);
>
> - make_edge (incr_bb, new_exit_bb, EDGE_FALSE_VALUE);
> + edge new_e = make_edge (incr_bb, new_exit_bb, EDGE_FALSE_VALUE);
> +
> + /* FIXME: Do we need to distribute probabilities for the conditional? */
> + new_e->probability = profile_probability::guessed_never ();
> /* The successor of incr_bb is already pointing to latch_bb; just
> change the flags.
> make_edge (incr_bb, latch_bb, EDGE_TRUE_VALUE); */
> Index: optabs.c
> ===================================================================
> --- optabs.c (revision 249769)
> +++ optabs.c (working copy)
> @@ -693,7 +693,8 @@ expand_doubleword_shift (machine_mode op
>
> NO_DEFER_POP;
> do_compare_rtx_and_jump (cmp1, cmp2, cmp_code, false, op1_mode,
> - 0, 0, subword_label, -1);
> + 0, 0, subword_label,
> + profile_probability::uninitialized ());
> OK_DEFER_POP;
>
> if (!expand_superword_shift (binoptab, outof_input, superword_op1,
> @@ -3187,7 +3188,8 @@ expand_abs (machine_mode mode, rtx op0,
> NO_DEFER_POP;
>
> do_compare_rtx_and_jump (target, CONST0_RTX (mode), GE, 0, mode,
> - NULL_RTX, NULL, op1, -1);
> + NULL_RTX, NULL, op1,
> + profile_probability::uninitialized ());
>
> op0 = expand_unop (mode, result_unsignedp ? neg_optab : negv_optab,
> target, target, 0);
> @@ -3979,7 +3981,8 @@ prepare_operand (enum insn_code icode, r
> we can do the branch. */
>
> static void
> -emit_cmp_and_jump_insn_1 (rtx test, machine_mode mode, rtx label, int prob)
> +emit_cmp_and_jump_insn_1 (rtx test, machine_mode mode, rtx label,
> + profile_probability prob)
> {
> machine_mode optab_mode;
> enum mode_class mclass;
> @@ -3994,13 +3997,13 @@ emit_cmp_and_jump_insn_1 (rtx test, mach
> gcc_assert (insn_operand_matches (icode, 0, test));
> insn = emit_jump_insn (GEN_FCN (icode) (test, XEXP (test, 0),
> XEXP (test, 1), label));
> - if (prob != -1
> + if (prob.initialized_p ()
> && profile_status_for_fn (cfun) != PROFILE_ABSENT
> && insn
> && JUMP_P (insn)
> && any_condjump_p (insn)
> && !find_reg_note (insn, REG_BR_PROB, 0))
> - add_int_reg_note (insn, REG_BR_PROB, prob);
> + add_int_reg_note (insn, REG_BR_PROB, prob.to_reg_br_prob_base ());
> }
>
> /* Generate code to compare X with Y so that the condition codes are
> @@ -4025,7 +4028,7 @@ emit_cmp_and_jump_insn_1 (rtx test, mach
> void
> emit_cmp_and_jump_insns (rtx x, rtx y, enum rtx_code comparison, rtx size,
> machine_mode mode, int unsignedp, rtx label,
> - int prob)
> + profile_probability prob)
> {
> rtx op0 = x, op1 = y;
> rtx test;
> @@ -5856,7 +5859,8 @@ expand_compare_and_swap_loop (rtx mem, r
>
> /* Mark this jump predicted not taken. */
> emit_cmp_and_jump_insns (success, const0_rtx, EQ, const0_rtx,
> - GET_MODE (success), 1, label, 0);
> + GET_MODE (success), 1, label,
> + profile_probability::guessed_never ());
> return true;
> }
>
> Index: optabs.h
> ===================================================================
> --- optabs.h (revision 249769)
> +++ optabs.h (working copy)
> @@ -247,7 +247,9 @@ extern rtx prepare_operand (enum insn_co
> /* Emit a pair of rtl insns to compare two rtx's and to jump
> to a label if the comparison is true. */
> extern void emit_cmp_and_jump_insns (rtx, rtx, enum rtx_code, rtx,
> - machine_mode, int, rtx, int prob=-1);
> + machine_mode, int, rtx,
> + profile_probability prob
> + = profile_probability::uninitialized ());
>
> /* Generate code to indirectly jump to a location given in the rtx LOC. */
> extern void emit_indirect_jump (rtx);
> Index: predict.c
> ===================================================================
> --- predict.c (revision 249769)
> +++ predict.c (working copy)
> @@ -404,11 +404,11 @@ optimize_loop_nest_for_size_p (struct lo
> bool
> predictable_edge_p (edge e)
> {
> - if (profile_status_for_fn (cfun) == PROFILE_ABSENT)
> + if (!e->probability.initialized_p ())
> return false;
> - if ((e->probability
> + if ((e->probability.to_reg_br_prob_base ()
> <= PARAM_VALUE (PARAM_PREDICTABLE_BRANCH_OUTCOME) * REG_BR_PROB_BASE / 100)
> - || (REG_BR_PROB_BASE - e->probability
> + || (REG_BR_PROB_BASE - e->probability.to_reg_br_prob_base ()
> <= PARAM_VALUE (PARAM_PREDICTABLE_BRANCH_OUTCOME) * REG_BR_PROB_BASE / 100))
> return true;
> return false;
> @@ -539,7 +539,7 @@ probability_reliable_p (int prob)
> bool
> edge_probability_reliable_p (const_edge e)
> {
> - return probability_reliable_p (e->probability);
> + return e->probability.reliable_p ();
> }
>
> /* Same predicate as edge_probability_reliable_p, working on notes. */
> @@ -859,12 +859,13 @@ set_even_probabilities (basic_block bb,
> if (!unlikely_executed_edge_p (e))
> {
> if (unlikely_edges != NULL && unlikely_edges->contains (e))
> - e->probability = PROB_VERY_UNLIKELY;
> + e->probability = profile_probability::very_unlikely ();
> else
> - e->probability = (REG_BR_PROB_BASE + c / 2) / c;
> + e->probability = profile_probability::guessed_always ()
> + .apply_scale (1, c);
> }
> else
> - e->probability = 0;
> + e->probability = profile_probability::never ();
> }
>
> /* Combine all REG_BR_PRED notes into single probability and attach REG_BR_PROB
> @@ -971,20 +972,23 @@ combine_predictions_for_insn (rtx_insn *
> conditional jump. */
> if (!single_succ_p (bb))
> {
> - BRANCH_EDGE (bb)->probability = combined_probability;
> + BRANCH_EDGE (bb)->probability
> + = profile_probability::from_reg_br_prob_base (combined_probability);
> FALLTHRU_EDGE (bb)->probability
> - = REG_BR_PROB_BASE - combined_probability;
> + = BRANCH_EDGE (bb)->probability.invert ();
> }
> }
> else if (!single_succ_p (bb))
> {
> int prob = XINT (prob_note, 0);
>
> - BRANCH_EDGE (bb)->probability = prob;
> - FALLTHRU_EDGE (bb)->probability = REG_BR_PROB_BASE - prob;
> + BRANCH_EDGE (bb)->probability
> + = profile_probability::from_reg_br_prob_base (prob);
> + FALLTHRU_EDGE (bb)->probability
> + = BRANCH_EDGE (bb)->probability.invert ();
> }
> else
> - single_succ_edge (bb)->probability = REG_BR_PROB_BASE;
> + single_succ_edge (bb)->probability = profile_probability::always ();
> }
>
> /* Edge prediction hash traits. */
> @@ -1129,6 +1133,8 @@ combine_predictions_for_bb (basic_block
> if (!first)
> first = e;
> }
> + else if (!e->probability.initialized_p ())
> + e->probability = profile_probability::never ();
>
> /* When there is no successor or only one choice, prediction is easy.
>
> @@ -1173,8 +1179,8 @@ combine_predictions_for_bb (basic_block
> nedges, bb->index);
> FOR_EACH_EDGE (e, ei, bb->succs)
> if (!unlikely_executed_edge_p (e))
> - dump_prediction (dump_file, PRED_COMBINED, e->probability,
> - bb, REASON_NONE, e);
> + dump_prediction (dump_file, PRED_COMBINED,
> + e->probability.to_reg_br_prob_base (), bb, REASON_NONE, e);
> }
> }
> return;
> @@ -1284,8 +1290,9 @@ combine_predictions_for_bb (basic_block
>
> if (!bb->count.initialized_p () && !dry_run)
> {
> - first->probability = combined_probability;
> - second->probability = REG_BR_PROB_BASE - combined_probability;
> + first->probability
> + = profile_probability::from_reg_br_prob_base (combined_probability);
> + second->probability = first->probability.invert ();
> }
> }
>
> @@ -3042,7 +3049,7 @@ propagate_freq (basic_block head, bitmap
> * BLOCK_INFO (e->src)->frequency /
> REG_BR_PROB_BASE); */
>
> - sreal tmp = e->probability;
> + sreal tmp = e->probability.to_reg_br_prob_base ();
> tmp *= BLOCK_INFO (e->src)->frequency;
> tmp *= real_inv_br_prob_base;
> frequency += tmp;
> @@ -3074,7 +3081,7 @@ propagate_freq (basic_block head, bitmap
> = ((e->probability * BLOCK_INFO (bb)->frequency)
> / REG_BR_PROB_BASE); */
>
> - sreal tmp = e->probability;
> + sreal tmp = e->probability.to_reg_br_prob_base ();
> tmp *= BLOCK_INFO (bb)->frequency;
> EDGE_INFO (e)->back_edge_prob = tmp * real_inv_br_prob_base;
> }
> @@ -3534,7 +3541,7 @@ estimate_bb_frequencies (bool force)
> mark_dfs_back_edges ();
>
> single_succ_edge (ENTRY_BLOCK_PTR_FOR_FN (cfun))->probability =
> - REG_BR_PROB_BASE;
> + profile_probability::always ();
>
> /* Set up block info for each basic block. */
> alloc_aux_for_blocks (sizeof (block_info));
> @@ -3546,7 +3553,8 @@ estimate_bb_frequencies (bool force)
>
> FOR_EACH_EDGE (e, ei, bb->succs)
> {
> - EDGE_INFO (e)->back_edge_prob = e->probability;
> + EDGE_INFO (e)->back_edge_prob
> + = e->probability.to_reg_br_prob_base ();
> EDGE_INFO (e)->back_edge_prob *= real_inv_br_prob_base;
> }
> }
> @@ -3898,16 +3906,18 @@ void
> force_edge_cold (edge e, bool impossible)
> {
> profile_count count_sum = profile_count::zero ();
> - int prob_sum = 0;
> + profile_probability prob_sum = profile_probability::never ();
> edge_iterator ei;
> edge e2;
> profile_count old_count = e->count;
> - int old_probability = e->probability;
> - int prob_scale = REG_BR_PROB_BASE;
> + profile_probability old_probability = e->probability;
> bool uninitialized_exit = false;
>
> + profile_probability goal = (impossible ? profile_probability::never ()
> + : profile_probability::very_unlikely ());
> +
> /* If edge is already improbably or cold, just return. */
> - if (e->probability <= (impossible ? PROB_VERY_UNLIKELY : 0)
> + if (e->probability <= goal
> && (!impossible || e->count == profile_count::zero ()))
> return;
> FOR_EACH_EDGE (e2, ei, e->src->succs)
> @@ -3917,24 +3927,26 @@ force_edge_cold (edge e, bool impossible
> count_sum += e2->count;
> else
> uninitialized_exit = true;
> - prob_sum += e2->probability;
> + if (e2->probability.initialized_p ())
> + prob_sum += e2->probability;
> }
>
> /* If there are other edges out of e->src, redistribute probabilitity
> there. */
> - if (prob_sum)
> + if (prob_sum > profile_probability::never ())
> {
> - e->probability
> - = MIN (e->probability, impossible ? 0 : PROB_VERY_UNLIKELY);
> + if (!(e->probability < goal))
> + e->probability = goal;
> if (impossible)
> e->count = profile_count::zero ();
> - else if (old_probability)
> - e->count = e->count.apply_scale (e->probability, old_probability);
> + else if (old_probability > profile_probability::never ())
> + e->count = e->count.apply_probability (e->probability
> + / old_probability);
> else
> e->count = e->count.apply_scale (1, REG_BR_PROB_BASE);
>
> - prob_scale = RDIV ((REG_BR_PROB_BASE - e->probability) * REG_BR_PROB_BASE,
> - prob_sum);
> + profile_probability prob_comp = prob_sum / e->probability.invert ();
> +
> if (dump_file && (dump_flags & TDF_DETAILS))
> fprintf (dump_file, "Making edge %i->%i %s by redistributing "
> "probability to other edges.\n",
> @@ -3946,15 +3958,14 @@ force_edge_cold (edge e, bool impossible
> {
> if (count_sum > 0)
> e2->count.apply_scale (count_sum2, count_sum);
> - e2->probability = RDIV (e2->probability * prob_scale,
> - REG_BR_PROB_BASE);
> + e2->probability /= prob_comp;
> }
> }
> /* If all edges out of e->src are unlikely, the basic block itself
> is unlikely. */
> else
> {
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> if (e->src->count == profile_count::zero ())
> return;
> if (count_sum == profile_count::zero () && !uninitialized_exit
> @@ -3989,7 +4000,8 @@ force_edge_cold (edge e, bool impossible
> This in general is difficult task to do, but handle special case when
> BB has only one predecestor. This is common case when we are updating
> after loop transforms. */
> - if (!prob_sum && count_sum == profile_count::zero ()
> + if (!(prob_sum > profile_probability::never ())
> + && count_sum == profile_count::zero ()
> && single_pred_p (e->src) && e->src->frequency > (impossible ? 0 : 1))
> {
> int old_frequency = e->src->frequency;
> Index: profile-count.c
> ===================================================================
> --- profile-count.c (revision 249769)
> +++ profile-count.c (working copy)
> @@ -31,6 +31,8 @@ along with GCC; see the file COPYING3.
> #include "data-streamer.h"
> #include "cgraph.h"
>
> +/* Dump THIS to F. */
> +
> void
> profile_count::dump (FILE *f) const
> {
> @@ -39,43 +41,53 @@ profile_count::dump (FILE *f) const
> else
> {
> fprintf (f, "%" PRId64, m_val);
> - if (m_quality == count_adjusted)
> - fprintf (f, "(adjusted)");
> - else if (m_quality == count_afdo)
> - fprintf (f, "(auto FDO)");
> - else if (m_quality == count_guessed)
> - fprintf (f, "(guessed)");
> + if (m_quality == profile_adjusted)
> + fprintf (f, " (adjusted)");
> + else if (m_quality == profile_afdo)
> + fprintf (f, " (auto FDO)");
> + else if (m_quality == profile_guessed)
> + fprintf (f, " (guessed)");
> }
> }
>
> +/* Dump THIS to stderr. */
> +
> void
> profile_count::debug () const
> {
> dump (stderr);
> + fprintf (stderr, "\n");
> }
>
> +/* Return true if THIS differs from OTHER; tolerate small diferences. */
> +
> bool
> profile_count::differs_from_p (profile_count other) const
> {
> if (!initialized_p () || !other.initialized_p ())
> return false;
> - if (m_val - other.m_val < 100 || other.m_val - m_val < 100)
> + if ((uint64_t)m_val - (uint64_t)other.m_val < 100
> + || (uint64_t)other.m_val - (uint64_t)m_val < 100)
> return false;
> if (!other.m_val)
> return true;
> - int64_t ratio = m_val * 100 / other.m_val;
> + int64_t ratio = (int64_t)m_val * 100 / other.m_val;
> return ratio < 99 || ratio > 101;
> }
>
> +/* Stream THIS from IB. */
> +
> profile_count
> profile_count::stream_in (struct lto_input_block *ib)
> {
> profile_count ret;
> ret.m_val = streamer_read_gcov_count (ib);
> - ret.m_quality = (profile_count_quality) streamer_read_uhwi (ib);
> + ret.m_quality = (profile_quality) streamer_read_uhwi (ib);
> return ret;
> }
>
> +/* Stream THIS to OB. */
> +
> void
> profile_count::stream_out (struct output_block *ob)
> {
> @@ -83,9 +95,102 @@ profile_count::stream_out (struct output
> streamer_write_uhwi (ob, m_quality);
> }
>
> +/* Stream THIS to OB. */
> +
> void
> profile_count::stream_out (struct lto_output_stream *ob)
> {
> streamer_write_gcov_count_stream (ob, m_val);
> streamer_write_uhwi_stream (ob, m_quality);
> }
> +
> +/* Dump THIS to F. */
> +
> +void
> +profile_probability::dump (FILE *f) const
> +{
> + if (!initialized_p ())
> + fprintf (f, "uninitialized");
> + else
> + {
> + /* Make difference between 0.00 as a roundoff error and actual 0.
> + Similarly for 1. */
> + if (m_val == 0)
> + fprintf (f, "never");
> + else if (m_val == max_probability)
> + fprintf (f, "always");
> + else
> + fprintf (f, "%3.1f%%", (double)m_val * 100 / max_probability);
> + if (m_quality == profile_adjusted)
> + fprintf (f, " (adjusted)");
> + else if (m_quality == profile_afdo)
> + fprintf (f, " (auto FDO)");
> + else if (m_quality == profile_guessed)
> + fprintf (f, " (guessed)");
> + }
> +}
> +
> +/* Dump THIS to stderr. */
> +
> +void
> +profile_probability::debug () const
> +{
> + dump (stderr);
> + fprintf (stderr, "\n");
> +}
> +
> +/* Return true if THIS differs from OTHER; tolerate small diferences. */
> +
> +bool
> +profile_probability::differs_from_p (profile_probability other) const
> +{
> + if (!initialized_p () || !other.initialized_p ())
> + return false;
> + if ((uint64_t)m_val - (uint64_t)other.m_val < 10
> + || (uint64_t)other.m_val - (uint64_t)m_val < 10)
> + return false;
> + if (!other.m_val)
> + return true;
> + int64_t ratio = m_val * 100 / other.m_val;
> + return ratio < 99 || ratio > 101;
> +}
> +
> +/* Return true if THIS differs significantly from OTHER. */
> +
> +bool
> +profile_probability::differs_lot_from_p (profile_probability other) const
> +{
> + if (!initialized_p () || !other.initialized_p ())
> + return false;
> + uint32_t d = m_val > other.m_val ? m_val - other.m_val : other.m_val - m_val;
> + return d > max_probability / 2;
> +}
> +
> +/* Stream THIS from IB. */
> +
> +profile_probability
> +profile_probability::stream_in (struct lto_input_block *ib)
> +{
> + profile_probability ret;
> + ret.m_val = streamer_read_uhwi (ib);
> + ret.m_quality = (profile_quality) streamer_read_uhwi (ib);
> + return ret;
> +}
> +
> +/* Stream THIS to OB. */
> +
> +void
> +profile_probability::stream_out (struct output_block *ob)
> +{
> + streamer_write_uhwi (ob, m_val);
> + streamer_write_uhwi (ob, m_quality);
> +}
> +
> +/* Stream THIS to OB. */
> +
> +void
> +profile_probability::stream_out (struct lto_output_stream *ob)
> +{
> + streamer_write_uhwi_stream (ob, m_val);
> + streamer_write_uhwi_stream (ob, m_quality);
> +}
> Index: profile-count.h
> ===================================================================
> --- profile-count.h (revision 249769)
> +++ profile-count.h (working copy)
> @@ -23,19 +23,19 @@ along with GCC; see the file COPYING3.
>
> /* Quality of the proflie count. Because gengtype does not support enums
> inside of clases, this is in global namespace. */
> -enum profile_count_quality {
> +enum profile_quality {
> /* Profile is based on static branch prediction heuristics. It may or may
> not reflect the reality. */
> - count_guessed = 0,
> + profile_guessed = 0,
> /* Profile was determined by autofdo. */
> - count_afdo = 1,
> + profile_afdo = 1,
> /* Profile was originally based on feedback but it was adjusted
> by code duplicating optimization. It may not precisely reflect the
> particular code path. */
> - count_adjusted = 2,
> + profile_adjusted = 2,
> /* Profile was read from profile feedback or determined by accurate static
> method. */
> - count_read = 3
> + profile_precise = 3
> };
>
> /* The base value for branch probability notes and edge probabilities. */
> @@ -43,6 +43,422 @@ enum profile_count_quality {
>
> #define RDIV(X,Y) (((X) + (Y) / 2) / (Y))
>
> +/* Data type to hold probabilities. It implement fixed point arithmetics
> + with capping so probability is always in range [0,1] and scaling requiring
> + values greater than 1 needs to be represented otherwise.
> +
> + In addition to actual value the quality of profile is tracked and propagated
> + through all operations. Special value UNINITIALIZED is used for probabilities
> + that has not been detemrined yet (for example bacause of
> + -fno-guess-branch-probability)
> +
> + Typically probabilities are derived from profile feedback (via
> + probability_in_gcov_type), autoFDO or guessed statically and then propagated
> + thorough the compilation.
> +
> + Named probabilities are available:
> + - never (0 probability)
> + - guessed_never
> + - very_unlikely (1/2000 probability)
> + - unlikely (1/5 probablity)
> + - even (1/2 probability)
> + - likely (4/5 probability)
> + - very_likely (1999/2000 probability)
> + - guessed_always
> + - always
> +
> + Named probabilities except for never/always are assumed to be statically
> + guessed and thus not necessarily acurate. The difference between never
> + and guessedn never is that the first one should be used only in case that
> + well behaving program will very likely not execute the "never" path.
> + For example if the path is going to abort () call or it exception handling.
> +
> + Alawyas and guessted_always probabilities are symmetric.
> +
> + For legacy code we support conversion to/from REG_BR_PROB_BASE based fixpoint
> + integer arithmetics. Once the code is converted to branch probabiitlies,
> + these conversions will probably go away because they are lossy.
> +*/
> +
> +class GTY((user)) profile_probability
> +{
> + /* For now use values in range 0...REG_BR_PROB_BASE. Later we can use full
> + precision of 30 bits available. */
> +
> + static const int n_bits = 30;
> + static const uint32_t max_probability = REG_BR_PROB_BASE;
> + static const uint32_t uninitialized_probability = ((uint32_t) 1 << n_bits) - 1;
> +
> + uint32_t m_val : 30;
> + enum profile_quality m_quality : 2;
> +
> + friend class profile_count;
> +public:
> +
> + /* Named probabilities. */
> + static profile_probability never ()
> + {
> + profile_probability ret;
> + ret.m_val = 0;
> + ret.m_quality = profile_precise;
> + return ret;
> + }
> + static profile_probability guessed_never ()
> + {
> + profile_probability ret;
> + ret.m_val = 0;
> + ret.m_quality = profile_guessed;
> + return ret;
> + }
> + static profile_probability very_unlikely ()
> + {
> + /* Be consistent with PROB_VERY_UNLIKELY in predict.h. */
> + profile_probability r
> + = profile_probability::always ().apply_scale (1, 2000);
> + r.m_val--;
> + return r;
> + }
> + static profile_probability unlikely ()
> + {
> + /* Be consistent with PROB_VERY_LIKELY in predict.h. */
> + profile_probability r
> + = profile_probability::always ().apply_scale (1, 5);
> + r.m_val--;
> + return r;
> + }
> + static profile_probability even ()
> + {
> + return profile_probability::always ().apply_scale (1, 2);
> + }
> + static profile_probability very_likely ()
> + {
> + return profile_probability::always () - very_unlikely ();
> + }
> + static profile_probability likely ()
> + {
> + return profile_probability::always () - unlikely ();
> + }
> + static profile_probability guessed_always ()
> + {
> + profile_probability ret;
> + ret.m_val = max_probability;
> + ret.m_quality = profile_guessed;
> + return ret;
> + }
> + static profile_probability always ()
> + {
> + profile_probability ret;
> + ret.m_val = max_probability;
> + ret.m_quality = profile_precise;
> + return ret;
> + }
> + /* Probabilities which has not been initialized. Either because
> + initialization did not happen yet or because profile is unknown. */
> + static profile_probability uninitialized ()
> + {
> + profile_probability c;
> + c.m_val = uninitialized_probability;
> + c.m_quality = profile_guessed;
> + return c;
> + }
> +
> +
> + /* Return true if value has been initialized. */
> + bool initialized_p () const
> + {
> + return m_val != uninitialized_probability;
> + }
> + /* Return true if value can be trusted. */
> + bool reliable_p () const
> + {
> + return initialized_p ();
> + }
> +
> + /* Conversion from and to REG_BR_PROB_BASE integer fixpoint arithmetics.
> + this is mostly to support legacy code and hsould go away. */
> + static profile_probability from_reg_br_prob_base (int v)
> + {
> + profile_probability ret;
> + gcc_checking_assert (v >= 0 && v <= REG_BR_PROB_BASE);
> + ret.m_val = RDIV (v * max_probability, REG_BR_PROB_BASE);
> + ret.m_quality = profile_guessed;
> + return ret;
> + }
> + int to_reg_br_prob_base () const
> + {
> + gcc_checking_assert (initialized_p ());
> + return RDIV (m_val * REG_BR_PROB_BASE, max_probability);
> + }
> +
> + /* Return VAL1/VAL2. */
> + static profile_probability probability_in_gcov_type
> + (gcov_type val1, gcov_type val2)
> + {
> + profile_probability ret;
> + gcc_checking_assert (val1 >= 0 && val2 > 0);
> + if (val1 > val2)
> + ret.m_val = max_probability;
> + else
> + ret.m_val = RDIV (val1 * max_probability, val2);
> + ret.m_quality = profile_precise;
> + return ret;
> + }
> +
> + /* Basic operations. */
> + bool operator== (const profile_probability &other) const
> + {
> + return m_val == other.m_val && m_quality == other.m_quality;
> + }
> + profile_probability operator+ (const profile_probability &other) const
> + {
> + if (other == profile_probability::never ())
> + return *this;
> + if (*this == profile_probability::never ())
> + return other;
> + if (!initialized_p () || !other.initialized_p ())
> + return profile_probability::uninitialized ();
> +
> + profile_probability ret;
> + ret.m_val = MIN ((uint32_t)(m_val + other.m_val), max_probability);
> + ret.m_quality = MIN (m_quality, other.m_quality);
> + return ret;
> + }
> + profile_probability &operator+= (const profile_probability &other)
> + {
> + if (other == profile_probability::never ())
> + return *this;
> + if (*this == profile_probability::never ())
> + {
> + *this = other;
> + return *this;
> + }
> + if (!initialized_p () || !other.initialized_p ())
> + return *this = profile_probability::uninitialized ();
> + else
> + {
> + m_val = MIN ((uint32_t)(m_val + other.m_val), max_probability);
> + m_quality = MIN (m_quality, other.m_quality);
> + }
> + return *this;
> + }
> + profile_probability operator- (const profile_probability &other) const
> + {
> + if (*this == profile_probability::never ()
> + || other == profile_probability::never ())
> + return *this;
> + if (!initialized_p () || !other.initialized_p ())
> + return profile_probability::uninitialized ();
> + profile_probability ret;
> + ret.m_val = m_val >= other.m_val ? m_val - other.m_val : 0;
> + ret.m_quality = MIN (m_quality, other.m_quality);
> + return ret;
> + }
> + profile_probability &operator-= (const profile_probability &other)
> + {
> + if (*this == profile_probability::never ()
> + || other == profile_probability::never ())
> + return *this;
> + if (!initialized_p () || !other.initialized_p ())
> + return *this = profile_probability::uninitialized ();
> + else
> + {
> + m_val = m_val >= other.m_val ? m_val - other.m_val : 0;
> + m_quality = MIN (m_quality, other.m_quality);
> + }
> + return *this;
> + }
> + profile_probability operator* (const profile_probability &other) const
> + {
> + if (*this == profile_probability::never ()
> + || other == profile_probability::never ())
> + return profile_probability::never ();
> + if (!initialized_p () || !other.initialized_p ())
> + return profile_probability::uninitialized ();
> + profile_probability ret;
> + ret.m_val = RDIV ((uint64_t)m_val * other.m_val, max_probability);
> + ret.m_quality = MIN (m_quality, other.m_quality);
> + return ret;
> + }
> + profile_probability &operator*= (const profile_probability &other)
> + {
> + if (*this == profile_probability::never ()
> + || other == profile_probability::never ())
> + return *this = profile_probability::never ();
> + if (!initialized_p () || !other.initialized_p ())
> + return *this = profile_probability::uninitialized ();
> + else
> + {
> + m_val = RDIV ((uint64_t)m_val * other.m_val, max_probability);
> + m_quality = MIN (m_quality, other.m_quality);
> + }
> + return *this;
> + }
> + profile_probability operator/ (const profile_probability &other) const
> + {
> + if (*this == profile_probability::never ())
> + return profile_probability::never ();
> + if (!initialized_p () || !other.initialized_p ())
> + return profile_probability::uninitialized ();
> + profile_probability ret;
> + if (m_val >= other.m_val)
> + ret.m_val = max_probability;
> + else if (!m_val)
> + ret.m_val = 0;
> + else
> + {
> + gcc_checking_assert (other.m_val);
> + ret.m_val = MIN (RDIV ((uint64_t)m_val * max_probability,
> + other.m_val),
> + max_probability);
> + }
> + ret.m_quality = MIN (m_quality, other.m_quality);
> + return ret;
> + }
> + profile_probability &operator/= (const profile_probability &other)
> + {
> + if (*this == profile_probability::never ())
> + return *this = profile_probability::never ();
> + if (!initialized_p () || !other.initialized_p ())
> + return *this = profile_probability::uninitialized ();
> + else
> + {
> + if (m_val > other.m_val)
> + m_val = max_probability;
> + else if (!m_val)
> + ;
> + else
> + {
> + gcc_checking_assert (other.m_val);
> + m_val = MIN (RDIV ((uint64_t)m_val * max_probability,
> + other.m_val),
> + max_probability);
> + }
> + m_quality = MIN (m_quality, other.m_quality);
> + }
> + return *this;
> + }
> +
> + gcov_type apply (gcov_type val) const
> + {
> + if (*this == profile_probability::uninitialized ())
> + return val / 2;
> + return RDIV (val * m_val, max_probability);
> + }
> +
> + /* Return 1-*THIS. */
> + profile_probability invert () const
> + {
> + return profile_probability::always() - *this;
> + }
> +
> + profile_probability combine_with_freq (int freq1, profile_probability other,
> + int freq2) const
> + {
> + profile_probability ret;
> +
> + if (*this == profile_probability::uninitialized ()
> + || other == profile_probability::uninitialized ())
> + return profile_probability::uninitialized ();
> +
> + gcc_checking_assert (freq1 >= 0 && freq2 >= 0);
> + if (!freq1 && !freq2)
> + {
> + ret.m_val = (m_val + other.m_val) / 2;
> + }
> + else
> + ret.m_val = RDIV (m_val * (uint64_t) freq1
> + + other.m_val * (uint64_t) freq2, freq1 + freq2);
> + ret.m_quality = MIN (m_quality, other.m_quality);
> + return ret;
> + }
> +
> + /* Return *THIS * NUM / DEN. */
> + profile_probability apply_scale (int64_t num, int64_t den) const
> + {
> + if (*this == profile_probability::never ())
> + return *this;
> + if (!initialized_p ())
> + return profile_probability::uninitialized ();
> + profile_probability ret;
> + ret.m_val = MIN (RDIV (m_val * num, den),
> + max_probability);
> + ret.m_quality = MIN (m_quality, profile_adjusted);
> + return ret;
> + }
> +
> + /* Return true when the probability of edge is reliable.
> +
> + The profile guessing code is good at predicting branch outcome (ie.
> + taken/not taken), that is predicted right slightly over 75% of time.
> + It is however notoriously poor on predicting the probability itself.
> + In general the profile appear a lot flatter (with probabilities closer
> + to 50%) than the reality so it is bad idea to use it to drive optimization
> + such as those disabling dynamic branch prediction for well predictable
> + branches.
> +
> + There are two exceptions - edges leading to noreturn edges and edges
> + predicted by number of iterations heuristics are predicted well. This macro
> + should be able to distinguish those, but at the moment it simply check for
> + noreturn heuristic that is only one giving probability over 99% or bellow
> + 1%. In future we might want to propagate reliability information across the
> + CFG if we find this information useful on multiple places. */
> +
> + bool probably_reliable_p () const
> + {
> + if (m_quality >= profile_adjusted)
> + return true;
> + if (!initialized_p ())
> + return false;
> + return m_val < max_probability / 100
> + || m_val > max_probability - max_probability / 100;
> + }
> +
> + /* Return false if profile_probability is bogus. */
> + bool verify () const
> + {
> + if (m_val == uninitialized_probability)
> + return m_quality == profile_guessed;
> + else
> + return m_val <= REG_BR_PROB_BASE;
> + }
> +
> + /* Comparsions are three-state and conservative. False is returned if
> + the inequality can not be decided. */
> + bool operator< (const profile_probability &other) const
> + {
> + return initialized_p () && other.initialized_p () && m_val < other.m_val;
> + }
> + bool operator> (const profile_probability &other) const
> + {
> + return initialized_p () && other.initialized_p () && m_val > other.m_val;
> + }
> +
> + bool operator<= (const profile_probability &other) const
> + {
> + return initialized_p () && other.initialized_p () && m_val <= other.m_val;
> + }
> + bool operator>= (const profile_probability &other) const
> + {
> + return initialized_p () && other.initialized_p () && m_val >= other.m_val;
> + }
> +
> + /* Output THIS to F. */
> + void dump (FILE *f) const;
> +
> + /* Print THIS to stderr. */
> + void debug () const;
> +
> + /* Return true if THIS is known to differ significantly from OTHER. */
> + bool differs_from_p (profile_probability other) const;
> + /* Return if difference is greater than 50%. */
> + bool differs_lot_from_p (profile_probability other) const;
> +
> + /* LTO streaming support. */
> + static profile_probability stream_in (struct lto_input_block *);
> + void stream_out (struct output_block *);
> + void stream_out (struct lto_output_stream *);
> +};
> +
> /* Main data type to hold profile counters in GCC. In most cases profile
> counts originate from profile feedback. They are 64bit integers
> representing number of executions during the train run.
> @@ -85,7 +501,7 @@ class GTY(()) profile_count
> static const uint64_t uninitialized_count = ((uint64_t) 1 << n_bits) - 1;
>
> uint64_t m_val : n_bits;
> - enum profile_count_quality m_quality : 2;
> + enum profile_quality m_quality : 2;
>
> /* Assume numbers smaller than this to multiply. This is set to make
> testsuite pass, in future we may implement precise multiplication in higer
> @@ -108,7 +524,7 @@ public:
> {
> profile_count c;
> c.m_val = uninitialized_count;
> - c.m_quality = count_guessed;
> + c.m_quality = profile_guessed;
> return c;
> }
>
> @@ -120,7 +536,7 @@ public:
> profile_count ret;
> gcc_checking_assert (v >= 0 && (uint64_t) v <= max_count);
> ret.m_val = v;
> - ret.m_quality = count_read;
> + ret.m_quality = profile_precise;
> return ret;
> }
>
> @@ -207,7 +623,7 @@ public:
> /* Return false if profile_count is bogus. */
> bool verify () const
> {
> - return m_val != uninitialized_count || m_quality == count_guessed;
> + return m_val != uninitialized_count || m_quality == profile_guessed;
> }
>
> /* Comparsions are three-state and conservative. False is returned if
> @@ -237,7 +653,7 @@ public:
> }
> bool operator>= (const profile_count &other) const
> {
> - return initialized_p () && m_val >= other.m_val;
> + return initialized_p () && other.initialized_p () && m_val >= other.m_val;
> }
> bool operator<= (const gcov_type other) const
> {
> @@ -261,7 +677,23 @@ public:
> return profile_count::uninitialized ();
> profile_count ret;
> ret.m_val = RDIV (m_val * prob, REG_BR_PROB_BASE);
> - ret.m_quality = MIN (m_quality, count_adjusted);
> + ret.m_quality = MIN (m_quality, profile_adjusted);
> + return ret;
> + }
> +
> + /* Scale counter according to PROB. */
> + profile_count apply_probability (profile_probability prob) const
> + {
> + if (*this == profile_count::zero ())
> + return *this;
> + if (prob == profile_probability::never ())
> + return profile_count::zero ();
> + if (!initialized_p ())
> + return profile_count::uninitialized ();
> + profile_count ret;
> + ret.m_val = RDIV (m_val * prob.m_val,
> + profile_probability::max_probability);
> + ret.m_quality = MIN (m_quality, prob.m_quality);
> return ret;
> }
> /* Return *THIS * NUM / DEN. */
> @@ -277,7 +709,7 @@ public:
> gcc_checking_assert ((num <= REG_BR_PROB_BASE
> || den <= REG_BR_PROB_BASE) || 1);
> ret.m_val = RDIV (m_val * num, den);
> - ret.m_quality = MIN (m_quality, count_adjusted);
> + ret.m_quality = MIN (m_quality, profile_adjusted);
> return ret;
> }
> profile_count apply_scale (profile_count num, profile_count den) const
> @@ -299,23 +731,27 @@ public:
> else
> ret.m_val = RDIV (m_val * RDIV (num.m_val * max_safe_multiplier,
> den.m_val), max_safe_multiplier);
> - ret.m_quality = MIN (m_quality, count_adjusted);
> + ret.m_quality = MIN (m_quality, profile_adjusted);
> return ret;
> }
>
> /* Return probability of event with counter THIS within event with counter
> OVERALL. */
> - int probability_in (profile_count overall)
> + profile_probability probability_in (const profile_count overall) const
> {
> if (!m_val)
> - return 0;
> - if (!initialized_p () || !overall.initialized_p ())
> - return REG_BR_PROB_BASE / 2;
> - if (overall < *this)
> - return REG_BR_PROB_BASE;
> - if (!overall.m_val)
> - return REG_BR_PROB_BASE / 2;
> - return RDIV (m_val * REG_BR_PROB_BASE, overall.m_val);
> + return profile_probability::never ();
> + if (!initialized_p () || !overall.initialized_p ()
> + || !overall.m_val)
> + return profile_probability::uninitialized ();
> + profile_probability ret;
> + if (overall < m_val)
> + ret.m_val = profile_probability::max_probability;
> + else
> + ret.m_val = RDIV (m_val * profile_probability::max_probability,
> + overall.m_val);
> + ret.m_quality = MIN (m_quality, overall.m_quality);
> + return ret;
> }
>
> /* Output THIS to F. */
> Index: profile.c
> ===================================================================
> --- profile.c (revision 249769)
> +++ profile.c (working copy)
> @@ -768,8 +768,8 @@ compute_branch_probabilities (unsigned c
> if (bb_gcov_count (bb))
> {
> FOR_EACH_EDGE (e, ei, bb->succs)
> - e->probability = GCOV_COMPUTE_SCALE (edge_gcov_count (e),
> - bb_gcov_count (bb));
> + e->probability = profile_probability::probability_in_gcov_type
> + (edge_gcov_count (e), bb_gcov_count (bb));
> if (bb->index >= NUM_FIXED_BLOCKS
> && block_ends_with_condjump_p (bb)
> && EDGE_COUNT (bb->succs) >= 2)
> @@ -784,7 +784,7 @@ compute_branch_probabilities (unsigned c
> if (!(e->flags & (EDGE_FAKE | EDGE_FALLTHRU)))
> break;
>
> - prob = e->probability;
> + prob = e->probability.to_reg_br_prob_base ();
> index = prob * 20 / REG_BR_PROB_BASE;
>
> if (index == 20)
> @@ -810,15 +810,17 @@ compute_branch_probabilities (unsigned c
> {
> FOR_EACH_EDGE (e, ei, bb->succs)
> if (!(e->flags & (EDGE_COMPLEX | EDGE_FAKE)))
> - e->probability = REG_BR_PROB_BASE / total;
> + e->probability
> + = profile_probability::guessed_always ().apply_scale (1, total);
> else
> - e->probability = 0;
> + e->probability = profile_probability::never ();
> }
> else
> {
> total += EDGE_COUNT (bb->succs);
> FOR_EACH_EDGE (e, ei, bb->succs)
> - e->probability = REG_BR_PROB_BASE / total;
> + e->probability
> + = profile_probability::guessed_always ().apply_scale (1, total);
> }
> if (bb->index >= NUM_FIXED_BLOCKS
> && block_ends_with_condjump_p (bb)
> Index: recog.c
> ===================================================================
> --- recog.c (revision 249769)
> +++ recog.c (working copy)
> @@ -3460,8 +3460,7 @@ peep2_attempt (basic_block bb, rtx_insn
> flags);
>
> nehe->probability = eh_edge->probability;
> - nfte->probability
> - = REG_BR_PROB_BASE - nehe->probability;
> + nfte->probability = nehe->probability.invert ();
>
> peep2_do_cleanup_cfg |= purge_dead_edges (nfte->dest);
> bb = nfte->src;
> Index: sched-ebb.c
> ===================================================================
> --- sched-ebb.c (revision 249769)
> +++ sched-ebb.c (working copy)
> @@ -648,7 +648,8 @@ schedule_ebbs (void)
> e = find_fallthru_edge (bb->succs);
> if (! e)
> break;
> - if (e->probability <= probability_cutoff)
> + if (e->probability.initialized_p ()
> + && e->probability.to_reg_br_prob_base () <= probability_cutoff)
> break;
> if (e->dest->flags & BB_DISABLE_SCHEDULE)
> break;
> Index: sched-rgn.c
> ===================================================================
> --- sched-rgn.c (revision 249769)
> +++ sched-rgn.c (working copy)
> @@ -507,7 +507,8 @@ find_single_block_region (bool ebbs_p)
> e = find_fallthru_edge (bb->succs);
> if (! e)
> break;
> - if (e->probability <= probability_cutoff)
> + if (e->probability.initialized_p ()
> + && e->probability.to_reg_br_prob_base () <= probability_cutoff)
> break;
> }
>
> @@ -1441,7 +1442,11 @@ compute_dom_prob_ps (int bb)
> FOR_EACH_EDGE (out_edge, out_ei, in_edge->src->succs)
> bitmap_set_bit (pot_split[bb], EDGE_TO_BIT (out_edge));
>
> - prob[bb] += combine_probabilities (prob[pred_bb], in_edge->probability);
> + prob[bb] += combine_probabilities
> + (prob[pred_bb],
> + in_edge->probability.initialized_p ()
> + ? in_edge->probability.to_reg_br_prob_base ()
> + : 0);
> // The rounding divide in combine_probabilities can result in an extra
> // probability increment propagating along 50-50 edges. Eventually when
> // the edges re-merge, the accumulated probability can go slightly above
> @@ -3171,8 +3176,10 @@ schedule_region (int rgn)
> sched_rgn_n_insns += sched_n_insns;
> realloc_bb_state_array (saved_last_basic_block);
> f = find_fallthru_edge (last_bb->succs);
> - if (f && f->probability * 100 / REG_BR_PROB_BASE >=
> - PARAM_VALUE (PARAM_SCHED_STATE_EDGE_PROB_CUTOFF))
> + if (f
> + && (!f->probability.initialized_p ()
> + || f->probability.to_reg_br_prob_base () * 100 / REG_BR_PROB_BASE >=
> + PARAM_VALUE (PARAM_SCHED_STATE_EDGE_PROB_CUTOFF)))
> {
> memcpy (bb_state[f->dest->index], curr_state,
> dfa_state_size);
> Index: sel-sched-ir.c
> ===================================================================
> --- sel-sched-ir.c (revision 249769)
> +++ sel-sched-ir.c (working copy)
> @@ -4747,7 +4747,9 @@ compute_succs_info (insn_t insn, short f
> sinfo->probs_ok.safe_push (
> /* FIXME: Improve calculation when skipping
> inner loop to exits. */
> - si.bb_end ? si.e1->probability : REG_BR_PROB_BASE);
> + si.bb_end && si.e1->probability.initialized_p ()
> + ? si.e1->probability.to_reg_br_prob_base ()
> + : REG_BR_PROB_BASE);
> sinfo->succs_ok_n++;
> }
> else
> @@ -4756,8 +4758,8 @@ compute_succs_info (insn_t insn, short f
> /* Compute all_prob. */
> if (!si.bb_end)
> sinfo->all_prob = REG_BR_PROB_BASE;
> - else
> - sinfo->all_prob += si.e1->probability;
> + else if (si.e1->probability.initialized_p ())
> + sinfo->all_prob += si.e1->probability.to_reg_br_prob_base ();
>
> sinfo->all_succs_n++;
> }
> Index: stmt.c
> ===================================================================
> --- stmt.c (revision 249769)
> +++ stmt.c (working copy)
> @@ -93,9 +93,9 @@ struct case_node
> tree low; /* Lowest index value for this label */
> tree high; /* Highest index value for this label */
> tree code_label; /* Label to jump to when node matches */
> - int prob; /* Probability of taking this case. */
> + profile_probability prob; /* Probability of taking this case. */
> /* Probability of reaching subtree rooted at this node */
> - int subtree_prob;
> + profile_probability subtree_prob;
> };
>
> typedef struct case_node *case_node_ptr;
> @@ -108,7 +108,8 @@ static void balance_case_nodes (case_nod
> static int node_has_low_bound (case_node_ptr, tree);
> static int node_has_high_bound (case_node_ptr, tree);
> static int node_is_bounded (case_node_ptr, tree);
> -static void emit_case_nodes (rtx, case_node_ptr, rtx_code_label *, int, tree);
> +static void emit_case_nodes (rtx, case_node_ptr, rtx_code_label *,
> + profile_probability, tree);
>
> /* Return the rtx-label that corresponds to a LABEL_DECL,
> creating it if necessary. */
> @@ -704,9 +705,8 @@ expand_naked_return (void)
> is the probability of jumping to LABEL. */
> static void
> do_jump_if_equal (machine_mode mode, rtx op0, rtx op1, rtx_code_label *label,
> - int unsignedp, int prob)
> + int unsignedp, profile_probability prob)
> {
> - gcc_assert (prob <= REG_BR_PROB_BASE);
> do_compare_rtx_and_jump (op0, op1, EQ, unsignedp, mode,
> NULL_RTX, NULL, label, prob);
> }
> @@ -722,7 +722,7 @@ do_jump_if_equal (machine_mode mode, rtx
>
> static struct case_node *
> add_case_node (struct case_node *head, tree low, tree high,
> - tree label, int prob,
> + tree label, profile_probability prob,
> object_allocator<case_node> &case_node_pool)
> {
> struct case_node *r;
> @@ -859,7 +859,7 @@ expand_switch_as_decision_tree_p (tree r
> static void
> emit_case_decision_tree (tree index_expr, tree index_type,
> case_node_ptr case_list, rtx_code_label *default_label,
> - int default_prob)
> + profile_probability default_prob)
> {
> rtx index = expand_normal (index_expr);
>
> @@ -902,14 +902,14 @@ emit_case_decision_tree (tree index_expr
>
> /* Return the sum of probabilities of outgoing edges of basic block BB. */
>
> -static int
> +static profile_probability
> get_outgoing_edge_probs (basic_block bb)
> {
> edge e;
> edge_iterator ei;
> - int prob_sum = 0;
> + profile_probability prob_sum = profile_probability::never ();
> if (!bb)
> - return 0;
> + return profile_probability::never ();
> FOR_EACH_EDGE (e, ei, bb->succs)
> prob_sum += e->probability;
> return prob_sum;
> @@ -922,16 +922,11 @@ get_outgoing_edge_probs (basic_block bb)
> BASE_PROB is the probability of reaching the branch instruction relative
> to the same basic block BB. */
>
> -static inline int
> -conditional_probability (int target_prob, int base_prob)
> +static inline profile_probability
> +conditional_probability (profile_probability target_prob,
> + profile_probability base_prob)
> {
> - if (base_prob > 0)
> - {
> - gcc_assert (target_prob >= 0);
> - gcc_assert (target_prob <= base_prob);
> - return GCOV_COMPUTE_SCALE (target_prob, base_prob);
> - }
> - return -1;
> + return target_prob / base_prob;
> }
>
> /* Generate a dispatch tabler, switching on INDEX_EXPR and jumping to
> @@ -960,12 +955,13 @@ emit_case_dispatch_table (tree index_exp
> rtx_code_label *table_label = gen_label_rtx ();
> bool has_gaps = false;
> edge default_edge = stmt_bb ? EDGE_SUCC (stmt_bb, 0) : NULL;
> - int default_prob = default_edge ? default_edge->probability : 0;
> - int base = get_outgoing_edge_probs (stmt_bb);
> + profile_probability default_prob = default_edge ? default_edge->probability
> + : profile_probability::never ();
> + profile_probability base = get_outgoing_edge_probs (stmt_bb);
> bool try_with_tablejump = false;
>
> - int new_default_prob = conditional_probability (default_prob,
> - base);
> + profile_probability new_default_prob = conditional_probability (default_prob,
> + base);
>
> if (! try_casesi (index_type, index_expr, minval, range,
> table_label, default_label, fallback_label,
> @@ -1030,15 +1026,16 @@ emit_case_dispatch_table (tree index_exp
> through the indirect jump or the direct conditional jump
> before that. Split the probability of reaching the
> default label among these two jumps. */
> - new_default_prob = conditional_probability (default_prob/2,
> + new_default_prob = conditional_probability (default_prob.apply_scale
> + (1, 2),
> base);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> base -= default_prob;
> }
> else
> {
> base -= default_prob;
> - default_prob = 0;
> + default_prob = profile_probability::never ();
> }
>
> if (default_edge)
> @@ -1047,12 +1044,12 @@ emit_case_dispatch_table (tree index_exp
> /* We have altered the probability of the default edge. So the probabilities
> of all other edges need to be adjusted so that it sums up to
> REG_BR_PROB_BASE. */
> - if (base)
> + if (base > profile_probability::never ())
> {
> edge e;
> edge_iterator ei;
> FOR_EACH_EDGE (e, ei, stmt_bb->succs)
> - e->probability = GCOV_COMPUTE_SCALE (e->probability, base);
> + e->probability /= base;
> }
>
> if (try_with_tablejump)
> @@ -1150,7 +1147,7 @@ expand_case (gswitch *stmt)
> default_label = jump_target_rtx
> (CASE_LABEL (gimple_switch_default_label (stmt)));
> edge default_edge = EDGE_SUCC (bb, 0);
> - int default_prob = default_edge->probability;
> + profile_probability default_prob = default_edge->probability;
>
> /* Get upper and lower bounds of case values. */
> elt = gimple_switch_label (stmt, 1);
> @@ -1213,7 +1210,7 @@ expand_case (gswitch *stmt)
> edge case_edge = find_edge (bb, case_bb);
> case_list = add_case_node (
> case_list, low, high, lab,
> - case_edge->probability / (intptr_t)(case_edge->aux),
> + case_edge->probability.apply_scale (1, (intptr_t)(case_edge->aux)),
> case_node_pool);
> }
> reset_out_edges_aux (bb);
> @@ -1310,7 +1307,8 @@ expand_sjlj_dispatch_table (rtx dispatch
> {
> tree elt = dispatch_table[i];
> rtx_code_label *lab = jump_target_rtx (CASE_LABEL (elt));
> - do_jump_if_equal (index_mode, index, zero, lab, 0, -1);
> + do_jump_if_equal (index_mode, index, zero, lab, 0,
> + profile_probability::uninitialized ());
> force_expand_binop (index_mode, sub_optab,
> index, CONST1_RTX (index_mode),
> index, 0, OPTAB_DIRECT);
> @@ -1332,7 +1330,10 @@ expand_sjlj_dispatch_table (rtx dispatch
> tree elt = dispatch_table[i];
> tree low = CASE_LOW (elt);
> tree lab = CASE_LABEL (elt);
> - case_list = add_case_node (case_list, low, low, lab, 0, case_node_pool);
> + case_list = add_case_node (case_list, low, low, lab,
> + profile_probability::guessed_always ()
> + .apply_scale (1, ncases),
> + case_node_pool);
> }
>
> emit_case_dispatch_table (index_expr, index_type,
> @@ -1576,12 +1577,12 @@ node_is_bounded (case_node_ptr node, tre
>
> static void
> emit_case_nodes (rtx index, case_node_ptr node, rtx_code_label *default_label,
> - int default_prob, tree index_type)
> + profile_probability default_prob, tree index_type)
> {
> /* If INDEX has an unsigned type, we must make unsigned branches. */
> int unsignedp = TYPE_UNSIGNED (index_type);
> - int probability;
> - int prob = node->prob, subtree_prob = node->subtree_prob;
> + profile_probability probability;
> + profile_probability prob = node->prob, subtree_prob = node->subtree_prob;
> machine_mode mode = GET_MODE (index);
> machine_mode imode = TYPE_MODE (index_type);
>
> @@ -1701,7 +1702,7 @@ emit_case_nodes (rtx index, case_node_pt
> subtree or the left subtree. Divide the probability
> equally. */
> probability = conditional_probability (
> - node->right->subtree_prob + default_prob/2,
> + node->right->subtree_prob + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> /* See if the value is on the right. */
> emit_cmp_and_jump_insns (index,
> @@ -1712,7 +1713,7 @@ emit_case_nodes (rtx index, case_node_pt
> GT, NULL_RTX, mode, unsignedp,
> label_rtx (test_label),
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
>
> /* Value must be on the left.
> Handle the left-hand subtree. */
> @@ -1743,7 +1744,7 @@ emit_case_nodes (rtx index, case_node_pt
> if (!node_has_low_bound (node, index_type))
> {
> probability = conditional_probability (
> - default_prob/2,
> + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> emit_cmp_and_jump_insns (index,
> convert_modes
> @@ -1753,7 +1754,7 @@ emit_case_nodes (rtx index, case_node_pt
> LT, NULL_RTX, mode, unsignedp,
> default_label,
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> }
>
> emit_case_nodes (index, node->right, default_label, default_prob, index_type);
> @@ -1785,7 +1786,7 @@ emit_case_nodes (rtx index, case_node_pt
> if (!node_has_high_bound (node, index_type))
> {
> probability = conditional_probability (
> - default_prob/2,
> + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> emit_cmp_and_jump_insns (index,
> convert_modes
> @@ -1795,7 +1796,7 @@ emit_case_nodes (rtx index, case_node_pt
> GT, NULL_RTX, mode, unsignedp,
> default_label,
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> }
>
> emit_case_nodes (index, node->left, default_label,
> @@ -1858,7 +1859,7 @@ emit_case_nodes (rtx index, case_node_pt
> test_label = build_decl (curr_insn_location (),
> LABEL_DECL, NULL_TREE, void_type_node);
> probability = conditional_probability (
> - node->right->subtree_prob + default_prob/2,
> + node->right->subtree_prob + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> emit_cmp_and_jump_insns (index,
> convert_modes
> @@ -1868,7 +1869,7 @@ emit_case_nodes (rtx index, case_node_pt
> GT, NULL_RTX, mode, unsignedp,
> label_rtx (test_label),
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> }
>
> /* Value belongs to this node or to the left-hand subtree. */
> @@ -1909,7 +1910,7 @@ emit_case_nodes (rtx index, case_node_pt
> if (!node_has_low_bound (node, index_type))
> {
> probability = conditional_probability (
> - default_prob/2,
> + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> emit_cmp_and_jump_insns (index,
> convert_modes
> @@ -1919,7 +1920,7 @@ emit_case_nodes (rtx index, case_node_pt
> LT, NULL_RTX, mode, unsignedp,
> default_label,
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> }
>
> /* Value belongs to this node or to the right-hand subtree. */
> @@ -1946,7 +1947,7 @@ emit_case_nodes (rtx index, case_node_pt
> if (!node_has_high_bound (node, index_type))
> {
> probability = conditional_probability (
> - default_prob/2,
> + default_prob.apply_scale (1, 2),
> subtree_prob + default_prob);
> emit_cmp_and_jump_insns (index,
> convert_modes
> @@ -1956,7 +1957,7 @@ emit_case_nodes (rtx index, case_node_pt
> GT, NULL_RTX, mode, unsignedp,
> default_label,
> probability);
> - default_prob /= 2;
> + default_prob = default_prob.apply_scale (1, 2);
> }
>
> /* Value belongs to this node or to the left-hand subtree. */
> Index: targhooks.c
> ===================================================================
> --- targhooks.c (revision 249769)
> +++ targhooks.c (working copy)
> @@ -60,6 +60,7 @@ along with GCC; see the file COPYING3.
> #include "stringpool.h"
> #include "tree-vrp.h"
> #include "tree-ssanames.h"
> +#include "profile-count.h"
> #include "optabs.h"
> #include "regs.h"
> #include "recog.h"
> Index: tracer.c
> ===================================================================
> --- tracer.c (revision 249769)
> +++ tracer.c (working copy)
> @@ -135,10 +135,8 @@ better_p (const_edge e1, const_edge e2)
> if (e1->count.initialized_p () && e2->count.initialized_p ()
> && !(e1->count == e2->count))
> return e1->count > e2->count;
> - if (e1->src->frequency * e1->probability !=
> - e2->src->frequency * e2->probability)
> - return (e1->src->frequency * e1->probability
> - > e2->src->frequency * e2->probability);
> + if (EDGE_FREQUENCY (e1) != EDGE_FREQUENCY (e2))
> + return EDGE_FREQUENCY (e1) > EDGE_FREQUENCY (e2);
> /* This is needed to avoid changes in the decision after
> CFG is modified. */
> if (e1->src != e2->src)
> @@ -160,7 +158,8 @@ find_best_successor (basic_block bb)
> best = e;
> if (!best || ignore_bb_p (best->dest))
> return NULL;
> - if (best->probability <= probability_cutoff)
> + if (best->probability.initialized_p ()
> + && best->probability.to_reg_br_prob_base () <= probability_cutoff)
> return NULL;
> return best;
> }
> Index: trans-mem.c
> ===================================================================
> --- trans-mem.c (revision 249769)
> +++ trans-mem.c (working copy)
> @@ -2934,9 +2934,9 @@ expand_transaction (struct tm_region *re
> join_bb->frequency = test_bb->frequency = transaction_bb->frequency;
> join_bb->count = test_bb->count = transaction_bb->count;
>
> - ei->probability = PROB_ALWAYS;
> - et->probability = PROB_LIKELY;
> - ef->probability = PROB_UNLIKELY;
> + ei->probability = profile_probability::always ();
> + et->probability = profile_probability::likely ();
> + ef->probability = profile_probability::unlikely ();
> et->count = test_bb->count.apply_probability (et->probability);
> ef->count = test_bb->count.apply_probability (ef->probability);
>
> @@ -2967,20 +2967,20 @@ expand_transaction (struct tm_region *re
> edge ei = make_edge (transaction_bb, test_bb, EDGE_FALLTHRU);
> test_bb->frequency = transaction_bb->frequency;
> test_bb->count = transaction_bb->count;
> - ei->probability = PROB_ALWAYS;
> + ei->probability = profile_probability::always ();
>
> // Not abort edge. If both are live, chose one at random as we'll
> // we'll be fixing that up below.
> redirect_edge_pred (fallthru_edge, test_bb);
> fallthru_edge->flags = EDGE_FALSE_VALUE;
> - fallthru_edge->probability = PROB_VERY_LIKELY;
> + fallthru_edge->probability = profile_probability::very_likely ();
> fallthru_edge->count = test_bb->count.apply_probability
> (fallthru_edge->probability);
>
> // Abort/over edge.
> redirect_edge_pred (abort_edge, test_bb);
> abort_edge->flags = EDGE_TRUE_VALUE;
> - abort_edge->probability = PROB_VERY_UNLIKELY;
> + abort_edge->probability = profile_probability::unlikely ();
> abort_edge->count = test_bb->count.apply_probability
> (abort_edge->probability);
>
> @@ -3020,13 +3020,13 @@ expand_transaction (struct tm_region *re
> // use the uninst path when falling back to serial mode.
> redirect_edge_pred (inst_edge, test_bb);
> inst_edge->flags = EDGE_FALSE_VALUE;
> - inst_edge->probability = REG_BR_PROB_BASE / 2;
> + inst_edge->probability = profile_probability::even ();
> inst_edge->count
> = test_bb->count.apply_probability (inst_edge->probability);
>
> redirect_edge_pred (uninst_edge, test_bb);
> uninst_edge->flags = EDGE_TRUE_VALUE;
> - uninst_edge->probability = REG_BR_PROB_BASE / 2;
> + uninst_edge->probability = profile_probability::even ();
> uninst_edge->count
> = test_bb->count.apply_probability (uninst_edge->probability);
> }
> Index: tree-call-cdce.c
> ===================================================================
> --- tree-call-cdce.c (revision 249769)
> +++ tree-call-cdce.c (working copy)
> @@ -752,10 +752,6 @@ gen_shrink_wrap_conditions (gcall *bi_ca
> return;
> }
>
> -
> -/* Probability of the branch (to the call) is taken. */
> -#define ERR_PROB 0.01
> -
> /* Shrink-wrap BI_CALL so that it is only called when one of the NCONDS
> conditions in CONDS is false. */
>
> @@ -916,14 +912,15 @@ shrink_wrap_one_built_in_call_with_conds
> basic_block src_bb = call_edge->src;
> gcc_assert (src_bb == nocall_edge->src);
>
> - call_edge->probability = REG_BR_PROB_BASE * ERR_PROB;
> + call_edge->probability = profile_probability::very_unlikely ();
> call_edge->count
> = src_bb->count.apply_probability (call_edge->probability);
> - nocall_edge->probability = inverse_probability (call_edge->probability);
> + nocall_edge->probability = profile_probability::always ()
> + - call_edge->probability;
> nocall_edge->count = src_bb->count - call_edge->count;
>
> - unsigned int call_frequency = apply_probability (src_bb->frequency,
> - call_edge->probability);
> + unsigned int call_frequency
> + = call_edge->probability.apply (src_bb->frequency);
>
> bi_call_bb->count += call_edge->count;
> bi_call_bb->frequency += call_frequency;
> Index: tree-cfg.c
> ===================================================================
> --- tree-cfg.c (revision 249769)
> +++ tree-cfg.c (working copy)
> @@ -2837,9 +2837,7 @@ gimple_split_edge (edge edge_in)
> new_bb = create_empty_bb (after_bb);
> new_bb->frequency = EDGE_FREQUENCY (edge_in);
> new_bb->count = edge_in->count;
> - new_edge = make_edge (new_bb, dest, EDGE_FALLTHRU);
> - new_edge->probability = REG_BR_PROB_BASE;
> - new_edge->count = edge_in->count;
> + new_edge = make_single_succ_edge (new_bb, dest, EDGE_FALLTHRU);
>
> e = redirect_edge_and_branch (edge_in, new_bb);
> gcc_assert (e == edge_in);
> @@ -7244,7 +7242,7 @@ move_sese_region_to_fn (struct function
> basic_block after, bb, *entry_pred, *exit_succ, abb;
> struct function *saved_cfun = cfun;
> int *entry_flag, *exit_flag;
> - unsigned *entry_prob, *exit_prob;
> + profile_probability *entry_prob, *exit_prob;
> unsigned i, num_entry_edges, num_exit_edges, num_nodes;
> edge e;
> edge_iterator ei;
> @@ -7282,7 +7280,7 @@ move_sese_region_to_fn (struct function
> num_entry_edges = EDGE_COUNT (entry_bb->preds);
> entry_pred = XNEWVEC (basic_block, num_entry_edges);
> entry_flag = XNEWVEC (int, num_entry_edges);
> - entry_prob = XNEWVEC (unsigned, num_entry_edges);
> + entry_prob = XNEWVEC (profile_probability, num_entry_edges);
> i = 0;
> for (ei = ei_start (entry_bb->preds); (e = ei_safe_edge (ei)) != NULL;)
> {
> @@ -7297,7 +7295,7 @@ move_sese_region_to_fn (struct function
> num_exit_edges = EDGE_COUNT (exit_bb->succs);
> exit_succ = XNEWVEC (basic_block, num_exit_edges);
> exit_flag = XNEWVEC (int, num_exit_edges);
> - exit_prob = XNEWVEC (unsigned, num_exit_edges);
> + exit_prob = XNEWVEC (profile_probability, num_exit_edges);
> i = 0;
> for (ei = ei_start (exit_bb->succs); (e = ei_safe_edge (ei)) != NULL;)
> {
> Index: tree-cfgcleanup.c
> ===================================================================
> --- tree-cfgcleanup.c (revision 249769)
> +++ tree-cfgcleanup.c (working copy)
> @@ -155,8 +155,6 @@ cleanup_control_expr_graph (basic_block
> }
> if (!warned)
> fold_undefer_and_ignore_overflow_warnings ();
> - if (taken_edge->probability > REG_BR_PROB_BASE)
> - taken_edge->probability = REG_BR_PROB_BASE;
> }
> else
> taken_edge = single_succ_edge (bb);
> Index: tree-eh.c
> ===================================================================
> --- tree-eh.c (revision 249769)
> +++ tree-eh.c (working copy)
> @@ -3244,9 +3244,7 @@ lower_resx (basic_block bb, gresx *stmt,
> }
>
> gcc_assert (EDGE_COUNT (bb->succs) == 0);
> - e = make_edge (bb, new_bb, EDGE_FALLTHRU);
> - e->count = bb->count;
> - e->probability = REG_BR_PROB_BASE;
> + e = make_single_succ_edge (bb, new_bb, EDGE_FALLTHRU);
> }
> else
> {
> @@ -3262,7 +3260,7 @@ lower_resx (basic_block bb, gresx *stmt,
> e = single_succ_edge (bb);
> gcc_assert (e->flags & EDGE_EH);
> e->flags = (e->flags & ~EDGE_EH) | EDGE_FALLTHRU;
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = bb->count;
>
> /* If there are no more EH users of the landing pad, delete it. */
> @@ -4283,7 +4281,7 @@ cleanup_empty_eh_move_lp (basic_block bb
>
> /* Clean up E_OUT for the fallthru. */
> e_out->flags = (e_out->flags & ~EDGE_EH) | EDGE_FALLTHRU;
> - e_out->probability = REG_BR_PROB_BASE;
> + e_out->probability = profile_probability::always ();
> e_out->count = e_out->src->count;
> }
>
> Index: tree-if-conv.c
> ===================================================================
> --- tree-if-conv.c (revision 249769)
> +++ tree-if-conv.c (working copy)
> @@ -2564,7 +2564,8 @@ version_loop_for_if_conversion (struct l
> /* At this point we invalidate porfile confistency until IFN_LOOP_VECTORIZED
> is re-merged in the vectorizer. */
> new_loop = loop_version (loop, cond, &cond_bb,
> - REG_BR_PROB_BASE, REG_BR_PROB_BASE,
> + profile_probability::always (),
> + profile_probability::always (),
> REG_BR_PROB_BASE, REG_BR_PROB_BASE, true);
> free_original_copy_tables ();
>
> Index: tree-inline.c
> ===================================================================
> --- tree-inline.c (revision 249769)
> +++ tree-inline.c (working copy)
> @@ -2291,10 +2291,44 @@ copy_edges_for_bb (basic_block bb, profi
> }
> }
>
> + bool update_probs = false;
> +
> if (gimple_code (copy_stmt) == GIMPLE_EH_DISPATCH)
> - make_eh_dispatch_edges (as_a <geh_dispatch *> (copy_stmt));
> + {
> + make_eh_dispatch_edges (as_a <geh_dispatch *> (copy_stmt));
> + update_probs = true;
> + }
> else if (can_throw)
> - make_eh_edges (copy_stmt);
> + {
> + make_eh_edges (copy_stmt);
> + update_probs = true;
> + }
> +
> + /* EH edges may not match old edges. Copy as much as possible. */
> + if (update_probs)
> + {
> + edge e;
> + edge_iterator ei;
> + basic_block copy_stmt_bb = gimple_bb (copy_stmt);
> +
> + FOR_EACH_EDGE (old_edge, ei, bb->succs)
> + if ((old_edge->flags & EDGE_EH)
> + && (e = find_edge (copy_stmt_bb,
> + (basic_block) old_edge->dest->aux))
> + && (e->flags & EDGE_EH))
> + {
> + e->probability = old_edge->probability;
> + e->count = old_edge->count;
> + }
> +
> + FOR_EACH_EDGE (e, ei, copy_stmt_bb->succs)
> + if ((e->flags & EDGE_EH) && !e->probability.initialized_p ())
> + {
> + e->probability = profile_probability::never ();
> + e->count = profile_count::zero ();
> + }
> + }
> +
>
> /* If the call we inline cannot make abnormal goto do not add
> additional abnormal edges but only retain those already present
> @@ -2317,7 +2351,8 @@ copy_edges_for_bb (basic_block bb, profi
> && gimple_call_arg (copy_stmt, 0) == boolean_true_node)
> nonlocal_goto = false;
> else
> - make_edge (copy_stmt_bb, abnormal_goto_dest, EDGE_ABNORMAL);
> + make_single_succ_edge (copy_stmt_bb, abnormal_goto_dest,
> + EDGE_ABNORMAL);
> }
>
> if ((can_throw || nonlocal_goto)
> @@ -2789,7 +2824,7 @@ copy_cfg_body (copy_body_data * id, prof
> if (new_entry)
> {
> edge e = make_edge (entry_block_map, (basic_block)new_entry->aux, EDGE_FALLTHRU);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = incoming_count;
> }
>
> Index: tree-parloops.c
> ===================================================================
> --- tree-parloops.c (revision 249769)
> +++ tree-parloops.c (working copy)
> @@ -2115,10 +2115,12 @@ create_parallel_loop (struct loop *loop,
> gcc_assert (exit == single_dom_exit (loop));
>
> guard = make_edge (for_bb, ex_bb, 0);
> + /* FIXME: What is the probability? */
> + guard->probability = profile_probability::guessed_never ();
> /* Split the latch edge, so LOOPS_HAVE_SIMPLE_LATCHES is still valid. */
> loop->latch = split_edge (single_succ_edge (loop->latch));
> single_pred_edge (loop->latch)->flags = 0;
> - end = make_edge (single_pred (loop->latch), ex_bb, EDGE_FALLTHRU);
> + end = make_single_succ_edge (single_pred (loop->latch), ex_bb, EDGE_FALLTHRU);
> rescan_loop_exit (end, true, false);
>
> for (gphi_iterator gpi = gsi_start_phis (ex_bb);
> @@ -2358,7 +2360,9 @@ gen_parallel_loop (struct loop *loop,
> /* We assume that the loop usually iterates a lot. */
> prob = 4 * REG_BR_PROB_BASE / 5;
> loop_version (loop, many_iterations_cond, NULL,
> - prob, REG_BR_PROB_BASE - prob,
> + profile_probability::from_reg_br_prob_base (prob),
> + profile_probability::from_reg_br_prob_base
> + (REG_BR_PROB_BASE - prob),
> prob, REG_BR_PROB_BASE - prob, true);
> update_ssa (TODO_update_ssa);
> free_original_copy_tables ();
> @@ -3132,6 +3136,8 @@ oacc_entry_exit_single_gang (bitmap in_l
> gsi_insert_after (&gsi2, cond, GSI_NEW_STMT);
>
> edge e3 = make_edge (bb, bb3, EDGE_FALSE_VALUE);
> + /* FIXME: What is the probability? */
> + e3->probability = profile_probability::guessed_never ();
> e->flags = EDGE_TRUE_VALUE;
>
> tree vdef = gimple_vdef (stmt);
> Index: tree-profile.c
> ===================================================================
> --- tree-profile.c (revision 249769)
> +++ tree-profile.c (working copy)
> @@ -436,16 +436,16 @@ gimple_gen_ic_func_profiler (void)
> edge true_edge = single_succ_edge (cond_bb);
> true_edge->flags = EDGE_TRUE_VALUE;
>
> - int probability;
> + profile_probability probability;
> if (DECL_VIRTUAL_P (current_function_decl))
> - probability = PROB_VERY_LIKELY;
> + probability = profile_probability::very_likely ();
> else
> - probability = PROB_UNLIKELY;
> + probability = profile_probability::unlikely ();
>
> true_edge->probability = probability;
> edge e = make_edge (cond_bb, single_succ_edge (update_bb)->dest,
> EDGE_FALSE_VALUE);
> - e->probability = REG_BR_PROB_BASE - true_edge->probability;
> + e->probability = true_edge->probability.invert ();
>
> /* Insert code:
>
> @@ -497,10 +497,10 @@ gimple_gen_time_profiler (unsigned tag,
>
> edge true_edge = single_succ_edge (cond_bb);
> true_edge->flags = EDGE_TRUE_VALUE;
> - true_edge->probability = PROB_UNLIKELY;
> + true_edge->probability = profile_probability::unlikely ();
> edge e
> = make_edge (cond_bb, single_succ_edge (update_bb)->dest, EDGE_FALSE_VALUE);
> - e->probability = REG_BR_PROB_BASE - true_edge->probability;
> + e->probability = true_edge->probability.invert ();
>
> gimple_stmt_iterator gsi = gsi_start_bb (cond_bb);
> tree original_ref = tree_coverage_counter_ref (tag, base);
> Index: tree-ssa-dce.c
> ===================================================================
> --- tree-ssa-dce.c (revision 249769)
> +++ tree-ssa-dce.c (working copy)
> @@ -1054,7 +1054,7 @@ remove_dead_stmt (gimple_stmt_iterator *
> e = e2;
> }
> gcc_assert (e);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = bb->count;
>
> /* The edge is no longer associated with a conditional, so it does
> Index: tree-ssa-ifcombine.c
> ===================================================================
> --- tree-ssa-ifcombine.c (revision 249769)
> +++ tree-ssa-ifcombine.c (working copy)
> @@ -363,18 +363,14 @@ update_profile_after_ifcombine (basic_bl
> inner_taken->count += outer2->count;
> outer2->count = profile_count::zero ();
>
> - inner_taken->probability = outer2->probability
> - + RDIV (outer_to_inner->probability
> - * inner_taken->probability,
> - REG_BR_PROB_BASE);
> - if (inner_taken->probability > REG_BR_PROB_BASE)
> - inner_taken->probability = REG_BR_PROB_BASE;
> - inner_not_taken->probability = REG_BR_PROB_BASE
> + inner_taken->probability = outer2->probability + outer_to_inner->probability
> + * inner_taken->probability;
> + inner_not_taken->probability = profile_probability::always ()
> - inner_taken->probability;
>
> - outer_to_inner->probability = REG_BR_PROB_BASE;
> + outer_to_inner->probability = profile_probability::always ();
> inner_cond_bb->frequency = outer_cond_bb->frequency;
> - outer2->probability = 0;
> + outer2->probability = profile_probability::never ();
> }
>
> /* If-convert on a and pattern with a common else block. The inner
> Index: tree-ssa-loop-im.c
> ===================================================================
> --- tree-ssa-loop-im.c (revision 249769)
> +++ tree-ssa-loop-im.c (working copy)
> @@ -1801,7 +1801,7 @@ execute_sm_if_changed (edge ex, tree mem
> int freq_sum = 0;
> profile_count count_sum = profile_count::zero ();
> int nbbs = 0, ncount = 0;
> - int flag_probability = -1;
> + profile_probability flag_probability = profile_probability::uninitialized ();
>
> /* Flag is set in FLAG_BBS. Determine probability that flag will be true
> at loop exit.
> @@ -1824,27 +1824,29 @@ execute_sm_if_changed (edge ex, tree mem
> if ((*it)->count.initialized_p ())
> count_sum += (*it)->count, ncount ++;
> if (dominated_by_p (CDI_DOMINATORS, ex->src, *it))
> - flag_probability = REG_BR_PROB_BASE;
> + flag_probability = profile_probability::always ();
> nbbs++;
> }
>
> - if (flag_probability != -1)
> + profile_probability cap = profile_probability::always ().apply_scale (2, 3);
> +
> + if (flag_probability.initialized_p ())
> ;
> else if (ncount == nbbs && count_sum > 0 && preheader->count >= count_sum)
> {
> flag_probability = count_sum.probability_in (preheader->count);
> - if (flag_probability > REG_BR_PROB_BASE * 2 / 3)
> - flag_probability = REG_BR_PROB_BASE * 2 / 3;
> + if (flag_probability > cap)
> + flag_probability = cap;
> }
> else if (freq_sum > 0 && EDGE_FREQUENCY (preheader) >= freq_sum)
> {
> - flag_probability = GCOV_COMPUTE_SCALE (freq_sum,
> - EDGE_FREQUENCY (preheader));
> - if (flag_probability > REG_BR_PROB_BASE * 2 / 3)
> - flag_probability = REG_BR_PROB_BASE * 2 / 3;
> + flag_probability = profile_probability::from_reg_br_prob_base
> + (GCOV_COMPUTE_SCALE (freq_sum, EDGE_FREQUENCY (preheader)));
> + if (flag_probability > cap)
> + flag_probability = cap;
> }
> else
> - flag_probability = REG_BR_PROB_BASE * 2 / 3;
> + flag_probability = cap;
>
> /* ?? Insert store after previous store if applicable. See note
> below. */
> @@ -1876,7 +1878,7 @@ execute_sm_if_changed (edge ex, tree mem
> old_dest = ex->dest;
> new_bb = split_edge (ex);
> then_bb = create_empty_bb (new_bb);
> - then_bb->frequency = apply_probability (new_bb->frequency, flag_probability);
> + then_bb->frequency = flag_probability.apply (new_bb->frequency);
> then_bb->count = new_bb->count.apply_probability (flag_probability);
> if (irr)
> then_bb->flags = BB_IRREDUCIBLE_LOOP;
> @@ -1901,13 +1903,11 @@ execute_sm_if_changed (edge ex, tree mem
> e1->flags |= EDGE_FALSE_VALUE | (irr ? EDGE_IRREDUCIBLE_LOOP : 0);
> e1->flags &= ~EDGE_FALLTHRU;
>
> - e1->probability = REG_BR_PROB_BASE - flag_probability;
> + e1->probability = flag_probability.invert ();
> e1->count = new_bb->count - then_bb->count;
>
> - then_old_edge = make_edge (then_bb, old_dest,
> + then_old_edge = make_single_succ_edge (then_bb, old_dest,
> EDGE_FALLTHRU | (irr ? EDGE_IRREDUCIBLE_LOOP : 0));
> - then_old_edge->probability = REG_BR_PROB_BASE;
> - then_old_edge->count = then_bb->count;
>
> set_immediate_dominator (CDI_DOMINATORS, then_bb, new_bb);
>
> Index: tree-ssa-loop-ivcanon.c
> ===================================================================
> --- tree-ssa-loop-ivcanon.c (revision 249769)
> +++ tree-ssa-loop-ivcanon.c (working copy)
> @@ -529,7 +529,7 @@ remove_exits_and_undefined_stmts (struct
> }
> if (!loop_exit_edge_p (loop, exit_edge))
> exit_edge = EDGE_SUCC (bb, 1);
> - exit_edge->probability = REG_BR_PROB_BASE;
> + exit_edge->probability = profile_probability::always ();
> exit_edge->count = exit_edge->src->count;
> gcc_checking_assert (loop_exit_edge_p (loop, exit_edge));
> gcond *cond_stmt = as_a <gcond *> (elt->stmt);
> @@ -642,7 +642,7 @@ unloop_loops (bitmap loop_closed_ssa_inv
> it in. */
> stmt = gimple_build_call (builtin_decl_implicit (BUILT_IN_UNREACHABLE), 0);
> latch_edge = make_edge (latch, create_basic_block (NULL, NULL, latch), flags);
> - latch_edge->probability = 0;
> + latch_edge->probability = profile_probability::never ();
> latch_edge->count = profile_count::zero ();
> latch_edge->flags |= flags;
> latch_edge->goto_locus = locus;
> @@ -1106,7 +1106,7 @@ try_peel_loop (struct loop *loop,
> }
> int scale = 1;
> if (loop->header->count > 0)
> - scale = entry_count.probability_in (loop->header->count);
> + scale = entry_count.probability_in (loop->header->count).to_reg_br_prob_base ();
> else if (loop->header->frequency)
> scale = RDIV (entry_freq * REG_BR_PROB_BASE, loop->header->frequency);
> scale_loop_profile (loop, scale, 0);
> Index: tree-ssa-loop-manip.c
> ===================================================================
> --- tree-ssa-loop-manip.c (revision 249769)
> +++ tree-ssa-loop-manip.c (working copy)
> @@ -1244,7 +1244,10 @@ tree_transform_and_unroll_loop (struct l
> scale_rest = REG_BR_PROB_BASE;
>
> new_loop = loop_version (loop, enter_main_cond, NULL,
> - prob_entry, REG_BR_PROB_BASE - prob_entry,
> + profile_probability::from_reg_br_prob_base
> + (prob_entry),
> + profile_probability::from_reg_br_prob_base
> + (REG_BR_PROB_BASE - prob_entry),
> scale_unrolled, scale_rest, true);
> gcc_assert (new_loop != NULL);
> update_ssa (TODO_update_ssa);
> @@ -1259,9 +1262,11 @@ tree_transform_and_unroll_loop (struct l
> /* Since the exit edge will be removed, the frequency of all the blocks
> in the loop that are dominated by it must be scaled by
> 1 / (1 - exit->probability). */
> - scale_dominated_blocks_in_loop (loop, exit->src,
> - REG_BR_PROB_BASE,
> - REG_BR_PROB_BASE - exit->probability);
> + if (exit->probability.initialized_p ())
> + scale_dominated_blocks_in_loop (loop, exit->src,
> + REG_BR_PROB_BASE,
> + REG_BR_PROB_BASE
> + - exit->probability.to_reg_br_prob_base ());
>
> bsi = gsi_last_bb (exit_bb);
> exit_if = gimple_build_cond (EQ_EXPR, integer_zero_node,
> @@ -1278,11 +1283,13 @@ tree_transform_and_unroll_loop (struct l
> new_exit->count = exit->count;
> new_exit->probability = exit->probability;
> new_nonexit = single_pred_edge (loop->latch);
> - new_nonexit->probability = REG_BR_PROB_BASE - exit->probability;
> + new_nonexit->probability = exit->probability.invert ();
> new_nonexit->flags = EDGE_TRUE_VALUE;
> new_nonexit->count -= exit->count;
> - scale_bbs_frequencies_int (&loop->latch, 1, new_nonexit->probability,
> - REG_BR_PROB_BASE);
> + if (new_nonexit->probability.initialized_p ())
> + scale_bbs_frequencies_int (&loop->latch, 1,
> + new_nonexit->probability.to_reg_br_prob_base (),
> + REG_BR_PROB_BASE);
>
> old_entry = loop_preheader_edge (loop);
> new_entry = loop_preheader_edge (new_loop);
> @@ -1368,24 +1375,29 @@ tree_transform_and_unroll_loop (struct l
> if (freq_e == profile_count::zero ())
> freq_e = profile_count::from_gcov_type (1);
> /* This should not overflow. */
> - scale = freq_e.probability_in (freq_h);
> + scale = freq_e.probability_in (freq_h).to_reg_br_prob_base ();
> scale_loop_frequencies (loop, scale, REG_BR_PROB_BASE);
> }
>
> exit_bb = single_pred (loop->latch);
> new_exit = find_edge (exit_bb, rest);
> new_exit->count = loop_preheader_edge (loop)->count;
> - new_exit->probability = REG_BR_PROB_BASE / (new_est_niter + 1);
> + new_exit->probability = profile_probability::always ()
> + .apply_scale (1, new_est_niter + 1);
>
> rest->count += new_exit->count;
> rest->frequency += EDGE_FREQUENCY (new_exit);
>
> new_nonexit = single_pred_edge (loop->latch);
> - prob = new_nonexit->probability;
> - new_nonexit->probability = REG_BR_PROB_BASE - new_exit->probability;
> + if (new_nonexit->probability.initialized_p ())
> + prob = new_nonexit->probability.to_reg_br_prob_base ();
> + else
> + prob = 0;
> + new_nonexit->probability = new_exit->probability.invert ();
> new_nonexit->count = exit_bb->count - new_exit->count;
> if (prob > 0)
> - scale_bbs_frequencies_int (&loop->latch, 1, new_nonexit->probability,
> + scale_bbs_frequencies_int (&loop->latch, 1,
> + new_nonexit->probability.to_reg_br_prob_base (),
> prob);
>
> /* Finally create the new counter for number of iterations and add the new
> Index: tree-ssa-loop-split.c
> ===================================================================
> --- tree-ssa-loop-split.c (revision 249769)
> +++ tree-ssa-loop-split.c (working copy)
> @@ -354,10 +354,10 @@ connect_loops (struct loop *loop1, struc
> }
>
> new_e->count = skip_bb->count;
> - new_e->probability = PROB_LIKELY;
> + new_e->probability = profile_probability::likely ();
> new_e->count = skip_e->count.apply_probability (PROB_LIKELY);
> skip_e->count -= new_e->count;
> - skip_e->probability = inverse_probability (PROB_LIKELY);
> + skip_e->probability = profile_probability::unlikely ();
>
> return new_e;
> }
> @@ -559,8 +559,11 @@ split_loop (struct loop *loop1, struct t
> them, and fix up SSA form for that. */
> initialize_original_copy_tables ();
> basic_block cond_bb;
> +
> + /* FIXME: probabilities seems wrong here. */
> struct loop *loop2 = loop_version (loop1, cond, &cond_bb,
> - REG_BR_PROB_BASE, REG_BR_PROB_BASE,
> + profile_probability::always (),
> + profile_probability::always (),
> REG_BR_PROB_BASE, REG_BR_PROB_BASE,
> true);
> gcc_assert (loop2);
> Index: tree-ssa-loop-unswitch.c
> ===================================================================
> --- tree-ssa-loop-unswitch.c (revision 249769)
> +++ tree-ssa-loop-unswitch.c (working copy)
> @@ -480,7 +480,7 @@ static struct loop *
> tree_unswitch_loop (struct loop *loop,
> basic_block unswitch_on, tree cond)
> {
> - unsigned prob_true;
> + profile_probability prob_true;
> edge edge_true, edge_false;
>
> /* Some sanity checking. */
> @@ -490,9 +490,13 @@ tree_unswitch_loop (struct loop *loop,
>
> extract_true_false_edges_from_block (unswitch_on, &edge_true, &edge_false);
> prob_true = edge_true->probability;
> + int p = prob_true.initialized_p () ? prob_true.to_reg_br_prob_base ()
> + : REG_BR_PROB_BASE / 2;
> return loop_version (loop, unshare_expr (cond),
> - NULL, prob_true, REG_BR_PROB_BASE - prob_true, prob_true,
> - REG_BR_PROB_BASE - prob_true, false);
> + NULL, prob_true,
> + prob_true.invert (),
> + p, REG_BR_PROB_BASE - p,
> + false);
> }
>
> /* Unswitch outer loops by hoisting invariant guard on
> @@ -818,10 +822,13 @@ hoist_guard (struct loop *loop, edge gua
> /* Create new loop pre-header. */
> e = split_block (pre_header, last_stmt (pre_header));
> if (dump_file && (dump_flags & TDF_DETAILS))
> - fprintf (dump_file, " Moving guard %i->%i (prob %i) to bb %i, "
> - "new preheader is %i\n",
> - guard->src->index, guard->dest->index, guard->probability,
> - e->src->index, e->dest->index);
> + {
> + fprintf (dump_file, " Moving guard %i->%i (prob ",
> + guard->src->index, guard->dest->index);
> + guard->probability.dump (dump_file);
> + fprintf (dump_file, ") to bb %i, new preheader is %i\n",
> + e->src->index, e->dest->index);
> + }
>
> gcc_assert (loop_preheader_edge (loop)->src == e->dest);
>
> @@ -854,23 +861,26 @@ hoist_guard (struct loop *loop, edge gua
> }
> new_edge->count = skip_count;
> if (dump_file && (dump_flags & TDF_DETAILS))
> - fprintf (dump_file, " Estimated probability of skipping loop is %i\n",
> - new_edge->probability);
> + {
> + fprintf (dump_file, " Estimated probability of skipping loop is ");
> + new_edge->probability.dump (dump_file);
> + fprintf (dump_file, "\n");
> + }
>
> /* Update profile after the transform:
>
> First decrease count of path from newly hoisted loop guard
> to loop header... */
> e->count -= skip_count;
> - e->probability = REG_BR_PROB_BASE - new_edge->probability;
> + e->probability = new_edge->probability.invert ();
> e->dest->count = e->count;
> e->dest->frequency = EDGE_FREQUENCY (e);
>
> /* ... now update profile to represent that original guard will be optimized
> away ... */
> - guard->probability = 0;
> + guard->probability = profile_probability::never ();
> guard->count = profile_count::zero ();
> - not_guard->probability = REG_BR_PROB_BASE;
> + not_guard->probability = profile_probability::always ();
> /* This count is wrong (frequency of not_guard does not change),
> but will be scaled later. */
> not_guard->count = guard->src->count;
> @@ -888,7 +898,10 @@ hoist_guard (struct loop *loop, edge gua
> {
> if (dump_file && (dump_flags & TDF_DETAILS))
> fprintf (dump_file, " %i", bb->index);
> - scale_bbs_frequencies_int (&bb, 1, e->probability, REG_BR_PROB_BASE);
> + if (e->probability.initialized_p ())
> + scale_bbs_frequencies_int (&bb, 1,
> + e->probability.to_reg_br_prob_base (),
> + REG_BR_PROB_BASE);
> }
> }
>
> Index: tree-ssa-phionlycprop.c
> ===================================================================
> --- tree-ssa-phionlycprop.c (revision 249769)
> +++ tree-ssa-phionlycprop.c (working copy)
> @@ -313,8 +313,6 @@ propagate_rhs_into_lhs (gimple *stmt, tr
> te->flags &= ~(EDGE_TRUE_VALUE | EDGE_FALSE_VALUE);
> te->flags &= ~EDGE_ABNORMAL;
> te->flags |= EDGE_FALLTHRU;
> - if (te->probability > REG_BR_PROB_BASE)
> - te->probability = REG_BR_PROB_BASE;
> }
> }
> }
> Index: tree-ssa-phiopt.c
> ===================================================================
> --- tree-ssa-phiopt.c (revision 249769)
> +++ tree-ssa-phiopt.c (working copy)
> @@ -374,7 +374,7 @@ replace_phi_edge_with_variable (basic_bl
> {
> EDGE_SUCC (cond_block, 0)->flags |= EDGE_FALLTHRU;
> EDGE_SUCC (cond_block, 0)->flags &= ~(EDGE_TRUE_VALUE | EDGE_FALSE_VALUE);
> - EDGE_SUCC (cond_block, 0)->probability = REG_BR_PROB_BASE;
> + EDGE_SUCC (cond_block, 0)->probability = profile_probability::always ();
> EDGE_SUCC (cond_block, 0)->count += EDGE_SUCC (cond_block, 1)->count;
>
> block_to_remove = EDGE_SUCC (cond_block, 1)->dest;
> @@ -384,7 +384,7 @@ replace_phi_edge_with_variable (basic_bl
> EDGE_SUCC (cond_block, 1)->flags |= EDGE_FALLTHRU;
> EDGE_SUCC (cond_block, 1)->flags
> &= ~(EDGE_TRUE_VALUE | EDGE_FALSE_VALUE);
> - EDGE_SUCC (cond_block, 1)->probability = REG_BR_PROB_BASE;
> + EDGE_SUCC (cond_block, 1)->probability = profile_probability::always ();
> EDGE_SUCC (cond_block, 1)->count += EDGE_SUCC (cond_block, 0)->count;
>
> block_to_remove = EDGE_SUCC (cond_block, 0)->dest;
> @@ -1017,7 +1017,7 @@ value_replacement (basic_block cond_bb,
> if (optimize_bb_for_speed_p (cond_bb)
> /* The special case is useless if it has a low probability. */
> && profile_status_for_fn (cfun) != PROFILE_ABSENT
> - && EDGE_PRED (middle_bb, 0)->probability < PROB_EVEN
> + && EDGE_PRED (middle_bb, 0)->probability < profile_probability::even ()
> /* If assign is cheap, there is no point avoiding it. */
> && estimate_num_insns (assign, &eni_time_weights)
> >= 3 * estimate_num_insns (cond, &eni_time_weights))
> Index: tree-ssa-reassoc.c
> ===================================================================
> --- tree-ssa-reassoc.c (revision 249769)
> +++ tree-ssa-reassoc.c (working copy)
> @@ -5843,7 +5843,7 @@ branch_fixup (void)
> gsi_insert_after (&gsi, g, GSI_NEW_STMT);
>
> edge etrue = make_edge (cond_bb, merge_bb, EDGE_TRUE_VALUE);
> - etrue->probability = REG_BR_PROB_BASE / 2;
> + etrue->probability = profile_probability::even ();
> etrue->count = cond_bb->count.apply_scale (1, 2);
> edge efalse = find_edge (cond_bb, then_bb);
> efalse->flags = EDGE_FALSE_VALUE;
> Index: tree-ssa-tail-merge.c
> ===================================================================
> --- tree-ssa-tail-merge.c (revision 249769)
> +++ tree-ssa-tail-merge.c (working copy)
> @@ -1592,9 +1592,10 @@ replace_block_by (basic_block bb1, basic
> else if (bb2->frequency && !bb1->frequency)
> ;
> else if (out_freq_sum)
> - e2->probability = GCOV_COMPUTE_SCALE (EDGE_FREQUENCY (e1)
> - + EDGE_FREQUENCY (e2),
> - out_freq_sum);
> + e2->probability = profile_probability::from_reg_br_prob_base
> + (GCOV_COMPUTE_SCALE (EDGE_FREQUENCY (e1)
> + + EDGE_FREQUENCY (e2),
> + out_freq_sum));
> out_sum += e2->count;
> }
> bb2->frequency += bb1->frequency;
> Index: tree-ssa-threadupdate.c
> ===================================================================
> --- tree-ssa-threadupdate.c (revision 249769)
> +++ tree-ssa-threadupdate.c (working copy)
> @@ -302,7 +302,7 @@ remove_ctrl_stmt_and_useless_edges (basi
> }
> else
> {
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = bb->count;
> ei_next (&ei);
> }
> @@ -546,11 +546,9 @@ static void
> create_edge_and_update_destination_phis (struct redirection_data *rd,
> basic_block bb, int idx)
> {
> - edge e = make_edge (bb, rd->path->last ()->e->dest, EDGE_FALLTHRU);
> + edge e = make_single_succ_edge (bb, rd->path->last ()->e->dest, EDGE_FALLTHRU);
>
> rescan_loop_exit (e, true, false);
> - e->probability = REG_BR_PROB_BASE;
> - e->count = bb->count;
>
> /* We used to copy the thread path here. That was added in 2007
> and dutifully updated through the representation changes in 2013.
> @@ -765,7 +763,8 @@ compute_path_counts (struct redirection_
> /* Handle incoming profile insanities. */
> if (total_count < path_in_count)
> path_in_count = total_count;
> - int onpath_scale = path_in_count.probability_in (total_count);
> + int onpath_scale
> + = path_in_count.probability_in (total_count).to_reg_br_prob_base ();
>
> /* Walk the entire path to do some more computation in order to estimate
> how much of the path_in_count will flow out of the duplicated threading
> @@ -919,7 +918,7 @@ recompute_probabilities (basic_block bb)
> get a flow verification error.
> Not much we can do to make counts/freqs sane without
> redoing the profile estimation. */
> - esucc->probability = REG_BR_PROB_BASE;
> + esucc->probability = profile_probability::guessed_always ();
> }
> }
>
> @@ -978,7 +977,8 @@ update_joiner_offpath_counts (edge epath
> among the duplicated off-path edges based on their original
> ratio to the full off-path count (total_orig_off_path_count).
> */
> - int scale = enonpath->count.probability_in (total_orig_off_path_count);
> + int scale = enonpath->count.probability_in (total_orig_off_path_count)
> + .to_reg_br_prob_base ();
> /* Give the duplicated offpath edge a portion of the duplicated
> total. */
> enonpathdup->count = total_dup_off_path_count.apply_probability (scale);
> @@ -1048,9 +1048,13 @@ freqs_to_counts_path (struct redirection
> /* Scale up the frequency by REG_BR_PROB_BASE, to avoid rounding
> errors applying the probability when the frequencies are very
> small. */
> - ein->count = profile_count::from_gcov_type
> - (apply_probability (ein->src->frequency * REG_BR_PROB_BASE,
> - ein->probability));
> + if (ein->probability.initialized_p ())
> + ein->count = profile_count::from_gcov_type
> + (apply_probability (ein->src->frequency * REG_BR_PROB_BASE,
> + ein->probability.to_reg_br_prob_base ()));
> + else
> + /* FIXME: this is hack; we should track uninitialized values. */
> + ein->count = profile_count::zero ();
> }
>
> for (unsigned int i = 1; i < path->length (); i++)
> @@ -2358,7 +2362,7 @@ duplicate_thread_path (edge entry, edge
> if (e)
> {
> rescan_loop_exit (e, true, false);
> - e->probability = REG_BR_PROB_BASE;
> + e->probability = profile_probability::always ();
> e->count = region_copy[n_region - 1]->count;
> }
>
> Index: tree-switch-conversion.c
> ===================================================================
> --- tree-switch-conversion.c (revision 249769)
> +++ tree-switch-conversion.c (working copy)
> @@ -103,7 +103,7 @@ hoist_edge_and_branch_if_true (gimple_st
>
> e_false->flags &= ~EDGE_FALLTHRU;
> e_false->flags |= EDGE_FALSE_VALUE;
> - e_false->probability = REG_BR_PROB_BASE - e_true->probability;
> + e_false->probability = e_true->probability.invert ();
> e_false->count = split_bb->count - e_true->count;
> new_bb->count = e_false->count;
>
> @@ -556,7 +556,7 @@ struct switch_conv_info
> basic_block final_bb;
>
> /* The probability of the default edge in the replaced switch. */
> - int default_prob;
> + profile_probability default_prob;
>
> /* The count of the default edge in the replaced switch. */
> profile_count default_count;
> @@ -1422,7 +1422,7 @@ gen_inbound_check (gswitch *swtch, struc
> /* flags and profiles of the edge for in-range values */
> if (!info->default_case_nonstandard)
> e01 = make_edge (bb0, bb1, EDGE_TRUE_VALUE);
> - e01->probability = REG_BR_PROB_BASE - info->default_prob;
> + e01->probability = info->default_prob.invert ();
> e01->count = info->other_count;
>
> /* flags and profiles of the edge taking care of out-of-range values */
> @@ -1434,7 +1434,7 @@ gen_inbound_check (gswitch *swtch, struc
> bbf = info->final_bb;
>
> e1f = make_edge (bb1, bbf, EDGE_FALLTHRU);
> - e1f->probability = REG_BR_PROB_BASE;
> + e1f->probability = profile_probability::always ();
> e1f->count = info->other_count;
>
> if (info->default_case_nonstandard)
> @@ -1442,7 +1442,7 @@ gen_inbound_check (gswitch *swtch, struc
> else
> {
> e2f = make_edge (bb2, bbf, EDGE_FALLTHRU);
> - e2f->probability = REG_BR_PROB_BASE;
> + e2f->probability = profile_probability::always ();
> e2f->count = info->default_count;
> }
>
> Index: tree-vect-loop-manip.c
> ===================================================================
> --- tree-vect-loop-manip.c (revision 249769)
> +++ tree-vect-loop-manip.c (working copy)
> @@ -540,7 +540,7 @@ slpeel_tree_duplicate_loop_to_edge_cfg (
> static edge
> slpeel_add_loop_guard (basic_block guard_bb, tree cond,
> basic_block guard_to, basic_block dom_bb,
> - int probability, bool irreducible_p)
> + profile_probability probability, bool irreducible_p)
> {
> gimple_stmt_iterator gsi;
> edge new_e, enter_e;
> @@ -571,7 +571,7 @@ slpeel_add_loop_guard (basic_block guard
> new_e->flags |= EDGE_IRREDUCIBLE_LOOP;
>
> enter_e->count -= new_e->count;
> - enter_e->probability = inverse_probability (probability);
> + enter_e->probability = probability.invert ();
> set_immediate_dominator (CDI_DOMINATORS, guard_to, dom_bb);
>
> /* Split enter_e to preserve LOOPS_HAVE_PREHEADERS. */
> @@ -1660,7 +1660,7 @@ vect_do_peeling (loop_vec_info loop_vinf
> edge e, guard_e;
> tree type = TREE_TYPE (niters), guard_cond;
> basic_block guard_bb, guard_to;
> - int prob_prolog, prob_vector, prob_epilog;
> + profile_probability prob_prolog, prob_vector, prob_epilog;
> int bound_prolog = 0, bound_scalar = 0, bound = 0;
> int vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo);
> int prolog_peeling = LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo);
> @@ -1670,10 +1670,11 @@ vect_do_peeling (loop_vec_info loop_vinf
> if (!prolog_peeling && !epilog_peeling)
> return NULL;
>
> - prob_vector = 9 * REG_BR_PROB_BASE / 10;
> + prob_vector = profile_probability::guessed_always ().apply_scale (9, 10);
> if ((vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo)) == 2)
> vf = 3;
> - prob_prolog = prob_epilog = (vf - 1) * REG_BR_PROB_BASE / vf;
> + prob_prolog = prob_epilog = profile_probability::guessed_always ()
> + .apply_scale (vf - 1, vf);
> vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo);
>
> struct loop *prolog, *epilog = NULL, *loop = LOOP_VINFO_LOOP (loop_vinfo);
> @@ -1718,9 +1719,11 @@ vect_do_peeling (loop_vec_info loop_vinf
> separately. Note in this case, the probability of epilog loop
> needs to be scaled back later. */
> basic_block bb_before_loop = loop_preheader_edge (loop)->src;
> - scale_bbs_frequencies_int (&bb_before_loop, 1, prob_vector,
> + if (prob_vector.initialized_p ())
> + scale_bbs_frequencies_int (&bb_before_loop, 1,
> + prob_vector.to_reg_br_prob_base (),
> REG_BR_PROB_BASE);
> - scale_loop_profile (loop, prob_vector, bound);
> + scale_loop_profile (loop, prob_vector.to_reg_br_prob_base (), bound);
> }
>
> tree niters_prolog = build_int_cst (type, 0);
> @@ -1762,15 +1765,17 @@ vect_do_peeling (loop_vec_info loop_vinf
> guard_to = split_edge (loop_preheader_edge (loop));
> guard_e = slpeel_add_loop_guard (guard_bb, guard_cond,
> guard_to, guard_bb,
> - inverse_probability (prob_prolog),
> + prob_prolog.invert (),
> irred_flag);
> e = EDGE_PRED (guard_to, 0);
> e = (e != guard_e ? e : EDGE_PRED (guard_to, 1));
> slpeel_update_phi_nodes_for_guard1 (prolog, loop, guard_e, e);
>
> - scale_bbs_frequencies_int (&bb_after_prolog, 1, prob_prolog,
> + scale_bbs_frequencies_int (&bb_after_prolog, 1,
> + prob_prolog.to_reg_br_prob_base (),
> REG_BR_PROB_BASE);
> - scale_loop_profile (prolog, prob_prolog, bound_prolog);
> + scale_loop_profile (prolog, prob_prolog.to_reg_br_prob_base (),
> + bound_prolog);
> }
> /* Update init address of DRs. */
> vect_update_inits_of_drs (loop_vinfo, niters_prolog);
> @@ -1834,7 +1839,7 @@ vect_do_peeling (loop_vec_info loop_vinf
> guard_to = split_edge (loop_preheader_edge (epilog));
> guard_e = slpeel_add_loop_guard (guard_bb, guard_cond,
> guard_to, guard_bb,
> - inverse_probability (prob_vector),
> + prob_vector.invert (),
> irred_flag);
> e = EDGE_PRED (guard_to, 0);
> e = (e != guard_e ? e : EDGE_PRED (guard_to, 1));
> @@ -1846,7 +1851,8 @@ vect_do_peeling (loop_vec_info loop_vinf
> guard_to->count = guard_bb->count;
> single_succ_edge (guard_to)->count = guard_to->count;
> /* Scale probability of epilog loop back. */
> - int scale_up = REG_BR_PROB_BASE * REG_BR_PROB_BASE / prob_vector;
> + int scale_up = REG_BR_PROB_BASE * REG_BR_PROB_BASE
> + / prob_vector.to_reg_br_prob_base ();
> scale_loop_frequencies (epilog, scale_up, REG_BR_PROB_BASE);
> }
>
> @@ -1875,7 +1881,7 @@ vect_do_peeling (loop_vec_info loop_vinf
> guard_to = split_edge (single_exit (epilog));
> guard_e = slpeel_add_loop_guard (guard_bb, guard_cond, guard_to,
> skip_vector ? anchor : guard_bb,
> - inverse_probability (prob_epilog),
> + prob_epilog.invert (),
> irred_flag);
> slpeel_update_phi_nodes_for_guard2 (loop, epilog, guard_e,
> single_exit (epilog));
> @@ -1883,13 +1889,13 @@ vect_do_peeling (loop_vec_info loop_vinf
> the guard_bb, which is the case when skip_vector is true. */
> if (guard_bb != bb_before_epilog)
> {
> - prob_epilog = (combine_probabilities (prob_vector, prob_epilog)
> - + inverse_probability (prob_vector));
> + prob_epilog = prob_vector * prob_epilog + prob_vector.invert ();
>
> - scale_bbs_frequencies_int (&bb_before_epilog, 1, prob_epilog,
> + scale_bbs_frequencies_int (&bb_before_epilog, 1,
> + prob_epilog.to_reg_br_prob_base (),
> REG_BR_PROB_BASE);
> }
> - scale_loop_profile (epilog, prob_epilog, bound);
> + scale_loop_profile (epilog, prob_epilog.to_reg_br_prob_base (), bound);
> }
> else
> slpeel_update_phi_nodes_for_lcssa (epilog);
> @@ -2171,7 +2177,10 @@ vect_loop_versioning (loop_vec_info loop
> /* We don't want to scale SCALAR_LOOP's frequencies, we need to
> scale LOOP's frequencies instead. */
> nloop = loop_version (scalar_loop, cond_expr, &condition_bb,
> - prob, REG_BR_PROB_BASE - prob,
> + profile_probability::guessed_always ().apply_scale
> + (prob, REG_BR_PROB_BASE),
> + profile_probability::guessed_always ().apply_scale
> + (REG_BR_PROB_BASE - prob, REG_BR_PROB_BASE),
> REG_BR_PROB_BASE, REG_BR_PROB_BASE - prob, true);
> scale_loop_frequencies (loop, prob, REG_BR_PROB_BASE);
> /* CONDITION_BB was created above SCALAR_LOOP's preheader,
> @@ -2200,7 +2209,10 @@ vect_loop_versioning (loop_vec_info loop
> }
> else
> nloop = loop_version (loop, cond_expr, &condition_bb,
> - prob, REG_BR_PROB_BASE - prob,
> + profile_probability::guessed_always ().apply_scale
> + (prob, REG_BR_PROB_BASE),
> + profile_probability::guessed_always ().apply_scale
> + (REG_BR_PROB_BASE - prob, REG_BR_PROB_BASE),
> prob, REG_BR_PROB_BASE - prob, true);
>
> if (version_niter)
> Index: tree-vect-loop.c
> ===================================================================
> --- tree-vect-loop.c (revision 249769)
> +++ tree-vect-loop.c (working copy)
> @@ -7121,21 +7121,25 @@ scale_profile_for_vect_loop (struct loop
> if (!(freq_e > profile_count::from_gcov_type (1)))
> freq_e = profile_count::from_gcov_type (1);
> /* This should not overflow. */
> - scale = freq_e.apply_scale (new_est_niter + 1, 1).probability_in (freq_h);
> + scale = freq_e.apply_scale (new_est_niter + 1, 1).probability_in (freq_h)
> + .to_reg_br_prob_base ();
> scale_loop_frequencies (loop, scale, REG_BR_PROB_BASE);
> }
>
> basic_block exit_bb = single_pred (loop->latch);
> edge exit_e = single_exit (loop);
> exit_e->count = loop_preheader_edge (loop)->count;
> - exit_e->probability = REG_BR_PROB_BASE / (new_est_niter + 1);
> + exit_e->probability = profile_probability::always ()
> + .apply_scale (1, new_est_niter + 1);
>
> edge exit_l = single_pred_edge (loop->latch);
> - int prob = exit_l->probability;
> - exit_l->probability = REG_BR_PROB_BASE - exit_e->probability;
> + int prob = exit_l->probability.initialized_p ()
> + ? exit_l->probability.to_reg_br_prob_base () : 0;
> + exit_l->probability = exit_e->probability.invert ();
> exit_l->count = exit_bb->count - exit_e->count;
> if (prob > 0)
> - scale_bbs_frequencies_int (&loop->latch, 1, exit_l->probability, prob);
> + scale_bbs_frequencies_int (&loop->latch, 1,
> + exit_l->probability.to_reg_br_prob_base (), prob);
> }
>
> /* Function vect_transform_loop.
> @@ -7658,7 +7662,7 @@ optimize_mask_stores (struct loop *loop)
> e->flags = EDGE_TRUE_VALUE;
> efalse = make_edge (bb, store_bb, EDGE_FALSE_VALUE);
> /* Put STORE_BB to likely part. */
> - efalse->probability = PROB_UNLIKELY;
> + efalse->probability = profile_probability::unlikely ();
> store_bb->frequency = PROB_ALWAYS - EDGE_FREQUENCY (efalse);
> make_edge (store_bb, join_bb, EDGE_FALLTHRU);
> if (dom_info_available_p (CDI_DOMINATORS))
> Index: ubsan.c
> ===================================================================
> --- ubsan.c (revision 249769)
> +++ ubsan.c (working copy)
> @@ -799,7 +799,7 @@ ubsan_expand_null_ifn (gimple_stmt_itera
> /* Make an edge coming from the 'cond block' into the 'then block';
> this edge is unlikely taken, so set up the probability accordingly. */
> e = make_edge (cond_bb, then_bb, EDGE_TRUE_VALUE);
> - e->probability = PROB_VERY_UNLIKELY;
> + e->probability = profile_probability::very_unlikely ();
>
> /* Connect 'then block' with the 'else block'. This is needed
> as the ubsan routines we call in the 'then block' are not noreturn.
> @@ -810,7 +810,7 @@ ubsan_expand_null_ifn (gimple_stmt_itera
> e = find_edge (cond_bb, fallthru_bb);
> e->flags = EDGE_FALSE_VALUE;
> e->count = cond_bb->count;
> - e->probability = REG_BR_PROB_BASE - PROB_VERY_UNLIKELY;
> + e->probability = profile_probability::very_likely ();
>
> /* Update dominance info for the newly created then_bb; note that
> fallthru_bb's dominance info has already been updated by
> @@ -873,13 +873,13 @@ ubsan_expand_null_ifn (gimple_stmt_itera
> this edge is unlikely taken, so set up the probability
> accordingly. */
> e = make_edge (cond1_bb, then_bb, EDGE_TRUE_VALUE);
> - e->probability = PROB_VERY_UNLIKELY;
> + e->probability = profile_probability::very_unlikely ();
>
> /* Set up the fallthrough basic block. */
> e = find_edge (cond1_bb, cond2_bb);
> e->flags = EDGE_FALSE_VALUE;
> e->count = cond1_bb->count;
> - e->probability = REG_BR_PROB_BASE - PROB_VERY_UNLIKELY;
> + e->probability = profile_probability::very_likely ();
>
> /* Update dominance info. */
> if (dom_info_available_p (CDI_DOMINATORS))
> Index: value-prof.c
> ===================================================================
> --- value-prof.c (revision 249769)
> +++ value-prof.c (working copy)
> @@ -104,11 +104,6 @@ along with GCC; see the file COPYING3.
> and gimple_value_profile_transformations table-driven, perhaps...
> */
>
> -static tree gimple_divmod_fixed_value (gassign *, tree, int, gcov_type,
> - gcov_type);
> -static tree gimple_mod_pow2 (gassign *, int, gcov_type, gcov_type);
> -static tree gimple_mod_subtract (gassign *, int, int, int, gcov_type,
> - gcov_type, gcov_type);
> static bool gimple_divmod_fixed_value_transform (gimple_stmt_iterator *);
> static bool gimple_mod_pow2_value_transform (gimple_stmt_iterator *);
> static bool gimple_mod_subtract_transform (gimple_stmt_iterator *);
> @@ -693,7 +688,7 @@ gimple_value_profile_transformations (vo
> alter the original STMT. */
>
> static tree
> -gimple_divmod_fixed_value (gassign *stmt, tree value, int prob,
> +gimple_divmod_fixed_value (gassign *stmt, tree value, profile_probability prob,
> gcov_type count, gcov_type all)
> {
> gassign *stmt1, *stmt2;
> @@ -753,16 +748,16 @@ gimple_divmod_fixed_value (gassign *stmt
> e12->count = profile_count::from_gcov_type (count);
>
> e13 = make_edge (bb, bb3, EDGE_TRUE_VALUE);
> - e13->probability = REG_BR_PROB_BASE - prob;
> + e13->probability = prob.invert ();
> e13->count = profile_count::from_gcov_type (all - count);
>
> remove_edge (e23);
>
> e24 = make_edge (bb2, bb4, EDGE_FALLTHRU);
> - e24->probability = REG_BR_PROB_BASE;
> + e24->probability = profile_probability::always ();
> e24->count = profile_count::from_gcov_type (count);
>
> - e34->probability = REG_BR_PROB_BASE;
> + e34->probability = profile_probability::always ();
> e34->count = profile_count::from_gcov_type (all - count);
>
> return tmp2;
> @@ -777,7 +772,7 @@ gimple_divmod_fixed_value_transform (gim
> enum tree_code code;
> gcov_type val, count, all;
> tree result, value, tree_val;
> - gcov_type prob;
> + profile_probability prob;
> gassign *stmt;
>
> stmt = dyn_cast <gassign *> (gsi_stmt (*si));
> @@ -816,9 +811,9 @@ gimple_divmod_fixed_value_transform (gim
>
> /* Compute probability of taking the optimal path. */
> if (all > 0)
> - prob = GCOV_COMPUTE_SCALE (count, all);
> + prob = profile_probability::probability_in_gcov_type (count, all);
> else
> - prob = 0;
> + prob = profile_probability::never ();
>
> if (sizeof (gcov_type) == sizeof (HOST_WIDE_INT))
> tree_val = build_int_cst (get_gcov_type (), val);
> @@ -855,7 +850,7 @@ gimple_divmod_fixed_value_transform (gim
> the temp; it does not replace or alter the original STMT. */
>
> static tree
> -gimple_mod_pow2 (gassign *stmt, int prob, gcov_type count, gcov_type all)
> +gimple_mod_pow2 (gassign *stmt, profile_probability prob, gcov_type count, gcov_type all)
> {
> gassign *stmt1, *stmt2, *stmt3;
> gcond *stmt4;
> @@ -918,16 +913,16 @@ gimple_mod_pow2 (gassign *stmt, int prob
> e12->count = profile_count::from_gcov_type (count);
>
> e13 = make_edge (bb, bb3, EDGE_TRUE_VALUE);
> - e13->probability = REG_BR_PROB_BASE - prob;
> + e13->probability = prob.invert ();
> e13->count = profile_count::from_gcov_type (all - count);
>
> remove_edge (e23);
>
> e24 = make_edge (bb2, bb4, EDGE_FALLTHRU);
> - e24->probability = REG_BR_PROB_BASE;
> + e24->probability = profile_probability::always ();
> e24->count = profile_count::from_gcov_type (count);
>
> - e34->probability = REG_BR_PROB_BASE;
> + e34->probability = profile_probability::always ();
> e34->count = profile_count::from_gcov_type (all - count);
>
> return result;
> @@ -942,7 +937,7 @@ gimple_mod_pow2_value_transform (gimple_
> enum tree_code code;
> gcov_type count, wrong_values, all;
> tree lhs_type, result, value;
> - gcov_type prob;
> + profile_probability prob;
> gassign *stmt;
>
> stmt = dyn_cast <gassign *> (gsi_stmt (*si));
> @@ -987,9 +982,9 @@ gimple_mod_pow2_value_transform (gimple_
> return false;
>
> if (all > 0)
> - prob = GCOV_COMPUTE_SCALE (count, all);
> + prob = profile_probability::probability_in_gcov_type (count, all);
> else
> - prob = 0;
> + prob = profile_probability::never ();
>
> result = gimple_mod_pow2 (stmt, prob, count, all);
>
> @@ -1009,7 +1004,8 @@ gimple_mod_pow2_value_transform (gimple_
> /* FIXME: Generalize the interface to handle NCOUNTS > 1. */
>
> static tree
> -gimple_mod_subtract (gassign *stmt, int prob1, int prob2, int ncounts,
> +gimple_mod_subtract (gassign *stmt, profile_probability prob1,
> + profile_probability prob2, int ncounts,
> gcov_type count1, gcov_type count2, gcov_type all)
> {
> gassign *stmt1;
> @@ -1079,7 +1075,7 @@ gimple_mod_subtract (gassign *stmt, int
>
> e12->flags &= ~EDGE_FALLTHRU;
> e12->flags |= EDGE_FALSE_VALUE;
> - e12->probability = REG_BR_PROB_BASE - prob1;
> + e12->probability = prob1.invert ();
> e12->count = profile_count::from_gcov_type (all - count1);
>
> e14 = make_edge (bb, bb4, EDGE_TRUE_VALUE);
> @@ -1091,14 +1087,14 @@ gimple_mod_subtract (gassign *stmt, int
> e23->flags &= ~EDGE_FALLTHRU;
> e23->flags |= EDGE_FALSE_VALUE;
> e23->count = profile_count::from_gcov_type (all - count1 - count2);
> - e23->probability = REG_BR_PROB_BASE - prob2;
> + e23->probability = prob2.invert ();
>
> e24 = make_edge (bb2, bb4, EDGE_TRUE_VALUE);
> e24->probability = prob2;
> e24->count = profile_count::from_gcov_type (count2);
> }
>
> - e34->probability = REG_BR_PROB_BASE;
> + e34->probability = profile_probability::always ();
> e34->count = profile_count::from_gcov_type (all - count1 - count2);
>
> return result;
> @@ -1113,7 +1109,7 @@ gimple_mod_subtract_transform (gimple_st
> enum tree_code code;
> gcov_type count, wrong_values, all;
> tree lhs_type, result;
> - gcov_type prob1, prob2;
> + profile_probability prob1, prob2;
> unsigned int i, steps;
> gcov_type count1, count2;
> gassign *stmt;
> @@ -1181,12 +1177,12 @@ gimple_mod_subtract_transform (gimple_st
> /* Compute probability of taking the optimal path(s). */
> if (all > 0)
> {
> - prob1 = GCOV_COMPUTE_SCALE (count1, all);
> - prob2 = GCOV_COMPUTE_SCALE (count2, all);
> + prob1 = profile_probability::probability_in_gcov_type (count1, all);
> + prob2 = profile_probability::probability_in_gcov_type (count2, all);
> }
> else
> {
> - prob1 = prob2 = 0;
> + prob1 = prob2 = profile_probability::never ();
> }
>
> /* In practice, "steps" is always 2. This interface reflects this,
> @@ -1316,7 +1312,7 @@ check_ic_target (gcall *call_stmt, struc
>
> gcall *
> gimple_ic (gcall *icall_stmt, struct cgraph_node *direct_call,
> - int prob, profile_count count, profile_count all)
> + profile_probability prob, profile_count count, profile_count all)
> {
> gcall *dcall_stmt;
> gassign *load_stmt;
> @@ -1386,7 +1382,7 @@ gimple_ic (gcall *icall_stmt, struct cgr
> /* The indirect call might be noreturn. */
> if (e_ij != NULL)
> {
> - e_ij->probability = REG_BR_PROB_BASE;
> + e_ij->probability = profile_probability::always ();
> e_ij->count = all - count;
> e_ij = single_pred_edge (split_edge (e_ij));
> }
> @@ -1402,7 +1398,7 @@ gimple_ic (gcall *icall_stmt, struct cgr
> e_cd->count = count;
>
> e_ci = make_edge (cond_bb, icall_bb, EDGE_FALSE_VALUE);
> - e_ci->probability = REG_BR_PROB_BASE - prob;
> + e_ci->probability = prob.invert ();
> e_ci->count = all - count;
>
> remove_edge (e_di);
> @@ -1414,12 +1410,12 @@ gimple_ic (gcall *icall_stmt, struct cgr
> else
> {
> e_dj = make_edge (dcall_bb, join_bb, EDGE_FALLTHRU);
> - e_dj->probability = REG_BR_PROB_BASE;
> + e_dj->probability = profile_probability::always ();
> e_dj->count = count;
>
> e_ij->count = all - count;
> }
> - e_ij->probability = REG_BR_PROB_BASE;
> + e_ij->probability = profile_probability::always ();
> }
>
> /* Insert PHI node for the call result if necessary. */
> @@ -1497,6 +1493,8 @@ gimple_ic (gcall *icall_stmt, struct cgr
> if (e_eh->flags & (EDGE_EH | EDGE_ABNORMAL))
> {
> e = make_edge (dcall_bb, e_eh->dest, e_eh->flags);
> + e->probability = e_eh->probability;
> + e->count = e_eh->count;
> for (gphi_iterator psi = gsi_start_phis (e_eh->dest);
> !gsi_end_p (psi); gsi_next (&psi))
> {
> @@ -1645,7 +1643,7 @@ interesting_stringop_to_profile_p (gcall
> assuming we'll propagate a true constant into ICALL_SIZE later. */
>
> static void
> -gimple_stringop_fixed_value (gcall *vcall_stmt, tree icall_size, int prob,
> +gimple_stringop_fixed_value (gcall *vcall_stmt, tree icall_size, profile_probability prob,
> gcov_type count, gcov_type all)
> {
> gassign *tmp_stmt;
> @@ -1709,16 +1707,16 @@ gimple_stringop_fixed_value (gcall *vcal
> e_ci->count = profile_count::from_gcov_type (count);
>
> e_cv = make_edge (cond_bb, vcall_bb, EDGE_FALSE_VALUE);
> - e_cv->probability = REG_BR_PROB_BASE - prob;
> + e_cv->probability = prob.invert ();
> e_cv->count = profile_count::from_gcov_type (all - count);
>
> remove_edge (e_iv);
>
> e_ij = make_edge (icall_bb, join_bb, EDGE_FALLTHRU);
> - e_ij->probability = REG_BR_PROB_BASE;
> + e_ij->probability = profile_probability::always ();
> e_ij->count = profile_count::from_gcov_type (count);
>
> - e_vj->probability = REG_BR_PROB_BASE;
> + e_vj->probability = profile_probability::always ();
> e_vj->count = profile_count::from_gcov_type (all - count);
>
> /* Insert PHI node for the call result if necessary. */
> @@ -1753,7 +1751,7 @@ gimple_stringops_transform (gimple_stmt_
> gcov_type count, all, val;
> tree dest, src;
> unsigned int dest_align, src_align;
> - gcov_type prob;
> + profile_probability prob;
> tree tree_val;
> int size_arg;
>
> @@ -1788,9 +1786,9 @@ gimple_stringops_transform (gimple_stmt_
> if (check_counter (stmt, "value", &count, &all, gimple_bb (stmt)->count))
> return false;
> if (all > 0)
> - prob = GCOV_COMPUTE_SCALE (count, all);
> + prob = profile_probability::probability_in_gcov_type (count, all);
> else
> - prob = 0;
> + prob = profile_probability::never ();
>
> dest = gimple_call_arg (stmt, 0);
> dest_align = get_pointer_alignment (dest);
> Index: value-prof.h
> ===================================================================
> --- value-prof.h (revision 249769)
> +++ value-prof.h (working copy)
> @@ -90,8 +90,8 @@ void gimple_move_stmt_histograms (struct
> void verify_histograms (void);
> void free_histograms (function *);
> void stringop_block_profile (gimple *, unsigned int *, HOST_WIDE_INT *);
> -gcall *gimple_ic (gcall *, struct cgraph_node *, int, profile_count,
> - profile_count);
> +gcall *gimple_ic (gcall *, struct cgraph_node *, profile_probability,
> + profile_count, profile_count);
> bool check_ic_target (gcall *, struct cgraph_node *);
>
>
More information about the Gcc-patches
mailing list