[LIPO merge][Augmented profile instrumentation and runtime call-graph analysis]

[Xinliang David Li]

On Mon, Oct 12, 2009 at 1:58 AM, Richard Guenther
<richard.guenther@gmail.com> wrote:
> On Wed, Sep 30, 2009 at 11:36 PM, Raksit Ashok <raksit@google.com> wrote:
>> Hi,
>>
>> This patch merges the "augmented instrumentation" and "runtime
>> call-graph analysis" parts of LIPO
>> (http://gcc.gnu.org/wiki/LightweightIpo) from the lw-ipo branch. LIPO,
>> in its entirety won't be merged into 4.5 -- but getting just this part
>> into 4.5 would be great. The IL Streaming based infrastructure
>> (introduced by LTO) could then leverage LIPO's runtime module-grouping
>> analysis. It would also make merging LIPO into 4.6 easier.
>>
>> This has been bootstrapped on x86_64-unknown-linux-gnu.
>> make check RUNTESTFLAGS="tree-prof.exp" passed.
>> Full regression-testing is underway....
>
> How is this useful to our users in that state?  I can't see that
> this dynamic call graph profile is used anywhere.

Indeed, this is intended as part of the staged checkin for LIPO
functionality --- all transformation part are delebrately stripped off
which makes this checkin
really safe, but in the meantime reduces its usage.  However, it sure
can be used at least as advisory tool to the user for their source
level repartitioning/re-organizing. Diego also indicated the potential
interaction with LTO.


>
> You probably recognize that you're trying to sneak that in a day
> before stage1 ends without ever sending this stuff for reviews
> or notifying people that you intend to merge this for 4.5?

Please be clear it is not intended to be  'Sneaked' in -- though the
decision to give a try did come in late after some internal
discussions.

Thanks,

David

>
> Thanks,
> Richard.
>
>> thanks,
>> raksit
>>
>>
>> 2009-09-30 Xinliang David Li <davidxl@google.com>
>>        Raksit Ashok <raksit@google.com>
>>
>>        * libgcc/Makefile.in: Add dyn-ipa.o to libgcov.
>>        * doc/invoke.texi: Document new flag -fripa.
>>        * dyn-ipa.c: New file.
>>        * value-prof.c (dump_histogram_value): Handle new profile histogram
>>        type, HIST_TYPE_INDIR_CALL_TOPN.
>>        (gimple_indirect_call_to_profile): Instrument indirect calls
>>        differently if -fripa is used.
>>        (gimple_find_values_to_profile): Handle new profile histogram type,
>>        HIST_TYPE_INDIR_CALL_TOPN.
>>        * value-prof.h (enum hist_type): Define new profile histogram type,
>>        HIST_TYPE_INDIR_CALL_TOPN.
>>        (struct profile_hooks): Add direct-call profiler hook.
>>        * tree-pass.h: Declare new direct-call profiling pass.
>>        * incpath.c (get_include_chains): Define new function.
>>        * incpath.h (get_include_chains): Declare new function.
>>        * toplev.h (compile_file): Add the second parameter to the
>>        coverage_init call.
>>        (compile_file): Call coverage_finish if -fripa is used.
>>        * gcov-io.c (gcov_read_module_info): Define new function.
>>        (gcov_truncate): Define new function.
>>        *gcov-io.h (FUNC_ID_WIDTH, FUNC_ID_MASK,
>>        EXTRACT_MODULE_ID_FROM_GLOBAL_ID, EXTRACT_FUNC_ID_FROM_GLOBAL_ID,
>>        GEN_FUNC_GLOBAL_ID): Define new macros.
>>        (GCOV_TAG_MODULE_INFO): Add new type of gcov tag.
>>        (GCOV_COUNTER_ICALL_TOPNV, GCOV_COUNTER_DIRECT_CALL): Add new types
>>        of gcov counters.
>>        (GCOV_COUNTER_NAMES): Add names of new counters to macro.
>>        (GCOV_ICALL_TOPN_VAL, GCOV_ICALL_TOPN_NCOUNTS,
>>        GCOV_MODULE_UNKNOWN_LANG, GCOV_MODULE_C_LANG, GCOV_MODULE_CPP_LANG,
>>        GCOV_MODULE_FORT_LANG): Define new macros.
>>        (GCOV_MERGE_FUNCTIONS): Add new merge functions to macro.
>>        (struct gcov_module_info): New struct.
>>        (struct gcov_info): Add new fields.
>>        (__gcov_merge_dc, __gcov_merge_icall_topn,
>>        __gcov_indirect_call_topn_profiler, __gcov_direct_call_profiler,
>>        __gcov_sort_n_vals, gcov_read_module_info, gcov_write_module_infos,
>>        gcov_get_sorted_import_module_array, gcov_truncate): Declare new
>>        functions.
>>        * opts.c (lipo_cl_args, num_lipo_cl_args): Define new variables.
>>        (lipo_save_cl_args): Define new function.
>>        (handle_options): Add call to lipo_save_cl_args.
>>        * opts.h (lipo_cl_args, num_lipo_cl_args): Declare new variables.
>>        (coverage_note_define): Declare new function.
>>        * function.h (FUNC_DECL_FUNC_ID): Define new macro.
>>        * profile.c (instrument_values): Handle new profile histogram type,
>>        HIST_TYPE_INDIR_CALL_TOPN.
>>        * coverage.c (struct cpp_def_list): Define new struct.
>>        (struct counts_entry): Change type of field "ident" from unsigned to
>>        unsigned HOST_WIDEST_INT.
>>        (da_base_file_name, main_input_file_name, cpp_defines_head,
>>        cpp_defines_tail, num_cpp_defines): New static variables.
>>        (get_gcov_unsigned_t): Make function non-static.
>>        (coverage_function_present, coverage_dc_end_function,
>>        build_inc_path_array_value, build_cpp_def_array_value,
>>        build_cl_args_array_value, build_gcov_module_info_value,
>>        coverage_note_define): Define new functions.
>>        (build_gcov_info): Store additional information when building the
>>        gcov_info struct.
>>        (get_da_file_name): Refactor this function from coverage_init.
>>        (coverage_init): Add a new function parameter.
>>        * coverage.h (coverage_init): Modify function declaration to take an
>>        additional parameter.
>>        (coverage_dc_end_function, coverage_function_present,
>>        get_gcov_unsigned_t): Declare new functions.
>>        * c-opts.c (finish_options): Call coverage_note_define to make a note
>>        of the -D/-U command-line options.
>>        * common.opt: Add new flag -fripa.
>>        * tree-profile.c (tree_init_ic_make_global_vars): Emit different
>>        instrumentation code based on -fripa.
>>        (tree_init_edge_profiler): Initialization related to new types of
>>        instrumentation counters.
>>        (tree_gen_ic_func_topn_profiler, tree_gen_dc_profiler,
>>        tree_gen_dc_func_profiler, do_direct_call_profiling,
>>        direct_call_profiling): Define new functions.
>>        (tree_profiling): Further conditionalize call to
>>        tree_gen_ic_func_profiler on !flag_dyn_ipa.
>>        (pass_direct_call_profile): Define new pass.
>>        (struct profile_hooks tree_profile_hooks): Add direct-call profiling
>>        hook.
>>        * Makefile.in: Update dependencies.
>>        * libgcov.c (THREAD_PREFIX): Define new macro.
>>        (__gcov_list): Rename gcov_list to __gcov_list and make this variable
>>        non-static.
>>        (gcov_cur_module_id, __gcov_direct_call_counters,
>>        __gcov_direct_call_callee, __gcov_indirect_call_topn_counters,
>>        __gcov_indirect_call_topn_callee): Define new variables.
>>        (gcov_strip_leading_dirs, gcov_sort_n_vals,
>>        gcov_sort_icall_topn_counter, gcov_write_import_file,
>>        __gcov_is_gid_insane, __gcov_merge_dc, __gcov_merge_icall_topn,
>>        __gcov_topn_value_profiler_body, __gcov_indirect_call_topn_profiler,
>>        __gcov_direct_call_profiler): Define new functions.
>>        (gcov_exit): Handle new counters, and do module-grouping analysis.
>>        * passes.c (init_optimization_passes): Insert new direct-call
>>        profiling pass.
>>        * gcov-dump.c (dump_aux_modules, tag_module_info): Define new
>>        functions.
>>        (flag_dump_aux_modules_only): New variable.
>>        (struct option options): Add new option for dumping the list of
>>        auxiliary modules.
>>        (main): Add new flag, -x, for dumping the list of auxiliary modules.
>>        (print_usage): Add usage string for new flag, -x.
>>
>