Re: [RFC] libgcov.c re-factoring and offline profile-tool

[Rong Xu <xur at google dot com>]

On Mon, Nov 11, 2013 at 7:17 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> Here is the patch that includes profile-tool.
>> Profile-tool now has two functions: merge and rewrite. I'll add diff later.
>>
>> Compiler is tested with spec2006 and profiledbootstrap.
>> profile-tool is tested with spec2006 profiles.
>
> Hi,
> it would be nice if you could elaborate bit more on the tool and what it does.
> I plan to implement relink only instrumentatino for LTO and for that I will need
> profile reading/handling that is independent on functio nbodies, so perhaps
> we can unify the infrastructure.  My plans always was to merge the gcov and
> profile.c profile handling logic and make it bit more generic.

This tool is designed to manipulate raw profile counters. We plan to
have the following functionality:

(1) merge the gcda file with weights. We have fairly complicated build
system. Sometime online merging in
current libgcov.a not feasible (or not desirable). We need offline
tool the merge the FDO profiles.
The program may have multiple training inputs with different
importance -- this is where weighted merging
comes from.

(2) diff the profile. We want to know how different of two profile
data. Some tolerance of mismatch is required here.
This is crucial for performance triaging.

(3) rewrite the gcda file. We can do some (user-direct) fix-up work in
the rewrite. This is may not be important for FDO. But
this will be very useful for LIPO (like fix-up the indirect-target due
to mismatch, rewrite module-grouping)

(4) better dumping: like dump top-n-hottest
functions/objects/indirectly-targets etc.

This tool reconstructs the gcov_info list from gcda files and then
uses significant part of libgcov.c code:
merge, compute summary/histo-gram, and dump gcda files etc.

The merging into current libgcov can also use this piece of code to do
the in-memory merge. The advantage
here is we will have precise histro-gram.

>
> Also I think profile-tool is a bit generic name. I.e. it does not realy say it
> is related to GCC's coverage profiling.  Maybe gcc-profile-tool or
> gcov-profile-tool may be better?

Yes. gcov-profile-tool seems to be better, or simply gcov-tool?

> Perhaps an resonable option would also be to bundle it into gcov binary...

Bundling into gcov is an interesting idea. I do plan to read gcno file
to extract more information (at least the function name).
Bungling into gcda means the tool can work on bb level, rather on the
raw counters. Let me think a bit more on this.

>> Index: libgcc/libgcov-tool.c
>> ===================================================================
>> --- libgcc/libgcov-tool.c     (revision 0)
>> +++ libgcc/libgcov-tool.c     (revision 0)
>> @@ -0,0 +1,800 @@
>> +/* GCC instrumentation plugin for ThreadSanitizer.
>> +   Copyright (C) 2011-2013 Free Software Foundation, Inc.
>> +   Contributed by Dmitry Vyukov <dvyukov@google.com>
>
> You need to update this...
> Why there needs to be libgcov changes?

David pointed out the copyright issue in an earlier email. I fixed it.

The changes in libgcov are
(1) ifdef the target headers. I need all the libgcov functionality to
read/write the gcda files, as well as summary and histo-gram. But the
binary is for host machines.
(2) change the merge function so that I can take an in-memory
gcov-info as input.

>
> Honza