Re: FDO usability patch -- cfg and lineno checksum

[Xinliang David Li <davidxl at google dot com>]

On Sun, Apr 17, 2011 at 8:36 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> Hi, ?in current FDO implementation, the source file version used in
>> profile-generate needs to strictly match the version used in
>> profile-use -- simple formating changes will invalidate the profile
>> data (use of it will lead to compiler error or ICE). There are two
>> main problems that lead to the weakness:
>>
>> 1) the function checksum is computed based on line number and number
>> of basic blocks -- this means any line number change will invalidate
>> the profile data. In fact, line number should play very minimal role
>> in profile matching decision. Another problem is that number of basic
>> blocks is also not a good indicator whether CFG matches or not.
>> 2) cgraph's pid is used as the key to find the matching profile data
>> for a function. If there is any new function added, or if the order of
>> the functions changes, the profile data is invalidated.
>>
>> The attached is a patch from google that improves this.
>> 1) function checksum is split into two parts: lineno checksum and cfg checksum
>> 2) function assembler name is used in profile hashing.
>
> Well, the overall idea was to make profile intolerant to pretty much any source
> level change, since you never know if even when the CFG shape match, the actual
> probabiliies did not changed.
> I however see that during development it is not bad to make compiler grok
> the profile as long as it seems consistent.
>

Make FDO usable in development cycle is one usage of the better
matching tolerance. There is another bigger use case which we are
looking at -- using sample profiling data from production binary
collected by the profile server to guide optimization. Of course there
will be lots of other problems (e.g., compiler optimizations can
drastically change control flow, etc.) before this can become reality.


> I did not looked in detail into the implementation yet, but it seems
> sane at first glance. ?However what about issuing a warning when line number
> information change telling user that profile might no longer be accurate
> and that he may want to remove it and train again?

Yes, this may be good to have.

Thanks,

David
>
> Honza
>