Clarification on CTF/BTF workings with LTO

[Indu Bhagat]

On 7/21/21 11:18 PM, Richard Biener wrote:
> On Wed, 21 Jul 2021, Indu Bhagat wrote:
> 
>> Hello,
>>
>> Wanted to follow up on the CTF/BTF debug info + LTO workings.
>>
>> To summarize, the current status/workflow on trunk is:
>>
>> - The CTF container is written out in the ctfout.c or btfout.c via the
>> ctf_debug_finalize () API.
>> - At this time, the ctf_debug_finalize () itself is called once in
>> dwarf2out_early_finish ().
>> - Until this time, the requirements of CTF and BTF are simple.
>>     - The generated .ctf/.BTF sections needs no demarcation of "early"/"late"
>> debug. All of it can be generated "early".
>>     - The generated .ctf/.BTF information does not need to be different for the
>> final assembly and the fat LTO IR.
>>     - The BPF CO-RE is not yet implemented on trunk.
>>
>> Writing out the CTF/BTF at dwarf2out_early_finish seems to work - there will
>> always be a .ctf/.BTF section whether it's fat or slim LTO objects (because
>> the emission is still in dwarf2out_early_finish on the trunk). And we have
>> functionality to copy over the .ctf/.BTF debug sections in
>> handle_lto_debug_sections (). However, reading through some of the past emails
>> on the CTF/BTF patch series, it seems that you have been pointing to the
>> CTF/BTF debug info generation being broken when used with LTO. If true, I am
>> most certainly missing some key point here.
>>
>> So, before we move to the next steps of supporting additional requirements of
>> BPF CO-RE etc., I would like to make sure that my current understanding is OK
>> and that the current state of CTF/BTF on trunk is functional -with LTO-. I
>> have tested some bits (with and without fat objects on x86_64) and have not
>> run into issues.
>>
>> Can you please confirm what you see amiss in the current workings of CTF/BTF
>> with LTO on trunk ?
> 
> So on the functional level it seems to do something, that is, I see
> .ctf sections in a LTO linked test program as well as in a non-LTO
> linked program from fat LTO objects.  When I dump the .ctf section
> with readelf I see type info that looks OK but I don't see any
> function objects (my test has a main and foo function).  It might
> be an artifact of the readelf version I have (2.36.1) since the
> same happens w/o LTO.
> 
> So yes, in principle it should work in case there's only info
> that needs to be emitted early.  ISTR that in the beginning you
> had pieces emitted from dwarf2out_finish and there my concerns
> were rooted.
> 
> For DWARF the "late" data (like anything that needs relocations
> to symbols or addresses) is emitted from dwarf2out_finish and
> the LTRANS unit where the info is emitted from does not have
> the DWARF DIE generated early in memory but instead it knows
> how to reference it by a symbol + offset relocation.  So it
> generates a DIE like
> 
>     DW_TAG_subprogram
>     DW_AT_abstract_origin $early_debug_symbol + offset
>     DW_AT_low_pc .LC0_begin
> ...
> 
> to amend the early DIE with additional information, creating the
> "concrete" instance of the subprogram, re-using the early
> generated DIE as "abstract" instance.
> 
> I understand that CTF doesn't work like this (have relocations
> or DIE offsets or some such) but you need some late annotation
> at least for BPF?
> 

Correct. At this time, CTF does not have any relocations, or any "late" 
debug info.

And with regards to BTF, there are two usecases that BTF is serving and 
each have different requirements:
   - [#1] BTF for any backend (including BPF _but_ excluding BPF CO-RE 
usecase): This just needs the .BTF section. This is currently being 
generated at dwarf2out_early_finish just like .ctf
   - [#2] BTF for the _CO-RE_usecase_of BPF backend: This will be 
invoked explicitly, by say -mcore, for BPF backend and the generated BTF 
debug info will include additional .BTF.ext section. This .BTF.ext 
section contains "CO-RE Relocation" records, each containing offset of 
BPF instruction to be patched, the BTF type ID of the data structure 
being accessed by the instruction, and an offset to the BTF string which 
encodes a series of field accesses to retrieve the field of interest in 
the instruction. A BPF program loader will use this .BTF.ext along with 
the associated .BTF and make the necessary changes to facilitate the BPF 
program to run across kernel versions and headers. More details on this 
when the patch set for BPF's CO-RE support is posted...

So, yes, it is the #2 - BTF generated for CO-RE (Compile Once - Run 
Everywhere) usecase of BPF which needs bits to be emitted from 
dwarf2out_finish. The approach to best deal with this requirement is 
still in the works at this time.

Thanks
Indu