Re: GSOC 2018 - Textual LTO dump tool project

[Richard Biener <richard dot guenther at gmail dot com>]

On Tue, Mar 13, 2018 at 5:30 AM, Hrishikesh Kulkarni
<hrishikeshparag@gmail.com> wrote:
> Hi,
>
> Thanks. I have tried to incorporate suggestions and prepared a revised draft
> of proposal for GSOC. Please find the same attached herewith. Your
> suggestions in regard with this draft would definitely help me to improve it
> further for submission.

Thanks, it looks very good now.  You have essentially duplicated items
in 1. and 2., namely --summary=<passname> and Dumping of IPA summaries.
I would move some of the 1. items over to 2., apart from --summary I'd also
move --cgraph-dot.

Richard.

>
> Drive link to Draft Proposal:
>
> https://docs.google.com/document/d/1-jYwwDWsHQwMaxVsHFBrJ9EiCAev7ljTDLS1xMwvK5w/edit
>
>
> Regards,
>
> Hrishikesh
>
>
>
>
> On Mon, Mar 12, 2018 at 4:45 PM, Richard Biener <richard.guenther@gmail.com>
> wrote:
>>
>> On Sun, Mar 11, 2018 at 8:23 PM, Hrishikesh Kulkarni
>> <hrishikeshparag@gmail.com> wrote:
>> > Hi,
>> >
>> > Greetings! Please find my draft proposal for GSOC attached herewith. I
>> > am
>> > very grateful to all of you for your inputs, suggestions and directions.
>> > I
>> > have tried to assimilate these inputs received from you to convert it
>> > into a
>> > proposal. Your suggestions in regard with this draft would definitely
>> > help
>> > me to convert it into final proposal for submission. In addition, could
>> > you
>> > please suggest the possible extensions those can be added to dump tool?
>> >
>> >
>> > Drive link to Draft Proposal:
>> >
>> >
>> > https://docs.google.com/document/d/1-jYwwDWsHQwMaxVsHFBrJ9EiCAev7ljTDLS1xMwvK5w/edit
>>
>> The proposal looks a bit sparse when giving details about the actual
>> project.
>> I'd suggest to clarify at least the deliverables.  I suggest for 1. add a
>> 1 c)
>> that specifies what should be working, for example
>>
>>  lto-dump -l
>>
>> should dump a list of variables and functions contained in the IL
>>
>>  lto-sump -s <id>
>>
>> should dump detailed info about the symbol <id> (using the symtab dump
>> infrastructure)
>>
>>  lto-dump -f <id>
>>
>> should dump the function body of the function with <id> (using the gimple
>> dump infrastructure)
>>
>> the lto-dump tool should be verified to work on both WPA-time and
>> LTRANS-time
>> objects.
>>
>> Thus your 2) a) should be possible with 1) already.  2) would then contain
>> dumping of IPA summaries as major part apart from visualizing the
>> callgraph.
>> For visualizing the (full) callgraph you need to handle multiple LTO
>> IL input files.
>> Those two pieces should be enough for 2) unless usability issues spill
>> over
>> from 1).
>>
>> In the introduction I miss some general words about the LTO IL, like that
>> it
>> is non-self-describing bytecode which is documented only by the code
>> reading/writing it and thus hard to debug.  It also misses to lay out the
>> overall structure of a LTO IL file (you are already going into detail with
>> IPA passes so this missing stands out).
>>
>> Richard.
>>
>> >
>> > Regards,
>> >
>> > Hrishikesh
>> >
>> >
>> >
>> > On Tue, Mar 6, 2018 at 8:59 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> >>
>> >> > On Tue, Mar 6, 2018 at 4:02 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> >> > >> On Tue, Mar 6, 2018 at 2:30 PM, Hrishikesh Kulkarni
>> >> > >> <hrishikeshparag@gmail.com> wrote:
>> >> > >> > Hi,
>> >> > >> >
>> >> > >> > Thank you Richard and Honza for the suggestions. If I understand
>> >> > >> > correctly,
>> >> > >> > the issue is that LTO file format keeps changing per compiler
>> >> > >> > versions, so
>> >> > >> > we need a more “stable” representation and the first step for
>> >> > >> > that
>> >> > >> > would be
>> >> > >> > to “stabilize” representations for lto-cgraph and symbol table ?
>> >> > >>
>> >> > >> Yes.  Note the issue is that the current format is a 1:1
>> >> > >> representation of
>> >> > >> the internal representation -- which means it is the internal
>> >> > >> representation
>> >> > >> that changes frequently across releases.  I'm not sure how Honza
>> >> > >> wants
>> >> > >> to deal with those changes in the context of a "stable" IL format.
>> >> > >> Given
>> >> > >> we haven't been able to provide a stable API to plugins I think
>> >> > >> it's
>> >> > >> much
>> >> > >> harder to provide a stable streaming format for all the IL
>> >> > >> details....
>> >> > >>
>> >> > >> > Could you
>> >> > >> > please elaborate on what initial steps need to be taken in this
>> >> > >> > regard, and
>> >> > >> > if it’s feasible within GSoC timeframe ?
>> >> > >>
>> >> > >> I don't think it is feasible in the GSoC timeframe (nor do I think
>> >> > >> it's feasible
>> >> > >> at all ...)
>> >> > >
>> >> > > I skipped this, with GSoC timeframe I fully agree.  With
>> >> > > feasibility
>> >> > > at all not so
>> >> > > much - LLVM documents its bitcode to reasonable extend
>> >> > > https://llvm.org/docs/BitCodeFormat.html
>> >> > >
>> >> > > Reason why i mentioned it is that I would like to use this as an
>> >> > > excuse to get
>> >> > > things incrementally cleaned up and it would be nice to keep it in
>> >> > > mind while
>> >> > > working on this.
>> >> >
>> >> > Ok.  It's probably close enough to what I recommended doing with
>> >> > respect
>> >> > to make the LTO bytecode "self-descriptive" -- thus start with making
>> >> > the
>> >> > structure documented and parseable without assigning semantics to
>> >> > every bit ;)  I think that can be achieved top-down in a very
>> >> > incremental
>> >> > way if you get the bottom implemented first (the data-streamer part).
>> >>
>> >> OK :)
>> >> I did not mean to document every bit either, at least not for the fancy
>> >> parts.
>> >> It would be nice to have clenned up i.e. the section headers/footers so
>> >> they
>> >> do not depend on endianity and slowly cleanup similar nonsences at
>> >> higher
>> >> levels.  So it may make sense to progress from both directions lower
>> >> hanging
>> >> fruits first.
>> >>
>> >> Honza
>> >
>> >
>
>