Re: [GSOC] LTO dump tool project

[Hrishikesh Kulkarni <hrishikeshparag at gmail dot com>]

Hi,

I have added command line options:

-body=
To dump gimple body (TDF_NONE) of specific function.

-optimized-blocks=
To dump gimple body (TDF_BLOCKS) of specific function.

-optimized-stats=
To dump gimple body (TDF_STATS) of specific function.

-optimized-vops=
To dump gimple body (TDF_VOPS) of specific function.

for example:

-body=foo  will dump

Gimple body of function: foo
foo (int a, int b)
{
  <bb 2> [local count: 1073741825]:
  _3 = a_1(D) + b_2(D);
  return _3;

}


-optimized-blocks=foo  will dump

Gimple body of function: foo
foo (int a, int b)
{
;;   basic block 2, loop depth 0
;;    pred:       ENTRY
  _3 = a_1(D) + b_2(D);
  return _3;
;;    succ:       EXIT

}


-optimized-stats=foo  will dump

Gimple body of function: foo
foo (int a, int b)
{
  <bb 2> [local count: 1073741825]:
  _3 = a_1(D) + b_2(D);
  return _3;

}


-optimized-vops=foo  will dump

Gimple body of function: foo
foo (int a, int b)
{
  <bb 2> [local count: 1073741825]:
  _3 = a_1(D) + b_2(D);
  # VUSE <.MEM_4(D)>
  return _3;

}

I have pushed the changes to the current branch. Please find the diff
file attached herewith.

I have tried to follow the coding conventions as far as possible in
this patch. Please let me know if I am missing out something so that I
can improve the same while refactoring and clean up as suggested in
the previous mail.

Regards,

Hrishikesh

On Wed, Jul 11, 2018 at 12:10 AM, Hrishikesh Kulkarni
<hrishikeshparag@gmail.com> wrote:
> Hi,
>
> Thanks for suggestions. I would start working on these points and will
> try to complete as early as possible.
>
> Regards,
>
> Hrishikesh
>
> On Mon, Jul 9, 2018 at 2:04 PM, Martin Liška <mliska@suse.cz> wrote:
>> On 07/09/2018 09:50 AM, Hrishikesh Kulkarni wrote:
>>> Hi,
>>>
>>> The command line option -gimple-stats will dump the statistics of
>>> gimple statements.
>>>
>>> For example:
>>>
>>> $ ../stage1-build/gcc/lto-dump test_hello.o -gimple-stats
>>>
>>> will dump:
>>>
>>> GIMPLE statements
>>> Kind                   Stmts      Bytes
>>> ---------------------------------------
>>> assignments                3        264
>>> phi nodes                  1        248
>>> conditionals               1         80
>>> everything else           12        704
>>> ---------------------------------------
>>> Total                     17       1296
>>> ---------------------------------------
>>>
>>> I have pushed the changes to the repo. Please find the diff file
>>> attached herewith.
>>>
>>> Regards,
>>>
>>> Hrishikesh
>>
>> Hi.
>>
>> Thanks for the work. I briefly took a look at the code and I would
>> focus now directly on refactoring:
>>
>> - please make a new branch, first copy lto-dump.c file and all the
>> Makefile needed stuff and commit that.
>> - next please rebase (squash) all your patches which you have on top
>> of it; you did some formatting corrections and it's very hard to read
>> - please fix coding style, it's mentioned here: https://gcc.gnu.org/codingconventions.html
>> the most problematic is usage of horizontal white spaces. We use 2 spaces
>> for indentation level, and 8 spaces are replaced with a tabular; without that reading your
>> code is very hard for me
>> - then please start refactoring functionality that is copied from lto.c and put shared
>> stuff into a header file that will be used by lto.c and lto-dump.c.
>>
>> Other observations:
>> - you use "\t\t%s\t\t%s\t\t%s" formats for prints; I think it would be better to
>> use fixed strings with spaces, try %20s. It's probably also used by tools like nm or objdump
>> - come up with more specific name for 'entry' and 'compare'
>> - 'entry' should have functions that will print names, ... according to options
>> (flag_lto_dump_demangle, ...); you can have overrides for functions and variables
>> - I would first put all symbols into the vector and then I would print it on a single place
>> - consider using vec from vec.h, hash_map from hash-map.h instead of std:: variants
>> - exit after functions like dump_list, dump_symbol,...
>> - remove dummy 'dump' function
>>
>> Martin
>>
>>>
>>> On Thu, Jul 5, 2018 at 10:41 PM, Hrishikesh Kulkarni
>>> <hrishikeshparag@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I have added new command line option:
>>>> -objects
>>>> which will dump size, offset and name of each section for all lto objects
>>>>
>>>> for example:
>>>> $ ../stage1-build/gcc/lto-dump test_hello.o test.o -objects
>>>>
>>>> gives output:
>>>>             LTO object name: test_hello.o
>>>>
>>>> NO.        OFFSET        SIZE        SECTION NAME
>>>>
>>>> 1        64        15        .gnu.lto_.profile.a7add72ac123628
>>>> 2        79        55        .gnu.lto_.icf.a7add72ac123628
>>>> 3        134        134        .gnu.lto_.jmpfuncs.a7add72ac123628
>>>> 4        268        116        .gnu.lto_.inline.a7add72ac123628
>>>> 5        384        24        .gnu.lto_.pureconst.a7add72ac123628
>>>> 6        408        306        .gnu.lto_foo.a7add72ac123628
>>>> 7        714        469        .gnu.lto_bar.a7add72ac123628
>>>> 8        1183        345        .gnu.lto_main.a7add72ac123628
>>>> 9        1528        88        .gnu.lto_.symbol_nodes.a7add72ac123628
>>>> 10        1616        15        .gnu.lto_.refs.a7add72ac123628
>>>> 11        1631        1205        .gnu.lto_.decls.a7add72ac123628
>>>> 12        2836        109        .gnu.lto_.symtab.a7add72ac123628
>>>> 13        2945        76        .gnu.lto_.opts
>>>>
>>>>             LTO object name: test.o
>>>>
>>>> NO.        OFFSET        SIZE        SECTION NAME
>>>>
>>>> 1        64        15        .gnu.lto_.profile.ffab9cb8eb84fc03
>>>> 2        79        30        .gnu.lto_.icf.ffab9cb8eb84fc03
>>>> 3        109        108        .gnu.lto_.jmpfuncs.ffab9cb8eb84fc03
>>>> 4        217        62        .gnu.lto_.inline.ffab9cb8eb84fc03
>>>> 5        279        21        .gnu.lto_.pureconst.ffab9cb8eb84fc03
>>>> 6        300        194        .gnu.lto_koo.ffab9cb8eb84fc03
>>>> 7        494        389        .gnu.lto_gain.ffab9cb8eb84fc03
>>>> 8        883        77        .gnu.lto_.symbol_nodes.ffab9cb8eb84fc03
>>>> 9        960        15        .gnu.lto_.refs.ffab9cb8eb84fc03
>>>> 10        975        966        .gnu.lto_.decls.ffab9cb8eb84fc03
>>>> 11        1941        58        .gnu.lto_.symtab.ffab9cb8eb84fc03
>>>> 12        1999        76        .gnu.lto_.opts
>>>>
>>>>
>>>> I have pushed the changes to the repo. Please find the diff file
>>>> attached herewith.
>>>>
>>>> Regards,
>>>>
>>>> Hrishikesh
>>>>
>>>> On Thu, Jul 5, 2018 at 12:24 AM, Hrishikesh Kulkarni
>>>> <hrishikeshparag@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I have:
>>>>> tried to do all the formatting and style corrections according to
>>>>> output given by check_GNU_style.py
>>>>> removed '-fdump-lto' prefix from the command line options
>>>>> added few necessary comments in the code
>>>>> added command line option -type-stats from previous branch (added a
>>>>> new percentage column to it)
>>>>> for e.g.
>>>>>     integer_type    3    25.00%
>>>>>     pointer_type    3    25.00%
>>>>>     array_type    1    8.33%
>>>>>     function_type    5    41.67%
>>>>> I have pushed the changes  to the repo. Please find the diff file
>>>>> attached herewith.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Hrishikesh
>>>>>
>>>>> On Tue, Jul 3, 2018 at 6:39 PM, Hrishikesh Kulkarni
>>>>> <hrishikeshparag@gmail.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for suggestions. I have started incorporating them. As a first:
>>>>>>
>>>>>> I have added command line options:
>>>>>> -print-size  to print the size of functions and variables
>>>>>> size of variables is in bits and size of functions is represented as
>>>>>> number of basic blocks.
>>>>>>
>>>>>> -print-value  to print the value of initialization of global variables.
>>>>>>
>>>>>> -size-sort  to sort the symbol names according to the size.
>>>>>>
>>>>>> for example command line options:
>>>>>>
>>>>>>  ../stage1-build/gcc/lto-dump test_hello.o -fdump-lto-list -print-size
>>>>>> -print-value -size-sort
>>>>>>
>>>>>> the dump is:
>>>>>>
>>>>>> Symbol Table
>>>>>>         Name         Type         Visibility        Size     Value
>>>>>>
>>>>>>         printf         function        default           0
>>>>>>         main         function        default           3
>>>>>>         foo            function        default           3
>>>>>>         bar            function        default           6
>>>>>>
>>>>>>         z               variable        default           8        97
>>>>>>         k               variable        default          32        5
>>>>>>         p               variable        default          32
>>>>>> 4.400000095367431640625e+0
>>>>>>
>>>>>> I have also tried to make memory allocation dynamic to the best of my knowledge.
>>>>>> I have pushed the changes to the repo. Please find the diff file
>>>>>> attached herewith.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Hrishikesh
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 29, 2018 at 12:55 PM, Martin Liška <mliska@suse.cz> wrote:
>>>>>>> On 06/27/2018 10:06 PM, Hrishikesh Kulkarni wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have added new command line options:
>>>>>>>> -no-demangle -> for the default non demangled output
>>>>>>>> -no-sort -> for the list of symbols in order of their occurrence
>>>>>>>> -alpha-sort -> for the list of symbols in their alphabetical order
>>>>>>>> -reverse-sort -> for the list of symbols in reversed order
>>>>>>>> -defined-only -> for only the defined symbols
>>>>>>>
>>>>>>> Hi.
>>>>>>>
>>>>>>> Good progress.
>>>>>>>
>>>>>>>>
>>>>>>>> for example:
>>>>>>>>
>>>>>>>> ../stage1-build/gcc/lto-dump test_hello.o -fdump-lto-list -alpha-sort
>>>>>>>> -demangle -defined-only -reverse-sort
>>>>>>>
>>>>>>> Now as you have a separate tool (lto-dump), you can strip 'fdump-lto' prefix
>>>>>>> from the older options.
>>>>>>>
>>>>>>>>
>>>>>>>> will dump
>>>>>>>>
>>>>>>>> Symbol Table
>>>>>>>>         Name         Type         Visibility         Size
>>>>>>>>         main        function        default
>>>>>>>>         k        variable        default
>>>>>>>>         foo        function        default
>>>>>>>>         bar        function        default
>>>>>>>>
>>>>>>>> It is a reversed alphabetical order of demangled symbol names which
>>>>>>>> have been defined(hence printf excluded).
>>>>>>>> Along with this I have also added all previous progress with reference
>>>>>>>> to symbol table to the new branch.
>>>>>>>>
>>>>>>>> For further options to add like -size-sort and -print-size I tried to
>>>>>>>> access size of the symbol with symtab node using
>>>>>>>> TREE_INT_CST_LOW(DECL_SIZE(node->decl));
>>>>>>>> but I am unable to do so.
>>>>>>>> So how should I proceed with it?
>>>>>>>
>>>>>>> Sent advises via instant messaging.
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>>>
>>>>>>>> Please find the diff file attached herewith. I have also pushed the
>>>>>>>> changes to the new branch.
>>>>>>>>
>>>>>>>> Please advise,
>>>>>>>>
>>>>>>>> Hrishikesh>
>>>>>>>> On Wed, Jun 27, 2018 at 1:15 AM, Hrishikesh Kulkarni
>>>>>>>> <hrishikeshparag@gmail.com> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have created another branch lto-dump-tool-improved as suggested.
>>>>>>>>> I have applied the patch for separation to lto-dump binary, made a few
>>>>>>>>> necessary changes in other files and it is running successfully.
>>>>>>>>> I will keep on adding previous work to this branch incrementally.
>>>>>>>>>
>>>>>>>>> Please find the diff file attached for dumping of TREE statistics and
>>>>>>>>> GIMPLE statistics.
>>>>>>>>>
>>>>>>>>> for example:
>>>>>>>>> (after configuring with --enable-gather-detailed-mem-stats)
>>>>>>>>> -fdump-lto-gimple-stats will dump
>>>>>>>>> GIMPLE statements
>>>>>>>>> Kind                   Stmts      Bytes
>>>>>>>>> ---------------------------------------
>>>>>>>>> assignments                0          0
>>>>>>>>> phi nodes                  0          0
>>>>>>>>> conditionals               0          0
>>>>>>>>> everything else            0          0
>>>>>>>>> ---------------------------------------
>>>>>>>>> Total                      0          0
>>>>>>>>> ---------------------------------------
>>>>>>>>>
>>>>>>>>> -fdump-lto-tree-stats will dump
>>>>>>>>>
>>>>>>>>> Tree Statistics
>>>>>>>>>
>>>>>>>>> Kind                   Nodes      Bytes
>>>>>>>>> ----------------------------------------
>>>>>>>>> decls                   4327     932672
>>>>>>>>> types                   1531     257208
>>>>>>>>> blocks                     0          0
>>>>>>>>> stmts                      0          0
>>>>>>>>> refs                       0          0
>>>>>>>>> exprs                      4        128
>>>>>>>>> constants                 82       2060
>>>>>>>>> identifiers             4430     177200
>>>>>>>>> vecs                      16      28544
>>>>>>>>> binfos                     0          0
>>>>>>>>> ssa names                  0          0
>>>>>>>>> constructors               0          0
>>>>>>>>> random kinds            7301     291952
>>>>>>>>> lang_decl kinds            0          0
>>>>>>>>> lang_type kinds            0          0
>>>>>>>>> omp clauses                0          0
>>>>>>>>> ----------------------------------------
>>>>>>>>> Total                  17691    1689764
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please advise,
>>>>>>>>>
>>>>>>>>> Hrishikesh
>>>>>>>>>
>>>>>>>>> On Wed, Jun 27, 2018 at 1:00 AM, Hrishikesh Kulkarni
>>>>>>>>> <hrishikeshparag@gmail.com> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I have created another branch lto-dump-tool-improved as suggested.
>>>>>>>>>> I have applied the patch for separation to lto-dump binary, made a few
>>>>>>>>>> necessary changes in other files and it is running successfully.
>>>>>>>>>> I will keep on adding previous work to this branch incrementally.
>>>>>>>>>>
>>>>>>>>>> Please find the diff file attached for dumping of TREE statistics and GIMPLE
>>>>>>>>>> statistics.
>>>>>>>>>>
>>>>>>>>>> for example:
>>>>>>>>>> (after configuring with --enable-gather-detailed-mem-stats)
>>>>>>>>>> -fdump-lto-gimple-stats will dump
>>>>>>>>>> GIMPLE statements
>>>>>>>>>> Kind                   Stmts      Bytes
>>>>>>>>>> ---------------------------------------
>>>>>>>>>> assignments                0          0
>>>>>>>>>> phi nodes                  0          0
>>>>>>>>>> conditionals               0          0
>>>>>>>>>> everything else            0          0
>>>>>>>>>> ---------------------------------------
>>>>>>>>>> Total                      0          0
>>>>>>>>>> ---------------------------------------
>>>>>>>>>>
>>>>>>>>>> -fdump-lto-tree-stats will dump
>>>>>>>>>>
>>>>>>>>>> Tree Statistics
>>>>>>>>>>
>>>>>>>>>> Kind                   Nodes      Bytes
>>>>>>>>>> ----------------------------------------
>>>>>>>>>> decls                   4327     932672
>>>>>>>>>> types                   1531     257208
>>>>>>>>>> blocks                     0          0
>>>>>>>>>> stmts                      0          0
>>>>>>>>>> refs                       0          0
>>>>>>>>>> exprs                      4        128
>>>>>>>>>> constants                 82       2060
>>>>>>>>>> identifiers             4430     177200
>>>>>>>>>> vecs                      16      28544
>>>>>>>>>> binfos                     0          0
>>>>>>>>>> ssa names                  0          0
>>>>>>>>>> constructors               0          0
>>>>>>>>>> random kinds            7301     291952
>>>>>>>>>> lang_decl kinds            0          0
>>>>>>>>>> lang_type kinds            0          0
>>>>>>>>>> omp clauses                0          0
>>>>>>>>>> ----------------------------------------
>>>>>>>>>> Total                  17691    1689764
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please advise,
>>>>>>>>>>
>>>>>>>>>> Hrishikesh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 22, 2018 at 2:29 PM, Martin Liška <mliska@suse.cz> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 06/18/2018 10:45 AM, Martin Jambor wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 17 2018, Hrishikesh Kulkarni wrote:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am trying to isolate the dump tool into real lto-dump tool. I have
>>>>>>>>>>>>> started with the copy&paste of lto.c into lto-dump.c and done the
>>>>>>>>>>>>> changes to Make-lang.in and config-lang.in suggested by Martin (patch
>>>>>>>>>>>>> attached). However when I try to build, I get the following error:
>>>>>>>>>>>>>
>>>>>>>>>>>>> In file included from ../../gcc/gcc/lto/lto-dump.c:24:0:
>>>>>>>>>>>>>
>>>>>>>>>>>>> ../../gcc/gcc/coretypes.h:397:24: fatal error: insn-modes.h: No such
>>>>>>>>>>>>>
>>>>>>>>>>>>> file or directory
>>>>>>>>>>>>>
>>>>>>>>>>>>> compilation terminated.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am unable to find the missing dependencies and would be grateful for
>>>>>>>>>>>>> suggestions on how to resolve the issue.
>>>>>>>>>>>>
>>>>>>>>>>>> insn-modes.h is one of header files which are generated at build time,
>>>>>>>>>>>> you will find it in the gcc subdirectory of your build directory (as
>>>>>>>>>>>> opposed to the source directory).
>>>>>>>>>>>>
>>>>>>>>>>>> Martin
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi.
>>>>>>>>>>>
>>>>>>>>>>> There's patch that works for me.
>>>>>>>>>>> Needed dependency is here:
>>>>>>>>>>>
>>>>>>>>>>> +lto/lto-dump.o: $(LTO_EXE)
>>>>>>>>>>>
>>>>>>>>>>> It's bit artificial, but for now it works. Please start working
>>>>>>>>>>> on separation of dump infrastructure to lto-dump.c. You'll need
>>>>>>>>>>> to factor out common code shared in between lto.c and lto-dump.c.
>>>>>>>>>>>
>>>>>>>>>>> Martin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>

diff --git a/gcc/lto/lang.opt b/gcc/lto/lang.opt
index 9e7d1fc..7c671c0 100644
--- a/gcc/lto/lang.opt
+++ b/gcc/lto/lang.opt
@@ -117,6 +117,15 @@ Dump the details of LTO objects.
 body=
 LTO Driver RejectNegative Joined Var(flag_lto_dump_body)
 
+optimized-blocks=
+LTO Driver RejectNegative Joined Var(flag_tree_optimized_blocks)
+
+optimized-stats=
+LTO Driver RejectNegative Joined Var(flag_tree_optimized_stats)
+
+optimized-vops=
+LTO Driver RejectNegative Joined Var(flag_tree_optimized_vops)
+
 type-stats
 LTO Var(flag_lto_dump_type_stats)
 Dump the statistics of tree types.
diff --git a/gcc/lto/lto-dump.c b/gcc/lto/lto-dump.c
index 40fdbbe..623c825 100644
--- a/gcc/lto/lto-dump.c
+++ b/gcc/lto/lto-dump.c
@@ -215,6 +215,66 @@ dump_symbol ()
 	  fprintf (stderr, "\n");
 }
 
+/* Dump gimple body (TDF_NONE) of specific function.  */
+void
+dump_body_none ()
+{
+	fprintf (stderr, "Gimple body of function: %s\n",
+	    flag_lto_dump_body);
+	cgraph_node *cnode;
+	FOR_EACH_FUNCTION (cnode)
+		if (!strcmp (cnode->name (), flag_lto_dump_body))
+		{
+			cnode->get_untransformed_body ();
+			debug_function (cnode->decl, 0);
+		}
+}
+
+/* Dump gimple body (TDF_BLOCKS) of specific function.  */
+void
+dump_body_blocks ()
+{
+	fprintf (stderr, "Gimple body of function: %s\n",
+	    flag_tree_optimized_blocks);
+	cgraph_node *cnode;
+	FOR_EACH_FUNCTION (cnode)
+		if (!strcmp (cnode->name (), flag_tree_optimized_blocks))
+		{
+			cnode->get_untransformed_body ();
+			debug_function (cnode->decl, TDF_BLOCKS);
+		}
+}
+
+/* Dump gimple body (TDF_STATS) of specific function.  */
+void
+dump_body_stats ()
+{
+	fprintf (stderr, "Gimple body of function: %s\n",
+	    flag_tree_optimized_stats);
+	cgraph_node *cnode;
+	FOR_EACH_FUNCTION (cnode)
+		if (!strcmp (cnode->name (), flag_tree_optimized_stats))
+		{
+			cnode->get_untransformed_body ();
+			debug_function (cnode->decl, TDF_STATS);
+		}
+}
+
+/* Dump gimple body (TDF_VOPS) of specific function.  */
+void
+dump_body_vops ()
+{
+	fprintf (stderr, "Gimple body of function: %s\n",
+		  flag_tree_optimized_vops);
+	cgraph_node *cnode;
+	FOR_EACH_FUNCTION (cnode)
+		if (!strcmp (cnode->name (), flag_tree_optimized_vops))
+		{
+			cnode->get_untransformed_body ();
+			debug_function (cnode->decl, TDF_VOPS);
+		}
+}
+
 /* Number of parallel tasks to run, -1 if we want to use GNU Make jobserver.  */
 static int lto_parallelism;
 
@@ -3580,6 +3640,22 @@ lto_main (void)
   if (flag_lto_dump_symbol)
     dump_symbol ();
 
+  /* Dump gimple body (TDF_NONE) of specific function.  */
+  if (flag_lto_dump_body)
+    dump_body_none ();
+
+  /* Dump gimple body (TDF_BLOCKS) of specific function.  */
+  if (flag_tree_optimized_blocks)
+    dump_body_blocks ();
+
+  /* Dump gimple body (TDF_STATS) of specific function.  */
+  if (flag_tree_optimized_stats)
+    dump_body_stats ();
+
+  /* Dump gimple body (TDF_VOPS) of specific function.  */
+  if (flag_tree_optimized_vops)
+    dump_body_vops ();
+
   timevar_stop (TV_PHASE_STREAM_IN);
 
   if (!seen_error ())