RE: [RFC] Cleaning up the pass manager

["Grigori Fursin" <gfursin at gmail dot com>]

Hi Diego,

Thanks a lot for doing this! I was a bit sad not to be able to continue
this work on pass selection and reordering but I would really like to see GCC pass 
manager improved in the future. I also forwarded your email to the cTuning mailing list
in case some of the ICI/MILEPOST GCC/cTuning CC users would want to provide
more feedback. 

By the way, one of the main reasons why I started developing ICI many years ago
was to be able to query GCC to tell me all available passes and then just use arbitrary 
selection and order of them for the whole program (IPO/LTO) or per function similar to 
what I could easily do with SUIF in my past research on empirical optimizations 
and what can be easily done in LLVM now. However, implementing it was really not easy
because:

* We have non-trivial (and not always fully documented) association between flags and passes, 
i.e. if I turn on unroll flag which turns on several passes, I can't later reproduce 
exactly the same behavior if I do not use any GCC flags but just try to turn on 
associated passes through pass manager.

* I believe that original idea of the pass manager introduced in GCC 4.x was to keep
a simple linked list of passes that are executed in a given order ONLY through documented
functions (API) and that can be turned on or off through the attribute in the list - 
this was a great idea and was one of the reasons why I finally moved to GCC from Open64 in 2004. 
However, I was a bit surprised to see in GCC 4.some explicit if statements inside pass manager 
that enabled some passes (for LTO) - in my opinion, it kills the main strength of the pass manager 
and also resulted that we had troubles porting ICI to the new GCC 4.5. 

* Lack of a table with full dependency info for each pass that can tell you at each stage of 
compilation, which passes can be selected next. I started working on that at the end of last 
year to get such info semi-empirically and also through the associated attributes (we presented 
preliminary results at GROW'10: http://ctuning.org/dissemination/grow10-08.pdf section 3.1), 
however again it was just before I moved to the new job so I couldn't finish it ...

* Well-known problem that we have some global variables shared between passes 
preventing arbitrary orders 

By the way, just to be clear, this is just a feedback based on the experience of my colleagues 
and myself and I do not want to say that these are the most important things for GCC right now
(though I think they are in a long term) or that someone should fix it particularly since right 
now personally I am not working in this area, so if someone thinks that it's not important/useless/obvious,
just skip it ;) ... I now see lots of effort going on to clean up GCC and to address some of the 
above issues so I think it's really great and I am sad that I can't help much at this stage. 
However, before moving to a new job, I released all the tools from my past research at cTuning.org
so maybe someone will find them useful to continue addressing the above issues ...

Cheers,
Grigori




By the way, here is some very brief feedback about why I needed for my reseafrom the R&D we did
at the beginning of this year just before I moved to the new job:
* 
-----Original Message-----
From: gcc-owner@gcc.gnu.org [mailto:gcc-owner@gcc.gnu.org] On Behalf Of Diego Novillo
Sent: Tuesday, June 15, 2010 4:03 AM
To: gcc@gcc.gnu.org
Subject: [RFC] Cleaning up the pass manager

I have been thinking about doing some cleanups to the pass manager.
The goal would be to have the pass manager be the central driver of
every action done by the compiler.  In particular, the front ends
should make use of it and the callgraph manager, instead of the
twisted interactions we have now.

Additionally, I would like to (at some point) incorporate some/most of
the functionality provided by ICI
(http://ctuning.org/wiki/index.php/CTools:ICI).  I'm not advocating
for integrating all of ICI, but leave enough hooks so such
experimentations are easier to do.

Initially, I'm going for some low hanging fruit:

- Fields properties_required, properties_provided and
properties_destroyed should Mean Something other than asserting
whether they exist.
- Whatever doesn't exist before a pass, needs to be computed.
- Pass scheduling can be done by simply declaring a pass and
presenting it to the pass manager.  The property sets should be enough
for the PM to know where to schedule a pass.
- dump_file and dump_flags are no longer globals.

Are there any particular pain points that people are currently
experiencing that fit this?


Thanks.  Diego.