[plugins-ici-cloning-instrumentation] new GCC plugin developements

["Grigori Fursin" <grigori dot fursin at inria dot fr>]

Hi all,

Just wanted to announce that we will now collaborative with Joern until the end of 
December to sync the GCC GSOC'09 developments and ICI with the mainline GCC. Joern 
can transfer all the code to FSF since he is covered by INRIA agreement 
until the end of December at least.

As a reminder, the goal of our developments is to sync our Interactive Compilation
Interface with the current plugin framework to enable already established research community
continue working with ICI directly in the official GCC without any changes. We would
also like to continue attracting more researchers to GCC and enable quick prototyping 
of research ideas in a real production quality compiler while avoiding a long process
of hacking it ;) ... This in turn should help us move useful research ideas back to
GCC much faster to improve program performance, compilation time, code size, etc ...

Furthermore, this can eventually help us clean and modularize compiler, and hopefully
eventually provide a clean interface and full documentation of all modules (I think we
all agreed on that in the last few years and there are already multiple on-going projects 
in that direction ...)

Our current plan of work:

* Create ICI branch, create related Wiki page on GCC website (or reuse/update the current one), 
agree on testing methodology, agree on plugin location

* Sync current ICI and pass manager hooks with the branch (Zbigniew can advise on that).
This will allow users select/deselect/reorder current passes to continue research on adaptive compilation 
technology to improve performance, code size, compilation time or add new passes (generic function cloning,
feature extraction from the MILEPOST project, analysis and instrumentation routines, passes written in MELT, etc).
Our latest plugins also use XML to dump current info about executed passes, change them and put them back to the compiler...

* Check that ICI uses the same calling convention as current plugins (compiler flags, etc)

* Sync function cloning and basic program instrumentation (GSOC'09 extensions by Liang and Yuanjie - they can help with that).
This will allow users create multiversioning applications to adapt to different architectures or program behavior at run-time,
or even create multi-target applications such as GPU/CELL/accelerators, etc. It is implemented as a generic IP pass that can
be called on demand through ICI and the clones can be optimized differently selecting/deselecting/reordering passes through ICI...

All the above is already in the reasonably stable stage, requires minimal modifications to GCC and mainly requires testing and 
moving to the mainline. Me and Yuri (in CC) are testing pass selection and reordering mechanism in GCC...

* Check fine-grain optimization selection in GCC through ICI (another GCC GSOC'09 development by Yunajie and Liang). The idea
is to enable selection and tuning of individual transformations such as unrolling, vectorization, tiling, inlining, etc through 
plugins. The prototype is available, however it seems that it slows down the compiler so we have to check what to do if we have
too many hooks in the compiler - we need some simple testing mechanism that will turn them off if not used ...
So, before moving those developments, we have to thoroughly investigate this matter since we can't afford to slow down GCC ;) ..

* Check/add MILEPOST feature extractor - it just a pass that is not invoked unless called by ICI so it is simple to add but 
we need to show the useful usage cases ...

The latest version of ICI and GSOC developments for GCC 4.4.0 is available at SVN:
http://gcc-ici.svn.sourceforge.net/viewvc/gcc-ici/branches/
(gcc-4.4.0-ici-2.0-adapt)

The stable version of MILEPOST GCC with ICI and feature extractor from IBM is available at:
http://sourceforge.net/projects/gcc-ici/files/

The latest documentation for ICI and GSOC developments is available here:
http://ctuning.org/wiki/index.php/CTools:ICI:Documentation
http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Function_cloning_and_program_instrumentation
http://ctuning.org/wiki/index.php/CTools:ICI:Projects:GSOC09:Fine_grain_tuning

The idea is to move all the these developments and the documentation to the GCC branch and GCC site now

By the way, just a small reminder that the submission deadline for the GROW'10 workshop
is coming soon: http://ctuning.org/workshop-grow10

We hope to have a 1-hour open discussion at the end of the GROW workshop about plugins, LTO and
further R&D opportunities in GCC ...

Also, until the end of December, the main contact point about these developments will be Joern ...

Will keep in touch,
Grigori
 

> -----Original Message-----
> From: Joern Rennecke [mailto:amylaar@spamcop.net]
> Sent: Monday, November 02, 2009 1:50 PM
> To: Grigori Fursin
> Cc: 'Zbigniew Chamski'; 'Richard Guenther'; 'Basile STARYNKEVITCH'; 'Ian Lance Taylor'; 'GCC
> Mailing List'; 'Albert Cohen'; ctuning-discussions@googlegroups.com; Yuri Kashnikoff
> Subject: RE: plugin hooks
> 
> Quoting Grigori Fursin <grigori.fursin@inria.fr>:
> > Also, I hope that we will start collaborating with Joern Rennecke in
> >  a few weeks to update the ICI
> > and GSOC'09
> >
> > developments based on the recent feedback to see if we can move it
> > to the mainline ...
> 
> We still need a branch name for that.  Since GCC is currently in phase 3,
> I suppose we should plan on re-baselining at least once - re-baselining
> from 4.6 experimental mainline after 4.5 is branched - so instead of
> a gcc version number a branch date seems more suitable.
> 
> I.e. if the branch were to be created today, I propose to use the name
> branches/ici-20091102-branch for the branch.