Re: new plugin events

[Basile STARYNKEVITCH <basile at starynkevitch dot net>]

Richard Guenther wrote:
> No, read what Diego said (emphasis mine): "Just that there _is_ some
> plugin that could take advantage of it."

Diego also said "could", not "can" (emphasis mine). I understood that the plugin is not supposed to be mature. It could 
even be in its design phase...

> As opposed to "There _might_ be some plugin that could take advantage
> of it."

The intermediate phrase is "There _will_ be some plugin...". I am mostly thinking of the brave guys wanting to make a 
plugin for GCC, and missing some particular event.

> I want to avoid the situation where we prematurely add a plugin hook
> and later recognize it should have been different in a sublte way once
> the first user appears.  Thus, I want to avoid 1) changing existing
> plugin callbacks, 2) removing existing plugin callbacks, or even worst
> 3) changing its semantics.  This all applies to between major and
> even more so minor GCC releases.

I agree with the list to avoid, but it is really premature today for 4.5. It will be a concern for 4.6 or 5.0!! And it 
probably won't be that much a concern. I would imagine that removing useless plugin events in 4.6 or 5.0 would be 
possible...

> If we don't have at least plugin API stability we could throw plugins
> down the toilet.

Could you define plugin API stability? For me it is premature to think of it.
First, the Plugin API is so new that it cannot be stable for 4.5. It could be stable if it existed before 4.5 which it 
didn't.

Now, if you think of stability of a given plugin -call it foo here, that is a foo1.c foo2.c foo.h compiled into foo.so 
and passed to gcc-4.5 -fplugin=foo.so- betweem eg 4.5 & 4.6 do you mean that

1.   the binary foo.so will work for both gcc-4.5 & gcc-4.6 ?
I strongly believe that is impossible, and should not be dreamed of.

2.   the binary foo.so will work for both gcc-4.5.1 & gcc-4.5.2 ?
I tend to believe that is not necessary. People (or more probably Linux distribution makers) will recompile foo for 
4.5.2. Plugins are free software, so users have the plugin sources (and distribution makers would recompile them).

3.  the sources foo*.[ch] can be recompiled without patches between gcc-4.5 & gcc-4.6?
I believe it is not realistic, but not because of any plugin API - whatever its stability. Looking into the past, I 
don't believe that a random (rarely used) pass which has been developped as a patch to 4.2 can work without changes on 
4.3. I admit I have no direct example. But look at today's passes. Many of them has been patched between 4.4 & 4.5... 
This problem has nothing to do with plugins API (I mean with gcc/gcc-plugin.h to be specific). It is related to the fact 
that GCC is not organised as a strict set of well defined interfaces. This is a social property of the GCC community, 
and it worked quite well in practice (GCC is not in a very bad shape today!). Of course, some files are more important 
than others. Very probably, adding fields inside gimple.h is much less common than adding some extra passes. And 
obviously, adding an important file like gimple.h is for elite GCC developers.

4. The sources foo*.[ch] can *usually* be recompiled without patches between gcc-4.5.1 & gcc-4.5.2 ?
This is my current intuition. I really believe that even for minor version changes like 4.5.1 -> 4.5.2, plugins will 
need to be at least recompiled. And if I understand correctly, the

> I especially want to avoid adding plugin callbacks in code that is
> known to be not the way it should be.  Instead GCC internals should
> be cleaned up and adjusted to be plugin-friendly.  This probably
> will apply to some of the pass-manager hooks people are going to
> propose.
>
> > So I understood that people could propose new plugin events, provided that
> > they are not too costly to implement (mostly meaning that the
> > invoke_register_plugins calling them does not slow down compilations not
> > using plugins), and provided people can explain clearly why they need it. It
> > is not necessary to have a useful plugin coded & runnable to show the need
> > of the event.
>
> Yes it is.  And I am going to voice this opinion whenever I see people
> are adding hooks because it is fun to do so.

Nobody is adding plugin hooks because it is fun. Mind you, I find pushing minuscule hooks [like the PLUGIN_PRAGMAS] a 
very stressing & boring task (and I warmly thank Rafael Espindola & Diego Novillo for their help). It is really not fun 
for me. However, if I really miss a hook, I won't be able to design a future plugin the way I am thinking of it today. 
My fun is not in adding a new PLUGIN_* identifier in the plugin_event enum of gcc-plugin.h. My fun is more in coding the 
plugin itself (or at least, it should be).

Actually, I tend to believe that the plugin infrastructure should ideally permit to replace almost every single file in 
GCC by an equivalent file in the plugin. Take for example the pass manager (file gcc/passes.c) I understood that ICI 
have their own pass manager (By the way, I Basile don't care about the pass manager). So ICI guys would probably be 
happy to bypass all the functions in the gcc-4.5/gcc/passes.c and provide their own variant of these. My intuition is 
that this should be doable.

I probably am happy enough with the current set of hooks. I might wish in a year some hooks for CPP, but I understood 
Diego need them, so I believe they will appear.

But I am not sure that every GCC experiment is pluginable today (I really don't know). Take for instance all the papers 
in the recent GCC summits (let's say after 2007). All of them describe code related to GCC. I am not sure that every 
such code could fit as a plugin. I believe it should.

Perhaps all of GCC code is bypassable by a plugin which would handle PLUGIN_START_UNIT, copy or call all of the existing 
code of GCC inside it, and exit without returning... So probably, plugin could do anything.

But I believe we should provide a lot more hooks for them.

Plugins are important for a very social, non-technical reason: nobody today is compiling GCC -except the GCC community & 
 the linux distribution makers. The art of compiling monster software like GCC is almost lost. People don't compile 
free software anymore. (this has changed since the 1980s. At the time of SunOS3.5, people did compile gcc & emacs). And 
I recently discovered that compiling GCC is too difficult a task for most GCC users (at least outside academia). GCC 
plugin will help experimental extensions towards this population of GCC users (not able or not wanting to compile GCC).
Another important motivation for plugins is the licence (more exactly the legal requirement of FSF ownership). Coding 
GPLv3 code is accepted by many managers & organizations (because the GPLv3 is a mainstream licence), but transfering 
copyright to FSF is painful to most big organizations - because it is an unusual procedure -.

This is why defining a large enough set of hooks is important *today* [especially for hooks or events easy to implement 
today]. I am pretty sure that gcc-4.5 will still be a lot used in 2013 or 2017, when the GCC community will be working 
on GCC-5.5 or even 6.5 (I know about some small SMEs in the embedded software economy which are using today a 2.95 
version of GCC, and not really able to compile a 4.3 version of GCC, even if produces better code for their embedded ARM 
processor). So the set of hooks today is important for many years!


Notice also another thing. there have been a lot of plugin experimentation (at least in GCC summit papers). And every 
experiment was about code which the developer thought would never be accepted into the trunk (and that intuition is 
right: nothing of all the plugin experiments landed into the trunk [for example, TreeHydra did not land inside GCC, 
etc.], except the idea of plugins and the small plugin API).

This means that plugins would probably appear mostly for things that have no chance to land into GCC core. Plugin will 
more proably do everything but the kitchen sink. I would expect AMD & Intel to continue contributing to the GCC core to 
target their newest processors - I cannot imagine why AMD or Intel would target their 2011 processor with only a plugin.
[there is a simple reason for that: plugin are a bit more expensive to use that straight GCC - you need extra options & 
extra installations-, and human nature is lazy; the interest of constructors is to sell chips, not GCC plugins!]. 
Toplevel processor makers will continue to work inside GCC.

What could happen if some plugin events are not added quickly? People interested in them will simply lose interest in 
GCC (they could for instance go towards LLVM if they feel it is easier for them). People waited much too long for 
plugins to appear... they won't wait till 4.6 for an event essential for them which is basically a dozen-line patch; 
they will just stop looking at (or inside) GCC.

If the GCC community cared about experimental languages front-end (stuff like Modula3, or Mercury, ...), we should also 
have hooks for other front-ends [I personally have no real need for that]. As a concrete example, GNU CLISP could use 
GCC instead of GNU lightning (to translate its custom bytecode into binary). No, I am not volunteering -and when needing 
a Common Lisp, I'll use SBCL.

So in  contrast to Richard Guenther, I don't percieve any danger today in adding several new plugin events (even if they 
later happen to be nearly useless). Precisely because the plugin API is newborn.

We should not be afraid of new plugin events. We could be afraid of people wanting to start coding plugins going 
elsewhere (to some other compiler technology).

I am not afraid of ICI or TreeHydra guys adding useless plugin events. I am afraid of ICI or TreeHydra guys losing 
interest in GCC (and perhaps going to other compilers). If that happens, it will be a loss for the GCC community.

Regards.

PS. It could also be the case that plugin hooks could be bypassed by sophisticated ld.so tricks like LD_PRELOAD.

--
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***