Defining a common plugin machinery

Brendon Costa bcosta@avdat.com.au
Fri Sep 19 09:35:00 GMT 2008


Hi all,

Firstly, do others think it would be helpful to summarise any of this
information on a wiki page, or will these emails be fine?

In this email I will give my opinion on the questions asked by Deigo and
ask a few additional questions that may be relevant.

------------------------------
What features do we want to support?

* Hooks
I like the sound of what Taras described in terms of the actual format
of defining the hooks throughout the GCC code. The hooks i currently use
for my project are (in GCC 4.0.4):
- c_common_init() : I use to initialise my module. Will be whatever we
decide for plugins.
- c_common_finish() : I use to finalise my module. Will be whatever we
decide for plugins.
- finalize() : I use this hook to write my data to a file. This is
before the asm output as one of my output format options is to insert my
data into a new ".edoc" section in the resulting binary. I do this by
appending to the asm file: ".section .edoc" and then defining a lot of
".byte" lines with the data.
- gimplify_function_tree() : To process the AST from the C/C++ front
ends before it is gimplified.

* Automatically loaded plugins as well as explicit user requested plugins
I would like to propose that we allow automatic loading of certain
plugins in addition to the explicit request for loading of plugins using
the -fplugin=<name> command line option.

The idea is that their are two "types" of plugin that are differentiated
only by the location in which they are installed and how they are loaded
by the GCC application(s). The first type of plugin as has been
discussed is loaded on explicit user request with -fplugin=xyz, the
second is a plugin that is always loaded automatically on execution of
the GCC application(s). A plugin found in for example an "autoload"
directory will be loaded every time the GCC application(s) are executed.

The main reason i see for such a feature is that it allows automatically
loaded plugins to become "transparent". I.e. There is nothing that the
user needs to do to make use of the new plugin but install it in the
correct location.

This could be a good thing if there is a desire in the future to move
core GCC features (such as existing optimization passes) into plugins
without having to require the user to specify -fplugin=<xyz>. Also if a
user finds a good optimisation plugin that they download and they want
to use it for everything they compile, they could just install it in the
autoload directory and not have to worry about ensuring that all
projects they compile somehow provide -fplugin=myoptim to GCC. I have
other reasons for wanting this with my project but it is are primarily
for convenience in my case (Using environment variables for turning
on/off features instead of command line options).


------------------------------
What should we export?
* I think that exporting as much as we can from GCC is good (say
everything but the main() function), not just a limited sub-set that
plugins can access. This gives the most flexibility for plugins (even if
we don't foresee people using all this at any point for now). Also it is
a lot simpler to do than defining a subset of interfaces to be exported
and making sure we export everything that future plugins *MAY* want to use.

Possible reasons against exporting everything:
* If we want to define a stable API for plugins then we would not export
everything. To be honest i think doing something like this would be too
much work for the benefit.
* It is possible that exporting everything could be a lot of work. In
particular i think for the win32 platform, we will need to mark exported
entities with __declspec(dllexport) and when used with
__declspec(dllimport) (Who in MS thought it was a good idea to have a
different markup for export and import....). I assume for non-win32
platforms we will be using default visibility, i.e. Not using:
-fvisibility=hidden?


------------------------------
What should be the user interface to incorporate plugin code?

It has been proposed that we use:
-fplugin=<name> -fplugin-arg=<arg>

I propose that we use slightly different format:
g++ -fplugin=<name> -f<name>-<arg>[=value]

for example:
g++ -fplugin=edoc -fedoc-file=blah.edc

My reasons for this are from the following cases:
1) multiple plugins with a single invocation
2) key/value arguments required for plugins
3) plugins that may want to be loaded automatically

1) This is not a big issue. I just clarity with the behavior.
Say we want to use two plugins in the same invocation. This might look like:

Original: g++ -fplugin=edoc -fplugin-arg=embed -fplugin=optim
-fplugin-arg=max
New:     g++ -fplugin=edoc -fedoc-embed -fplugin=optim -foptim-max

If using the original method, the order of the specification matters,
with the new method it does not.

2) With my EDoc++ project one of the arguments i look for has a format
like: -fedoc-file=blah.edc What would this look like with the
-fplugin-arg= format?

Possibilities with Original format:
g++ -fplugin=edoc -fplugin-arg=file:blah.edc
g++ -fplugin=edoc -fplugin-arg-file=blah.edc
g++ -fplugin=edoc -fplugin-arg=file=blah.edc
g++ -fplugin=edoc -fplugin-key=file -fplugin-value=blah.edc

New:
g++ -fplugin=edoc -fedoc-file=blah.edc

Personally the "original" method seems to be a little awkward and "bulky".

3) Automatic loading of plugins

If we allow automatic loading of plugins as i propose elsewhere in this
email. Then passing arguments to those plugins might be a bit awkward.
In particular (Assuming plugin edoc is loaded automatically):

Original method:
g++ -fplugin-arg=file:blah.edc (or one of the methods described above)

New method:
g++ -fedoc-file=blah.edc

In this case, the original method, it is not obvious which plugin the
argument belongs to and in particular if multiple plugins are loaded
automatically it is not possible to differentiate between the various
plugins and which should recieve the argument.. In the method i am
proposing it is.

There are some problems though with my proposal. In particular:
* A plugins name can not be the same as an existing -fxyz option.
* Implementing it may be a bit more work


------------------------------
At what point in the compilation stage should it kick in?

I think plugins should be loaded as soon as possible. I would propose
loading plugins immediately after the processing of command line
arguments or maybe even at the same time as command line argument
processing depending on what might be easier/cleaner.


------------------------------
Some extra questions.
------------------------------
What platforms do we want to support? I can think of the following
categories:

* Old platforms (No dlopen support)
Do we use the libltdl dlpre-opening to at least support "plugins" that
may be officially distributed with GCC?

* Windows (cygwin/mingw)
As i understand the issue (I am not very familiar with this) you can't
have unresolved references in a plugin back to the GCC executable. I.e.
Building GCC with -rdynamic will not work on this platform. Do we move
most of the GCC implementation into a "library/DLL" having a "stub"
main() that just calls the library implementation. Then both the
application AND the plugins can link with this library/DLL in a way that
will work on Windows.
Or are we going for the quick solution of using -rdynamic and not
supporting this platform (Not my preferred option)?

* "Newer" ELF platforms
These should be fine regardless of the method we use i think.


------------------------------
How do we search for plugins and make sure we don't load incompatible ones?

We want to avoid loading plugins for different/incompatible
builds/versions of GCC. Things i think we need to take into account
* Multiple installed versions of GCC
* Multiple installed builds for the same version of GCC (Is this worth
catering for?)
* Cross compilers
* Finding the directory where to install a plugin should not be TOO
difficult for an external plugin project

A few methods i can think of achieving this include:
* Enforced plugin naming conventions
* Searching for plugins only in a directory that expects plugins for a
particular version/build
* Embedding a version symbol in plugin binaries that can be queried
before dlopening the plugin

With that in mind, i assume the best option will be to define a specific
directory from where a specific build of GCC will search for plugins.
Would the following directory be one of the better locations to put
those plugins or is this a directory for objects that get linked into
generated binaries?
$libdir/gcc/i386-unknown-netbsdelf3.0/4.3.2/plugins/

Then comes the issue of thinking about how does an external project
locate that directory? Do we add an option to the command line of GCC to
obtain this directory name?

------------------------------
How/where do we install headers and the library for external plugin
projects to use?


Thanks,
Brendon.



More information about the Gcc mailing list