Re: Plan for removing global state from GCC's internals

[David Malcolm <dmalcolm at redhat dot com>]

On Wed, 2013-06-26 at 15:19 -0400, Dmitry Mikushin wrote:
> FWIW, we also needed to perform multiple invocations of toplev_main from
> a single execution of GCC frontend, which seems to be quite similar. 

Interesting.  Yes, this sounds very similar to the kinds of use-cases
I'm considering.  Am I right in thinking you're using GCC (and LLVM) to
target GPUs?

> The
> dirty dirty hack is to save the backup the content of .data and .bss
> symbols with ELF API before the first call to toplev_main and reset them
> to backup values before each subsequent call. And it works. 

Thanks.  I went looking for your code; for reference, is this what
you're referring to?
https://hpcforge.org/scm/viewvc.php/trunk/patches/gcc.patch?revision=1918&root=kernelgen&view=markup
(i.e. the changes to "main"?)

> Would be
> great to get rid of global state in a better way, maybe our approach
> could be useful for transition period.

The "backup of .data and .bss" approach allows for repeated calls to
toplev_main, but it doesn't allow for multiple threads to be running
simultaneously within one process.

As you say, it's a "dirty dirty hack" - I'm glad it works for you, but
it seems very fragile: e.g. what happens about GCC plugins and other
DSOs linked into the process: presumably they also have state, which
isn't going to get handled if you're only blowing away the state of the
main executable.

I'm interested in cleaning this up "properly"... for some definition of
that word, of course!



> - D.
> 
> On 06/26/2013 02:46 PM, David Malcolm wrote:
> > I've been looking at removing global state from GCC with a view to
> > making it be usable as a shared library.
> > 
> > I've been posting various patches relating to this, but I thought it was
> > time to post a comprehensive plan so you can see how I think it all
> > ought to fit together.
> > 
> > You can see an HTML version of my proposal at:
> >   http://dmalcolm.fedorapeople.org/gcc/global-state/
> > 
> > and the source for the doc can be seen at:
> >   https://github.com/davidmalcolm/gcc-global-state/
> > (restructured text, using the Sphinx toolchain).
> > 
> > I've gone through all of GCC's passes, identifying internal state, and
> > also looked at the most-frequently used global variables in GCC.  You
> > can see detailed notes on these in the appendices.
> > 
> > It's still a work-in-progress - there are still quite a few TODOs in
> > there, but it seemed time to post to this list.
> > 
> > A single-paragraph summary is that I want to move global variables and
> > functions into classes: these classes will be "singletons" in the normal
> > build, and will have multiple instances in a shared library build,
> > allowing for multiple "parallel universes" of state within one GCC
> > process.  There are various tricks to (a) maintain the performance of
> > the standard "monolithic binaries" use case and (b) minimize the
> > patching and backporting pain relative to older GCC source trees.  In
> > particular, it introduces a new compiler attribute "force_static" which
> > gets used in stages 2 and 3 of the bootstrap to eliminate "this" from
> > methods of the various singleton classes.
> > 
> > I hope the plan seems reasonable to the core GCC maintainers, and I'm
> > keen to get moving on this for GCC 4.9.   However I appreciate that I'm
> > a relative newcomer to GCC development (albeit the author/maintainer of
> > the gcc python plugin, for the last 2 years).
> > 
> > There are various questions e.g. what can go into trunk vs a branch?
> > various naming decisions etc.
> > 
> > The "trunk vs branch" question is the one I'm most keen to resolve.  In
> > particular, I'm aware that Andrew MacLeod recently posted another
> > architectural proposal:
> >   http://gcc.gnu.org/ml/gcc/2013-06/msg00163.html
> > AIUI his proposal and mine are mostly orthogonal to each other: his
> > makes changes to the insides to "tree", whereas mine bundles up global
> > variables and functions into classes.  Both proposals involve touching a
> > lot of code but both can largely be done incrementally, and, I hope
> > independently of each other - but I want to avoid painful branch
> > mergers, of course.
> > 
> > BTW, I will be at the GNU Tools Cauldron next month.
> > 
> > Thoughts?
> > Dave
> > 
>