Re: Plan for removing global state from GCC's internals

[Dmitry Mikushin <dmitry at kernelgen dot org>]

Yes, generation of both binary code and LLVM IR in a single GCC
invocation. So, first toplev_main goes as usual, and another one - with
DragonEgg plugin enabled. LLVM IR ends up as GPU kernels code a bit later.

Yes, that is the right patch.

Of course, not thread-safe, not generally portable and very fragile.
That's why I'm saying "FWIW", meaning it might be useful for some
internal transitioning during your very useful effort.

- D.

On 06/26/2013 03:54 PM, David Malcolm wrote:
> On Wed, 2013-06-26 at 15:19 -0400, Dmitry Mikushin wrote:
>> FWIW, we also needed to perform multiple invocations of toplev_main from
>> a single execution of GCC frontend, which seems to be quite similar. 
> 
> Interesting.  Yes, this sounds very similar to the kinds of use-cases
> I'm considering.  Am I right in thinking you're using GCC (and LLVM) to
> target GPUs?
> 
>> The
>> dirty dirty hack is to save the backup the content of .data and .bss
>> symbols with ELF API before the first call to toplev_main and reset them
>> to backup values before each subsequent call. And it works. 
> 
> Thanks.  I went looking for your code; for reference, is this what
> you're referring to?
> https://hpcforge.org/scm/viewvc.php/trunk/patches/gcc.patch?revision=1918&root=kernelgen&view=markup
> (i.e. the changes to "main"?)
> 
>> Would be
>> great to get rid of global state in a better way, maybe our approach
>> could be useful for transition period.
> 
> The "backup of .data and .bss" approach allows for repeated calls to
> toplev_main, but it doesn't allow for multiple threads to be running
> simultaneously within one process.
> 
> As you say, it's a "dirty dirty hack" - I'm glad it works for you, but
> it seems very fragile: e.g. what happens about GCC plugins and other
> DSOs linked into the process: presumably they also have state, which
> isn't going to get handled if you're only blowing away the state of the
> main executable.
> 
> I'm interested in cleaning this up "properly"... for some definition of
> that word, of course!
> 
> 
> 
>> - D.
>>
>> On 06/26/2013 02:46 PM, David Malcolm wrote:
>>> I've been looking at removing global state from GCC with a view to
>>> making it be usable as a shared library.
>>>
>>> I've been posting various patches relating to this, but I thought it was
>>> time to post a comprehensive plan so you can see how I think it all
>>> ought to fit together.
>>>
>>> You can see an HTML version of my proposal at:
>>>   http://dmalcolm.fedorapeople.org/gcc/global-state/
>>>
>>> and the source for the doc can be seen at:
>>>   https://github.com/davidmalcolm/gcc-global-state/
>>> (restructured text, using the Sphinx toolchain).
>>>
>>> I've gone through all of GCC's passes, identifying internal state, and
>>> also looked at the most-frequently used global variables in GCC.  You
>>> can see detailed notes on these in the appendices.
>>>
>>> It's still a work-in-progress - there are still quite a few TODOs in
>>> there, but it seemed time to post to this list.
>>>
>>> A single-paragraph summary is that I want to move global variables and
>>> functions into classes: these classes will be "singletons" in the normal
>>> build, and will have multiple instances in a shared library build,
>>> allowing for multiple "parallel universes" of state within one GCC
>>> process.  There are various tricks to (a) maintain the performance of
>>> the standard "monolithic binaries" use case and (b) minimize the
>>> patching and backporting pain relative to older GCC source trees.  In
>>> particular, it introduces a new compiler attribute "force_static" which
>>> gets used in stages 2 and 3 of the bootstrap to eliminate "this" from
>>> methods of the various singleton classes.
>>>
>>> I hope the plan seems reasonable to the core GCC maintainers, and I'm
>>> keen to get moving on this for GCC 4.9.   However I appreciate that I'm
>>> a relative newcomer to GCC development (albeit the author/maintainer of
>>> the gcc python plugin, for the last 2 years).
>>>
>>> There are various questions e.g. what can go into trunk vs a branch?
>>> various naming decisions etc.
>>>
>>> The "trunk vs branch" question is the one I'm most keen to resolve.  In
>>> particular, I'm aware that Andrew MacLeod recently posted another
>>> architectural proposal:
>>>   http://gcc.gnu.org/ml/gcc/2013-06/msg00163.html
>>> AIUI his proposal and mine are mostly orthogonal to each other: his
>>> makes changes to the insides to "tree", whereas mine bundles up global
>>> variables and functions into classes.  Both proposals involve touching a
>>> lot of code but both can largely be done incrementally, and, I hope
>>> independently of each other - but I want to avoid painful branch
>>> mergers, of course.
>>>
>>> BTW, I will be at the GNU Tools Cauldron next month.
>>>
>>> Thoughts?
>>> Dave
>>>
>>
> 
>