Re: [trunk r143197] patch adding optional extra marking to GGC

[Basile STARYNKEVITCH <basile at starynkevitch dot net>]

Ian Lance Taylor wrote:
> Basile STARYNKEVITCH <basile@starynkevitch.net> writes:
>
>   
> > The idea is to add, for some peculiar users (I am explicitly thinking
> > of future plugin facilities), the ability to invoke a marking routine
> > on some data during GGC collection.
> >     

I forgot to insist that I am talking about calling the GGC collector 
from *inside* passes. It being called from the pass manager is ok for 
me, but there are situations where it is not enough. I repeat that my 
focus is the rare situations where invoking the GGC collector inside a 
pass is useful. I have no issues with the way it is called from the pass 
manager.
> This seems quite difficult to use correctly, and I think it would be
> quite a pain if all plugin authors have to get it right. 
I really don't understand this argument. A plugin author who would need 
to call the proposed ggc_collect_extra_marking should by definition 
understand what it is doing. This is why I also documented it. Is my 
documentation not clear enough? Constructive comments welcome here 
(remember that I am not a native English speaker, and my english is poor).

The only way when my proposed ggc_collect_extra_marking gets called is 
when a pass explicitly calls it (I see no use of it outside passes' 
bodies). And the pass should then do it rightly.

If GGC is invoked outside passes (as is mostly the case today), it is 
enough to follow the static & global roots as done today.
And if plugins only provide whole passes, sinces passes do not call 
directly each other, I don't see any issues if they want to call 
ggc_collect_extra_marking.
On the contrary, I believe it is a clean mechanism to use the GGC 
collector inside passes - it is better to have such pass collect some 
single particular local (or heap or whatever) data, than to have it use 
static data (which for example should be cleaned explicitly, adding a 
burdom to the pass developer).

Of course, plugin machinery (if it becomes accepted one day in the trunk 
- I believe there are still unsettled legal issues, which I cannot 
contribute to) should have some machinery to add extra static  roots. 
This is required, unless one adds the severe convention that plugins 
cannot have any GTY-ed static data.

>  Would your
> needs be satisfied if there were a way to dynamically add roots to the
> GC?
>   
It would be satisfied if I can add dynamically a marking *routine* (not 
only plain struct-s static data). I believe this could be hided as 
adding an extra root [a fake structure] and providing its marking 
routine, perhaps using tricky mark_hook or if_marked GTY options. 
http://gcc.gnu.org/onlinedocs/gccint/GTY-Options.html#GTY-Options But I 
really don't think it is simpler or safer. My  proposed patch 
ggc_collect_extra_marking is extremely short (most of the patch is 
documentation or comments, the real added code is 18 lines).

Actually, MELT has its own copying GC (working quite well) backed up by 
the GGC mark&sweep garbage collector. There is a routine which marks all 
the local variables in MELT code [routine basilys_extra_marking in 
MELT's file gcc/basilys.c line 513] and this routine is mostly 
automatically generated (so no stress on the developer coding in MELT, 
more precisely basilys_extra_marking calls automatically generated 
routines at line 534). For information, I had a MELT paper accepted at 
the GROW09 workshop explaining the technicalities in more details.


So I still don't understand what Ian is thinking when he says that 
adding static roots is always simpler than adding a marking routine & 
data thru my proposed ggc_collect_extra_marking, and I cannot believe 
that potential (but few) plugin developers would have more pain with it 
that thru additionnal static roots.

I really don't understand why Ian is thinking that the proposed 
ggc_collect_extra_marking is difficult to use. My feeling is on the 
contrary that the existing ggc_collect is very difficult to use (and the 
fact that it was not documented before is significant), precisely 
because it has the very peculiar and unusual requirement that every data 
should be static or global (ie because in technical terms it is a 
precise collector without any ability to add local roots). I know no 
other garbage collector which has such a strange requirement or 
misfeature! There might be even a psychological chicken&egg issue here: 
ggc_collect is so difficult to use [precisely because of the lack of 
local roots, which is so counterintuitive] that most GCC developers hate 
it, avoid using it at all costs -even if that means coding many hundreds 
of lines with potential memory leaks, and overgeneralize by rejecting 
the whole idea of any garbage collector inside GCC.

So Ian, please explain why you think that ggc_collect_extra_marking is 
difficult to use *inside passes* (I don't claim anything about 
ggc_collect being called from the pass manager) while the existing 
ggc_collect is (apparently in your view) easy to use *inside passes* - I 
believe on the contrary that using the GGC collector inside passes is 
extremely tricky and under-documented.

Again, my focus is on calling the garbage collector from inside passes. 
I have no objection on it being called by the pass manager as today, but 
I feel that are occasions where this might not be enough.

So I believe of course one might avoid the proposed 
ggc_collect_extra_marking when calling GGC from inside passes, but doing 
this require very tricky and in my view very brittle code, while the 
proposed ggc_collect_extra_marking is: very simple to implement, easy to 
explain (my proposed patch has more documentation than code!) and 
hopefully easy to use for the few developers who need to call the GGC 
collector from inside their pass. If you feel that I have to improve my 
documentation, please say so (but help me: my english is poor and a 
foreign language to me). If explaining what is a mark&sweep collector is 
required in doc/gty.texi for my ggc_collect_extra_marking patch to be 
accepted, I could do that.

For Ian and other C++ enthusiasts: once you have a way to mark local 
roots, as ggc_collect_extra_marking provides, adding a C++ collected 
smart pointer template is really easy (with also a new operator to 
interface to GGC, which would invoke the collect when memory is heavily 
used.); basically, such smart pointers would be made into an internal 
linked list handled by C++ constructors & destructors. A contrario, 
providing such a template without any way to mark local roots is impossible.

When commenting on my short proposal, don't forget that my focus is 
calling the collector from inside passes.

Regards.

--
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
membre de l'APRIL "promouvoir et défendre le logiciel libre"
Rejoignez maitenant pplus de 3900 adhérents http://www.april.org