Speeing up __cxa_atexit

[Mark Mitchell]

--On Thursday, June 06, 2002 09:29:55 AM -0400 Jack Howarth 
<howarth@bromo.msbb.uc.edu> wrote:

> Mark,
>     Jakub was concerned that introducing -fuse-cxa-atexit into
> code that didn't need it would add significant startup overhead...
>
> -----------------------------------------------------------------
> Is this because of [basic.stat.term]/3?
> It is pretty expensive for startup time, especially with current glibc
> implementation which is not designed for lots of atexit registered
> handlers (as compared to DT_FINI resp. DT_FINI_ARRAY). atexit time is
> O(number_of_currently_registered_atexit_handlers), and even if it wasn't,
> it is pthread_lock + malloc + pthread_unlock + some data moving around
> as opposed to doing nothing at constructor time. The destruction times
> will be equal I think.
>
>         Jakub
> ------------------------------------------------------------------

Jack forwarded me the above message, which I had either misse for
hadn't gotten to yet.

We just don't have a choice for the standard library.

See my point about what happens if users replace functions like malloc.
It is only safe to use -fno-cxa-atexit with V3 if this condition is met:

- No dynamic initialization, nor its associated destruction, involves a
  direct or indirect call to any function which can be overridden by
  the user.

This is not an easy condition to analyze by looking at the code, nor is
it one that is necessarily going to remain true tomorrow just because
it is true today.

We are looking for headaches if we go this route.

(Note that the condition is met vacuously if there are no dynamic
initializations, which is what was (indirectly) alleged by the person
who did "nm | grep" and saw no atexit calls.  But, in that case, there
is no reason to use -fuse-cxa-atexit.)

Regarding efficiency:

If atexit is quadratic, it should be fixed.

Clearly the malloc calls can be amortized into big allocations.

Furthermore, there are rarely multiple threads when initialization is
happenning, so the pthread_lock calls should be cheap.  (You can keep
an unlocked boolean variable which tells you if there is more than one
thread; since you have to create a new thread by calling a function
you can make that function set the boolean variable. You don't need a
lock because you always start with a single thread.)

I think that if there are multiple threads you can get rid of the locks
in the following way.

Keep an array of atexit lists indexed by thread id. Also keep a global
counter, which is not locked.  When someone calls atexit, add an entry
to the appropriate list (containing the function to call at exit time
and the current value of the counter), and bump the counter.

The counter should have the property that multiple bumps at once result
in the counter having some value between its original value and the
original value plus the number of bumps and that reads during this
period never show the counter going backwards.  Using sigatomic_t on
most platforms should work.

This guarantee tells you that all the entries on the list for a
particular thread will be monotonically non-decreasing.

When you get to "exit", grab a lock and sort all the lists by the
counter (primary key) and by the order they appeared on the list
for a particular thread-id (secondary key).

You can only sort in a different order from the actual temporal order
if there were multiple atexit calls at once in different threads.  If
that happenned, it doesn't matter which order you sort things in; the
program had no guarantee about the order anyhow.  (A program that wants
deterministic atexit order has to grab locks around the calls to atexit.)

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com