Bug 54601 - AIX uses atexit which causes unloading of shared modules to break
Summary: AIX uses atexit which causes unloading of shared modules to break
Status: NEW
Alias: None
Product: gcc
Classification: Unclassified
Component: c++ (show other bugs)
Version: 4.5.2
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
Depends on:
Reported: 2012-09-16 14:36 UTC by Perry Smith
Modified: 2015-11-11 07:59 UTC (History)
4 users (show)

See Also:
Known to work:
Known to fail:
Last reconfirmed: 2012-10-03 00:00:00

git diff -c of base 4.5.2 tarball against new files and changes (8.83 KB, application/octet-stream)
2012-09-16 14:36 UTC, Perry Smith
Educational program demonstrating when atexit is used. (316 bytes, application/octet-stream)
2012-09-16 14:39 UTC, Perry Smith
cxa_atexit implementation in libgcc (2.89 KB, text/plain)
2013-01-30 16:59 UTC, David Edelsohn
Revised __cxa_atexit for libgcc (2.90 KB, patch)
2013-01-30 21:57 UTC, David Edelsohn
Details | Diff
__cxa_atexit with new crt file (3.18 KB, patch)
2013-01-31 06:11 UTC, David Edelsohn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Perry Smith 2012-09-16 14:36:23 UTC
Created attachment 28200 [details]
git diff -c of base 4.5.2 tarball against new files and changes

The bug was hit when using Apache httpd with a module called passenger.  passenger uses some c++ code.  httpd does an unload of all of the modules that it loads when it exits.  The symptom is a core dump because by the time the atexit is run, the TOC entries as well as the code to execute which are registered as the DTORs have been unloaded.

Using the x1.cpp attached, I determined that the atexit is used to register a DTOR for a static object with a non-trivial DTOR that is declared inside a function.  The DTOR is registered only if the function is called.  (x1.cpp does not demonstrate the problem.  It was just used to educate me about when gcc used the atexit service.)

I am also supplying a patch to the base 4.5.2 tarball that fixes the problem.  I stole the cxa_atexit and cxa_finialize from GNU's glibc-2.16.0 implementation.  I believe I have removed glibc's locking and successfully added back in the locking that gcc uses so that the file is built properly on the four different platforms: 32 bit, 32 bit threaded, 64 bit and 64 bit threaded.

The changes to the configure and makefiles will need to be ported to the real files.  I added references to the two new files introduced into the makefile and changed the configure script to assume cxa_atexit is present on AIX systems so that it will honor the --enable-__cxa_atexit configure option.  The options I used for configure are:

    "--with-gmp=${PUBLIC_BASE}" \
    "--with-mpfr=${PUBLIC_BASE}" \
    "--with-mpc=${PUBLIC_BASE}" \
    "--disable-nls" \
    "--enable-threads=aix" \
    "--with-libiconv-prefix=/usr" \
    "--enable-__cxa_atexit" \

If all goes well, I propose to have --enable-__cxa_atexit be the default for AIX.

The final big change is to collect2.  The changes are rather large.  I tried to keep the proper style.  There may be some debugging that can be removed.  A new c stub is created in the first link pass so that __dso_handle can be declared to allow the first link to complete.  I generalized a few things in the process.  I hope all that is ok.

I've used this new compiler for a month now but only on the original problem.  All seems to be working.
Comment 1 Perry Smith 2012-09-16 14:39:05 UTC
Created attachment 28201 [details]
Educational program demonstrating when atexit is used.

Run it without an argument, the function that declares the static is not called -- thus the DTOR is not registered.  Run it with an argument to call the function that declares the static.
Comment 2 Perry Smith 2012-09-16 14:40:25 UTC
Forgot to mention.  A long thread discussing the issue, etc is here:

Comment 3 Jonathan Wakely 2012-09-16 18:47:11 UTC
To be included the patch needs to be against trunk.
Comment 4 Perry Smith 2012-09-17 14:18:34 UTC
(In reply to comment #3)
> To be included the patch needs to be against trunk.

I don't mind trying to redo this against trunk but I need some help learning how to build against the trunk.  I'm assume it is documented somewhere but I don't know where.  I looked (e.g. site:gcc.gnu.org/onlinedocs build trunk) doesn't hit anything useful.  The pdf does not have "autoconf" nor "automake" which I'm sure are needed.  So I got stumped.  Sorry.
Comment 5 Paolo Carlini 2012-10-01 16:31:10 UTC
Adding David as AIX maintainer.

In terms of building, nothings extraordinary happened between 4.5 and current mainline. Make sure you have the required libraries installed (minimally, gmp, mpfr, mpc) and build as usual. Maybe David can help you about AIX-specific details.
Comment 6 David Edelsohn 2012-10-03 17:52:48 UTC
Perry and I have been discussing this for a while.  As others mentioned, the patch should be against mainline and this discussion of the path should happen on gcc-patches, not Bugzilla.
Comment 7 David Edelsohn 2013-01-30 16:59:13 UTC
Created attachment 29307 [details]
cxa_atexit implementation in libgcc

This version of the patch implements __cxa_atexit and __cxa_finalize in libgcc, not libsupc++, with no modifications to collect2.  I am not sure if using a low priority C destructor to run __cxa_finalize early in the destructor list is correct. The original collect2 patch ran it last, after other destructors, which I believe is incorrect and does not match crtstuff.c semantics for ELF.  When I configured GCC with the patch and --enable__cxa-atexit, I saw a few additional C++ errors in the testsuite.
Comment 8 David Edelsohn 2013-01-30 21:57:52 UTC
Created attachment 29311 [details]
Revised __cxa_atexit for libgcc

The revised patch uses LIB2ADDEH to add the functions to libgcc_s.a and libgcc_eh.a so that only one copy should exist in a process.
Comment 9 David Edelsohn 2013-01-31 04:52:51 UTC
This is going to be more difficult because __dso_handle needs to be provided by a new crt file supplied by GCC, not by the libraries. That is the way that every module ends up with a unique value. Otherwise the single __dso_handle is shared across the entire process.
Comment 10 Perry Smith 2013-01-31 05:50:00 UTC
Can you keep the libraries like you have them with the functions and private structures for the functions coming from the library but just put the definition of __dso_handle in crt?
Comment 11 David Edelsohn 2013-01-31 06:11:31 UTC
Created attachment 29312 [details]
__cxa_atexit with new crt file

I think it needs something like this new patch, which adds crtdso.o providing __dso_handle.  collect2 searches the file properly and finds the destructor. But I am not sure if the driver will find the file and if the file is installed correctly yet.
Comment 12 David Edelsohn 2013-02-01 20:26:37 UTC
Author: dje
Date: Fri Feb  1 20:26:24 2013
New Revision: 195675

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=195675
        PR target/54601
        * config.host (powerpc-ibm-aix[56789]): Add t-aix-cxa to tmake_file.
        Add crtcxa to extra_parts.
        * config/rs6000/exit.h: New file.
        * config/rs6000/cxa_atexit.c: New file.
        * config/rs6000/cxa_finalize.c: New file.
        * config/rs6000/crtcxa.c: New file.
        * config/rs6000/t-aix-cxa: New file.
        * config/rs6000/libgcc-aix-cxa.ver: New file.

        * configure.ac (cxa_atexit): Add AIX.
        * configure: Regenerate.

        * config/rs6000/aix61.h (STARTFILE_SPEC): Add crtcxa.o.

Comment 13 David Edelsohn 2015-11-10 15:01:02 UTC
The recent additions to GCC cxa atexit support on AIX may fix this.
Comment 14 Hugo Koblmueller 2015-11-10 15:57:25 UTC
David, which version does/will include these recent additions?

I recently encountered a crash on program exit in AIX 6.1, in a setup where I used a static C++ objects inside functions within a shared library (accessed via dlfcn interfaces & closed via dlclose), compiled with a gcc-4.6.3.
Having the gcc configure flag "--enable-__cxa_atexit" in place and working (meaning cxa_atexit is present) shall fix this issue.
Comment 15 David Edelsohn 2015-11-10 16:06:10 UTC
GCC development trunk and it will be in GCC 5.3.
Comment 16 Hugo Koblmueller 2015-11-11 07:59:09 UTC
Thanks, David! 
Any idea when 5.3 will be released? Did not find any info in the gcc dev-plan (https://gcc.gnu.org/develop.html).

Richard writes something about October... (https://gcc.gnu.org/ml/gcc/2015-07/msg00197.html)