User account creation filtered due to spam.
Created attachment 30784 [details]
Patch to protect libgomp thread pool against fork()
The problem is discussed in . To summarize if a process uses OpenMP
features and then calls fork, the threads from the OpenMP thread pool
of the parent process are not copied to the child process (which is
expected). Then later if the child process uses some OpenMP feature
again it will hang, waiting for threads that don't exist in its own
In practice this can often happen in Python programs that import
modules that use OpenMP internally while also using the
`multiprocessing` module. This module is in the Python standard
library and uses Unix fork for handling multi-core concurrency
efficiently at the Python level.
I attach the patch to this report and a test that checks that the fix actually works. The patch can also be visualized on this github branch .
When running the example snippet from  saved in a file called
`openmp_fork.c` I get the expected output:
$ gcc-head -fopenmp -o openmp_fork openmp_fork.c && ./openmp_fork
instead of a hanging process.
Note that the OpenMP implementation of ICC does not hang either when using fork.
Note 2: this problem is related to (a duplicate of)
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52303 which was deemed
invalid as the POSIX standard states that it's unsafe to use threading after a fork and prior to calling exec. However as the `pthread_atfork` protection from this patch is rather non-invasive and should not impact any process not initializing the libgomp runtime prior to a fork. Interpreted language implementations such as CPython that (ab)use Unix fork for efficient concurrency would really benefit from such a protection against libraries using OpenMP internally with the caller not necessarily being aware of it.
The PR52303 comment about what you are trying to do being invalid of course applies here too, and the patch isn't anything close to non-invasive, it is just wrong, because it will break valid OpenMP programs.
What kind of breakage does this introduce? It's a real question, I am not an experienced OpenMP developer.
Do you see any solution that would prevent libgomp-based programs such as mentioned in  to not hang after a fork().
Is it worth for me to spend time trying to work a better and safer solution (maybe with some guidance)?
Please read OpenMP 4.0, section 2.14.2 (threadprivate directive), or corresponding sections in older standards.
The implementation must preserve values of threadprivate variables in certain cases, which your hack violates. If somebody (validly) does:
#pragma omp threadprivate (v)
#pragma omp parallel num_threads (4)
v = omp_get_thread_num () * 16;
pid = fork ();
if (pid == 0)
/* Valid fork child, only calling functions POSIX allows it to. */
#pragma omp parallel num_threads (4)
if (v != omp_get_thread_num () * 16)
then the implementation must preserve the threadprivate values, but with your patch all the threads but the initial one will be lost during fork and thus v will be 0 instead of the desired value later on.
There is no point trying to hack around bugs in your code inside of libgomp, simply follow the requirements how you can use fork in multithreaded apps.
Thanks for the explanation. Would you consider a solution that would preserve the state of the parent process and would just reset the thread pool data on the child?
Otherwise we will have to consider that the way fork() is used in Python's multiprocessing module is really an abuse and that there is no way to safely use both openmp-libraries and Python multiprocessing in the same program safely.
Having a pthread_atfork child hook that would do freeing of memory, or pthread_mutex_init etc. would only make invalid any OpenMP program using fork, even those that use it correctly.
Alright thanks again. For reference I just discovered that the issue has recently been fixed in Python 3.4 by adding a new `forkserver` option to multiprocessing.