segfault with inquire()

[Andrew Benson]

Thanks for that suggestion. Running the code from my previous email with -
fsanitize=thread I get a lot of warnings about potential deadlocks:

==================
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=9982)
  Cycle in lock order graph: M26 (0x7f02bb31c340) => M84 (0x7b58002efee0) => 
M26

  Mutex M84 acquired here while holding mutex M26 in thread T63:
    #0 pthread_mutex_lock ../../../../gcc-trunk/libsanitizer/sanitizer_common/
sanitizer_common_interceptors.inc:4148 (libtsan.so.0+0x5095a)
    #1 __gthread_mutex_lock ../libgcc/gthr-default.h:749 (libgfortran.so.
5+0x1e25c9)
    #2 insert_unit ../../../gcc-trunk/libgfortran/io/unit.c:244 
(libgfortran.so.5+0x1e25c9)
    #3 gomp_thread_start ../../../gcc-trunk/libgomp/team.c:123 (libgomp.so.
1+0x198a5)

  Mutex M26 previously acquired by the same thread here:
    #0 pthread_mutex_lock ../../../../gcc-trunk/libsanitizer/sanitizer_common/
sanitizer_common_interceptors.inc:4148 (libtsan.so.0+0x5095a)
    #1 __gthread_mutex_lock ../libgcc/gthr-default.h:749 (libgfortran.so.
5+0x1e265c)
    #2 get_gfc_unit ../../../gcc-trunk/libgfortran/io/unit.c:332 
(libgfortran.so.5+0x1e265c)
    #3 gomp_thread_start ../../../gcc-trunk/libgomp/team.c:123 (libgomp.so.
1+0x198a5)

  Mutex M26 acquired here while holding mutex M84 in thread T63:
    #0 pthread_mutex_lock ../../../../gcc-trunk/libsanitizer/sanitizer_common/
sanitizer_common_interceptors.inc:4148 (libtsan.so.0+0x5095a)
    #1 __gthread_mutex_lock ../libgcc/gthr-default.h:749 (libgfortran.so.
5+0x1e336b)
    #2 close_unit_1 ../../../gcc-trunk/libgfortran/io/unit.c:735 
(libgfortran.so.5+0x1e336b)
    #3 gomp_thread_start ../../../gcc-trunk/libgomp/team.c:123 (libgomp.so.
1+0x198a5)

  Mutex M84 previously acquired by the same thread here:
    #0 pthread_mutex_trylock ../../../../gcc-trunk/libsanitizer/tsan/
tsan_interceptors_posix.cpp:1246 (libtsan.so.0+0x4340a)
    #1 __gthread_mutex_trylock ../libgcc/gthr-default.h:758 (libgfortran.so.
5+0x1e26f1)
    #2 get_gfc_unit ../../../gcc-trunk/libgfortran/io/unit.c:380 
(libgfortran.so.5+0x1e26f1)
    #3 gomp_thread_start ../../../gcc-trunk/libgomp/team.c:123 (libgomp.so.
1+0x198a5)

  Thread T63 (tid=10047, running) created by main thread at:
    #0 pthread_create ../../../../gcc-trunk/libsanitizer/tsan/
tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5bb22)
    #1 gomp_team_start ../../../gcc-trunk/libgomp/team.c:839 (libgomp.so.
1+0x19eba)
    #2 main <null> (a.out+0x400ef9)

SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) ../libgcc/
gthr-default.h:749 in __gthread_mutex_lock
==================

-Andrew


On Wednesday, December 4, 2019 7:22:20 PM PST Tobias Burnus wrote:
> Using -fsanitize=thread did help in the past to find some race
> conditions with threads. I don't know what exactly it tests for and it
> might very well be that your issue is not covered – but you could also
> give it a try.
> 
> See also:
> https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanit
> ize_003dthread
> 
> Tobias
> 
> On 12/4/19 7:08 PM, Andrew Benson wrote:
> > I compiled this with -O0 to ensure that the inquire didn't get optimized
> > away, and verified that it actually calls the inquire code in libgfortran
> > by setting a break point in gdb. It runs without any problems.
> > 
> > I'll keep trying a few other variations around this idea and see if I can
> > reproduce the issue.
> > 
> > For what it's worth, in my code that triggers this problem I wrapped the
> > "inquire()" inside an OpenMP critical section so that no more than one
> > OpenMP thread can be calling inquire() at a time. So far it's been
> > running for about 20 hours which is the longest it's run without crashing
> > since this problem began to appear. So, I take this as another piece of
> > evidence that there's some thread conflict, but it doesn't really help to
> > narrow down where it happens!
> > 
> > Thanks,
> > Andrew


-- 

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus