Bug 82635 - std::thread's join broken on FreeBSD with all GCCs >= 5
Summary: std::thread's join broken on FreeBSD with all GCCs >= 5
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libgcc (show other bugs)
Version: 8.0
: P3 normal
Target Milestone: ---
Assignee: Andreas Tobler
URL:
Keywords:
: 82634 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-10-20 13:13 UTC by Hannes Hauswedell
Modified: 2017-11-05 19:43 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2017-10-20 00:00:00


Attachments
test program (360 bytes, text/x-csrc)
2017-10-20 13:14 UTC, Hannes Hauswedell
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hannes Hauswedell 2017-10-20 13:13:30 UTC
See the attached code. It compiles "fine" everywhere, but the resulting binary crashes in some situations:

It runs fine on all compilers on Linux; it runs fine on all versions of clang on FreeBSD and on GCC-4.9 on FreeBSD.

With GCC>=5 (including current 8-snapshot) the program crashes after 5 (sometimes 6) iterations. This may depend on the hardware, I can't say for sure. It is also strange how it is only triggered after multiple iterations.

The problem must have been introduced in the last months, I know for sure that it did not happen in June. However it seems strange that it affects all GCCs down to version 5. Maybe something that was backported?

I am CCing FreeBSD's port maintainer in case something happened on the packaging side of things.
Comment 1 Hannes Hauswedell 2017-10-20 13:14:41 UTC
Created attachment 42411 [details]
test program
Comment 2 Jonathan Wakely 2017-10-20 14:23:04 UTC
*** Bug 82634 has been marked as a duplicate of this bug. ***
Comment 3 Jonathan Wakely 2017-10-20 14:37:12 UTC
https://gcc.gnu.org/bugs/

What we need:
...
- the complete command line that triggers the bug;

Are you linking to libpthread?

Where does it crash?


(In reply to Hannes Hauswedell from comment #0)
> The problem must have been introduced in the last months, I know for sure
> that it did not happen in June. However it seems strange that it affects all
> GCCs down to version 5. Maybe something that was backported?

Nope.

The code is largely the same for GNU/Linux and FreeBSD, and I'm not aware of any problems in it, and TSan doesn't show any, which suggests a problem in the FreeBSD pthreads implementation.

I do recall something in FreeBSD's pthreads impl being non-conforming, where the static PTHREAD_MUTEX_INITIALIZER (or maybe PTHREAD_COND_INITIALIZER) creates an invalid object that then requires dynamic initialization on the first use. There could be a race in that code (or I could be totally misremembering).
Comment 4 Hannes Hauswedell 2017-10-20 14:59:17 UTC
> the complete command line that triggers the bug;

/usr/local/bin/g++7 -Wl,-rpath -Wl,/usr/local/lib/gcc7/ -std=c++11 -pthread test_thread.cpp
(g++7 could be g++6 g++6 or g++8)

> Are you linking to libpthread?

I am calling -pthread (or -lpthread), but ldd says:
% ldd a.out
a.out:
        libstdc++.so.6 => /usr/local/lib/gcc7//libstdc++.so.6 (0x800825000)
        libm.so.5 => /lib/libm.so.5 (0x800bba000)
        libgcc_s.so.1 => /usr/local/lib/gcc7//libgcc_s.so.1 (0x800de5000)
        libthr.so.3 => /lib/libthr.so.3 (0x800ffc000)
        libc.so.7 => /lib/libc.so.7 (0x801224000)

The broken combinations crash after a few iterations:
% ./a.out
Hallo
Waited successfully. Was joinable: 1
Hallo
Waited successfully. Was joinable: 1
Hallo
Waited successfully. Was joinable: 1
Hallo
Waited successfully. Was joinable: 1
Hallo
Waited successfully. Was joinable: 1
Hallo
[1]    2613 segmentation fault (core dumped)  ./a.out

The crash happens when calling .joinable(). If we skip the joinable check it crashes on .join().
Comment 5 Jonathan Wakely 2017-10-20 15:25:17 UTC
(In reply to Hannes Hauswedell from comment #4)
> The crash happens when calling .joinable(). If we skip the joinable check it
> crashes on .join().

That's not very precise. Calling it where? From the destructor? It doesn't make sense anyway, since joinable() just compares two pthread_t values for equality. No pointers are dereferenced.

Could we get a stack trace please?
Comment 6 Hannes Hauswedell 2017-10-20 16:12:14 UTC
Sorry, I forgot the stack trace. It wasn't helpful to me:
(gdb) bt
#0  uw_frame_state_for (context=context@entry=0x801c00e20, fs=fs@entry=0x801c00b70) at /wrkdirs/usr/ports/lang/gcc7-devel/work/gcc-7-20171012/libgcc/unwind-dw2.c:1257
#1  0x0000000800df4f87 in _Unwind_ForcedUnwind_Phase2 (exc=exc@entry=0x801816730, context=context@entry=0x801c00e20) at /wrkdirs/usr/ports/lang/gcc7-devel/work/gcc-7-20171012/libgcc/unwind.inc:155
#2  0x0000000800df5525 in _Unwind_ForcedUnwind (exc=0x801816730, stop=0x8010127b0, stop_argument=<optimized out>) at /wrkdirs/usr/ports/lang/gcc7-devel/work/gcc-7-20171012/libgcc/unwind.inc:207
#3  0x0000000801012613 in ?? () from /lib/libthr.so.3
#4  0x000000080101242b in pthread_exit () from /lib/libthr.so.3
#5  0x0000000801004bcd in ?? () from /lib/libthr.so.3
#6  0x0000000801a01000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x801c01000

Anything else I can do?
Comment 7 Andrew Pinski 2017-10-20 17:44:09 UTC
Looks like the crash is inside libgcc.  Maybe the new libgcc is not fully compatible with freebsd.
Comment 8 Jonathan Wakely 2017-10-20 21:20:12 UTC
Thanks for the info, I'll try to reproduce this on freebsd but I'll confirm it anyway.
Comment 9 Andreas Tobler 2017-10-20 21:32:18 UTC
Could you please tell us the FreeBSD version and arch you run on?
uname -ra
Comment 10 Hannes Hauswedell 2017-10-21 01:35:17 UTC
> Could you please tell us the FreeBSD version and arch you run on?
uname -ra

FreeBSD celegans.imp.fu-berlin.de 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

For completeness, these are the GCC versions installed:
% pkg info | grep gcc
gcc-ecj-4.5                    Eclipse Java Compiler used to build GCC Java
gcc49-4.9.4_5                  GNU Compiler Collection 4.9
gcc5-5.4.0_4                   GNU Compiler Collection 5
gcc6-6.4.0_2                   GNU Compiler Collection 6
gcc7-devel-7.2.1.s20171012     GNU Compiler Collection 7
gcc8-devel-8.0.0.s20171008     GNU Compiler Collection 8

gcc49-4.9.4_5 works, the later ones don't.
Comment 11 Andreas Tobler 2017-10-21 20:48:44 UTC
For the record, in gcc5 this one went in:
md_unwind_header=i386/freebsd-unwind.h
Before we had no MD_FALLBACK_FRAME_STATE_FOR.

Will investigate.
Comment 12 Andreas Tobler 2017-10-31 21:19:30 UTC
Will soon commit a fix, for gcc6/7/8 on FreeBSD > 9.3. Older gcc's and OS releases will not be supported by this fix.
Comment 13 Hannes Hauswedell 2017-11-01 19:23:24 UTC
(In reply to Andreas Tobler from comment #12)
> Will soon commit a fix, for gcc6/7/8 on FreeBSD > 9.3. Older gcc's and OS
> releases will not be supported by this fix.

Thanks a lot!

But is there no chance of getting a fix for gcc5? Not even via patch in the FreeBSD port (if not doable via gcc5 trunk)?
Because we would need to selectively disable gcc5 then (while gcc49 and >5 would still work which is kind of akward).
Comment 14 Andreas Tobler 2017-11-01 19:43:27 UTC
The gcc5 branch is closed, so I can not commit there. In the ports tree we're on gcc6 as default gcc. We still can build gcc5. I certainly can talk to Gerald to make an exception or whatever and patch the gcc5 port.
Comment 15 Andreas Tobler 2017-11-04 19:40:55 UTC
Author: andreast
Date: Sat Nov  4 19:40:23 2017
New Revision: 254411

URL: https://gcc.gnu.org/viewcvs?rev=254411&root=gcc&view=rev
Log:
2017-11-04  Andreas Tobler  <andreast@gcc.gnu.org>

    PR libgcc/82635
    * config/i386/freebsd-unwind.h (MD_FALLBACK_FRAME_STATE_FOR): Use a
    sysctl to determine whether we're in a trampoline.
    Keep the pattern matching method for systems without
    KERN_PROC_SIGTRAMP sysctl.

Modified:
    trunk/libgcc/ChangeLog
    trunk/libgcc/config/i386/freebsd-unwind.h
Comment 16 Andreas Tobler 2017-11-05 19:31:13 UTC
Author: andreast
Date: Sun Nov  5 19:30:41 2017
New Revision: 254431

URL: https://gcc.gnu.org/viewcvs?rev=254431&root=gcc&view=rev
Log:
2017-11-05  Andreas Tobler  <andreast@gcc.gnu.org>

    Backport from mainline
    2017-11-04  Andreas Tobler  <andreast@gcc.gnu.org>

    PR libgcc/82635
    * config/i386/freebsd-unwind.h (MD_FALLBACK_FRAME_STATE_FOR): Use a
    sysctl to determine whether we're in a trampoline.
    Keep the pattern matching method for systems without
    KERN_PROC_SIGTRAMP sysctl.

Modified:
    branches/gcc-7-branch/libgcc/ChangeLog
    branches/gcc-7-branch/libgcc/config/i386/freebsd-unwind.h
Comment 17 Andreas Tobler 2017-11-05 19:34:16 UTC
Author: andreast
Date: Sun Nov  5 19:33:44 2017
New Revision: 254432

URL: https://gcc.gnu.org/viewcvs?rev=254432&root=gcc&view=rev
Log:
2017-11-05  Andreas Tobler  <andreast@gcc.gnu.org>

    Backport from mainline
    2017-11-04  Andreas Tobler  <andreast@gcc.gnu.org>

    PR libgcc/82635
    * config/i386/freebsd-unwind.h (MD_FALLBACK_FRAME_STATE_FOR): Use a
    sysctl to determine whether we're in a trampoline.
    Keep the pattern matching method for systems without
    KERN_PROC_SIGTRAMP sysctl.

Modified:
    branches/gcc-6-branch/libgcc/ChangeLog
    branches/gcc-6-branch/libgcc/config/i386/freebsd-unwind.h
Comment 18 Andreas Tobler 2017-11-05 19:43:13 UTC
Trunk and active branches are fixed.