Re: Behaviour of __forced_unwind with noexcept

[Jonathan Wakely <jwakely dot gcc at gmail dot com>]

On 15 August 2017 at 16:21, Ron <ron@bitbabbler.org> wrote:
> On Tue, Aug 15, 2017 at 01:31:10PM +0200, Richard Biener wrote:
>> On Tue, Aug 15, 2017 at 1:28 PM, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>> > On 15 August 2017 at 11:24, Richard Biener <richard.guenther@gmail.com> wrote:
>> >> On Tue, Aug 15, 2017 at 6:44 AM, Ron <ron@bitbabbler.org> wrote:
>> >>> On Mon, Aug 14, 2017 at 06:22:39PM +0100, Jonathan Wakely wrote:
>> >>>> On 13 August 2017 at 19:20, Ron wrote:
>> >>>> >
>> >>>> > Hi,
>> >>>> >
>> >>>> > I'm looking for some clarification of how the __forced_unwind thread
>> >>>> > cancellation exceptions intersect with noexcept.  I've long been a
>> >>>> > big fan of the __forced_unwind idiom, but now that C++14 is the default
>> >>>> > since GCC 6.1, and many methods including destructors are implicitly
>> >>>> > noexcept, using it safely appears to have become a lot more tricky.
>> >>>> >
>> >>>> > The closest I've found so far to an "authoritative" statement of the
>> >>>> > expected behaviour is the comments from Jonathan Wakely here:
>> >>>> >
>> >>>> > https://stackoverflow.com/questions/14268080/cancelling-a-thread-that-has-a-mutex-locked-does-not-unlock-the-mutex
>> >>>> >
>> >>>> > In particular: "It interacts with noexcept as you'd expect:
>> >>>> > std::terminate() is called if a __forced_unwind escapes a noexcept
>> >>>> > function, so noexcept functions are really noexcept, they won't
>> >>>> > unexpectedly throw some 'special' type"
>> >>>> >
>> >>>> > Which does seem logical, but unless I'm missing something this makes
>> >>>> > it unsafe to perform any operation in a destructor which might cross
>> >>>> > a cancellation point, unless that destructor is noexcept(false).
>> >>>>
>> >>>> Unfortunately I still think that's true.
>> >>>>
>> >>>> This was also raised in https://gcc.gnu.org/ml/gcc-help/2015-08/msg00040.html
>> >>>
>> >>> Ouch.  Had you considered the option of having any scope that is
>> >>> noexcept(true) also be treated as if it was implicitly in a scoped
>> >>> pthread_setcancelstate(PTHREAD_CANCEL_DISABLE), restoring the
>> >>> old state when it leaves that scope?
>> >>>
>> >>> Would it be feasible for the compiler to automatically generate that?
>> >>>
>> >>> For any toolchain which does use the unwinding exceptions extension,
>> >>> that also seems like a logical extension to the noexcept behaviour,
>> >>> since allowing cancellation will otherwise result in an exception and
>> >>> process termination.  If people really need cancellation in such
>> >>> scopes, then they can more manageably mark just those noexcept(false).
>> >>>
>> >>>
>> >>> It would need to be done by the compiler, since in user code I can't
>> >>> do that in a destructor in a way that will also protect unwinding
>> >>> members of a class (which may have destructors in code I don't
>> >>> control).
>> >>>
>> >>> I can't even completely mitigate this by just always using -std=c++03
>> >>> because presumably I'm also exposed to (at least) libstdc++.so being
>> >>> built with the new compiler default of C++14 or later.
>> >>>
>> >>>
>> >>> I'd be really sad to lose the stack unwinding we currently have when
>> >>> a thread is cancelled.  I've always known it was an extension (and I'm
>> >>> still a bit surprised it hasn't become part of the official standard),
>> >>> but it is fairly portable in practice.
>> >>>
>> >>> On Linux (or on Debian at least) clang also supports it.  It's also
>> >>> supported by gcc on FreeBSD and MacOS (though not by clang there).
>> >>> It's supported by mingw for Windows builds.  OpenBSD is currently
>> >>> the only platform I know of where even its gcc toolchain doesn't
>> >>> support this (but they're also missing support for standard locale
>> >>> functionality so it's a special snowflake anyway).
>> >>>
>> >>>
>> >>> It seems that we need to find some way past the status-quo though,
>> >>> because "don't ever use pthread_cancel" is the same as saying that
>> >>> there's no longer any use for the forced_unwind extension.  Or that
>> >>> "you can have a pthread_cancel which leaks resources, or none at all".
>> >>>
>> >>> Having a pthread_cancel that only works on cancellation points that
>> >>> aren't noexcept seems like a reasonable compromise and extension to
>> >>> the shortcomings of the standard to me.  Am I missing something there
>> >>> which makes that solution not a viable option either?
>> >>
>> >> Have glibc override the abort () from the forced_unwind if in pthread_cancel
>> >> context?
>> >
>> > If the forced_unwind exception escapes a noexcept function then the
>> > compiler calls std::terminate(). That can be replaced by the user so
>> > that it doesn't call abort(). It must not return, but a user-supplied
>> > terminate handler could trap or raise SIGKILL or something else.
>> >
>> > Required behavior: A terminate_handler shall terminate execution of
>> > the program without returning
>> > to the caller.
>> > Default behavior: The implementation’s default terminate_handler calls abort().
>> >
>> > I don't think glibc can help, I think the compiler would need to
>> > change to not call std::terminate().
>>
>> Maybe it could call an unwinder provided hook so that forced_unwind can
>> set it to sth stopping the unwinding and signalling an error rather than
>> abort()ing.
>
> The trick there is that we don't actually want to just stop the unwinding
> at the first place where it hits a noexcept function (or we're back to
> the situation that force_unwind was created to avoid, resources higher up
> the stack may be leaked).
>
> And my gut feeling is that we don't want to diverge from the standard as
> to when a real error (an exception leaving a noexcept context) occurs.
>
> But it seems to me that we can avoid the problem case, and stay within
> the letter and spirit of the standard just by always ensuring that
> pthread_cancel is never acted upon in a noexcept context.  That way we
> simply never throw an exception, and don't need to do anything tricky
> about handling it.  The cancel request will just get acted upon at the
> next cancellation point which isn't in a noexcept scope.  All other
> errant exceptions will still behave exactly as expected.

Define "noexcept scope". For the following there would be no leaks:

void f() noexcept
{
  try {
    g();
  } catch (...) {
    throw;
  }
}

The forced_unwind would be caught, at which point the stack would have
been unwound to that point, and then it would terminate. In theory we
could allow cancellation during the execution of g(). Your proposal
would prevent that.

Also currently std::thread runs the supplied function object inside a
noexcept function. With your proposal cancellation would be blocked in
any thread created by a std::thread, i.e. you could only cancel in the
main() thread, or threads created by pthread_create.


> All we'd be adding is a simple extension to noexcept, which isn't in
> direct violation of the standard, to complement the stack unwinding
> extension that the standard didn't account for with noexcept.
>
>
> Is changing the cancellation state really an expensive operation?
> Moreso than the checking which I assume already needs to be done for
> noexcept to trap errant exceptions?

Changing the cancel state would have to happen in the hot path, just
in case a cancellation happens.

Terminating only has to happen in the cold path if an exception is
thrown, during stack unwinding.

> If it really is, I guess we could also have an attribute which declares
> a stronger guarantee than noexcept, to claim there are no cancellation
> points in that scope, if people have something in a hot path where a few
> cycles really matter to them and this protection is not actually needed.
> Which could also be an automatic optimisation if the compiler is able to
> prove there are no cancellation points?
>
>   Ron
>
>