[Bug driver/103949] gcc fails to provide a standard conforming C11 or C++17 environment even when specifying -std=c11 or -std=c++17

[manx-bugzilla at problemloesungsmaschine dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103949

--- Comment #17 from Jörn Heusipp <manx-bugzilla at problemloesungsmaschine dot de> ---

(In reply to Jonathan Wakely from comment #9)
> (In reply to Jörn Heusipp from comment #8)
> > > Note in newer versions of glibc, libpthread is all intergrated into libc and
> > > there is no issues again.
> > > 
> > > For Mac OS X/darwin you don't need -lm -pthread because libc has it.
> > 
> > Two examples where it works by chance is not sufficient to invalidate my
> > point about the general case.
> 
> But it does mean the problem you're raising doesn't exist on two of GCC's
> primary platforms. So the problem is "fixing itself" for many users.

The overall problem hurts users the most when porting between different
platforms, thus major platforms having one way of doing things that will not
work on minor platforms does not really solve the problem. This frustrates
users. And that is what I am (amongst other things) intending to address here.
And it's not really fixing itself. Others fix it, because GCC thus far has
refused to fix it.
Also, what about libm?



About POSIX, as brought up in the atomics issue: I do not particularly care
what POSIX allows or demands. My software also targets multiple platforms that
are not POSIX. I use GCC on platforms that are not POSIX. If POSIX proposes a
bad and broken interface, that is not a reason to only do it as bad as even
remotely allowed by POSIX. Only because POSIX does not *require* you to
auto-link -lm, this does not imply that POSIX *forbids* you to do so.



threads:

> > > >gcc libstdc++ implements std::thread. So what's your point again?
> > > 
> > > Because it implements it on top of pthreads.
> > 
> > Yeah, so that is an implementation detail because of which I am required to
> > pass -pthread. Why should the user of gcc even care how std::thread is
> > implemented? It literally makes no sense. You are supporting my point.
> 
> It's like that because linking to libpthread has historically caused
> performance degradation for C++ programs, due to reference counting in
> shared_ptr and the COW std::string using slower atomic ops if libpthread is
> linked into the executable. So if the compiler implicitly added -lpthread to
> every C++ program using C++11 or later, it would hurt the performance of
> single-threaded programs.
>
> It matters less for the std::string reference counting now, because most
> people use the SSO string not the COW string, but it still affects
> std::shared_ptr.

Thanks for the explanation, however I feel this indeed is a pure
quality-of-implementation issue and in particular still very much an
implementation detail, and thus something GCC or the platform should solve
themselves, instead of offloading the problem of knowing about such intricate
details onto the user.

This questionable (see later) optimization also completely misses the common
case of using a shared_ptr in a multi-threaded application but only in a
limited single-thread context (which was the situation for almost every single
time I have ever used shared_ptr). C++ does need a non-thread-safe flavour of
std::shared_ptr, but that's an entirely different discussion, let's not go
there.
It certainly is not a reason to complicate things for the user.

> And with very recent versions, you don't
> need -pthread at all.

That's progress, however I still do need it on Debian Testing, which is about
as recent and bleeding edge as I am willing to invest time into.

And if it is so, and actually fixed in a "proper" way, what again is the reason
why GCC still wants to stick to the old way of doing things and offloads the
(now unneeded on modern platforms) knowledge requirement onto each user? If you
really think a platform-specific global flag in the C library is the proper
solution to detect multi-threadedness (I could not agree less, but that's not
really relevant here), just over-link on other platforms and put them under
pressure to adapt a similar solution (if they even care about this
optimization).
"The 'bug' is fixed, let's still hurt users with the work-around."

> > Debian MinGW-w64 gcc is built in 2 flavours. One with Win32 and one with
> > posix threading model support. The version with posix threading model
> > support uses pthreads, and knows how to implement std::thread. Why is GCC
> > assuming single-threaded in that context?
> 
> The fact the runtime is capable of supporting threads doesn't mean a given
> program compiled by the user actually uses threads.

Isn't it still important for correctness in an application vs library situation
where one is multi-threaded and the other isn't? When considering C++11
threadsafe-statics, the compiler must always know whether the resulting binary
could ever be used multi-threaded, doesn't it? I think, it should, for
standards-compliance reasons, assume so, unless explicitly opted-out by the
user via -fno-threads as I suggested (indeed very similar in semantics and
implications as -fno-exceptions or -fno-rtti currently are, and it already
exists in a more limited form as -fno-threadsafe-statics). That however maybe
even more so implies that the user would be required to pass -pthread, even if
they, in their local context, might not even care about threads. Maybe. Or is
it always using threadsafe-initialization no matter what? I honestly do not
know. Documentation does not tell. (see below)

I actually do have precisely that use case: A shared library, which itself does
not touch any threads at all, however, in 1 place relies on C++11
thread-safe-statics in order to one-time-initialize some internal tables.

> The docs do not say that -std=c++17 is sufficient to get a fully C++17
> conforming environment, so strictly speaking this report is not a bug.
> You're complaining about something we don't aim for anyway. The
> implementation documents that you need to add certain other flags for some
> features:
> 
> https://gcc.gnu.org/onlinedocs/libstdc++/manual/using.html#manual.intro.using.flags

Well, I guess one could call it a "partially documented bug" :)

And it's not even accurately documented, it seems. It tells me that I need to
pass -pthread, however building my example for MinGW-w64-posix (after working
around the __STDCPP_THREADS__ issue) does not appear to require -pthread. It
builds and links and appears to work properly without it. Adding -pthread also
does not fix the missing __STDCPP_THREADS__ definition. So for MinGW-w64, it
does do the right thing and automatically links pthread. Why only there?
However, -mthreads is not automatically set for MinGW, and not documented at
the URL you linked, yet still required for standard compliance according to
other documentation?!? Why all these inconsistencies?
But, it actually turns out to not be required in the end, because it does
basically nothing whatsoever at all for MinGW-w64 (see below). Why does it even
exist there? Documentation is again seriously lacking.

> -latomic	Linking to libatomic is required for some uses of ISO C++11 <atomic>.

That also is not exactly precise. Which are these "some" uses? If GCC, as it
seems, is so concerned about not linking atomic ever when it is not strictly
required, yet it is giving users only a very vague indication of when it could
be required, the user is unable to make an informed decision here. If
overlinking is problematic, the user needs to know the precise conditions. If
it is not, GCC really should just link it.

Also, it's documented for libstdc++, so I do argue that GCC should have read
libstdc++'s documentation and acted accordingly to implement -std=c++17. It's
GCC (as in the driver, not the project) that is using libstdc++. It is thus
GCC's responsibility to drag in the required transitive dependencies. That's
how transitive dependencies are handled EVERYWHERE ELSE.


Let's look at all these thread-related options in detail, sorted by category as
in https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html#Option-Summary

c++:

-fno-threadsafe-statics

> Do not emit the extra code to use the routines specified in the C++ ABI for thread-safe initialization of local statics. You can use this option to reduce code size slightly in code that doesn’t need to be thread-safe.

So I may deduce, that otherwise thread-safe initialization is always used, even
if !__STDCPP_THREADS__ (where it would not be required by the standard, I
think). Or may I not?!?
If we look at a platform that is actually single-threaded (DJGPP), it looks
like GCC is providing "threadsafe" statics even there by default, which is
useless.

optimization:

-fthread-jumps

> Perform optimizations that check to see if a jump branches to a location where another comparison subsumed by the first is found. If so, the first branch is redirected to either the destination of the second branch or a point immediately following it, depending on whether the condition is known to be true or false.

Not even thread-related. Unfortunate overloading of the same word.

preprocessor:

-pthread

> Define additional macros required for using the POSIX threads library. You should use this option consistently for both compilation and linking. This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86 Cygwin and MinGW targets.

Why does the user who is only using C11 thrd or C++11 std::thread need be know
that GCC (or the standard C library that GCC implies) implements these with
POSIX threads? That's an implementation detail. As a user, I am concluding,
that I do in fact not have to pass that option, because I myself am not using
POSIX threads whatsoever at all.
On which platforms does it define additional macros? And which ones? Does this
have implications for the C standard library headers? Does it have implications
for the C++ standard library headers? Does it have implications for
__STDC_NO_THREADS__ or __STDCPP_THREADS__?

linker:

-pthread

> Link with the POSIX threads library. This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86 Cygwin and MinGW targets. On some targets this option also sets flags for the preprocessor, so it should be used consistently for both compilation and linking.

See above.

-lpthread

Is that different to -pthread for the linker? If so, how? If not, why does
-pthread exist?

hppa:

-threads

> Add support for multithreading with the dce thread library under HP-UX. This option sets flags for both the preprocessor and linker.

Same issues as with -pthread.

solaris2:

-pthreads

> This is a synonym for -pthread.

I guess compatibility with sunc?

x86:

-mthreads

> Support thread-safe exception handling on MinGW. Programs that rely on thread-safe exception handling must compile and link all code with the -mthreads option. When compiling, -mthreads defines -D_MT; when linking, it links in a special thread helper library -lmingwthrd which cleans up per-thread exception-handling data.

Why is this not the default for a multi-threaded standard version? Or is it?
Documentation is not clear. Documentation is also arguably just wrong for
MinGW-w64 (see below).

x86 windows:

-mthread
-mthreads

Inconsistently documented as either -mthread
(<https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html#Option-Summary>) or
-mthreads
(<https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html#x86-Windows-Options>)

> This option is available for MinGW targets. It specifies that MinGW-specific thread support is to be used.

I would assume (based on Windows platform knowledge), that this could imply
linking the multi-threaded version of the MSVCRT library, however documentation
is rather vague in my opinion. However, the shared library version of the
MSVCRT only ever comes in a multi-threaded version.
Also, this is either the same option as the identically named -mthreads, or it
is not. The user is seriously confused here.

-mthread is not accepted by my MinGW GCC.
-mthreads does nothing. I did check the implementation: The only measurable
thing it actually does for MinGW-w64, is define _MT globally. This macro is
checked nowhere. And it is set by any MinGW-w64 C library header
unconditionally anyway. The mingwthrd library in MinGW-w64 does nothing:
<https://github.com/mirror/mingw-w64/blob/d8842c6f535a91698c3b26a8e5b86d0062ea340f/mingw-w64-crt/libsrc/mingwthrd_mt.c#L1>.
I did not check original MinGW thoroughly, however a quick test revealed that
even there, _CRT_MT (see
<https://github.com/gcc-mirror/gcc/blob/6795e6ae66096d52a62e20ed33a47599233ab3d5/libgcc/config/i386/gthr-win32.h#L370>)
is always !=0 for me.


I would not need to ask ***any*** of these question, if gcc, or at least -std=,
actually did the reasonable thing by default in the first place.



math:

> https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html
> -nolibc
> Do not use the C library or system libraries tightly coupled with it when linking. Still link with the startup files, libgcc or toolchain provided language support libraries such as libgnat, libgfortran or libstdc++ unless options preventing their inclusion are used as well. This typically removes -lc from the link command line, as well as system libraries that normally go with it and become meaningless when absence of a C library is assumed, for example -lpthread or -lm in some configurations. This is intended for bare-board targets when there is indeed no C library available.

The docs say, it would not auto-link libm and libpthread if I pass -nolibc,
however that really is only valuable information if it would actually do so by
default in the first place when not passing -nolibc. Given that math and
threads do belong to the standard, I still do think that these libraries should
be implied by default if required on the respective platform (and excluded with
-nolibc, as documented). GCC chooses the libc it links. It is thus responsible
to (if possible) make up for missing functionality in this libc.



atomics:

Oh, and what about the --as-needed argument brought up in the atomics issue? Do
you really think GCC itself should strive for minimal linking on a particular
platform that does not provide that facility at all in the first place? That's
solely a platform issue, and nothing GCC has to worry about. It is even
completely irrelevant for correctness. This whole argument is just pre-mature
optimization. Linking a huge glibc.so (50% of which is random POSIX and kernel
stuff that I did not even ask for) and libstdc++.so, and then over-engineering
around and argueing about a 30kB/~110symbols library for atomics? I mean,
REALLY?
I assume the real reason here is again wider atomics dragging in POSIX locks.
And the actual real problem, again, is C++ missing a shared_ptr optimized for
single-threaded usage.
Please name only one single program, that *requires atomics*, and *is not using
wide atomics*, and *is single-threaded*, and *uses shared_ptr*, and *cares
about shared_ptr performance*, and *is not important enough to suggest a
non-atomic-refcount shared_ptr for ISO C++*. Name only 1. Because that is the
absurdly narrow case you are optimizing for here, and burdening the user with
your implementation details for. It really makes no sense. If you are unable to
provide an optimization without requiring user knowledge in the
non-optimization case, the correct solution is to not provide that optimization
by default, but only when a user specifically asks for it. The few people
desiring the optimized single-threaded shared_ptr are currently benefiting from
that optimization at the cost of everyone else. That's a wrong tradeoff to
make.



> I understand your complaint, but I think it's unlikely we're going to change
> anything in GCC.
> You're complaining about something we don't aim for anyway.
> This isn't ideal, but it's not easy to fix. And as already stated, some
> platforms are changing so it Just Works anyway.

Thanks for acknowledging the problem. However it's sad that you still do not
appear to realize the amount of grief it causes for users. I honestly think you
should aim for solving these issues. Currently, everything related to these
issues gets offloaded onto every single user. Solving it in the one single
place that inevitably already needs to know these details anyway would be a
more scalable solution for everyone.



All these issues are a tremendous user experience nightmare, and it sadly looks
like I absolutely have to point out every single one of them explicitly by
asking every single detail question possible, so that you actually can in fact
feel and realize the mess you have created for users to deal with. It ***IS***
a bug, and it's not primarily about documentation. It's about sane defaults.
The mess GCC has created here wastes every user's time. You seem to expect
users to know all the intricate details about every platform that you sometimes
have documented in a few words without further explanation, and sometimes not
even that. That's honestly not a reasonable expectation in my opinion. All
these are implementation details that a user of the ISO standards should not
need to have to know about. That's seriously not how standards are supposed to
be used. Standards exist to shield users from implementation details.