[WIP RFC] libstdc++: add module std

[Jonathan Wakely]

On 14/11/24 23:09 -0500, Jason Merrill wrote:
>On 10/18/24 9:38 AM, Jason Merrill wrote:
>>This patch is not ready for integration, but I'd like to get feedback on the
>>approach (and various specific questions below).
>
>An updated version:

>From 533c496ae39ba2cf1783136e1da5ae9f0f61d4f7 Mon Sep 17 00:00:00 2001
>From: Jason Merrill <jason@redhat.com>
>Date: Thu, 10 Oct 2024 17:27:50 -0400
>Subject: [PATCH] libstdc++: add module std [PR106852]
>To: gcc-patches@gcc.gnu.org
>
>This patch introduces an installed source form of module std and std.compat.
>To find them, we install a libstdc++.modules.json file alongside
>libstdc++.so, which tells the build system where the files are and any
>special flags it should use when compiling them (none, in our case).  The
>format is from a proposal in SG15.
>
>The build system can find this file with
>gcc -print-file-name=libstdc++.modules.json
>
>It seems preferable to use a relative path from this file to the sources so
>that moving the installation doesn't break the reference, but I didn't see
>any obvious way to compute that without relying on coreutils, perl, or
>python, so I wrote a POSIX shell script for it.
>
>Currently this installs the sources under $(gxx_include_dir)/bits/,
>i.e. /usr/include/c++/15/bits.  So with my -fsearch-include-path change,
>std.cc can be compiled with g++ -fsearch-include-path bits/std.cc.

Would there be a way to give a diagnostic if a translation unit does
#include <bits/std.cc> ?

If we install it there, somebody will try to include it like a header.
I'm not terribly concerned about that, but it would be nice to warn or
error if we can.

Maybe:

#if __INCLUDE_LEVEL__ != 0
# warning "stop it"
#endif

Or even:

#ifndef _GLIBCXX_ALLOW_WEIRD_INCLUDE_WHAT_ARE_YOU_EVEN_DOING
static_assert(std::string_view(__BASE_FILE__).ends_with("std.cc"));
#endif


>The sources currently have the extension .cc, like other source files.
>
>std.cc started with m.cencora's implementation in PR114600.  I've made some
>adjustments, but more is probably desirable, e.g. of the <algorithm>
>handling of namespace ranges, and to remove exports of templates that are
>only specialized in a particular header.
>
>I've filled in a bunch of missing exports, and added some FIXMEs where I
>noticed bits that are not implemented yet.
>
>Since bits/stdc++.h also intends to include the whole standard library, I
>include it rather than duplicate it.  But stdc++.h comments out <execution>,
>so I include it separately.  Alternatively, we could uncomment it in
>stdc++.h.

It's not included by default because it pulls in a dependency on
libtbb.so even if you don't use the parallel algos:

$ g++ -include algorithm -include execution -x c++ - <<< 'int main() { }'
/usr/bin/ld: /tmp/ccA76qrz.o: in function `tbb::detail::d1::execution_slot(tbb::detail::d1::execution_data const&)':
<stdin>:(.text._ZN3tbb6detail2d114execution_slotERKNS1_14execution_dataE[_ZN3tbb6detail2d114execution_slotERKNS1_14execution_dataE]+0x14): undefined reference to `tbb::detail::r1::execution_slot(tbb::detail::d1::execution_data const*)'
/usr/bin/ld: /tmp/ccA76qrz.o: in function `tbb::detail::d1::current_thread_index()':
<stdin>:(.text._ZN3tbb6detail2d120current_thread_indexEv[_ZN3tbb6detail2d120current_thread_indexEv]+0xe): undefined reference to `tbb::detail::r1::execution_slot(tbb::detail::d1::execution_data const*)'
collect2: error: ld returned 1 exit status

The way we arranged the PSTL inclusion is that you only get a
dependeny on TBB if you include a header containing a parallel algo
(<algorithm>, <numeric>) *and* include <execution>. Since including
<execution> is formally required by the standard in order to pass one
of std::execution::par, std::execution::seq etc. to a parallel algo,
we switch on whether <execution> has been included or not to detect
whether the user actually wants to use parallel algos.

Is this not a problem for 'import std;'?

Maybe we should figure out how to fix it for #include <execution>
anyway. When we eventually support senders/receivers in <execution>
for C++26, users are going to want to include that for reasons other
than using the parallel algos and their execution policies.

>It seemed most convenient for the two files to be monolithic so we don't
>need to worry about include paths.  So the C library names that module
>std.compat exports in both namespace std and :: in module are a block of
>code that is identical in both files, adjusted based on whether the macro
>STD_COMPAT is defined before the block.

We could split std.cc and std.compat.cc into several *.in files and
build them on the fly during installation, so that the verbatim part
is not duplicated. Something like:

cat std.head.in std.clib.in std.tail.in > std.cc
cat std.compat.head.in std.clib.in > std.compat.cc

>In this implementation std.compat imports std; it would also be valid for it
>to duplicate everything in std.  I see the libc++ std.compat also imports
>std.

That seems fine.

>As discussed in the PR, module std is supported in C++20 mode even though it
>was added in C++23.

Great.

>Changes to test module std will follow in a separate patch.  In my testing
>I've noticed a few compiler bugs that break various testcases, so I don't
>expect to enable module std testing by default at first.
>
>	PR libstdc++/106852
>
>libstdc++-v3/ChangeLog:
>
>	* include/bits/version.def: Add __cpp_lib_modules.
>	* include/bits/version.h: Regenerate.
>	* src/c++23/Makefile.am: Add module std/std.compat.
>	* src/c++23/Makefile.in: Regenerate.
>	* src/c++23/std.cc: New file.
>	* src/c++23/std.compat.cc: New file.
>	* src/c++23/libstdc++.modules.json.in: New file.
>
>contrib/ChangeLog:
>
>	* relpath.sh: New file.
>---
> libstdc++-v3/include/bits/version.h           |   10 +
> libstdc++-v3/src/c++23/std.cc                 | 3925 +++++++++++++++++
> libstdc++-v3/src/c++23/std.compat.cc          |  705 +++
> contrib/relpath.sh                            |   83 +
> libstdc++-v3/include/bits/version.def         |    9 +
> libstdc++-v3/src/c++23/Makefile.am            |   27 +
> libstdc++-v3/src/c++23/Makefile.in            |  137 +-
> .../src/c++23/libstdc++.modules.json.in       |   17 +
> 8 files changed, 4897 insertions(+), 16 deletions(-)
> create mode 100644 libstdc++-v3/src/c++23/std.cc
> create mode 100644 libstdc++-v3/src/c++23/std.compat.cc
> create mode 100755 contrib/relpath.sh
> create mode 100644 libstdc++-v3/src/c++23/libstdc++.modules.json.in
>
>diff --git a/libstdc++-v3/include/bits/version.h b/libstdc++-v3/include/bits/version.h
>index a0e3b36864b..c556aca38fa 100644
>--- a/libstdc++-v3/include/bits/version.h
>+++ b/libstdc++-v3/include/bits/version.h
>@@ -2075,4 +2075,14 @@
> #endif /* !defined(__cpp_lib_to_string) && defined(__glibcxx_want_to_string) */
> #undef __glibcxx_want_to_string
> 
>+#if !defined(__cpp_lib_modules)
>+# if (__cplusplus >= 202002L) && (__cpp_modules)
>+#  define __glibcxx_modules 202207L
>+#  if defined(__glibcxx_want_all) || defined(__glibcxx_want_modules)
>+#   define __cpp_lib_modules 202207L
>+#  endif
>+# endif
>+#endif /* !defined(__cpp_lib_modules) && defined(__glibcxx_want_modules) */
>+#undef __glibcxx_want_modules
>+
> #undef __glibcxx_want_all
>diff --git a/libstdc++-v3/src/c++23/std.cc b/libstdc++-v3/src/c++23/std.cc
>new file mode 100644
>index 00000000000..4311c8dc5cb
>--- /dev/null
>+++ b/libstdc++-v3/src/c++23/std.cc
>@@ -0,0 +1,3925 @@
>+// -*- C++ -*- [std.modules] module std
>+
>+// Copyright The GNU Toolchain Authors.
>+//
>+// This file is part of the GNU ISO C++ Library.  This library is free
>+// software; you can redistribute it and/or modify it under the
>+// terms of the GNU General Public License as published by the
>+// Free Software Foundation; either version 3.
>+
>+// This library is distributed in the hope that it will be useful,
>+// but WITHOUT ANY WARRANTY; without even the implied warranty of
>+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>+// GNU General Public License for more details.
>+
>+// Under Section 7 of GPL version 3, you are granted additional
>+// permissions described in the GCC Runtime Library Exception, version
>+// 3.1, as published by the Free Software Foundation.
>+
>+// You should have received a copy of the GNU General Public License and
>+// a copy of the GCC Runtime Library Exception along with this program;
>+// see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
>+// <http://www.gnu.org/licenses/>.
>+
>+module;
>+
>+#include <bits/stdc++.h>
>+
>+// ??? Commented out in stdc++.h
>+#include <execution>
>+
>+#undef __DEPRECATED

Could this be done with:
#pragma GCC diagnostic ignored "-Wdeprecated"
instead?

>+#include <strstream>
>+
>+export module std;
>+
>+// **** Any changes to the C library section below must also be made to the
>+// **** verbatim copy in std.compat.cc.
>+
>+// C standard library headers [tab:headers.cpp.c]
>+

[...]

>diff --git a/contrib/relpath.sh b/contrib/relpath.sh
>new file mode 100755
>index 00000000000..94301d2778c
>--- /dev/null
>+++ b/contrib/relpath.sh
>@@ -0,0 +1,83 @@
>+#!/bin/sh
>+
>+if [ "$1" == "--help" ]; then

Single = here.

>+    echo Usage: relpath.sh FROM TO
>+    echo Print the relative path from FROM to TO
>+    echo FROM must be a directory, but need not exist
>+    exit 0
>+fi
>+
>+if [ -f "$1" ]; then

Why not [ ! -d "$1" ] ?

>+    echo first argument must be a directory

Redirect >&2

>+    exit 1
>+fi
>+
>+from="${1%%/}"
>+to="${2%%/}"
>+
>+# The parent directory of a pathname, handling ..
>+parent() {
>+    name=$(basename "$1")
>+    path=$(dirname "$1")
>+    top=$(basename "$path")
>+    if [ "$top" = ".." ]; then
>+	path=$(parent "$path")
>+    fi
>+    if [ "$name" = ".." ]; then
>+	path=$(parent "$path")
>+    fi
>+    echo $path
>+}
>+
>+# Canonicalize a directory that contains '..'.
>+canonicalize() {
>+    path=$1
>+    suffix=
>+    while ! [ -d "$path" ]; do
>+	name=$(basename "$path")
>+	path=$(parent "$path")
>+	suffix="/$name$suffix"
>+    done
>+    if [ -d "$path" ]; then
>+	echo $(cd "$path"; pwd)$suffix
>+    else
>+	echo $1
>+    fi
>+}
>+
>+case "$to$from" in
>+    *..* )
>+	from=$(canonicalize "$from")
>+	to=$(canonicalize "$to")
>+	;;
>+esac
>+case "$to$from" in
>+    *..* )
>+	echo unable to canonicalize ..

Redirect to >&2 ?

>+	exit 1
>+	;;
>+esac
>+
>+back=
>+while [ "${to#$from}" = "$to" ]; do
>+    #echo $from too long
>+    from=$(dirname $from);
>+    back=../$back
>+
>+    if [ "$from" = "/" ]; then
>+	echo $to
>+	exit 0
>+    fi
>+done
>+
>+to=${to#$from}
>+to=${to##/}
>+back=${back%%/}
>+
>+if [ -n "$to" ] && [ -n "$back" ]; then
>+    echo $back/$to
>+elif [ -n "$back$to" ]; then
>+    echo $back$to
>+else
>+    echo .
>+fi