C++ Modules

A module system is coming to C++, this page describes the GCC implementation state.

The goal of the module system is to avoid huge header files, thus speeding up compilation. What distinguishes it from things like precompiled headers are:

Implementation State

Development branch: 'c++-modules' (svn://gcc.gnu.org/svn/gcc/branches/c++-modules). Reporting bugs

The branch was created, by Nathan Sidwell, Jan 2017, the specification, design and implementation are in flux.

Notable events:

Invoking the Compiler

There are several new options for modules:

More complete documentation is in the GCC manual under 'C++ Modules'.

Bugs

Due to the experimental nature of the implementation, I'm not very interested in bug reports just yet, sorry. If you're directly working with me, you'll already know how to get my attention.

Here's a list of known not-working significant features:

ATOM

The ATOM proposal deviates from the TS in a few ways. Now (October 2018) ATOM is no longer a separate flag, having been merged in p1103r3. The 4 distinguishing features of ATOM are:

* All imports must be a single block at the start of file (just after the module declaration, if there is one) [only for modules] * Module interfaces may be partitioned (replacing proclaimed ownership decls) [merged] * The global module is replaced by legacy header imports and an associate compilation mode for them. [both] * The type model is simpler. [merged]

BMI Location

When a BMI file is needed a module mapper is queried. Communication is via a text-based protocol, which provides mechanism without policy. As such the compiler itself is completely agnostic about where BMIs are or how they are named. Build systems may provide a build-specific mapper. If no module mapper has been specified, a default is provided. It is expected that the behaviour of the default mapper will mature.

The -fmodule-mapper value may be one of:

* =socket A local socket

* hostname:port or :port An IPV6 socket

* |program args... A program to invoke and communicate over its stdin/stdout

* file A text file of space-separated module-name/file-name tuples, one per line

The first four may specify a cookie to provide to the remote mapper, by affixing a trailing '?cookie' to the first component of the argument. For instance '=/tmp/mybuild?shibboleth7', or '|build-mapper?shibboleth8 bob'. If no cookie is specifed, the name of the main source file is used in its place. It is expected that parallel builds will use the cookie to distinguish connections from different compiler instances.

The protocol is documented in the GCC manual.

Design

There are two main pieces of work, (a) streaming to disk, (b) name lookup.

The original plan was to try and reuse LTO's streaming technology for the former. But that turned out to be impractal as there is not much overlap. LTO streams GIMPLE and language-agnostic type information. Modules need AST representation and FE type information. So I went the hand-written auto-numbering streaming route.

Name lookup started by abusing inline namespaces, but that too proved impractical. We'd need the ability to turn these namespaces on and off, and to do that requires changes to name-lookup. Once you're making that kind of change, one may as well do it properly. As a benefit, name-lookup has gotten a lot cleaner.

Mangling

Name mangling needs to be adjusted to deal with module-linkage. This is a compiler-interoperability and toolchain issue, as we want objects from different compilers to be link-compatible, and the debugger able to understand module symbols.

Current thoughts are described in module-abi-2017-09-01.pdf.

Interface Designation

At the start of implementation, there was no special syntax for denoting the interface TU of a module. But implementations need to know immediately after seeing the module declaration whether the TU is the interface or one of the implementation TUs -- they cannot defer that decision. This has now been resolved with the 'export module foo;' syntax designating the interface TU.

Compiling the interface TU generates a Binary Module Interface. This BMI is read in by each implementation TU and each importer of the module. There's clearly a dependency between these things, which is different from header files because we have to invoke the compiler to generate the BMI. I have now implemented a hook in the compiler that can determine what to do if a BMI is not found. The default implementation of this wrapper script invokes the compiler to generate the BMI.

The BMI is not a distributable artifact. Think of it as a cache, that can be recreated as needed.

Module Linkage

I am not presuming any new linker technology. Module ownership is a new concept, and at least for module-linkage na/mes, must be reflected in the name mangling. Exported names need not reflect this ownership.

I am working with the Clang developers to define interoperable changes here. To facilitate migration of code, mangling of exported entities does not change from what they would have outside of a module.

Binary Module Interface Files

As mentioved above, a BMI is generated during the compilation of a module interface unit. For GCC I'm generating it as an on-the-side entity, but it could be stashed as a special section in the output assembly file, or even be a new stage of compilation. (Clang is taking this last approach.) The data is encapsulated in an ELF-like file. You can use 'readelf -S' to get at the sections it contains, and 'readelf -p gnu.c++.README' to get at its human-readable section. There are several specially-named sections, which generate the set of namespace-scope bindings. The actual binding values are held in sections named by a decl within them. We support lazy loading via cooperation with the name-lookup machinery. If it finds a lazy binding, it invokes the loader to load that binding. We take care to make sure things are not recursive here (this is non-trivial with C++).

The BMI does not contain timestamps. Thus recompiling a TU with exactly the same options will produce an identical BMI -- that's what you want with a cache. It does contain CRCs, which are used to detect corruption. I've not made corruption detection cryptographically strong or anything. If we detect corruption, you should get an error and then compilation terminates with a fatal error -- the likelihood of any further diagnostics being meaningful is negligible.

BMIs are relocatable within the file system, or copyable to another machine, which you might want with a distributed build. They refer to their own imports by reference, naming both the import module, and the (relative) location of the BMI that was loaded. If you copy a BMI you must recursively copy all its imports and recreate the same file structure.

Example

Put the following in hello.cc: 

module;
#include <stdio.h>
export module hello;
export void greeter (const char *name)
{
  printf ("Hello %s!\n", name);
}

and put the following in main.cc: 

import hello;
int main (void)
{
  greeter ("world");
  return 0;
}

Now compile with: 

   g++ -fmodules-ts main.cc hello.cc

You can run the a.out: 

nathans@devvm2186:161>./a.out
Hello world!

Global Module

Declarations before the module-declaration are in the global module. While this is clear enough, it has a complicated interaction with a module interface: 

module;
void Foo ();
export module Quux;
export void Bar ();
void Baz ();

module Quux; // implementation of Quux
void Bar () {
  Baz (); // Baz's declaration visible from purview Quux interface
  Foo (); // ERROR global module decls from interface NOT visible
}

import Quux; // user of Quux
void Baz ()
{
  Bar (); // Quux's Bar
  Baz (); // ERROR: Quux's non-exported Baz not visible.
  Foo (); // ERROR: Foo not visible from Quux interface
}

This is not implemented -- simple cases may work 'by accident'.

Random Cleanups

I've been making some random cleanups to the code base:

Timeline

Documentation

None: cxx-modules (last edited 2018-10-21 22:43:04 by NathanSidwell)