Coarrays via GNU Fortran's Coarray Communication Library



Coarray Compilation Modes

GNU Fortran currently supports three coarray modes, which can be selected via the -fcoarray= flag:

A shared-memory version is considered.

Coarray Communication Libraries

Currently, four GNU Fortran coarray communication libraries exist. Part of the compiler is

The single library version is automatically installed with GCC. Thus it can be directly linked with -lcaf_single. The library is located in the libgfortran/caf of the GCC source code.

The OpenCoarrays (implementation status) project provides

See also the performance comparison with other vendors.

By default, all libraries use static linkage. However, using dynamic linkage and the same shared-library name, it is possible to switch the library at execution time of the program. However, static linkage is recommended.

Compiling Coarray Programs

Simply compile the Fortran files as usually using the -fcoarray=lib flag.

Running Coarray Programs

Combining Coarray Parallelization with OpenMP, MPI, pthreads, etc.

In principle, combining coarrays with other means of parallelization should work, but it is the users responsibility to avoid race conditions and other issues.

For the MPI version of the coarray library: Make sure that the user code does not call MPI_Init if the library is already intialized; e.g.

integer :: ierror
logical :: init
call MPI_Initialized (init, ierror)
if (.not. init) call MPI_Init (ierror)

Coarray Fortran with a Non-Fortran Main Program (for -fcoarray=lib)

If the main program is not written in Fortran, the initialization and finalization of the coarray communication library needs to be called manually. The other prototypes are listed for references and should usually not be needed. The value of the named constants and function prototypes can be found in the C header file libcaf.h; also the single-image implementation and the multi-image MPI/GASNet/ARMCI versions can be used as starting point for new libraries.

/!\ Note that this documentation is still in a flux and incomplete. The source code of the compiler and the libraries is the authoritative source; however, the API is not yet finished and will keep changing for a while.

List of additional ABIs, not yet covered in the documentation:

Calling SYNC ALL

The SYNC ALL statement has the following prototype:

void
_gfortran_caf_sync_all (int *stat, char *errmsg, int errmsg_len);

Calling SYNC IMAGES

The SYNC IMAGES statement has the following prototype:

void
_gfortran_caf_sync_images (int count, int images[],
                           int *stat, char *errmsg, int errmsg_len);

Calling SYNC MEMORY

This memory barrier is implemented using __sync_synchronize.

The STOP statement

The STOP statement is handled as in serial programs; if the file has been compiled with -fcoarray=lib, before the stop statement _gfortran_caf_finalize is invoked.

The ERROR STOP statement

For the ERROR STOP statement, the library is called and a graceful stop is tried before ending the program forcefully. The functions do not return. Two library functions are implemented, their prototype are

void
_gfortran_caf_error_stop_str (const char *string, int32_t len);

void
_gfortran_caf_error_stop (int32_t error)

Implementation Details

Argument handling

If a coarray is passed as actual argument to a coarray dummy, additional information needs to be transferred.

Front-end internal representation

Descriptor-free coarrays are arrays of the type GFC_ARRAY_TYPE_P; scalar types are normal scalars with the language-specific node attached. In this lang-specific node, the corank (GFC_TYPE_ARRAY_CORANK), the cobounds (GFC_TYPE_ARRAY_LBOUND and GFC_TYPE_ARRAY_UBOUND), token (GFC_TYPE_ARRAY_CAF_TOKEN) and offset (GFC_TYPE_ARRAY_CAF_OFFSET) is saved.

Allocatable coarrays use a descriptor - contrary to normal allocatables, also scalar coarrays have one. The descriptor contains additional elements as outlined above.

The passed token and offset arguments for assumed-shape coarrays are stored in the language-specific declaration (DECL_LANG_SPECIFIC) as GFC_DECL_TOKEN and DECL_LANG_SPECIFIC.

The token is an opaque object (of type "void*"), which contains in some direct or indirect way the base address of the coarray. The details are left to the library implementation. Possible choices would be the base address itself (e.g. for libcaf_single.a) or the base address of the coarray on all images (e.g. for libcaf_mpi.a).

None: CoarrayLib (last edited 2014-08-22 07:11:20 by TobiasBurnus)