• Coarray

Coarray* support in gfortran as specified in the Fortran 2008 standard

Coarrays are also known as CAF = Co(-)array Fortran

Coarrays are an extension of Fortran, which allows to write parallel programs using a Partitioned Global Address Space (PGAS) following the SPMD (single program, multiple data) parallelization scheme. Each process (called image) has its own private variables. Variables which have a so-called codimension are addressable from other images. This extension is part of the Fortran 2008 standard (ISO/IEC 1539-1:2010).

See also http://users.physik.fu-berlin.de/~tburnus/coarray/README.txt

Coarray Syntax and Further Reading

The crucial ingredient are coarrays, which are variables which are visible on all images and have a codimension. They are declared using for instance

  CODIMENSION[1:4,5,*] :: variable

where the highest subscript needs to be a "*" (unless the variable is allocatable - in this case, as usual, only ":" have to be used). A remote coarray can be accessed using the bracket syntax, e.g.

  A[4] = 5
  A = 7

which writes "5" to the scalar coarray "A" on the (remote) image corresponding to the cosubscript [4] and "7" to the local coarray of the executing image. Note that only a single image can be addressed at a given time; e.g. simultaneously broadcasting to collecting from all images is not supported (as of Fortran 2008).

The image index can be obtained using the this_image() intrinsic, the total number of images with num_images(). If an instruction should only be performed on a single image, an explicit

  if(this_image() == ...) then; ...; end if

has to be used.

Besides the coarrays themselves, there exist also image control statements such as for barriers,

• SYNC ALL()

• SYNC IMAGES(<scalar image index, array with image indexes or "*">)

• SYNC MEMORY

or blocks which ensure that instructions are only executed on one image at a time

• CRITICAL; ...; END CRITICAL

Note that the STOP statements only terminates one image; if all images should be stopped because of an error use ERROR STOP.

More details can be found in

• Fortran 2008 [Final Draft International Standard (FDIS)], ftp://ftp.nag.co.uk/sc22wg5/N1801-N1850/N1830.pdf

• Coarrays in the next Fortran Standard by John Reid, ftp://ftp.nag.co.uk/sc22wg5/N1801-N1850/N1824.pdf

• Rational for Co-Arrays in Fortran 2008 by Aleksandar Donev, ftp://ftp.nag.co.uk/sc22wg5/N1701-N1750/N1702.pdf

• Early draft for a Technical Report (TR) to enhance the coarray capabilities (teams, collective intrinsics, NOTIFY/QUERY image control, multi-image I/O). Modification proposals by WG5, atomic proposal, collective proposal

Historic note: Co-Array Fortran (CAF) has been developed in the 1990s by Robert Numrich and John Reid; it has been implemented as non-standard extension in some compilers (e.g. Cray [since version 3.1] or Rice). When coarrays were incorporated in the Fortran 2008 standard, the term coarrays did not not only loose its hyphen, but also some syntax was changed (e.g. SYNC ALL is now a statement instead of a subroutine). The last change was the renaming of ALL STOP to ERROR STOP in February 2010. Some functionality was removed when CAF were added to Fortran 2008 and has been deferred into a yet to be written technical report; this includes collective operations.

Examples

Coarray examples can be found on an extra page.

Current Implementation Status in GCC Fortran on the GCC Trunk [4.7 (experimental)]

GCC 4.6: Only single-image support (i.e. num_images() == 1) but many features did not work.

GCC 4.7: Also multi-image support via a communication library. Comprehensive support for a single image, but most features do not yet work with num_images() > 1

Supported features

See also below for the exceptions.

• -fcoarray=<none/single/lib> compile-time flag

• Declaration of coarrays
• Coarrays in expressions/coindexed expressions including allocatable coarrays

• Partial support for polymorphic scalar/array coarrays (cf. OOP status)

• Image-control statements:
  

  • ERROR STOP

  • CRITICAL block

  • LOCK and UNLOCK

  • SYNC ALL/MEMORY/IMAGES

• Intrinsics
  

  • num_images, this_image, ucobound, lcobound and image_index.

  • atomic_define()/atomic_ref()

• ISO_FORTRAN_ENV module: The integer parameters atomic_int_kind, atomic_logical_kind, iostat_inquire_internal_unit, stat_locked, stat_locked_other_image, stat_stopped_image, and stat_unlocked - and the derived type LOCK_TYPE.

Supported by the library (-fcoarray=lib)

There are no compile time errors and only limited run-time diagnostic if unimplemented features are used. Hence, use -fcoarray=lib with num_image() > 1 with uttermost care and read the restrictions carefully.

• For a single image (num_images() == 1) everything should work, which works with -fcoarray=single. This applies to both libcaf_single and libcaf_mpi (with mpiexec -np 1)

• Nonlibrary-based functionality such as ucobounds/this_image etc. works with any number of images
• Registering and deregistering of coarrays - static and allocatable
• SYNC ALL and SYNC MEMORY
• ERROR STOP

Not implemented (TODO)

• Polymorphic coarrays have still several issues
• Assignment to derived types should not reallocate allocatable coarray components

• Some smaller bugs and more -fcheck= run-time diagnostic

TODO for -fcoarray=lib

• Request processing loop for libcaf_mpi.

• Any communication (with very few exceptions). In particular, no communication is performed when using:

  • lock, atomic, data push/pull
  • sync images, critical
• friendlier error stop, better run-time diagnostic

Usage

• For single-image (serial) programs: Simply add the flag -fcoarray=single and compile, link and run as usual.

• For multi-image programs: See library-based coarray documentation