OpenCoarrays integration with gfortran

[Jerry DeLisle <jvdelisle at charter dot net>]

My apologies for kidnapping this thread:
On 9/20/18 1:01 PM, Thomas Koenig wrote:
> Hi Damian,
>
> > On a related note, two Sourcery Institute developers have attempted to 
> > edit
> > the GCC build system to make the downloading and building of OpenCoarrays
> > automatically part of the gfortran build process.  Neither developer
> > succeeded.
>
> We addressed integrating OpenCoarray into the gcc source tree at the
> recent Gcc summit during the gfortran BoF session.
>
> Feedback from people working for big Linux distributions was that they
> would prefer to package OpenCoarrays as a separate library.
> (They also mentioned it was quite hard to build.)

I would like to put in my humble 2 cents worth here.

OpenCoarrays was/is intended for a very broad audience, various large 
systems such as Cray, etc. I think this influenced heavily the path of 
its development, which is certainly OK.

It was/is intended to interface libraries such as OpenMPI or MPICH to 
gfortran as well as other Fortran compilers.

The actual library source code is contained mostly in one source file. 
After all the attempts to integrate into the GNU build systems without 
much success my thinking has shifted. Keep in mind that the OpenCoarrays 
implementation is quite dependent on gfortran and in fact has to do 
special things in the build dependent on the version of gcc/gfortran a 
user happens to use.  I dont think this is a good situation.

So I see two realistic strategies.  The first is already talked about a 
lot and is the cleanest approach for gfortran:

1) Focus on distribution packages such as Fedora, Debian, Ubuntu, 
Windows, etc. Building of these packages needs to be automated into the 
distributions. I think mostly this is what is happening and relies on 
the various distribution maintainers to do so.  Their support is greatly 
appreciated and this really is the cleanest approach.

The second option is not discussed as much because it leaves 
OpenCoarrays behind in a sense and requires an editing cycle in two 
places to fix bugs or add features.

2) Take the one source file, edit out all the macros that define 
prefixes to function calls, hard code the gfortran prefixes etc and fork 
it directly into the libgfortran library under GPL with attributions to 
the original developers as appropriate.

Strategy 2 would lock into specific current standard versions of the MPI 
interface and would support less bleeding edge changes.  It would also 
require either OpenMPI or MPICH as a new gfortran dependency for 
building, which not all users may need. So we would need some 
configuration magic to enable or disable this portion of the build. 
Something like --with-MPI-support would do the trick.

Strategy 2 does add burden to gfortran maintainers who are already 
overloaded. But, as the code matures the burden would decrease, 
particularly once TEAMS are finished.

Strategy 2 does have some advantages. For example, eliminating the need 
for separate CAF and CAFRUN scripts which are a wrapper on gfortran. 
The coarray features are part of the Fortran language and gfortran 
should just "handle it" transparently using an environment variable to 
define the number of images at run time. It would also actually 
eliminate the need to manage all of the separate distribution packages. 
So from a global point of view the overall maintanance effort would be 
reduced.

Strategy 2 would enable a set of users who are not focused so much on 
distributions and loading packages, etc etc and those who are dependent 
on getting through bureaucratic administrations who already are loading 
gfortran on systems and would not have to also get another package 
approved.  People would just have to stop thinking about it and just use it.

So I think there are real advantages to Strategy 2 as well as Strategy 1 
and think it should be at least included in discussions. I would even 
suggest there is likely a combination of 1 and 2 that may hit the mark. 
For example, keeping OpenCoarrays as a separate package for bleeding 
edge development and migrating the stable features into libgfortran on a 
less frequent cycle.

As I said, my 2 cents worth.

Regards to all,

Jerry