Mixed language i/o sychronization

[Janne Blomqvist]

On Tue, Aug 09, 2005 at 10:34:46AM +0200, Erik Schnetter wrote:
> On Tuesday 09 August 2005 10:15, FX Coudert wrote:
> > It is important to me, too. But I think it's wrong to ask one
> > compiler (in this case, gfortran), to be responsible for handling the
> > possible mistakes of other parts of the code. Plus, fwrite() has
> > nothing universal: some people use the POSIX write() functions in
> > their C code, I don't know how this will behave for mixed
> > Java-Fortran code, etc. Basically, I think the flaw in this scheme is
> > that even if we wanted to do some bugware, people will still have
> > problems with that.
> >
> > Plus, on the "fundamental" side of the issue, I think sometimes we
> > have to tell people: "this code is not written well, you'll have to
> > remove these programming errors".
> 
> I agree with that statement.  On the other hand, the whole issue of 
> calling Fortran from C and vice versa is currently compiler dependent, 
> and people still need to do it.  If you call write() from C, then you 
> do unbuffered I/O, thus if you also use unbuffered I/O from Fortran (by 
> whatever means), you could expect things to interoperate.  There is no 
> standard for buffered I/O interoperability,

... and that is exactly the problem. Even if we get the case of
C<->Fortran right, what about C++, Java, python, perl and whatever
other languages people might want to use?

> and the only thing that is 
> "guaranteed" to work is then to do all I/O via C, by calling C routines 
> from Fortran if you want to write to a file or to the screen.

Or all I/O from Fortran, for that matter. 

The only way that is guaranteed to work without flushing all the time,
is to access the same file descriptor via one method only.

> Many compiler suites that come with both a C and a Fortran compiler 
> (Intel, PGI, gcc/g77) have given in to user pressure and made buffered 
> I/O from C and Fortran work together.

I don't think the situation for commercial compilers can be compared
that way. AFAIK commercial compilers ship with their own C (and
Fortran) library (or a subset of it, at the least, depending on the
system C library for the actual kernel entry stuff). While C and
Fortran buffered I/O might very well use the same buffer
implementation under the covers, it doesn't mean that the Fortran I/O
library is implemented via fwrite() and friends.

gfortran OTOH aims for portability to many C libraries, and IMHO it
doesn't make sense to ship a partial reimplementation of the C library
that must be used instead of the system library when doing gfortran/C
mixed programming.

>  Whether we want 
> that thing in gcc is something that should be decided by the 
> maintainers, depending how burdensome it is to implement and maintain 
> it.

I think maintenance and debugging are the big problems. There are
already enough problems with having both mmap and fd methods (which is
why I proposed dropping the mmap() code path a while ago), do we
really want to add a third code path to that?

> What was the original reason for not using libc to do buffered I/O?  If 
> I remember correctly, the person who took the first stab at a Fortran 
> I/O library in g90 (quite some time ago) decided that libc I/O would be 
> too inefficient, and that something custom-made would be better.  If 
> that is still the sole reason, then one could imagine switching, if 
> someone makes the effort of converting the I/O subsystem of gfortran.  
> Does anybody know?

The gfortran I/O library is based on the paper

Krieger et. al., The Alloc Stream Facility: A Redesign of
Application-Level Stream I/O, IEEE Computer (1994), downloadable from
http://citeseer.ist.psu.edu/krieger94alloc.html
 
In that paper there are some figures suggesting that the alloc stream
facility (ASF), based on use of mmap(), is faster than the normal
read/write I/O. The author also argues that especially for parallel
computers the advantage might grow significantly.

However, currently the gfortran I/O library is very inefficient due to
other reasons (fine-grained interface between the compiled code and
the library, mainly), so any potential performance advantage of ASF is
not seen. Also, the library isn't thread-safe so any advantage with
respect to parallel performance are also not achievable.

As far as reimplementing the library with fwrite() and friends, the
main advantage IMHO would be simplifying the library, as well as C
compatibility. Still, it's no weekend task as IIRC the library
currently is about 10 kLOC. However, a pitfall with a fwrite
implementation would be parallel programming. AFAICS C I/O only
supports whole file locking, and that within a single process. If you
want to lock part of a file, or lock a file over multiple processes,
you need fcntl() locking, which is incompatible with C buffered I/O.


-- 
Janne Blomqvist