[Bug fortran/56981] Slow I/O: Unformatted 5x slower, large sys component; formatted slow as well

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56981

--- Comment #5 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-04-17 14:50:16 UTC ---
(In reply to comment #4)
> The reason why gfortran is slow here is that for non-regular files we use
> unbuffered I/O. If you write to a regular file instead of /dev/null, you'll 
> see us doing ~8 KB writes at a time.
> 
> The reason for this is that non-regular files (a.k.a. special files) are
> special in many ways wrt seeking. Some allow seeking just fine, some always
> return 0, some return an error (and which special files behave in which way is
> to some extent different on different OS'es).

I do not understand the argument regarding seek. If seek doesn't work - why
should there be a problem with buffering but not without? At least with
SEQUENTIAL one cannot do without (buffer exceeded or no buffering) and with
STREAM no seek should be required.

> Also, for special files users often expect non-buffered IO, e.g. they want
> output on the terminal directly instead of waiting until the 8 KB buffer fills
> up, programs communicating via pipes can deadlock if data sits in the buffers,
> etc.

But the code should be able to wait until a complete record has been written?
That should be rather quick, unless one write a 2GB array. I am not talking
about flushing the data only when 8kB are filled or when the file is closed.
And doing buffering within a record avoids seeks.

> One could of course make "unbuffered" I/O in gfortran really mean "flush
> the buffer at the end of each I/O statement" rather than not using a buffer at
> all.

We should consider this.

 * * *

I have now updated timings with writing to a file.

Results for the example in comment 0, but writing to a file ("test.dat",
tmpfs). Unformatted is much faster with a normal file, but some others
compilers are still significantly faster. And for formatted, all other
compilers are significantly faster.

---- Timing in sec ------------------------------------------------
Unformatted  Formatted
real / user  real / user  Compiler
-----------  -----------  -----------------------------------------
0.378/0.352  2.815/2.804  GCC 4.8.0 (-Ofast, 20130308, Rev. 196547)
0.307/0.296  1.303/1.288  g95 4.0.3 (g95 0.93!) Aug 17 2010 (-O3)
0.210/0.196  0.555/0.532  Sun Fortran 95 8.3 Linux_i386 2007/05/03
0.208/0.184  0.920/0.888  PathScale 3.2.99
0.176/0.152  2.185/2.168  NAGWare Fortran 5.1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0.127/0.125  1.091/1.080  GCC 4.9 (trunk, -Ofast)
0.120/0.118  0.465/0.459  g95 4.0.3 (g95 0.94!) Dec 17 2012
0.136/0.131  0.527/0.524  PathScale EKOPath 4.9.0
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0.335/0.316  2.866/2.860  GCC 4.7.2 20120920 (Cray Inc.)
0.204/0.188  0.659/0.628  Cray Fortran : Version 8.1.6
0.881/0.328  1.281/0.672  Intel 64, Version 13.1.1.163
0.444/0.432  0.884/0.864  pgf90 12.10-0
-------------------------------------------------------------------