[Patch, libgfortran, 4.5] PR25561, 37754: New low level I/O library

[Janne Blomqvist]

Hi all,

attached is a substantially reworked low level I/O library for gfortran.
The general idea is to replace the "Alloc Stream Facility" with a simple
low level interface that provides more or less exactly the POSIX
semantics (read/write/seek/truncate/close). This is provided by the
raw_* functions in unix.c. Then there is another implementation of the
same interface in the buf_* functions, which as the name implies, use a
buffer to improve performance. Attached is gfortranio.rst with some more
justification for the design choices.

The patch also changes the mid level I/O library to use the format
buffer (fbuf) machinery for reads as well as writes (for 4.4 we use fbuf
for writes).

Some other minor cleanup have been done as well, e.g. changing
end-of-file detection to "lazy", i.e. it assumes everything is going
well until a read operation hits EOF (and generates an error at that
point) rather than trying to pro-actively determine that the next read
will hit EOF.

So far I have done very little performance tuning, mainly to make
fbuf_getc() an inline function that calls a fbuf_getc_refill non-inline
function to refill the buffer if necessary (a bit like getc() and
fgetc() in C stdio). But that being said, initial performance results
are promising. For the countlines.f benchmark (see PR37754) with the
patched trunk vs. 4.3 (I don't have vanilla 4.4 at the moment, but
according to the PR it's about 10-20 % slower than 4.3):

./countlines.gf44  2.90s user 0.08s system 98% cpu 3.022 total

./countlines.gf43  4.58s user 0.07s system 99% cpu 4.684 total

(results above are best of 10 runs each)

Attached is also a simple test that measures unformatted sequential
write performance with different record sizes. The new implementation
does quite well because the new buffering implementation is good about
reducing unnecessary syscalls like lseek(). Compared with strace e.g. to
gfortran 4.3 that seems to do an lseek() for every record, the patched
version avoids seeking as long as the record + record markers are small
enough to fit into the buffer (currently the buffer is 8 KB like for
previous gfortran versions). For the patched 4.4 a representative run
with the us_perf shows the following on my system:

 Unformatted sequential write performance test
 Record size                 MB/s
 ================================
           4   7.5802262656833790
           8   15.873509918930392
          16   25.410357328092356
          32   39.385463811119422
          64   60.307900379571777
         128   90.606550636531679
         256   124.39827155195042
         512   148.53833492807280
        1024   161.33197189039470
        2048   165.30075386586569
        4096   168.64479316414403
        8192   174.79361499332097
       16384   244.82046680142199
       32768   294.28299059919226
       65536   308.46720850537616
      131072   291.61666653611849
      262144   164.22273724046653
      524288   158.71373187189036

whereas with 4.3

 Unformatted sequential write performance test
 Record size                 MB/s
 ================================
           4   5.3134706580816573
           8   12.652808318693332
          16   16.467640435883737
          32   25.677765810986461
          64   41.438541428006481
         128   69.465166864727536
         256   100.52410645875071
         512   120.68720373185609
        1024   143.79566880191507
        2048   136.48643666988502
        4096   125.20637961368544
        8192   165.25923002902420
       16384   244.82046680142199
       32768   261.98301434601660
       65536   285.61729202322573
      131072   163.07498967544379
      262144   166.28830213878092
      524288   150.22626328225456

With records bigger than 8 KB there is no big difference between the
two, as in that case both 4.3 and trunk+patch bypasses the buffering.

The patch passes the gfortran and NIST testsuites on i686-pc-linux-gnu.
For NIST FM907 there is a single extra whitespace change compared to the
reference, but I don't think it's actually an error.

So far, however, there has been very little real application testing.

As this is a rather large and invasive patch, I'd like to get it in
relatively early in 4.5 in order to have plenty of time to fix all
remaining issues.

Ok for trunk once 4.4 branches?

-- 
Janne Blomqvist
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ChangeLog
URL: <http://gcc.gnu.org/pipermail/fortran/attachments/20090105/f7e699be/attachment.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: us_perf.f90
Type: text/x-fortran
Size: 1268 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/fortran/attachments/20090105/f7e699be/attachment.bin>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gfortranio.rst
URL: <http://gcc.gnu.org/pipermail/fortran/attachments/20090105/f7e699be/attachment-0001.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pr25561-part2-8.diff.gz
Type: application/x-gzip
Size: 28202 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/fortran/attachments/20090105/f7e699be/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://gcc.gnu.org/pipermail/fortran/attachments/20090105/f7e699be/attachment.sig>