[Bug fortran/22519] Memory and binary disk layout disagree for real*10

["tkoenig at gcc dot gnu dot org" <gcc-bugzilla at gcc dot gnu dot org>]

------- Additional Comments From tkoenig at gcc dot gnu dot org  2005-07-18 14:35 -------
(In reply to comment #2)


> ultimately, things have to be written by a system call, and a system call is 
> expensive.  (One system call per array element is out of the question.)

With the current implementation (which may not be the best for performance)
we use memory-mapped I/O.  It then becomes a question of how efficient
you transfer your data into the memory you mmapped.

> Thus 
> we either dump the whole array to disk, byte for byte as it is in memory, or 
> we have to transform it into a temporary buffer and then write this buffer. 

Agreed.
 
> C's fwrite uses a buffer (probably), fprintf does for sure. 
>  
> If you remove the padding, then you have to allocate and deallocate these 
> buffers

If you use read() instead of mmap(), it is enough to allocate a
single buffer, which can be reused.

>and pack or unpack the array elements explicitly.  Depending on your 
> I/O library, that may be a lot of work.

I tried to show in my little benchmark above that packing things may
be a little less work than actually copying the whole thing.  This doesn't
take disk I/O into account, which is also fairly slow.

> And in the end you are still 
> incompatibly to C. 

Fortran unformatted I/O is incompatible to C anyway, because of the record
markers, this is not really a big deal.

I think a few benchmarks could help there.

> (I'm afraid we may be going to repeat a discussion here that was already held 
> on the mailing list.) 

If there's a PR, I think it's better if people can read the discussion in its
audit trail.

The biggest problems in I/O performance at the moment are the overhead
of individual function calls for unformatted array I/O.  I think the best
way to address that would be to let the library use array descriptors.
For the general case, this would require pack / unpack operations, so
a conversion to 10 bytes for numbers aligned to 16 bytes would only
help.

Hmm....

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519