[patch, fortran] Unformatted array IO performance improvement

[Janne Blomqvist]

On Sun, Sep 25, 2005 at 12:09:10PM +0200, Thomas Koenig wrote:
> With your patch, the following program causes a segfault on
> i686-pc-linux-gnu:
> 
> $ cat arrayio_7.f90
> program main
>   character(len=2**25) :: a(2)
>   a = 'x'
>   open(10, form="unformatted", access="sequential")
>   write (10) a
> end
> $ gfortran -g arrayio_7.f90
> $ gdb ./a.out
> GNU gdb 6.3-debian
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-linux"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
> 
> (gdb) r
> Starting program: /home/ig25/Krempel/Array/a.out
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0xb7dbd5ef in memcpy () from /lib/tls/libc.so.6
> (gdb) bt
> #0  0xb7dbd5ef in memcpy () from /lib/tls/libc.so.6
> #1  0xb7f0f3d9 in unformatted_write (type=BT_CHARACTER, source=0x8049b60,
>     length=-33554432, nelems=2)
>     at ../../../gcc-4.1/libgfortran/io/transfer.c:356
> #2  0xb7f0f623 in *_gfortran_transfer_array (desc=0xbfb40810)
>     at ../../../gcc-4.1/libgfortran/io/transfer.c:996
> #3  0x0804876c in MAIN__ () at arrayio_7.f90:5
> #4  0x080487a3 in main (argc=1, argv=0xbfb408c4)
>     at ../../../gcc-4.1/libgfortran/fmain.c:18
> (gdb)
> 
> This is probably a signed issue, as the .t02.original file shows
> 
>     parm.1.dtype = -2147483599;

What's this .t02.original file?

Anyway, I found the reason: The size of the array elements is stored
in "index_type dtype" in the descritor, and the size is retrieved by
shifting 6 bits to the right. Thus on 32-bit platforms the maximum
size is 2**24, or 16 MB. Is there any practical need for arrays of >16
MB characters, or can we say this is an implementation limit and
nobody will care?

> Can you pass the length of a character array the same way that
> the recent patches to the reshape arrays introduced?

You mean a transfer_array_char library entry point with an extra
argument specifying the length of the characters, and leaving the size
in the descriptor for sizeof(char)=1? Yes, I guess that would be one
way. Another could be to simply make the dtype field 64 bits on all
platforms.

>  I think this
> would also be more consistent with the size of an array not being
> consistent with its string length.

Hmm, I think support for non-8-bit chars will require some major
surgery of the formatted transfer routines. I'm not sure I want to go
there now.

-- 
Janne Blomqvist