Re: io library bug?

[Fulvio Ciriaco <oivulf at gmail dot com>]

Hallo,
I guess I will need to patch the application to use smaller
records, and you are right of course about the memory
requirements.
Indeed I was misled by all the noise on google about gfortran
support for >2GB record length.
Thanks a lot
Fulvio

At Thu, 27 May 2010 10:18:57 +0200,
Tobias Burnus wrote:
> 
> On 05/27/2010 08:53 AM, Arjen Markus wrote:
> > 2010/5/27 Fulvio Ciriaco <oivulf@gmail.com>:
> >   
> >>      anint=865271640
> >>      open(unit=7,file='a',status='unknown',access='direct',
> >>     *     form='unformatted',recl=8*anint)
> >>     
> 
> That's an enormous record size of 6,601 MiB!
> 
> >> >gfortran -fdefault-integer-8 test.f
> >> a.out
> >>           6922173120
> >> At line 7 of file test.f (unit = 7, file = '')
> >> Fortran runtime error: RECL parameter is non-positive in OPEN statemen
> > Even though with the compiler option all integer variables become 8
> > bytes in length, that
> > does not necessarily mean that the record length will be taken as an
> > integer of that kind.
> > It is safest to stick to 4-bytes integers.
> >
> > Fortran 2003 introduces the possibility to specify the kind for
> > various such sizes
> 
> Actually, already Fortran 95 allows all integer kinds:
> 
> R905 connect-spec
>     is ...
>     or RECL = scalar-int-expr
> 
> (Otherwise "scalar-default-int-expr" were used.) Thus, in terms of the
> standard everything is fine and, indeed, all my compilers are able to
> compile that file. However, at run time all fail - either with out of
> memory (Intel's ifort, NAG's f95) or with the positive number error
> (gfortran, g95, pathf95).
> 
> As I/O is such defined that almost everything can produce an error, the
> compilers are standard conform. (Though, I think on systems which have
> large-file support, it could be supported.)
> 
> The real question is: Do you really need record sizes of 6.4 GiB? I can
> see reasons to store data of several hundreds of GiB - but I fail to
> come up with a good application where one accesses blocks of data
> consisting of several GiB. How much memory do you have (per process or
> core)? 20 GiB? If you have less per core, I fail to see why you then
> want to use such huge records if you cannot read them into the memory.
> (For instance, our x86-64 cluster has more than 100,000 cores but "only"
> 24 GiB per node and thus 3 GiB per core (minus memory needed by the
> operating system, queuing system, MPI etc.).
> 
> Tobias
> 
> PS: Just for completeness: The size given to RECL= is in "file storage
> units"; in Fortran 2003 you can find their value (in bits) via the
> "FILE_STORAGE_SIZE" parameter, which is part of the intrinsic
> "iso_fortran_env" module. For gfortran the value of is FILE_STORAGE_SIZE
> is 8 bits, i.e. RECL= takes the size in bytes. Depending on the version
> of the compiler and the options, other compilers might have a different
> value. For instance, ifort uses long-word (=4 bytes) for unformatted
> files unless "-assume byterecl" is used.