This is the mail archive of the
mailing list for the GCC project.
Re: [patch, libfortran] Adjust block size for libgfortran for unformatted reads
- From: Janne Blomqvist <blomqvist dot janne at gmail dot com>
- To: Manfred Schwarb <manfred99 at gmx dot ch>
- Cc: Thomas Koenig <tkoenig at netcologne dot de>, "fortran at gcc dot gnu dot org" <fortran at gcc dot gnu dot org>, gcc-patches <gcc-patches at gcc dot gnu dot org>, David Edelsohn <dje dot gcc at gmail dot com>
- Date: Mon, 8 Jul 2019 16:02:17 +0300
- Subject: Re: [patch, libfortran] Adjust block size for libgfortran for unformatted reads
- References: <email@example.com> <firstname.lastname@example.org>
On Mon, Jul 8, 2019 at 11:18 AM Manfred Schwarb <email@example.com> wrote:
> Am 07.07.19 um 22:13 schrieb Thomas Koenig:
> > Hello world,
> > the attached patch sets the I/O block size for unformatted files to
> > 2**17 and makes this, and the block size for formatted files,
> > adjustable via environment variables.
> > The main reason is that -fconvert=big-endian was quite slow on
> > some HPC systems. A bigger buffer should eliminate that. Also,
> > People who use unformatted files are likely to write large amounts
> > of data, so this seems like a good fit. Finally, some benchmarking
> > showed that 131072 seemed like a good value to use. Thanks to Jerry
> > for support.
> > I didn't change the value for formatted files because, frankly, we are
> > using a lot of CPU for converting numbers there, so any gain
> > negligible (unless somebody comes up with a benchmark which says
> > otherwise).
> formatted write: Did you try writing to an USB stick or similar? I guess
> for flash based devices anything below 64k will slow down operation.
> Computers like Raspberry Pi and the like often have flash based storage,
> and it is not extremely unrealistic to run fortran programs on them.
Good point. If you happen to have a USB stick handy, can you try the
simple C benchmark program at
(the kernel will coalesce IO's by itself, so the granularity of IO
syscalls is not necessarily the same as the actual IO to devices.
Network filesystems like NFS/Lustre/GPFS may have less latitude here
due to coherency requirements etc.)