[Bug libfortran/21820] Really, really, horrible IO performance

["sgk at troutmask dot apl dot washington dot edu" <gcc-bugzilla at gcc dot gnu dot org>]

------- Additional Comments From sgk at troutmask dot apl dot washington dot edu  2005-05-30 16:20 -------
Subject: Re:  Really, really, horrible IO performance

On Mon, May 30, 2005 at 03:20:18PM -0000, pinskia at gcc dot gnu dot org wrote:
> 
> I see the same thing on powerpc-darwin but it seems like on freebsd,
> it is actually paging in the memory which seems wrong, I almost want
> to say you should report it to freebsd as their performance bug
> (Yes working around in libgfortran is still a good idea).

It appears to be a faulty assumption in how gfortran uses mmap.
gfortran appears to assume Linux's behavior:

   (From some version of Redhat)
   MAP_SHARED   Share this mapping with all other processes that map this
                object.  Storing to the region is equivalent to writing to
                the file.  The file may not actually be updated until
                msync(2) or munmap(2) are called.

   (From FreeBSD)
   MAP_SHARED   Modifications are shared.

   MAP_NOSYNC   Causes data dirtied via this VM map to be flushed to
                physical media only when necessary (usually by the
                pager) rather than gratuitously.  Typically this pre-
                vents the update daemons from flushing pages dirtied
                through such maps and thus allows efficient sharing of
                memory across unassociated processes using a file-
                backed shared memory map.  (Much snipped, including a
                big warning about ftruncate extending a file).

   (From http://www.opengroup.org/onlinepubs/7990989775/xsh/mmap.html)
   If MAP_SHARED is specified, write references change the underlying object. 

It appears that Linux's MAP_SHARED is equivalent to FreeBSD's
(MAP_SHARED | MAP_NOSYNC).  NetBSD and OpenBSD man pages do not
have the MAP_NOSYNC flag, so they may not be affected.  Note 
POSIX doesn't state anything about when and how the underlying object 
is updated.  POSIX also doesn't include a MAP_NOSYNC flag.

I've verified that the following patch works and improves
performance, but I prefer the original O_TRUNC patch because
O_TRUNC is a POSIX flag and AFAIK most OS's implement it.

Index: unix.c
===================================================================
RCS file: /cvs/gcc/gcc/libgfortran/io/unix.c,v
retrieving revision 1.26
diff -c -p -r1.26 unix.c
*** unix.c      17 May 2005 16:54:51 -0000      1.26
--- unix.c      30 May 2005 16:06:54 -0000
*************** mmap_alloc (unix_stream * s, gfc_offset 
*** 622,628 ****
  
    length = ((where - offset) & page_mask) + 2 * page_size;
  
!   p = mmap (NULL, length, s->prot, MAP_SHARED, s->fd, offset);
    if (p == (char *) MAP_FAILED)
      return FAILURE;
  
--- 622,628 ----
  
    length = ((where - offset) & page_mask) + 2 * page_size;
  
!   p = mmap (NULL, length, s->prot, MAP_SHARED | MAP_NOSYNC, s->fd, offset);
    if (p == (char *) MAP_FAILED)
      return FAILURE;




-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820