Re: i/o synchronization revisited

[Jack Howarth <howarth at bromo dot med dot uc dot edu>]

On Fri, Mar 19, 2010 at 04:54:43PM +0200, Janne Blomqvist wrote:
> On Fri, Mar 19, 2010 at 15:45, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> > ? I have been porting xplor-nih to x86_64 on darwin10
> > since the author doesn't have a EMT64 capable intel Mac. The
> > result was an improvement in the execution time of the
> > xplor testsuite from 58 seconds with a 32-bit build
> > using gcc 4.2 to 50.8 seconds for a 64-bit with gcc trunk.
> > In both cases, the compilations used -O3 -ffast-math -funroll-loops.
> > These results are still almost twice as slow as what we see on
> > equivalent hardware with a build on linux with the Intel
> > compilers (~28 seconds).
> 
> Have you tried a recent gfortran on that Linux machine, in order to
> provide a relevant comparison to ifort?
> 
> > ? ?I am wondering how feasible an option like GFORTRAN_SYNC_PRECONNECTED
> > would be. The existing code for GFORTRAN_UNBUFFERED_PRECONNECTED would
> > be leveraged to leave the unconnected IO buffered and only attempt
> > to synchronize the preconnected IO.
> 
> I don't understand, what do you mean by unconnected IO and how is
> synchronized supposed to be different from unbuffered? What is the
> proposed GFORTRAN_SYNC_PRECONNECTED supposed to do, and how is it
> supposed to be different from GFORTRAN_UNBUFFERED_PRECONNECTED?

Janne,
   Sorry, I meant IO that libgfortran detects as not being 
preconnected. My understanding was that compilers like ifort
had a system to synchronize the IO from the different languages
without resorting to flushes.

> 
> To recap, when opening a unit, gfortran decides whether to use
> buffered or unbuffered I/O based on the following test:
> 
>   if (isatty (s->fd) || options.all_unbuffered
>       ||(options.unbuffered_preconnected &&
>          (s->fd == STDIN_FILENO
>           || s->fd == STDOUT_FILENO
>           || s->fd == STDERR_FILENO)))
>     raw_init (s);
>   else
>     buf_init (s);
> 

   So from the above code, it appears that stdout is never
buffered with GFORTRAN_UNBUFFERED_PRECONNECTED, correct?
If true, I guess I can test my hypothesis that the main
culprit in the performance loss when going from buffered
to GFORTRAN_UNBUFFERED_PRECONNECTED in xplor-nih with the
following change...

Index: libgfortran/io/unix.c
===================================================================
--- libgfortran/io/unix.c	(revision 157578)
+++ libgfortran/io/unix.c	(working copy)
@@ -815,7 +815,6 @@
   if (isatty (s->fd) || options.all_unbuffered
       ||(options.unbuffered_preconnected && 
          (s->fd == STDIN_FILENO 
-          || s->fd == STDOUT_FILENO 
           || s->fd == STDERR_FILENO)))
     raw_init (s);
   else

If this recovers all or most of the performance loss under
GFORTRAN_UNBUFFERED_PRECONNECTED compared to buffered io
it would suggest the real world code could benefit from at
least finding a way to synchronize stdout across the languages.
              Jack

> Also note that for formatted IO you can never get completely
> unbuffered IO, it's always buffered at least at IO statement level
> (this is required to correctly support things like TL edit descriptors
> on non-seekable devices).
> 
> > Considering that going from
> > a i686 gcc 4.2 build to a x86_64 gcc 4.5 build only obtained a
> > 14% performance increase, the potential performance increase from
> > synchronizing the remaining preconnected IO (18%) seems like
> > a very significant improvement. Far more than we are likely to
> > obtain from any upcoming optimizations (save perhaps Milepost).
> 
> Most programs seem to follow the common sense of not doing IO to the
> same file(s) from different languages with different IO libraries, and
> failing that, by appropriately inserting fflush(), FLUSH or equivalent
> things. So, I'm not convinced there is a huge demand for yet another
> special case solution of the "how to mix IO from different languages"
> problem.
> 
> 
> -- 
> Janne Blomqvist