I found that if I write some binary file operation in gfortran4.6.1, the io becomes very slow, but gfortran4.5.* is very fast.
Created attachment 24942 [details] test program
gfortran -O3 writebin.f90 gfortran-4.5 only cost 0.725s gfortran-4.6 cost very long time.
I can not reproduce this on my system. I did notice a warning about line truncation on the long line in the example. You may want to use line continuations for that one like this: write(13,rec=jj*i+j)nlgs(1,j,i),dlgs(200,j,i),nlgs(2,j,i),dlgs(199,j,i),& & nlgs(3,j,i),dlgs(198,j,i),nlgs(4,j,i),dlgs(197,j,i),nlgs(5:8,j,i),& & dlgs(154:190,j,i) Sometimes if you get truncated in just the right place, the program will still compile and run, but not the way you think it is.
(In reply to comment #3) > I can not reproduce this on my system. I did notice a warning about line > truncation on the long line in the example. You may want to use line > continuations for that one like this: > > write(13,rec=jj*i+j)nlgs(1,j,i),dlgs(200,j,i),nlgs(2,j,i),dlgs(199,j,i),& > & nlgs(3,j,i),dlgs(198,j,i),nlgs(4,j,i),dlgs(197,j,i),nlgs(5:8,j,i),& > & dlgs(154:190,j,i) > > Sometimes if you get truncated in just the right place, the program will still > compile and run, but not the way you think it is. Yea, I modify it as your follows, but it will also take a very long time (so I break it when it runs more than 1 minute). At the same time, gcc 4.5.3 will only take some seconds. btw, I use Win7 64bit, the CRT is MinGW64 project's CRT. But my gcc4.5.3 also use MinGW64 CRT, so I think that's the gcc4.6's reason, not CRT's.
Created attachment 24946 [details] fixed test
Maybe this is MingW specific? I haven't tested this, but here goes my guess: Since in the write loop we're jumping back and forth across the file, the buffer must be frequently flushed; in libgfortran/io/unix.c (buf_flush) we have in the trunk and 4.6 branch: #ifdef _WIN32 _commit (s->fd); #endif If this is some kind of win32 equivalent to fsync(), i.e. commit stuff to disk, then it's no surprise it's slow? This code was added to buf_flush() as a response to http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44698 .
(In reply to comment #6) > Maybe this is MingW specific? > > I haven't tested this, but here goes my guess: > > Since in the write loop we're jumping back and forth across the file, the > buffer must be frequently flushed; in libgfortran/io/unix.c (buf_flush) we have > in the trunk and 4.6 branch: > > #ifdef _WIN32 > _commit (s->fd); > #endif > > If this is some kind of win32 equivalent to fsync(), i.e. commit stuff to disk, > then it's no surprise it's slow? > > This code was added to buf_flush() as a response to > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44698 . That sounds reasonable. So this bug is fixed in the trunk, but haven't been backported to 4.6?
(In reply to comment #7) > (In reply to comment #6) > > Maybe this is MingW specific? > > > > I haven't tested this, but here goes my guess: > > > > Since in the write loop we're jumping back and forth across the file, the > > buffer must be frequently flushed; in libgfortran/io/unix.c (buf_flush) we have > > in the trunk and 4.6 branch: > > > > #ifdef _WIN32 > > _commit (s->fd); > > #endif > > > > If this is some kind of win32 equivalent to fsync(), i.e. commit stuff to disk, > > then it's no surprise it's slow? > > > > This code was added to buf_flush() as a response to > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44698 . > > That sounds reasonable. > So this bug is fixed in the trunk, but haven't been backported to 4.6? Sorry, I'm wrong. I think I can try 4.7 to test this.
CC list add Kai.
I test it using gcc4.7.0 on Win7, and it also takes a very long time. ps: that works fine on Linux.
(In reply to comment #3) > I can not reproduce this on my system. I did notice a warning about line > truncation on the long line in the example. You may want to use line > continuations for that one like this: > > write(13,rec=jj*i+j)nlgs(1,j,i),dlgs(200,j,i),nlgs(2,j,i),dlgs(199,j,i),& > & nlgs(3,j,i),dlgs(198,j,i),nlgs(4,j,i),dlgs(197,j,i),nlgs(5:8,j,i),& > & dlgs(154:190,j,i) > > Sometimes if you get truncated in just the right place, the program will still > compile and run, but not the way you think it is. you can compile it using "-ffree-line-length-none":)
I would like to see if Kai has any thought on this. Unfortunately I am not very familiar with the windows OS.
See also discussion started at http://gcc.gnu.org/ml/fortran/2011-10/msg00079.html
Author: burnus Date: Tue Oct 18 12:58:42 2011 New Revision: 180138 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=180138 Log: 2011-10-18 Tobias Burnus <burnus@net-b.de> Janne Blomqvist <jb@gcc.gnu.org> PR fortran/50016 * io/file_pos.c (st_flush): Call _commit on MinGW(-w64). * io/intrinsics.c (flush_i4, flush_i8): Ditto. * io/unix.c (flush_all_units_1, flush_all_units): Ditto. (buf_flush): Remove _commit call. * io/inquire.c (inquire_via_unit): Flush internal buffer and call file_length instead of invoking stat via file_size. Modified: branches/gcc-4_6-branch/libgfortran/ChangeLog branches/gcc-4_6-branch/libgfortran/io/file_pos.c branches/gcc-4_6-branch/libgfortran/io/inquire.c branches/gcc-4_6-branch/libgfortran/io/intrinsics.c branches/gcc-4_6-branch/libgfortran/io/unix.c
[4.6] FIXED (comment 14) or at least mitigated the issue on the 4.6 branch - in time for the 4.6.2 release. See also http://gcc.gnu.org/ml/fortran/2011-10/msg00132.html * * * For the 4.7 trunk, we are still discussion, when to call _commit. MinGW/MinGW-w64 users of GCC 4.7 could consider using that patch in the meanwhile. Start of the discussion: http://gcc.gnu.org/ml/fortran/2011-10/threads.html#00079 Patch version 1: Keep _commit only for FLUSH() subroutine/statement http://gcc.gnu.org/ml/fortran/2011-10/msg00132.html Patch version 2: Remove _commit completely, use internal file-size when inquiring open files. (The latter is an independent separate issue, but was the reason for adding _commit at the first place; cf. PR 44698.) http://gcc.gnu.org/ml/fortran/2011-10/msg00094.html * * * Xunxun: Thanks for the report and sorry for making the Windows version that slow and for the two-months delay in fixing it.
Good job. I will extract the patch to 4.6.1 release. ( I don't use gcc4.6 latest branch temporarily because PR 50664 ) Thanks.
Author: burnus Date: Wed Oct 19 17:27:03 2011 New Revision: 180199 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=180199 Log: 2011-10-19 Janne Blomqvist <jb@gcc.gnu.org> Kai Tietz <ktietz@redhat.com> Tobias Burnus <burnus@net-b.de> PR fortran/50016 * io/unix.h (flush_sync): Add new libgfortran-internal * prototype. * io/unix.c (flush_sync): New function, which calls sflush and on MinGW(-w64) also _commit. (flush_all_units, flush_all_units_1): Replace sflush/_commit by flush_sync. * io/file_pos.c (st_flush): Replace sflush/_commit by * flush_sync. * io/intrinsics.c (flush_i4, flush_i8): Ditto. Modified: branches/gcc-4_6-branch/libgfortran/ChangeLog branches/gcc-4_6-branch/libgfortran/io/file_pos.c branches/gcc-4_6-branch/libgfortran/io/intrinsics.c branches/gcc-4_6-branch/libgfortran/io/unix.c branches/gcc-4_6-branch/libgfortran/io/unix.h
(In reply to comment #17) > Author: burnus > Date: Wed Oct 19 17:27:03 2011 > New Revision: 180199 > > URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=180199 > Log: > 2011-10-19 Janne Blomqvist <jb@gcc.gnu.org> > Kai Tietz <ktietz@redhat.com> > Tobias Burnus <burnus@net-b.de> > > PR fortran/50016 > * io/unix.h (flush_sync): Add new libgfortran-internal > * prototype. > * io/unix.c (flush_sync): New function, which calls sflush and > on MinGW(-w64) also _commit. > (flush_all_units, flush_all_units_1): Replace sflush/_commit by > flush_sync. > * io/file_pos.c (st_flush): Replace sflush/_commit by > * flush_sync. > * io/intrinsics.c (flush_i4, flush_i8): Ditto. > > > Modified: > branches/gcc-4_6-branch/libgfortran/ChangeLog > branches/gcc-4_6-branch/libgfortran/io/file_pos.c > branches/gcc-4_6-branch/libgfortran/io/intrinsics.c > branches/gcc-4_6-branch/libgfortran/io/unix.c > branches/gcc-4_6-branch/libgfortran/io/unix.h Will the rev 180199 merge into gcc4.6.2 release? Because the first 4.6.2 RC doesn't contain the modify, gcc-4.6.2-RC-20111019 libgfortran build on mingw/mingw64 can cause: libtool: compile: i686-w64-mingw32-gcc -L/mingw/i686-w64-mingw32/lib -L/mingw/mingw/lib -isystem /mingw/i686-w64-mingw32/include -isystem /mingw/mingw/include -DHAVE_CONFIG_H -I. -I../.././libgfortran -iquote../.././libgfortran/io -I../.././libgfortran/../gcc -I../.././libgfortran/../gcc/config -I../.././libgfortran/../libquadmath -I../../host-i686-w64-mingw32/gcc -D_GNU_SOURCE -std=gnu99 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings -fcx-fortran-rules -ffunction-sections -fdata-sections -g -pipe -O2 -I/mingw/include -MT file_pos.lo -MD -MP -MF .deps/file_pos.Tpo -c ../.././libgfortran/io/file_pos.c -o file_pos.o ../.././libgfortran/io/file_pos.c: In function 'st_flush': ../.././libgfortran/io/file_pos.c:457:20: error: 'stream' has no member named 'fd' I would hope the next 4.6.2 RC or release fix the problem. Thanks.
Author: jb Date: Wed Nov 9 15:46:15 2011 New Revision: 181207 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=181207 Log: PR 50016 Slow I/O on MingW due to _commit frontend ChangeLog: 2011-11-09 Janne Blomqvist <jb@gcc.gnu.org> PR libfortran/50016 * gfortran.texi (Data consistency and durability): New section. testsuite ChangeLog: 2011-11-09 Janne Blomqvist <jb@gcc.gnu.org> PR libfortran/50016 * gfortran.dg/inquire_size.f90: Don't flush the unit. libgfortran ChangeLog: 2011-11-09 Janne Blomqvist <jb@gcc.gnu.org> PR libfortran/50016 * io/inquire.c (inquire_via_unit): Flush the unit and use ssize. * io/unix.c (buf_flush): Don't call _commit. Modified: trunk/gcc/fortran/ChangeLog trunk/gcc/fortran/gfortran.texi trunk/gcc/testsuite/ChangeLog trunk/gcc/testsuite/gfortran.dg/inquire_size.f90 trunk/libgfortran/ChangeLog trunk/libgfortran/io/inquire.c trunk/libgfortran/io/unix.c
Fixed, closing.