I have a relatively stable, relatively large fortran program (500,000+ lines). This program works correctly on linux, 64 bit linux, solaris, and windows when compiled with g77 3.4.4. When I complile it with gfortran 4.1.0, 26 out of 384 of my regerssion tests fail because of i/o errors. Basically, I have databases which are accesses via direct access, unformatted io. All transactions pass through two routines: a writter and a reader. I have attached a tar file which contains the two routines. Basically, they write two words at the beginning of each record - the current numbers of "saus" (sort integers) and the record which continues this record. Also attached is a file, trans, which shows the reads and writes. Each line in the file contains the action (dba_read or dba_write), the fortran unit number, the record number, the number of short integers on this record, the continuation record, and finally the record length. (The print statements that created this file are commentd out in the source). If you notice the last line in this file says that record 2958 on logical unit 15 has 1 short integer, but 4 lines up, it has 122 and is continued on 2959. Notice that there have been writes to this file, but at different record numbers. I would really like to give you a better report, but the fact that 350 tests work properly says that any attempt to create a simple test will probably fail (the code will work properly). I will be happy to do anything you would like to help in the resolution of this problem. In particular, it you would like for me to test any changes, please just send me the library and I can get back to you within a couple of hours.
Created attachment 11505 [details] source and results
I will take a look at this one. This does seem odd. Can you tell us what platform you are running on and what version of gfortran. gfortran -v will tell us useful info.
Subject: Re: Problems with direct access io On Wed, 25 May 2006, jvdelisle at gcc dot gnu dot org wrote: > > > ------- Comment #2 from jvdelisle at gcc dot gnu dot org 2006-05-25 00:51 ------- > I will take a look at this one. This does seem odd. Can you tell us what > platform you are running on and what version of gfortran. > > gfortran -v will tell us useful info. > > Actually I included this information 4.1.0. The results submitted were run on an AMD x86_64 machine, but the exactly same thing occurs on an Intel dual core P4 with Fedora Core 4 (a 32 bit OS). Thank you and if I can do anything to help, please let me know. Thanks, Ray
If you are running a gfortran straight from the FC 4 distribution it probably does not have the latest fixes to the IO that we have in the 4.1 branch. These have not been released yet. Release 4.1.1 is being prepared now. If you are up to it you could grab a gcc snapshot and build a 4.1.1 or 4.2 version and try it. 4.2 may not be very stable at the moment so a 4.1 snapshot latest would be a good bet. Let me know what you think and we will keep trying to help you with this.
Subject: Re: Problems with direct access io On Wed, 25 May 2006, jvdelisle at gcc dot gnu dot org wrote: > > > ------- Comment #4 from jvdelisle at gcc dot gnu dot org 2006-05-25 01:20 ------- > If you are running a gfortran straight from the FC 4 distribution it probably > does not have the latest fixes to the IO that we have in the 4.1 branch. These > have not been released yet. Release 4.1.1 is being prepared now. > > If you are up to it you could grab a gcc snapshot and build a 4.1.1 or 4.2 > version and try it. 4.2 may not be very stable at the moment so a 4.1 snapshot > latest would be a good bet. > > Let me know what you think and we will keep trying to help you with this. > I am really a big boy. Tried to get the latest snapshot but could not find it! Send url, I will make it, test it, and report tomorrow. Thanks again, Ray
http://gcc.gnu.org/mirrors.html
The following patch causes no regressions. I can't reproduce the problem here, but I have a hunch. Ray, can you try this and see if it resolves the problem? Thanks for tests and reports. Index: io/transfer.c =================================================================== *** io/transfer.c (revision 114105) --- io/transfer.c (working copy) *************** st_write_done (st_parameter_dt *dtp) *** 2416,2421 **** --- 2416,2425 ---- break; } + if (dtp->u.p.current_unit != NULL + && dtp->u.p.current_unit->flags.access == ACCESS_DIRECT) + flush (dtp->u.p.current_unit->s); + free_format_data (dtp); free_ionml (dtp); if (dtp->u.p.scratch != NULL)
Here is some good news. I came up with a test case that fails with gfortran and passes with intel. The bad news is that I have not fixed it yet. At least I have something to work with now and its ugly. program testdirect implicit none integer, dimension(100) :: a integer :: i,j,k,ier real :: x a = 0 call random_seed() open(unit=15,file="testdirectio",access="direct",form="unformatted",recl=4) do i=1,100 call random_number(x) k= int(x * 100)+1 a(i)=k write(unit=15, rec=k) k enddo do j=1,100 read(unit=15, rec=a(j), iostat=ier) k if (ier.ne.0) then print *, "No Record: ", j else print *, "Bad Record at ",a(j), k endif enddo close(unit=15, status="delete") end program testdirect
Subject: Re: Problems with direct access io Good work, knew this would be hard! Ray On Sat, 27 May 2006, jvdelisle at gcc dot gnu dot org wrote: > > > ------- Comment #8 from jvdelisle at gcc dot gnu dot org 2006-05-27 05:10 ------- > Here is some good news. I came up with a test case that fails with gfortran > and passes with intel. The bad news is that I have not fixed it yet. At least > I have something to work with now and its ugly. > > program testdirect > implicit none > integer, dimension(100) :: a > integer :: i,j,k,ier > real :: x > > a = 0 > call random_seed() > open(unit=15,file="testdirectio",access="direct",form="unformatted",recl=4) > do i=1,100 > call random_number(x) > k= int(x * 100)+1 > a(i)=k > write(unit=15, rec=k) k > enddo > do j=1,100 > read(unit=15, rec=a(j), iostat=ier) k > if (ier.ne.0) then > print *, "No Record: ", j > else > print *, "Bad Record at ",a(j), k > endif > enddo > close(unit=15, status="delete") > end program testdirect > > > -- > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27757 > > ------- You are receiving this mail because: ------- > You reported the bug, or are watching the reporter. > > -------------------+ Spam Scoring Results +------------------------ > Content analysis details: (0.0 hits, 5.0 required) > 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60% > [score: 0.5002] > > -------------------+ End Spam Scoring Results +----------------------- > Ultramarine, Inc. http://www.ultramarine.com Phone: 713-975-8146 Fax: 713-975-8179
This bug does not occur with g77, as this version of the test program shows: $ cat testdirect.f program testdirect implicit none integer a (100) integer i,j,k,ier real x integer myrand(100) data myrand / 13, 9, 34, 41, 25, 98, 6, 12, 11, 44, 79, 3, & 64, 61, 77, 57, 59, 2, 92, 38, 71, 64, 31, 60, 28, 90, 26, & 97, 47, 26, 48, 96, 95, 82, 100, 90, 45, 71, 71, 67, 72, & 76, 94, 49, 85, 45, 100, 22, 96, 48, 13, 23, 40, 14, 76, 99, & 96, 90, 65, 2, 8, 60, 96, 19, 45, 1, 100, 48, 91, 20, 92, & 72, 81, 59, 24, 37, 43, 21, 54, 68, 31, 19, 79, 63, 41, & 42, 12, 10, 62, 43, 9, 30, 9, 54, 35, 4, 5, 55, 3, 94 / do i=1,100 a(i) = 0 end do C call random_seed() open(unit=15,file="testdirectio",access="direct", & form="unformatted",recl=4) do i=1,100 k = myrand(i) ! k= int(x * 100)+1 a(i)=k write(unit=15, rec=k) k end do do j=1,100 read(unit=15, rec=a(j), iostat=ier) k if (ier.ne.0) then print *, "No Record: ", j else if (a(j) .ne. k) print *, "Bad Record at ",a(j), k endif enddo close(unit=15, status="delete") end $ g77 testdirect.f $ ./a.out $ gfortran testdirect.f $ ./a.out Bad Record at 13 14 Bad Record at 34 0 Bad Record at 25 26 Bad Record at 98 99 Bad Record at 6 11 Bad Record at 11 0 Bad Record at 44 45 Bad Record at 64 65 Bad Record at 61 0 Bad Record at 77 0 Bad Record at 57 59 Bad Record at 2 0 Bad Record at 38 0 Bad Record at 71 72 Bad Record at 64 65 Bad Record at 60 0 Bad Record at 28 0 Bad Record at 90 98 Bad Record at 26 34 Bad Record at 97 0 Bad Record at 47 48 Bad Record at 26 34 Bad Record at 96 97 Bad Record at 95 96 Bad Record at 82 0 Bad Record at 90 98 Bad Record at 45 0 Bad Record at 71 72 Bad Record at 71 72 Bad Record at 67 0 Bad Record at 76 0 Bad Record at 49 0 Bad Record at 85 0 Bad Record at 45 0 Bad Record at 22 23 Bad Record at 96 97 Bad Record at 13 14 Bad Record at 23 0 Bad Record at 40 0 Bad Record at 14 0 Bad Record at 76 0 Bad Record at 99 100 Bad Record at 96 97 Bad Record at 90 98 Bad Record at 65 0 Bad Record at 2 0 Bad Record at 8 0 Bad Record at 60 0 Bad Record at 96 97 Bad Record at 45 0 $ g77 -v Reading specs from /usr/lib/gcc/i486-linux-gnu/3.4.6/specs Configured with: ../src/configure -v --enable-languages=c,c++,f77,pascal --prefix=/usr --libexecdir=/usr/lib --with-gxx-include-dir=/usr/include/c++/3.4 --enable-shared --with-system-zlib --enable-nls --without-included-gettext --program-suffix=-3.4 --enable-__cxa_atexit --enable-clocale=gnu --enable-libstdcxx-debug --with-tune=i686 i486-linux-gnu Thread model: posix gcc version 3.4.6 (Debian 3.4.6-1) $ gfortran -v Using built-in specs. Target: i686-pc-linux-gnu Configured with: ../../gcc/trunk/configure --prefix=/home/ig25 --enable-languages=c,fortran Thread model: posix gcc version 4.2.0 20060525 (experimental)
Subject: Re: Problems with direct access io I believe I said that in the bug report. On Sat, 27 May 2006, tkoenig at gcc dot gnu dot org wrote: > > > ------- Comment #10 from tkoenig at gcc dot gnu dot org 2006-05-27 19:10 ------- > This bug does not occur with g77, as this version of the > test program shows: > > $ cat testdirect.f > program testdirect > implicit none > integer a (100) > integer i,j,k,ier > real x > integer myrand(100) > > data myrand / 13, 9, 34, 41, 25, 98, 6, 12, 11, 44, 79, 3, > & 64, 61, 77, 57, 59, 2, 92, 38, 71, 64, 31, 60, 28, 90, 26, > & 97, 47, 26, 48, 96, 95, 82, 100, 90, 45, 71, 71, 67, 72, > & 76, 94, 49, 85, 45, 100, 22, 96, 48, 13, 23, 40, 14, 76, 99, > & 96, 90, 65, 2, 8, 60, 96, 19, 45, 1, 100, 48, 91, 20, 92, > & 72, 81, 59, 24, 37, 43, 21, 54, 68, 31, 19, 79, 63, 41, > & 42, 12, 10, 62, 43, 9, 30, 9, 54, 35, 4, 5, 55, 3, 94 / > > do i=1,100 > a(i) = 0 > end do > C call random_seed() > open(unit=15,file="testdirectio",access="direct", > & form="unformatted",recl=4) > do i=1,100 > k = myrand(i) > ! k= int(x * 100)+1 > a(i)=k > write(unit=15, rec=k) k > end do > > do j=1,100 > read(unit=15, rec=a(j), iostat=ier) k > if (ier.ne.0) then > print *, "No Record: ", j > else > if (a(j) .ne. k) print *, "Bad Record at ",a(j), k > endif > enddo > close(unit=15, status="delete") > end > $ g77 testdirect.f > $ ./a.out > $ gfortran testdirect.f > $ ./a.out > Bad Record at 13 14 > Bad Record at 34 0 > Bad Record at 25 26 > Bad Record at 98 99 > Bad Record at 6 11 > Bad Record at 11 0 > Bad Record at 44 45 > Bad Record at 64 65 > Bad Record at 61 0 > Bad Record at 77 0 > Bad Record at 57 59 > Bad Record at 2 0 > Bad Record at 38 0 > Bad Record at 71 72 > Bad Record at 64 65 > Bad Record at 60 0 > Bad Record at 28 0 > Bad Record at 90 98 > Bad Record at 26 34 > Bad Record at 97 0 > Bad Record at 47 48 > Bad Record at 26 34 > Bad Record at 96 97 > Bad Record at 95 96 > Bad Record at 82 0 > Bad Record at 90 98 > Bad Record at 45 0 > Bad Record at 71 72 > Bad Record at 71 72 > Bad Record at 67 0 > Bad Record at 76 0 > Bad Record at 49 0 > Bad Record at 85 0 > Bad Record at 45 0 > Bad Record at 22 23 > Bad Record at 96 97 > Bad Record at 13 14 > Bad Record at 23 0 > Bad Record at 40 0 > Bad Record at 14 0 > Bad Record at 76 0 > Bad Record at 99 100 > Bad Record at 96 97 > Bad Record at 90 98 > Bad Record at 65 0 > Bad Record at 2 0 > Bad Record at 8 0 > Bad Record at 60 0 > Bad Record at 96 97 > Bad Record at 45 0 > $ g77 -v > Reading specs from /usr/lib/gcc/i486-linux-gnu/3.4.6/specs > Configured with: ../src/configure -v --enable-languages=c,c++,f77,pascal > --prefix=/usr --libexecdir=/usr/lib --with-gxx-include-dir=/usr/include/c++/3.4 > --enable-shared --with-system-zlib --enable-nls --without-included-gettext > --program-suffix=-3.4 --enable-__cxa_atexit --enable-clocale=gnu > --enable-libstdcxx-debug --with-tune=i686 i486-linux-gnu > Thread model: posix > gcc version 3.4.6 (Debian 3.4.6-1) > $ gfortran -v > Using built-in specs. > Target: i686-pc-linux-gnu > Configured with: ../../gcc/trunk/configure --prefix=/home/ig25 > --enable-languages=c,fortran > Thread model: posix > gcc version 4.2.0 20060525 (experimental) > > > -- > > tkoenig at gcc dot gnu dot org changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |tkoenig at gcc dot gnu dot > | |org > OtherBugsDependingO| |19292 > nThis| | > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27757 > > ------- You are receiving this mail because: ------- > You reported the bug, or are watching the reporter. > > -------------------+ Spam Scoring Results +------------------------ > Content analysis details: (-2.6 hits, 5.0 required) > -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% > [score: 0.0000] > > -------------------+ End Spam Scoring Results +----------------------- > Ultramarine, Inc. http://www.ultramarine.com Phone: 713-975-8146 Fax: 713-975-8179
I am confirming this is a regression. The test program works fine on 4.0.2
Subject: Re: [4.1/4.2 Regression] Problems with direct access io On Sat, 28 May 2006, jvdelisle at gcc dot gnu dot org wrote: > > > ------- Comment #12 from jvdelisle at gcc dot gnu dot org 2006-05-28 00:40 ------- > I am confirming this is a regression. The test program works fine on 4.0.2 > Thanks, I thought it had worked before but could not find out when! Ray
Here is the fix folks. Unbelievable! Index: io/unix.c =================================================================== *** io/unix.c (revision 114207) --- io/unix.c (working copy) *************** fd_seek (unix_stream * s, gfc_offset off *** 569,574 **** --- 569,575 ---- } s->physical_offset = s->logical_offset = offset; + s->active = 0; return (lseek (s->fd, offset, SEEK_SET) < 0) ? FAILURE : SUCCESS; }
Subject: Bug 27757 Author: jvdelisle Date: Mon May 29 23:39:53 2006 New Revision: 114220 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114220 Log: 2006-05-29 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/27757 * io/unix.c (fd_seek): Set active to zero. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/unix.c
Subject: Bug 27757 Author: jvdelisle Date: Mon May 29 23:42:55 2006 New Revision: 114221 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114221 Log: 2006-05-29 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/27757 * gfortran.dg/direct_io_5.f90: New test. Added: trunk/gcc/testsuite/gfortran.dg/direct_io_5.f90 Modified: trunk/gcc/testsuite/ChangeLog
Subject: Bug 27757 Author: jvdelisle Date: Tue May 30 02:51:26 2006 New Revision: 114227 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114227 Log: 2006-05-29 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/27757 * io/unix.c (fd_seek): Set active to zero. Modified: branches/gcc-4_1-branch/libgfortran/ChangeLog branches/gcc-4_1-branch/libgfortran/io/unix.c
Subject: Bug 27757 Author: jvdelisle Date: Tue May 30 02:54:29 2006 New Revision: 114228 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114228 Log: 2006-05-29 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/27757 * gfortran.dg/direct_io_5.f90: New test. Added: branches/gcc-4_1-branch/gcc/testsuite/gfortran.dg/direct_io_5.f90 Modified: branches/gcc-4_1-branch/gcc/testsuite/ChangeLog
Fixed on 4.1 and 4.1. Ray also confirmed by separate email that the original application problem is fixed as well.
and 4.2 that is. :)