User account creation filtered due to spam.

Bug 27757 - [4.1/4.2 Regression] Problems with direct access io
Summary: [4.1/4.2 Regression] Problems with direct access io
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libfortran (show other bugs)
Version: 4.1.0
: P3 normal
Target Milestone: 4.1.2
Assignee: Jerry DeLisle
URL:
Keywords:
Depends on:
Blocks: 19292
  Show dependency treegraph
 
Reported: 2006-05-24 14:57 UTC by Ray Nachlinger
Modified: 2006-05-30 23:02 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2006-05-27 02:35:03


Attachments
source and results (72.36 KB, application/octet-stream)
2006-05-24 14:58 UTC, Ray Nachlinger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ray Nachlinger 2006-05-24 14:57:13 UTC
I have a relatively stable, relatively large fortran program (500,000+ lines). This program
works correctly on linux, 64 bit linux, solaris, and windows when compiled with g77 3.4.4.

When I complile it with gfortran 4.1.0, 26 out of 384 of my regerssion tests fail because of i/o
errors. Basically, I have databases which are accesses via direct access, unformatted io.
All transactions pass through two routines: a writter and a reader. I have attached a tar file
which contains the two routines. Basically, they write two words at the beginning of each
record - the current numbers of "saus" (sort integers) and the record which continues this
record. Also attached is a file, trans, which shows the reads and writes. Each line in the
file contains the action (dba_read or dba_write), the fortran unit number, the record 
number, the number of short integers on this record, the continuation record, and finally
the record length. (The print statements that created this file are commentd out in the
source).

If you notice the last line in this file says that record 2958 on logical unit 15 has 1 short integer,
but 4 lines up, it has 122 and is continued on 2959. Notice that there have been writes
to this file, but at different record numbers.

I would really like to give you a better report, but the fact that 350 tests work properly says
that any attempt to create a simple test will probably fail (the code will work properly).
I will be happy to do anything you would like to help in the resolution of this problem. In
particular, it you would like for me to test any changes, please just send me the library
and I can get back to you within a couple of hours.
Comment 1 Ray Nachlinger 2006-05-24 14:58:58 UTC
Created attachment 11505 [details]
source and results
Comment 2 Jerry DeLisle 2006-05-25 00:51:47 UTC
I will take a look at this one.  This does seem odd.  Can you tell us what platform you are running on and what version of gfortran.

gfortran -v will tell us useful info.
Comment 3 Ray Nachlinger 2006-05-25 01:03:42 UTC
Subject: Re:  Problems with direct access io

On Wed, 25 May 2006, jvdelisle at gcc dot gnu dot org wrote:

> 
> 
> ------- Comment #2 from jvdelisle at gcc dot gnu dot org  2006-05-25 00:51 -------
> I will take a look at this one.  This does seem odd.  Can you tell us what
> platform you are running on and what version of gfortran.
> 
> gfortran -v will tell us useful info.
> 
> 

  Actually I included this information 4.1.0. The results submitted
  were run on an AMD x86_64 machine, but the exactly same thing 
  occurs on an Intel dual core P4 with Fedora Core 4 (a 32 bit
  OS). Thank you and if I can do anything to help, please let
  me know.

  Thanks,

  Ray
Comment 4 Jerry DeLisle 2006-05-25 01:20:42 UTC
If you are running a gfortran straight from the FC 4 distribution it probably does not have the latest fixes to the IO that we have in the 4.1 branch.  These have not been released yet.  Release 4.1.1 is being prepared now.

If you are up to it you could grab a gcc snapshot and build a 4.1.1 or 4.2 version and try it.  4.2 may not be very stable at the moment so a 4.1 snapshot latest would be a good bet.

Let me know what you think and we will keep trying to help you with this.
Comment 5 Ray Nachlinger 2006-05-25 02:17:30 UTC
Subject: Re:  Problems with direct access io

On Wed, 25 May 2006, jvdelisle at gcc dot gnu dot org wrote:

> 
> 
> ------- Comment #4 from jvdelisle at gcc dot gnu dot org  2006-05-25 01:20 -------
> If you are running a gfortran straight from the FC 4 distribution it probably
> does not have the latest fixes to the IO that we have in the 4.1 branch.  These
> have not been released yet.  Release 4.1.1 is being prepared now.
> 
> If you are up to it you could grab a gcc snapshot and build a 4.1.1 or 4.2
> version and try it.  4.2 may not be very stable at the moment so a 4.1 snapshot
> latest would be a good bet.
> 
> Let me know what you think and we will keep trying to help you with this.
> 

  I am really a big boy. Tried to get the latest snapshot but
  could not find it! Send url, I will make it, test it, and 
  report tomorrow.

  Thanks again,


  Ray
Comment 6 Jerry DeLisle 2006-05-25 05:08:49 UTC
http://gcc.gnu.org/mirrors.html
Comment 7 Jerry DeLisle 2006-05-27 02:35:03 UTC
The following patch causes no regressions.  I can't reproduce the problem here, but I have a hunch.

Ray, can you try this and see if it resolves the problem?

Thanks for tests and reports.

Index: io/transfer.c
===================================================================
*** io/transfer.c	(revision 114105)
--- io/transfer.c	(working copy)
*************** st_write_done (st_parameter_dt *dtp)
*** 2416,2421 ****
--- 2416,2425 ----
  	break;
        }
  
+   if (dtp->u.p.current_unit != NULL 
+       && dtp->u.p.current_unit->flags.access == ACCESS_DIRECT)
+     flush (dtp->u.p.current_unit->s);
+     
    free_format_data (dtp);
    free_ionml (dtp);
    if (dtp->u.p.scratch != NULL)
Comment 8 Jerry DeLisle 2006-05-27 05:10:06 UTC
Here is some good news.  I came up with a test case that fails with gfortran and passes with intel.  The bad news is that I have not fixed it yet.  At least I have something to work with now and its ugly.

program testdirect
  implicit none
  integer, dimension(100) :: a
  integer :: i,j,k,ier
  real    :: x
  
  a = 0
  call random_seed()
  open(unit=15,file="testdirectio",access="direct",form="unformatted",recl=4)
  do i=1,100
    call random_number(x)
    k= int(x * 100)+1
    a(i)=k
    write(unit=15, rec=k) k
  enddo
  do j=1,100
    read(unit=15, rec=a(j), iostat=ier) k
    if (ier.ne.0) then
      print *, "No Record:  ", j
    else
      print *, "Bad Record at ",a(j), k
    endif
  enddo
  close(unit=15, status="delete")
end program testdirect  
Comment 9 Ray Nachlinger 2006-05-27 12:21:05 UTC
Subject: Re:  Problems with direct access io


Good work, knew this would be hard!

Ray

On Sat, 27 May 2006, jvdelisle at gcc 
dot gnu dot org wrote:

> 
> 
> ------- Comment #8 from jvdelisle at gcc dot gnu dot org  2006-05-27 05:10 -------
> Here is some good news.  I came up with a test case that fails with gfortran
> and passes with intel.  The bad news is that I have not fixed it yet.  At least
> I have something to work with now and its ugly.
> 
> program testdirect
>   implicit none
>   integer, dimension(100) :: a
>   integer :: i,j,k,ier
>   real    :: x
> 
>   a = 0
>   call random_seed()
>   open(unit=15,file="testdirectio",access="direct",form="unformatted",recl=4)
>   do i=1,100
>     call random_number(x)
>     k= int(x * 100)+1
>     a(i)=k
>     write(unit=15, rec=k) k
>   enddo
>   do j=1,100
>     read(unit=15, rec=a(j), iostat=ier) k
>     if (ier.ne.0) then
>       print *, "No Record:  ", j
>     else
>       print *, "Bad Record at ",a(j), k
>     endif
>   enddo
>   close(unit=15, status="delete")
> end program testdirect  
> 
> 
> -- 
> 
> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27757
> 
> ------- You are receiving this mail because: -------
> You reported the bug, or are watching the reporter.
> 
> -------------------+  Spam Scoring Results   +------------------------
> Content analysis details:   (0.0 hits, 5.0 required)
>  0.0 BAYES_50               BODY: Bayesian spam probability is 40 to 60%
>                             [score: 0.5002]
> 
> -------------------+ End Spam Scoring Results +-----------------------
> 


Ultramarine, Inc.
http://www.ultramarine.com
Phone: 713-975-8146
Fax:   713-975-8179
Comment 10 Thomas Koenig 2006-05-27 19:10:58 UTC
This bug does not occur with g77, as this version of the
test program shows:

$ cat testdirect.f
      program testdirect
      implicit none
      integer a (100)
      integer  i,j,k,ier
      real x
      integer myrand(100)

      data myrand / 13, 9, 34, 41, 25, 98, 6, 12, 11, 44, 79, 3,
     &    64, 61, 77, 57, 59, 2, 92, 38, 71, 64, 31, 60, 28, 90, 26,
     &    97, 47, 26, 48, 96, 95, 82, 100, 90, 45, 71, 71, 67, 72,
     &    76, 94, 49, 85, 45, 100, 22, 96, 48, 13, 23, 40, 14, 76, 99,
     &    96, 90, 65, 2, 8, 60, 96, 19, 45, 1, 100, 48, 91, 20, 92,
     &    72, 81, 59, 24, 37, 43, 21, 54, 68, 31, 19, 79, 63, 41,
     &    42, 12, 10, 62, 43, 9, 30, 9, 54, 35, 4, 5, 55, 3, 94 /

      do i=1,100
         a(i) = 0
      end do
C     call random_seed()
      open(unit=15,file="testdirectio",access="direct",
     &     form="unformatted",recl=4)
      do i=1,100
         k = myrand(i)
         ! k= int(x * 100)+1
         a(i)=k
         write(unit=15, rec=k) k
      end do

      do j=1,100
         read(unit=15, rec=a(j), iostat=ier) k
         if (ier.ne.0) then
            print *, "No Record:  ", j
         else
            if (a(j) .ne. k) print *, "Bad Record at ",a(j), k
         endif
      enddo
      close(unit=15, status="delete")
      end
$ g77 testdirect.f
$ ./a.out
$ gfortran testdirect.f
$ ./a.out
 Bad Record at           13          14
 Bad Record at           34           0
 Bad Record at           25          26
 Bad Record at           98          99
 Bad Record at            6          11
 Bad Record at           11           0
 Bad Record at           44          45
 Bad Record at           64          65
 Bad Record at           61           0
 Bad Record at           77           0
 Bad Record at           57          59
 Bad Record at            2           0
 Bad Record at           38           0
 Bad Record at           71          72
 Bad Record at           64          65
 Bad Record at           60           0
 Bad Record at           28           0
 Bad Record at           90          98
 Bad Record at           26          34
 Bad Record at           97           0
 Bad Record at           47          48
 Bad Record at           26          34
 Bad Record at           96          97
 Bad Record at           95          96
 Bad Record at           82           0
 Bad Record at           90          98
 Bad Record at           45           0
 Bad Record at           71          72
 Bad Record at           71          72
 Bad Record at           67           0
 Bad Record at           76           0
 Bad Record at           49           0
 Bad Record at           85           0
 Bad Record at           45           0
 Bad Record at           22          23
 Bad Record at           96          97
 Bad Record at           13          14
 Bad Record at           23           0
 Bad Record at           40           0
 Bad Record at           14           0
 Bad Record at           76           0
 Bad Record at           99         100
 Bad Record at           96          97
 Bad Record at           90          98
 Bad Record at           65           0
 Bad Record at            2           0
 Bad Record at            8           0
 Bad Record at           60           0
 Bad Record at           96          97
 Bad Record at           45           0
$ g77 -v
Reading specs from /usr/lib/gcc/i486-linux-gnu/3.4.6/specs
Configured with: ../src/configure -v --enable-languages=c,c++,f77,pascal --prefix=/usr --libexecdir=/usr/lib --with-gxx-include-dir=/usr/include/c++/3.4 --enable-shared --with-system-zlib --enable-nls --without-included-gettext --program-suffix=-3.4 --enable-__cxa_atexit --enable-clocale=gnu --enable-libstdcxx-debug --with-tune=i686 i486-linux-gnu
Thread model: posix
gcc version 3.4.6 (Debian 3.4.6-1)
$ gfortran -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../../gcc/trunk/configure --prefix=/home/ig25 --enable-languages=c,fortran
Thread model: posix
gcc version 4.2.0 20060525 (experimental)
Comment 11 Ray Nachlinger 2006-05-27 20:53:51 UTC
Subject: Re:  Problems with direct access io



I believe I said that in the bug report.

On Sat, 27 May 2006, tkoenig at 
gcc dot gnu dot org wrote:

> 
> 
> ------- Comment #10 from tkoenig at gcc dot gnu dot org  2006-05-27 19:10 -------
> This bug does not occur with g77, as this version of the
> test program shows:
> 
> $ cat testdirect.f
>       program testdirect
>       implicit none
>       integer a (100)
>       integer  i,j,k,ier
>       real x
>       integer myrand(100)
> 
>       data myrand / 13, 9, 34, 41, 25, 98, 6, 12, 11, 44, 79, 3,
>      &    64, 61, 77, 57, 59, 2, 92, 38, 71, 64, 31, 60, 28, 90, 26,
>      &    97, 47, 26, 48, 96, 95, 82, 100, 90, 45, 71, 71, 67, 72,
>      &    76, 94, 49, 85, 45, 100, 22, 96, 48, 13, 23, 40, 14, 76, 99,
>      &    96, 90, 65, 2, 8, 60, 96, 19, 45, 1, 100, 48, 91, 20, 92,
>      &    72, 81, 59, 24, 37, 43, 21, 54, 68, 31, 19, 79, 63, 41,
>      &    42, 12, 10, 62, 43, 9, 30, 9, 54, 35, 4, 5, 55, 3, 94 /
> 
>       do i=1,100
>          a(i) = 0
>       end do
> C     call random_seed()
>       open(unit=15,file="testdirectio",access="direct",
>      &     form="unformatted",recl=4)
>       do i=1,100
>          k = myrand(i)
>          ! k= int(x * 100)+1
>          a(i)=k
>          write(unit=15, rec=k) k
>       end do
> 
>       do j=1,100
>          read(unit=15, rec=a(j), iostat=ier) k
>          if (ier.ne.0) then
>             print *, "No Record:  ", j
>          else
>             if (a(j) .ne. k) print *, "Bad Record at ",a(j), k
>          endif
>       enddo
>       close(unit=15, status="delete")
>       end
> $ g77 testdirect.f
> $ ./a.out
> $ gfortran testdirect.f
> $ ./a.out
>  Bad Record at           13          14
>  Bad Record at           34           0
>  Bad Record at           25          26
>  Bad Record at           98          99
>  Bad Record at            6          11
>  Bad Record at           11           0
>  Bad Record at           44          45
>  Bad Record at           64          65
>  Bad Record at           61           0
>  Bad Record at           77           0
>  Bad Record at           57          59
>  Bad Record at            2           0
>  Bad Record at           38           0
>  Bad Record at           71          72
>  Bad Record at           64          65
>  Bad Record at           60           0
>  Bad Record at           28           0
>  Bad Record at           90          98
>  Bad Record at           26          34
>  Bad Record at           97           0
>  Bad Record at           47          48
>  Bad Record at           26          34
>  Bad Record at           96          97
>  Bad Record at           95          96
>  Bad Record at           82           0
>  Bad Record at           90          98
>  Bad Record at           45           0
>  Bad Record at           71          72
>  Bad Record at           71          72
>  Bad Record at           67           0
>  Bad Record at           76           0
>  Bad Record at           49           0
>  Bad Record at           85           0
>  Bad Record at           45           0
>  Bad Record at           22          23
>  Bad Record at           96          97
>  Bad Record at           13          14
>  Bad Record at           23           0
>  Bad Record at           40           0
>  Bad Record at           14           0
>  Bad Record at           76           0
>  Bad Record at           99         100
>  Bad Record at           96          97
>  Bad Record at           90          98
>  Bad Record at           65           0
>  Bad Record at            2           0
>  Bad Record at            8           0
>  Bad Record at           60           0
>  Bad Record at           96          97
>  Bad Record at           45           0
> $ g77 -v
> Reading specs from /usr/lib/gcc/i486-linux-gnu/3.4.6/specs
> Configured with: ../src/configure -v --enable-languages=c,c++,f77,pascal
> --prefix=/usr --libexecdir=/usr/lib --with-gxx-include-dir=/usr/include/c++/3.4
> --enable-shared --with-system-zlib --enable-nls --without-included-gettext
> --program-suffix=-3.4 --enable-__cxa_atexit --enable-clocale=gnu
> --enable-libstdcxx-debug --with-tune=i686 i486-linux-gnu
> Thread model: posix
> gcc version 3.4.6 (Debian 3.4.6-1)
> $ gfortran -v
> Using built-in specs.
> Target: i686-pc-linux-gnu
> Configured with: ../../gcc/trunk/configure --prefix=/home/ig25
> --enable-languages=c,fortran
> Thread model: posix
> gcc version 4.2.0 20060525 (experimental)
> 
> 
> -- 
> 
> tkoenig at gcc dot gnu dot org changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |tkoenig at gcc dot gnu dot
>                    |                            |org
> OtherBugsDependingO|                            |19292
>               nThis|                            |
> 
> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27757
> 
> ------- You are receiving this mail because: -------
> You reported the bug, or are watching the reporter.
> 
> -------------------+  Spam Scoring Results   +------------------------
> Content analysis details:   (-2.6 hits, 5.0 required)
> -2.6 BAYES_00               BODY: Bayesian spam probability is 0 to 1%
>                             [score: 0.0000]
> 
> -------------------+ End Spam Scoring Results +-----------------------
> 


Ultramarine, Inc.
http://www.ultramarine.com
Phone: 713-975-8146
Fax:   713-975-8179
Comment 12 Jerry DeLisle 2006-05-28 00:40:39 UTC
I am confirming this is a regression.  The test program works fine on 4.0.2
Comment 13 Ray Nachlinger 2006-05-28 19:06:40 UTC
Subject: Re:  [4.1/4.2 Regression] Problems with direct
 access io

On Sat, 28 May 2006, jvdelisle at gcc dot gnu dot org wrote:

> 
> 
> ------- Comment #12 from jvdelisle at gcc dot gnu dot org  2006-05-28 00:40 -------
> I am confirming this is a regression.  The test program works fine on 4.0.2
> 


  Thanks, I thought it had worked before but could not find
  out when!

  Ray
Comment 14 Jerry DeLisle 2006-05-29 20:49:49 UTC
Here is the fix folks.  Unbelievable!

Index: io/unix.c
===================================================================
*** io/unix.c   (revision 114207)
--- io/unix.c   (working copy)
*************** fd_seek (unix_stream * s, gfc_offset off
*** 569,574 ****
--- 569,575 ----
      }

    s->physical_offset = s->logical_offset = offset;
+   s->active = 0;

    return (lseek (s->fd, offset, SEEK_SET) < 0) ? FAILURE : SUCCESS;
  }
Comment 15 Jerry DeLisle 2006-05-29 23:40:00 UTC
Subject: Bug 27757

Author: jvdelisle
Date: Mon May 29 23:39:53 2006
New Revision: 114220

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114220
Log:
2006-05-29  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/27757
	* io/unix.c (fd_seek): Set active to zero.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unix.c

Comment 16 Jerry DeLisle 2006-05-29 23:43:01 UTC
Subject: Bug 27757

Author: jvdelisle
Date: Mon May 29 23:42:55 2006
New Revision: 114221

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114221
Log:
2006-05-29  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/27757
	* gfortran.dg/direct_io_5.f90: New test.

Added:
    trunk/gcc/testsuite/gfortran.dg/direct_io_5.f90
Modified:
    trunk/gcc/testsuite/ChangeLog

Comment 17 Jerry DeLisle 2006-05-30 02:51:36 UTC
Subject: Bug 27757

Author: jvdelisle
Date: Tue May 30 02:51:26 2006
New Revision: 114227

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114227
Log:
2006-05-29  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/27757
	* io/unix.c (fd_seek): Set active to zero.

Modified:
    branches/gcc-4_1-branch/libgfortran/ChangeLog
    branches/gcc-4_1-branch/libgfortran/io/unix.c

Comment 18 Jerry DeLisle 2006-05-30 02:54:36 UTC
Subject: Bug 27757

Author: jvdelisle
Date: Tue May 30 02:54:29 2006
New Revision: 114228

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=114228
Log:
2006-05-29  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/27757
	* gfortran.dg/direct_io_5.f90: New test.

Added:
    branches/gcc-4_1-branch/gcc/testsuite/gfortran.dg/direct_io_5.f90
Modified:
    branches/gcc-4_1-branch/gcc/testsuite/ChangeLog

Comment 19 Jerry DeLisle 2006-05-30 23:02:11 UTC
Fixed on 4.1 and 4.1.  Ray also confirmed by separate email that the original application problem is fixed as well.
Comment 20 Jerry DeLisle 2006-05-30 23:02:56 UTC
and 4.2 that is.  :)