Reduced test case:

       program test
       dimension data(100)
       read(11,end=        1000 )data
 1000  continue
       backspace 11
       backspace 11
       end

I will have to check to see if this is legal code.

Well, since it relies on uninitialized variables, it's a bit hard to reproduce. The following code (I believe it is legal) reproducibly fails for me on i686-linux:

      integer dat(5)
      dat = (/ 0, 0, 0, 0, 18651460 /)
      write(11) dat,dat,dat
      rewind 11
      write(11) dat,dat,dat,dat
      rewind 11
      write(11) dat
      rewind 11
      write(11) dat
      rewind 11
      read(11) dat
      read(11,end=1008) dat
 1008 continue
      backspace 11
      write(11) dat
      backspace 11
      read(11) dat
      read(11,end=1011) dat
 1011 continue
      backspace 11
      backspace 11
      end

I think it has to do with array transfer, and overwriting the record markes with data. Adding Janne in CC list for opinion.

The testcase can be reduced further:

      integer dat(5)
      dat = (/ 0, 0, 0, 0, 1 /)
      write(11) dat,dat,dat,dat
      rewind 11
      write(11) dat
      read(11,end=1008) dat
 1008 continue
      backspace 11
      write(11) dat
      read(11,end=1011) dat
 1011 continue
      backspace 11
      backspace 11
      end

At line 13 of file testio2.f
Fortran runtime error: Invalid argument


From looking at the fort.11 file, I think the output is correct at every point of the program. Also note that if we comment two lines of this testcase, like that:

      integer dat(5)
      dat = (/ 0, 0, 0, 0, 1 /)
      write(11) dat,dat,dat,dat
      rewind 11
      write(11) dat
      read(11,end=1008) dat
 1008 continue
!      backspace 11
!      write(11) dat
      read(11,end=1011) dat
 1011 continue
      backspace 11
      backspace 11
      end

then we get another error message (which shouldn't happen):

At line 10 of file testio2.f
Fortran runtime error: Read past ENDFILE record

I don't know yet if these bugs have the same cause.

It doesn't have anything to do with array transfers. The following test program is equivalent to the one in #3, but uses the scalar transfer_integer instead of transfer_array:

program test3
  integer dat(5)
  dat = (/ 1, 2, 3, 4, 5 /)
  write(11)  dat(1), dat(2), dat(3), dat(4), dat(5), &
       dat(1), dat(2), dat(3), dat(4), dat(5), &
       dat(1), dat(2), dat(3), dat(4), dat(5), &
       dat(1), dat(2), dat(3), dat(4), dat(5)
  rewind 11
  write(11) dat(1), dat(2), dat(3), dat(4), dat(5)
  read(11,end=1008) dat(1), dat(2), dat(3), dat(4), dat(5)
1008 continue
  backspace 11
  write(11)   dat(1), dat(2), dat(3), dat(4), dat(5)
  read(11,end=1011)  dat(1), dat(2), dat(3), dat(4), dat(5)
1011 continue
  backspace 11
  backspace 11
end program test3

Depending on whether the two lines after 1008 continue are commented or not, it shows the same errors as the ones in #3.

There are two errors on this one. The first is not a regression (happens with 4.0.3):

$ cat a.f90
  integer :: i = 1
  open(11,status="replace",form="unformatted")
  read(11,end=1008) i
1008 continue
  read(11,end=1011) i
1011 continue
  end
$ gfortran a.f90 && ./a.out
At line 5 of file a.f90
Fortran runtime error: Read past ENDFILE record


The second one is a regression (doesn't happen with 4.0.3).

$ cat a.f90                              
  integer :: i = 1
  open(11,status="replace",form="unformatted")
  write(11) dat
  read(11,end=1008) i
1008 continue
  backspace 11
  backspace 11
  backspace 11
  end
$ gfortran a.f90 && ./a.out
At line 8 of file a.f90
Fortran runtime error: Invalid argument

PS: sorry Janne for hinting that it might have been related to your patch. Reducing this has been a bit painful.

I found the problem in the second one, which is fixed by the one-line patch:

Index: io/file_pos.c
===================================================================
--- io/file_pos.c       (revision 108353)
+++ io/file_pos.c       (working copy)
@@ -109,13 +109,14 @@
 
   length = sizeof (gfc_offset);
 
-  p = salloc_r_at (u->s, &length,
-                  file_position (u->s) - length);
+  /* length is modified by this function call, and most probably
+     set to zero.  */
+  p = salloc_r_at (u->s, &length, file_position (u->s) - length);
   if (p == NULL)
     goto io_error;
 
   memcpy (&m, p, sizeof (gfc_offset));
-  new = file_position (u->s) - m - 2*length;
+  new = file_position (u->s) - m - sizeof (gfc_offset) - length;
   if (sseek (u->s, new) == FAILURE)
     goto io_error;
 

(well, more than one line since I added a comment so that we don't do this mistake again). Since this does not fixed the other problem, I filed it separately under PR 25340.

This patch works for the reduced test case, but the original test case still fails for me -


[dranta:~/tests/gfortran-D] dir% gfortran -o write08 write08.f
[dranta:~/tests/gfortran-D] dir% write08
[dranta:~/tests/gfortran-D] dir% cat write08.f
      integer :: i = 1
      open(11,status="replace",form="unformatted")
      write(11) dat
      read(11,end=1008) i
1008  continue
      backspace 11
      backspace 11
      backspace 11
      end

[dranta:~/tests/gfortran-D] dir% gfortran -o testio2 testio2.f
[dranta:~/tests/gfortran-D] dir% testio2
At line 72 of file testio2.f
Fortran runtime error: Invalid argument
[dranta:~/tests/gfortran-D] dir% cat testio2.f
      program test
      dimension data(100)
       read(11,end=        1000 )data
 1000  continue
       backspace 11
       backspace 11
       backspace 11
       read(11,end=        1001 )data
 1001  continue
       backspace 11
       rewind 11
       backspace 11
       backspace 11
       backspace 11
       rewind 11
       write(11)data
       backspace 11
       backspace 11
       backspace 11
       backspace 11
       backspace 11
       backspace 11
       rewind 11
       backspace 11
       write(11)data
       write(11)data
       write(11)data
       backspace 11
       read(11,end=        1002 )data
 1002  continue
       backspace 11
       rewind 11
       backspace 11
       read(11,end=        1003 )data
 1003  continue
       write(11)data
       write(11)data
       write(11)data
       rewind 11
       read(11,end=        1004 )data
 1004  continue
       rewind 11
       rewind 11
       read(11,end=        1005 )data
 1005  continue
       backspace 11
       write(11)data
       backspace 11
       rewind 11
       backspace 11
       rewind 11
       read(11,end=        1006 )data
 1006  continue
       rewind 11
       write(11)data
       backspace 11
       read(11,end=        1007 )data
 1007  continue
       read(11,end=        1008 )data
 1008  continue
       backspace 11
       read(11,end=        1009 )data
 1009  continue
       backspace 11
       write(11)data
       backspace 11
       read(11,end=        1010 )data
 1010  continue
       read(11,end=        1011 )data
 1011  continue
       backspace 11
       backspace 11
      stop
      end
[dranta:~/tests/gfortran-D] dir% gfortran --v
Using built-in specs.
Target: powerpc-apple-darwin8.3.0
Configured with: ./configure --prefix=/Users/dir/gfortran --enable-languages=c,f95
Thread model: posix
gcc version 4.2.0 20051212 (experimental)
[dranta:~/tests/gfortran-D] dir% 

Looking into unformatted_backspace, there does not seem to be anything there to prevent "u->last_record" from going negative - may be that is problem.

I generated random I/O sequences doing 100,000 tests at each I/O operation count staring at five here is the shortest one to fail (somewhere in the eights) -

[dranta:~/tests/gfortran-D] dir% gfortran -o write10 write10.f
[dranta:~/tests/gfortran-D] dir% write10
At line 18 of file write10.f
Fortran runtime error: Invalid argument
[dranta:~/tests/gfortran-D] dir% cat write10.f
      integer :: i = 1
      dimension data(500)
      do 10 i=1,500
      data(i)=i
   10 continue
      open(unit=11,status='scratch',form='unformatted')
       write(11)data
       write(11)data
       rewind 11
       write(11)data
       backspace 11
       read(11,end=        1000 )data
 1000  continue
       read(11,end=        1001 )data
 1001  continue
       backspace 11
       backspace 11
       backspace 11
      close(11)
      end

Removing the constraint of no read after write gives a slightly shorter one -

[dranta:~/tests/gfortran-D] dir%  gfortran -o write11 write11.f
[dranta:~/tests/gfortran-D] dir% write11
At line 14 of file write11.f
Fortran runtime error: Invalid argument
[dranta:~/tests/gfortran-D] dir% cat write11.f
      integer :: i = 1
      dimension data(5)
      do 10 i=1,5
      data(i)=i
   10 continue
      open(unit=11,status='scratch',form='unformatted')
       write(11)data
       read(11,end=        1000 )data
 1000  continue
       backspace 11
       rewind 11
       read(11,end=        1001 )data
 1001  continue
       read(11,end=        1002 )data
 1002  continue
      close(11)
      end

When the array of size 2045 or larger the error goes away -

[dranta:~/tests/gfortran-D] dir% gfortran -o write11 write11.f
[dranta:~/tests/gfortran-D] dir% write11
At line 14 of file write11.f
Fortran runtime error: Invalid argument
[dranta:~/tests/gfortran-D] dir% cat write11.f
      integer :: i = 1
      dimension data(2044)
      do 10 i=1,5
      data(i)=i
   10 continue
      open(unit=11,status='scratch',form='unformatted')
       write(11)data
       read(11,end=        1000 )data
 1000  continue
c      backspace 11
       rewind 11
       read(11,end=        1001 )data
 1001  continue
       read(11,end=        1002 )data
 1002  continue
      close(11)
      end

[dranta:~/tests/gfortran-D] dir% gfortran -o write11 write11.f
[dranta:~/tests/gfortran-D] dir% write11
[dranta:~/tests/gfortran-D] dir% cat write11.f
      integer :: i = 1
      dimension data(2045)
      do 10 i=1,5
      data(i)=i
   10 continue
      open(unit=11,status='scratch',form='unformatted')
       write(11)data
       read(11,end=        1000 )data
 1000  continue
c      backspace 11
       rewind 11
       read(11,end=        1001 )data
 1001  continue
       read(11,end=        1002 )data
 1002  continue
      close(11)
      end

Removing patch keyword and reassigning it to nobody as my patch doesn't fix it and I won't have free time before january. Sorry for not doing this earlier.

Fortran problems are not release critical: P5.

After tracing the errors for a while, it became clean that "active" pointer into the read/write buffer was not being correctly updated. Adding one line ( s->active=0; ) to the bottom of routine "fd_truncate" in file "unix.c" fixes these errors. All the examples here run OK with this patch and I built 16 of my programs and ran their many test problems with no errors. I tried to run the fortran testsuite with "make check-gfortran", but check-gfortran is not in the makefile. It would be nice if just fortran testsuite could be run.



*** unix.c      Wed Dec 21 08:11:12 2005
--- unixoriginal.c      Wed Dec 21 08:10:27 2005
***************
*** 601,608 ****
      }
  
    s->physical_offset = s->file_length = s->logical_offset;
-   
-   s->active=0;
  
    return SUCCESS;
  }
--- 601,606 ----

(In reply to comment #14)
> I tried to run the
> fortran testsuite with "make check-gfortran", but check-gfortran is not
> in the
> makefile. It would be nice if just fortran testsuite could be run.
> 

Dale, move into the gcc/ directory of your build directory and run
the "make check-gfortran" command.  Yes, it's an unexpected PITA,
but it should work.

I down loaded gfortran and built it on the Macintosh with -
 
svn -q checkout svn://gcc.gnu.org/svn/gcc/trunk gcc
cd gcc
configure --prefix=/Users/dir/gfortran --enable-languages=c,f95
make -j 4
make install

When, I try the "check-gfortran" in the directory 'gcc' where I did the configure,make -j 4,make install, I get -

[dranta:~/gfortran/gcc] dir% make check-gfortran
make: *** No rule to make target `check-gfortran'.  Stop.

When I move to the directory 'gcc' below that, I get -

[dranta:~/gfortran/gcc] dir% cd gcc
[dranta:~/gfortran/gcc/gcc] dir% make check-gfortran
make: *** No rule to make target `check-gfortran'.  Stop.
[dranta:~/gfortran/gcc/gcc] dir% 

Is there another directory called gcc or does it need to be build some other way ?

(In reply to comment #16)
> When, I try the "check-gfortran" in the directory 'gcc' where I did the
> configure,make -j 4,make install, I get

Dale, you should really stop building your compiler inside the source tree. This isn't very actively supported (actually, it breaks every so often) and it makes everything harder. For example, I usually create a directory parent directory build/, including build/gcc which is the source tree and build/ibin which is the build. Inside build/ibin, do:

../gcc/configure [...] && make && make install

What Steve is saying is that you have to go inside the build/ibin/gcc directory and run make check-gfortran *there*.

> Is there another directory called gcc or does it need to be build some other
> way ?

I guess with your scheme, you need to go inside gcc/gcc/ and run make check-gfortran. But please, don't do in-tree builds :)

This is great!  I have regression tested and NIST tested on i686 and all pass.  I have not tried some of the problem cases yet, but will do later tonight.

I was just getting ready to start working on this one and I am tickled pink that it appears to be a fix.

Who will approve and who will commit this.  I will be happy to commit with a test case if needed.

Ran some test cases.  The second example in Comment #3 fails. With or without FX patch given in Comment #6

At line 10 of file back3.f
Fortran runtime error: Read past ENDFILE record

Dale, are you getting this?

Thanks FX - You gave me the first good explaination of how it should be done.

The second example in comment 3 should fail as it does - since it does try read beyond the end of file. As far as I can tell, the FX patch does not fix any of the test cases on the Macintosh and I think that it actually is incorrect. It should actually read -

   new = file_position (u->s) - m - 2*sizeof (gfc_offset);

as "2*sizeof (gfc_offset)" is the size of the green words before and after the data and m is the size of the data. (m+2*sizeof (gfc_offset)) is the total size of the binary record that is being skipped backwards over.

(In reply to comment #21)
>    new = file_position (u->s) - m - 2*sizeof (gfc_offset);
> 
> as "2*sizeof (gfc_offset)" is the size of the green words before and after the
> data and m is the size of the data. (m+2*sizeof (gfc_offset)) is the total size
> of the binary record that is being skipped backwards over.

Did you test that patch? Does it work? I would be very surprised if it did. You need to consider that the previous read (or salloc_r_at, to be exact) might or might not read all length bytes. So, after that call, length might be sizeof(gfc_offset), or it might be 0, or it might be something in between.

You need to do something that takes into account all these cases, and that's what I thought my patch did. I don't have time now to make all the reasonning again, but I just wanted to underline that issue so that you're aware of it.

What is happening here is actually quite simple. The program reads the green word for the previous record from the file from location "file_position (u->s) - length)", that word gives the length of the previous record in bytes. It then pulls that number out of the buffer and puts it into correct integer form for the current machine, that is the number 'm'. The length of a binary record is always the length of the two green words plus the length of the data or (m + 2*sizeof (gfc_offset)). The position before the currect record is then file_position (u->s) - m - 2*sizeof (gfc_offset).
    I did not change the current coding and it works for all of my tests. Any time that p is not NULL, length returns the number of bytes read, which should be sizeof (gfc_offset). If length is not equal to sizeof (gfc_offset), there is an error in the file data - so may be that should be put in as an error test.

Also, I just noticed that, if length != sizeof (gfc_offset) then 'm' has not been correctly read - so the routine should quit. Using a bad number to calculate 'new' will not likely useful.

The failure in the second example in comment #3 is that the reading past the record is not being caught by the end specifier in the read statement.  It should go to 1011 and continue.  This is a different bug.  I will try to hunt this down this weekend unless Dale does.  Let me know off bugzilla.

I ran the "check-gfortran" tests before and after putting in the two changes that I suggested on the Macintosh and the results were identical and my real programs now run Ok with the changes, but someone is currectly making changes to the fortran io routines and some of the tests given here are failing in new ways -

Test Run By dir on Thu Dec 22 09:47:44 2005
Native configuration is powerpc-apple-darwin8.3.0

                === gfortran tests ===

Schedule of variations:
    unix

Running target unix
Using /usr/local/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /usr/local/share/dejagnu/config/unix.exp as generic interface file for target.
Using /Users/dir/gfortran/gcc/gcc/testsuite/config/default.exp as tool-and-target-specific interface file.
Running /Users/dir/gfortran/gcc/gcc/testsuite/gfortran.dg/dg.exp ...
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
FAIL: gfortran.dg/large_real_kind_2.F90 execution test
Running /Users/dir/gfortran/gcc/gcc/testsuite/gfortran.dg/vect/vect.exp ...
Running /Users/dir/gfortran/gcc/gcc/testsuite/gfortran.fortran-torture/compile/compile.exp ...
Running /Users/dir/gfortran/gcc/gcc/testsuite/gfortran.fortran-torture/execute/execute.exp ...

                === gfortran Summary ===

# of expected passes            10985
# of unexpected failures        8
# of expected failures          7
# of unsupported tests          42

In the second example in comment #3, the first read reads the end of file and the end=1008 works correctly. On the second read, the end of file has been passed so that the end=1011 does not and should not catch it - it is an error condition and a correct error message is printed.  If err=1011 was used, then the error message should not be printed and the programs should jump to 1011 with out comment.

On i686-pc-linux-gnu:

$ cat back3.f
      integer dat(5)
      dat = (/ 0, 0, 0, 0, 1 /)
      write(11) dat,dat,dat,dat
      rewind 11
      write(11) dat
      read(11,end=1008) dat
      call abort()
 1008 continue
!      backspace 11
!      write(11) dat
      read(11,end=1011) dat
      print *, 'failed'
 1011 continue
      backspace 11
      backspace 11
      end

$ gfc back3.f
$ ./a.out
At line 11 of file back3.f
Fortran runtime error: Read past ENDFILE record
$ ifc back3.f
$ ./a.out
$
gfortran and ifort give different results with those two lines commented out.

The other cases work fine.

Dale, do you have copyright assignment and/or commit priviliges?  Also, I have gone to comp.lang.fortran for an interpretation on my comment #28.  I see what you are saying and I see what ifort is doing.  Ifort is usually right, but not always, so we will se what others say.  BTW Good work.

Based on feedback on comp.lang.fortran, the code in #28 is illegal and so the behavior is acceptable and probably appropriate.  Based on this I believe we have a working solution to this bug.  We just need to get it committed.

Thanks Dale.

  Hi Jerry,

  Would you go ahead and commit this ? A test case something like that in Comment #9 shows the problem before and the fix after or may be you have a better one from NIST. The change suggest in comment #14 fixes the problem that started all of this for me. There are other bugs, that I have hit looking into this, but I think that they should be in seperate bug reports.

Yes, I will work this up with a proper test case and submit for approval.

Subject: Bug 25139

Author: jvdelisle
Date: Wed Dec 28 20:58:08 2005
New Revision: 109122

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=109122
Log:
2005-12-28  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/25139
	* io/unix.c (fd_truncate): Set s->active to zero.
	PR libgfortran/25510
	* libgfortran.h: Add ERROR_INTERNAL and ERROR_INTERNAL_UNIT.
	* runtime/error.c (translate_error): Add messages for new errors.
	* io/list_read.c (next_char): Use new errors.
	* io/transfer.c (next_record_r) (next_record_w): Use new errors.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unix.c
    trunk/libgfortran/libgfortran.h
    trunk/libgfortran/runtime/error.c

Subject: Bug 25139

Author: jvdelisle
Date: Wed Dec 28 21:12:52 2005
New Revision: 109123

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=109123
Log:
2005-12-28  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/25139
	* gfortran.dg/backspace_2.f: New test.

Added:
    trunk/gcc/testsuite/gfortran.dg/backspace_2.f
Modified:
    trunk/gcc/testsuite/ChangeLog

Subject: Bug 25139

Author: jvdelisle
Date: Sun Jan  1 03:49:00 2006
New Revision: 109212

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=109212
Log:
2005-12-31  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/25139
	* io/unix.c (fd_truncate): Set s->active to zero.
	PR libgfortran/25510
	* libgfortran.h: Add ERROR_INTERNAL and ERROR_INTERNAL_UNIT.
	* runtime/error.c (translate_error): Add messages for new errors.
	* io/list_read.c (next_char): Use new errors.
	* io/transfer.c (next_record_r) (next_record_w): Use new errors.

Modified:
    branches/gcc-4_1-branch/libgfortran/ChangeLog
    branches/gcc-4_1-branch/libgfortran/io/list_read.c
    branches/gcc-4_1-branch/libgfortran/io/transfer.c
    branches/gcc-4_1-branch/libgfortran/io/unix.c
    branches/gcc-4_1-branch/libgfortran/libgfortran.h
    branches/gcc-4_1-branch/libgfortran/runtime/error.c

Subject: Bug 25139

Author: jvdelisle
Date: Sun Jan  1 03:53:12 2006
New Revision: 109213

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=109213
Log:
2005-12-31  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR libgfortran/25139
	* gfortran.dg/backspace_2.f: New test.

Added:
    branches/gcc-4_1-branch/gcc/testsuite/gfortran.dg/backspace_2.f
Modified:
    branches/gcc-4_1-branch/gcc/testsuite/ChangeLog

Fixed on 4.1 and 4.2