User account creation filtered due to spam.

Bug 30009 - [4.1 only] Unformatted reads exceeding storage units gives EOF instead of ERR
Summary: [4.1 only] Unformatted reads exceeding storage units gives EOF instead of ERR
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libfortran (show other bugs)
Version: 4.3.0
: P3 normal
Target Milestone: 4.2.0
Assignee: Not yet assigned to anyone
URL:
Keywords: diagnostic
: 30010 (view as bug list)
Depends on: 29568
Blocks: 19292
  Show dependency treegraph
 
Reported: 2006-11-28 21:34 UTC by Tobias Burnus
Modified: 2007-01-02 04:48 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Known to work: 4.2.0 4.3.0
Known to fail: 4.1.0
Last reconfirmed: 2006-11-28 22:23:23


Attachments
first attempt at a patch (1.82 KB, patch)
2006-12-03 21:02 UTC, Thomas Koenig
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Burnus 2006-11-28 21:34:21 UTC
See http://gcc.gnu.org/ml/fortran/2006-11/msg00708.html
Reported by Chris Talley.

Tested with current 4.3 and also with 4.1.

Testcase, see URL; essential parts of the test case (form="unformatted"):
       WRITE(1) 1
       WRITE(1) 2, 3, 4
       WRITE(1) 5, 6, 7
       REWIND(1)
       READ(1,ERR=10) I,J,K
The read should read "1" of the first record and then "2" and "3" of the second record. gfortran however reads "1", hits then the end of the record and gives an end-of-file error.

Note that none of the end-of-file conditions given in 9.10 of Fortran 2003 (cf. also 9.4.1.5f. of Fortran 95) is met. Note further that an end-of-record condition cannot occure for unformatted files.
Comment 1 Brooks Moses 2006-11-28 21:49:31 UTC
*** Bug 30010 has been marked as a duplicate of this bug. ***
Comment 2 Thomas Koenig 2006-11-28 22:23:23 UTC
(In reply to comment #0)
> See http://gcc.gnu.org/ml/fortran/2006-11/msg00708.html
> Reported by Chris Talley.
> 
> Tested with current 4.3 and also with 4.1.
> 
> Testcase, see URL; essential parts of the test case (form="unformatted"):
>        WRITE(1) 1
>        WRITE(1) 2, 3, 4
>        WRITE(1) 5, 6, 7
>        REWIND(1)
>        READ(1,ERR=10) I,J,K
> The read should read "1" of the first record and then "2" and "3" of the second
> record.

I don't think so.

F 2003, 9.5.3.4.1 (Unformatted data transfer):

# On input from a file connected for sequential or direct access,
# the number of file storage units required by the input list shall
# be less than or equal to the number of file storage units in the record.

This is not a constraint, so the burden is on the programmer.
In other words, the program is non-conforming.  We are definitely
not required to read past the end of the current record.

> gfortran however reads "1", hits then the end of the record and gives
> an end-of-file error.

End of file is misleading, true.  In the "Intel subrecord" patch
(PR 29568), an error condition is raised for this case.  I have

Changing keyword to "diagnostic", and assigning to myself.
Comment 3 Tobias Burnus 2006-11-28 22:58:13 UTC
(In reply to comment #2)
> > The read should read "1" of the first record and then "2" and "3" of the
> > second record.
> I don't think so.

You are of cause right. I should have really read

> F 2003, 9.5.3.4.1 (Unformatted data transfer):

more carefully (I did twice and missed that part). The bug is really that if one reads too much data END= and not ERR= is triggered.

A quick test with ifort shows that it outputs the message "input statement requires too much data" if one removes ERR=.
Comment 4 Thomas Koenig 2006-12-02 21:56:17 UTC
g77 gets this right:

$ cat eor2.f
      program eor
      WRITE(1) 1
      REWIND(1)
      READ(1,ERR=10) I,J,K
      print *,"no error"
      stop
 10   print *,"error value"
      end
$ g77 eor2.f && ./a.out
 error value

Comment 5 Jerry DeLisle 2006-12-02 23:31:57 UTC
Please note the subtle difference with this case while working this:

       WRITE(1) 1
       REWIND(1)
       READ(1,ERR=10) I
       READ(1,END=10) J
       print *,"no error"
       stop
 10    print *,"error value"
       end

This is a true EOF condition and gfortran gets that right.  Ifort and gfortran agree on this behavior as well.

Also note that EOF and EOR are not the same as ERR.  ERR would imply something more serious then EOF or EOR which would be considered normal things to encounter, like reading until you find the end of a file and then stopping as opposed to a bad sector on a disk.

Comment 6 Tobias Burnus 2006-12-03 08:32:31 UTC
For
       READ(1,ERR=10) J ! Read beyond EOF
there are two possible implementations one finds:
  ifort: forrtl: severe (24): end-of-file during read
  gfortran: Fortran runtime error: End of file
  g95: Internal Error: EOF condition not handled-- END= tag needed
  sunf95: End-of-file:  -1
or
  NAG f95: error value
  g77:  error value
Comment 7 Thomas Koenig 2006-12-03 14:06:25 UTC
(In reply to comment #6)
> For
>        READ(1,ERR=10) J ! Read beyond EOF
> there are two possible implementations one finds:
>   ifort: forrtl: severe (24): end-of-file during read

Really?  With ifort 8.1, I get

$ cat end-of-file.f
       WRITE(1) 1
       REWIND(1)
       READ(1) I
       READ(1,END=10) J
       print *,"no error"
       stop
 10    print *,"error value"
       end

$ ifort -v && ifort end-of-file.f && ./a.out
Version 8.1
 error value

I do think this is the correct behavior, and I'll take care
to maintain this behavior.
Comment 8 Tobias Burnus 2006-12-03 14:17:31 UTC
> >        READ(1,ERR=10) J ! Read beyond EOF
> >   ifort: forrtl: severe (24): end-of-file during read
> 
> Really?
Yes, really. Note: END /= ERR in the two examples.

> With ifort 8.1, I get
>        READ(1,END=10) J
>  error value
Comment 9 Thomas Koenig 2006-12-03 18:56:21 UTC
I've looked at the F 2003 standard, and at least there
the wording is clear:

9.10:

# An end-of-file condition occurs in the following cases:
# 
# (1) When an endfile record is encountered during reading of a file
#     for sequential access.

This is not the case here, when we try to read too many items from a record.

# (2) When an attempt is made to read beyond the end of an internal file.
# 
# (3) When an attempt is made to read beyond the end of a stream file.

So we need an error condition here.
Comment 10 Thomas Koenig 2006-12-03 21:02:23 UTC
Created attachment 12734 [details]
first attempt at a patch

This should fix this PR, and PR 30056 as well.
Comment 11 Jerry DeLisle 2006-12-04 02:18:43 UTC
Patch reviewed and tested.  Looks good to go.  I assume you tested the part of the corrupted file somehow.  I suppose we could right a test case uses stream I/O to build a bogus file and then try to read it and confirm that error.  That could be another testcase.
Comment 12 patchapp@dberlin.org 2006-12-05 19:01:03 UTC
Subject: Bug number PR 30009

A patch for this bug has been added to the patch tracker.
The mailing list url for the patch is http://gcc.gnu.org/ml/gcc-patches/2006-12/msg00334.html
Comment 13 Thomas Koenig 2006-12-06 19:26:01 UTC
Subject: Bug 30009

Author: tkoenig
Date: Wed Dec  6 19:25:44 2006
New Revision: 119592

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=119592
Log:
2006-12-06  Thomas Koenig  <Thomas.Koenig@online.de>

	PR libfortran/30009
	PR libfortran/30056
	* gfortran.dg/read_eof_4.f90:  Add tests.
	* gfortran.dg/readwrite_unf_direct_eor_1.f90:  New test.
	* gfortran.dg/unf_read_corrupted_1.f90: New test.

2006-12-06  Thomas Koenig  <Thomas.Koenig@online.de>

	PR libfortran/30009
	PR libfortran/30056
	* libgfortran.h:  Add ERROR_CORRUPT_FILE to error_codes.
	* runtime/error.c (translate_error):  Add handling for
	ERROR_CORRUPT_FILE.
	* io/transfer.c (read_block_direct):  Add comment about
	EOR for stream files.
	Remove test for no bytes left for direct access files.
	Generate an ERROR_SHORT_RECORD if the read was short.
	For unformatted sequential files:  Check endfile condition.
	Remove test for no bytes left.  End of file here means
	that the file structure has been corrupted.  Pre-position
	the file for the next record in case of error.
	(write_buf):  Whitespace fix.  Subtract the number of bytes
	written from bytes_left.


Added:
    trunk/gcc/testsuite/gfortran.dg/readwrite_unf_direct_eor_1.f90
    trunk/gcc/testsuite/gfortran.dg/unf_read_corrupted_1.f90
Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gfortran.dg/read_eof_4.f90
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/libgfortran.h
    trunk/libgfortran/runtime/error.c

Comment 14 Thomas Koenig 2006-12-10 22:16:45 UTC
Subject: Bug 30009

Author: tkoenig
Date: Sun Dec 10 22:16:14 2006
New Revision: 119712

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=119712
Log:
2006-12-10  Thomas Koenig  <Thomas.Koenig@online.de>

	Backport from mainline
	PR libfortran/29568
	* gfortran.dg/convert_implied_open.f90:  Change to
	new default record length.
	* gfortran.dg/unf_short_record_1.f90:  Adapt to
	new error message.
	* gfortran.dg/unformatted_subrecords_1.f90:  New test.
	PR libfortran/30009
	PR libfortran/30056
	* gfortran.dg/read_eof_4.f90:  Add tests.
	* gfortran.dg/readwrite_unf_direct_eor_1.f90:  New test.
	* gfortran.dg/unf_read_corrupted_1.f90: New test.

2006-12-10  Thomas Koenig  <Thomas.Koenig@online.de>

	PR libfortran/29568
	* gfortran.h (gfc_option_t):  Add max_subrecord_length.
	(top level): Define MAX_SUBRECORD_LENGTH.
	* lang.opt:  Add option -fmax-subrecord-length=.
	* trans-decl.c:  Add new function set_max_subrecord_length.
	(gfc_generate_function_code): If we are within the main
	program and max_subrecord_length has been set, call
	set_max_subrecord_length.
	* options.c (gfc_init_options):  Add defaults for
	max_subrecord_lenght, convert and record_marker.
	(gfc_handle_option):  Add handling for
	-fmax_subrecord_length.
	* invoke.texi:  Document the new default for
	-frecord-marker=<n>.

2006-12-10  Thomas Koenig  <Thomas.Koenig@online.de>

	PR libfortran/29568
	* libgfortran/libgfortran.h (compile_options_t):  Add
	record_marker. (top level):  Define GFC_MAX_SUBRECORD_LENGTH.
	* runtime/compile_options.c (set_record_marker):  Change
	default to four-byte record marker.
	(set_max_subrecord_length):  New function.
	* runtime/error.c (translate_error):  Change error message
	for short record on unformatted read.
	* io/io.h (gfc_unit):  Add recl_subrecord, bytes_left_subrecord
	and continued.
	* io/file_pos.c (unformatted_backspace):  Change default of record
	marker size to four bytes.  Loop over subrecords.
	* io/open.c:  Default recl is max_offset.  If
	compile_options.max_subrecord_length has been set, set set
	u->recl_subrecord to its value, to the maximum value otherwise.
	* io/transfer.c (top level):  Add prototypes for us_read, us_write,
	next_record_r_unf and next_record_w_unf.
	(read_block_direct):  Separate codepaths for unformatted direct
	and unformatted sequential.  If a recl has been set by the
	user, use the number of bytes left for the record if it is smaller
	than the read request.  Loop over subrecords.  Set an error if the
	user has set a recl and the read was short.
	(write_buf):  Separate codepaths for unformatted direct and
	unformatted sequential. If a recl has been set by the
	user, use the number of bytes left for the record if it is smaller
	than the read request.  Loop over subrecords.  Set an error if the
	user has set a recl and the read was short.
	(us_read):  Add parameter continued (to indicate that bytes_left
	should not be intialized).  Change default of record marker size
	to four bytes. Use subrecord.  If the subrecord length is smaller than
	zero, this indicates a continuation.
	(us_write):  Add parameter continued (to indicate that the continued
	flag should be set).  Use subrecord.
	(pre_position):  Use 0 for continued on us_write and us_read calls.
	(skip_record):  New function.
	(next_record_r_unf):  New function.
	(next_record_r):  Use next_record_r_unf.
	(write_us_marker):  Default size for record markers is four bytes.
	(next_record_w_unf):  New function.
	(next_record_w):  Use next_record_w_unf.
	PR libfortran/30009
	PR libfortran/30056
	* libgfortran.h:  Add ERROR_CORRUPT_FILE to error_codes.
	* runtime/error.c (translate_error):  Add handling for
	ERROR_CORRUPT_FILE.
	* io/transfer.c (read_block_direct):  Add comment about
	EOR for stream files.
	Remove test for no bytes left for direct access files.
	Generate an ERROR_SHORT_RECORD if the read was short.
	For unformatted sequential files:  Check endfile condition.
	Remove test for no bytes left.  End of file here means
	that the file structure has been corrupted.  Pre-position
	the file for the next record in case of error.
	(write_buf):  Whitespace fix.  Subtract the number of bytes
	written from bytes_left.


Added:
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/readwrite_unf_direct_eor_1.f90
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/unf_read_corrupted_1.f90
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/unformatted_subrecord_1.f90
Modified:
    branches/gcc-4_2-branch/gcc/fortran/ChangeLog
    branches/gcc-4_2-branch/gcc/fortran/gfortran.h
    branches/gcc-4_2-branch/gcc/fortran/lang.opt
    branches/gcc-4_2-branch/gcc/fortran/options.c
    branches/gcc-4_2-branch/gcc/fortran/trans-decl.c
    branches/gcc-4_2-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/convert_implied_open.f90
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/read_eof_4.f90
    branches/gcc-4_2-branch/gcc/testsuite/gfortran.dg/unf_short_record_1.f90
    branches/gcc-4_2-branch/libgfortran/ChangeLog
    branches/gcc-4_2-branch/libgfortran/io/file_pos.c
    branches/gcc-4_2-branch/libgfortran/io/io.h
    branches/gcc-4_2-branch/libgfortran/io/open.c
    branches/gcc-4_2-branch/libgfortran/io/transfer.c
    branches/gcc-4_2-branch/libgfortran/libgfortran.h
    branches/gcc-4_2-branch/libgfortran/runtime/compile_options.c
    branches/gcc-4_2-branch/libgfortran/runtime/error.c

Comment 15 Thomas Koenig 2006-12-15 22:26:59 UTC
I have had a look at this for 4.1, and the effort
to backport this patch and the one for PR 30056 will
be fairly large.

Because 4.1 has neither stream I/O nor the subrecod
patch, fixing would require a substantial rewrite
of the patch.

Comments?  Does anybody feel this is important?
Comment 16 Jerry DeLisle 2006-12-16 00:34:51 UTC
I think I would not risk potential further breakage and leave 4.1 alone.  4.2 will be coming along soon enough.
Comment 17 Thomas Koenig 2006-12-16 08:18:14 UTC
I'm leaving this open for now, but
unassigning myself.  If anybody wants
to tackle this for 4.1, feel free.
Comment 18 Jerry DeLisle 2007-01-02 04:48:48 UTC
Closing, we need to focus resources on current trunk if we can.