Hi, the attached problem showed up after PR 30918 got fixed. (Thanks, Jerry!) When reading a namelist again and again, even after EOF, then suddenly the IOSTAT values jumps from -1 to 5008. This does not happen with other compilers. The namelist file fort.10 is the same as in PR 30918. Output of test run: NAMELIST /REPORT/ after iteration 1: &REPORT TYPE=SYNOP , USE=active , MAX_PROC= 20, / Number of reports: 1 Last status = -1 Number of reports: 0 Last status = -1 Number of reports: 0 Last status = 5008 Cheers, -ha
Created attachment 13148 [details] Demo code The namelist file is here: http://gcc.gnu.org/bugzilla/attachment.cgi?id=13088
The value 5008 is listed in libgfortran.h as ERROR_ENDFILE. The -1 corresponds to ERROR_END. So, the return value of 5008 is telling you that you are trying to (initiate a?) read beyond the end of the file, which you admit to doing. AFAIK, the IOSTAT values aren't documented in gfortran.texi. Someday, someone will get around to documenting them.
(In reply to comment #2) > The value 5008 is listed in libgfortran.h as ERROR_ENDFILE. The > -1 corresponds to ERROR_END. So, the return value of 5008 is > telling you that you are trying to (initiate a?) read beyond > the end of the file, which you admit to doing. I actually do not want to read beyond the end of file, I want gfortran to stay at the end of the file when trying to read the next record or namelist. All other compilers I have checked (xlf, ifort 7.x-9.x, g95) stay at the end of file. I find their behavior consistent with the F2003 standard, section 9.10.4: Execution of an input/output statement containing the IOSTAT= specifier causes the scalar-int-variable in the IOSTAT= specifier to become defined with (3) The processor-dependent negative integer value of the constant IOSTAT_END (13.8.2.5) if an end-of-file condition occurs and no error condition occurs, or (...) > AFAIK, the IOSTAT values aren't documented in gfortran.texi. > Someday, someone will get around to documenting them. The standard separates end-of-file, end-of-record, and error conditions. I don't think that EOF is necessarily an error condition. It would be nice if gfortran would conform with these other compilers.
(In reply to comment #3) > All other compilers I have checked (xlf, ifort 7.x-9.x, g95) > stay at the end of file. I stand corrected with regard to xlf. It returns a documented "recoverable error" condition. I have to find out why the code in question that lead to the problem report does not properly recover with gfortran.
(In reply to comment #4) > I have to find out why the code in question that lead to > the problem report does not properly recover with gfortran. It might be an interference with BACKSPACE, as the full code that lead me to the investigation does a scan through a namelist file and uses BACKSPACE here and then. Removing the BACKSPACE on the unit when the EOF condition occured helps somewhat, but then I encounter a funny error on a NAMELIST read: Fortran runtime error: Missing format for FORMATTED data transfer I thought that NAMELIST reads are list-directed. Anyway, I guess I have to produce a better testcase.
Subject: Re: Bad IOSTAT values when readings NAMELISTs past EOF On Tue, Mar 06, 2007 at 08:20:23AM -0000, anlauf at gmx dot de wrote: > > > ------- Comment #3 from anlauf at gmx dot de 2007-03-06 08:20 ------- > (In reply to comment #2) > > The value 5008 is listed in libgfortran.h as ERROR_ENDFILE. The > > -1 corresponds to ERROR_END. So, the return value of 5008 is > > telling you that you are trying to (initiate a?) read beyond > > the end of the file, which you admit to doing. > > I actually do not want to read beyond the end of file, > I want gfortran to stay at the end of the file when trying to > read the next record or namelist. > > All other compilers I have checked (xlf, ifort 7.x-9.x, g95) > stay at the end of file. I find their behavior consistent > with the F2003 standard, section 9.10.4: > > Execution of an input/output statement containing the IOSTAT= > specifier causes the scalar-int-variable in the IOSTAT= specifier > to become defined with > > (3) The processor-dependent negative integer value of the constant > IOSTAT_END (13.8.2.5) if an end-of-file condition occurs and no > error condition occurs, or (...) > > > AFAIK, the IOSTAT values aren't documented in gfortran.texi. > > Someday, someone will get around to documenting them. > > The standard separates end-of-file, end-of-record, and > error conditions. I don't think that EOF is necessarily > an error condition. It would be nice if gfortran would > conform with these other compilers. For whatever reason, you are hitting (2). (2) A processor-dependent positive integer value if an error condition occurs. At a guess, you get a -1 if you start a read from within the the file, and you hit the EOF. Once the EOF is hit, the file is probably positioned at a point beyond the EOF, the next attempt of a read is considered an error condition, so the 5008 is return. I saw your other email about BACKSPACE. I don't know what the expected behavior is with BACKSPACE and a file position that is at or exceeds the EOF marker. At a guess and knowing the vagueness of the Standard, this is probably processor-dependent. I think you're getting hit by 9.10.2(4) for F2003: (4) If the file specified in the input statement is an external record file, it is positioned after the endfile record. and BACKSPACE isn't doing its job with respect to Note 9.54. This is an area that Jerry knows. Perhaps, he has an opinion.
Created attachment 13153 [details] Revised demo Revised demo. New namelist for this demo follows. At some place, the input read gets garbled with gfortran, see the comment before the print statement in "position_nml". Removing a few lines in the main program will hide this bug. Somehow the cycle REWIND/READ/BACKSPACE/(READ) confuses the runtime.
Created attachment 13154 [details] Namelist
(In reply to comment #8) In the new example, when I comment out the BACKSPACE in "position_nml" for ios<0 (EOF condition), the garbled output goes away. But in the large program (where this was extracted) this does not help; it dies later. (See comment #5).
Replying to comment #6. I am forming an opinion. Section 9.7 of the F2003 standard states: "Backspacing over records written using list-directed or namelist formatting is prohibited." This implies to me that one should not mix BACKSPACE with a namelist file. However, I want to study the demo program a bit. This might be one of those "correct behavior, outside the standard" problems. Generally, reading past the last record gives an EOF and trying to read again is an error. The concept of a "record" with a namelist file does not seem to be compatible to me though. I will respond further after I have a chance to explore this a bit more.
I will take this one, but I am not confirming its a bug yet.
The garbled text is because gfortran does not default initialize variable as some compilers do. The 'line' variable is not getting read because gfortran is getting an error on the read, so the it is not set to anything. Put a: line="" at the top of the subroutine to get around that. After this you can see that IOS is set to -1, so 'line' is not read. I am now going to study the REWIND and BACKSPACE and see what we should be doing differently if anything.
AFAICT A backspace is not allowed for namelist files according to 9.7.1 I interpret this to mean this is invalid code. Further, Note 9.54 states that if the preceding record read is the EOF record, the BACKSPACE would position the file to "before" the EOF. You can think of this as in between the last record and the EOF record. So subsequent reads will again read the EOF record. From the demo program, you can see the repeated values of ios=-1, or the EOF condition returned by gfortran. So I believe the gfortran behavior is correct. Regardless, I do think REWIND should clear the EOF condition for namelist and I don't think it is. I confirm this as a bug. I am working on it.
Here is a patch. Herald, do you want to see if this fixes it for you. I tested here, but it does not hurt to test some more. Index: file_pos.c =================================================================== --- file_pos.c (revision 122529) +++ file_pos.c (working copy) @@ -311,7 +311,6 @@ st_rewind (st_parameter_filepos *fpp) u->endfile = NO_ENDFILE; u->current_record = 0; - u->bytes_left = 0; u->strm_pos = 1; u->read_bad = 0; test_endfile (u); I will prepare a submittal to the list and a test case in the next few days, but this will get you going I hope.
(In reply to comment #14) Jerry, I don't have the ressources for a (re)build, so I will wait until it shows up in FX's daily binaries. Anyway, many thanks, and feel free to adjust the PR summary appropriately. From your patch it appears that the real culprit was the REWIND rather than the NAMELIST read.
hmm, I goofed, the patch in #14 does not fix this, scratch that. However, the problem is that test_endfile in st_rewind is incorrectly setting the AT_ENDFILE condition. I have a patch in progress.
Subject: Bug number PR31052 A patch for this bug has been added to the patch tracker. The mailing list url for the patch is http://gcc.gnu.org/ml/gcc-patches/2007-03/msg00892.html
Subject: Bug 31052 Author: jvdelisle Date: Sun Mar 18 18:13:50 2007 New Revision: 123038 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123038 Log: 2007-03-18 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * io/file_position (st_rewind): Fix comments. Remove use of test_endfile. Don't seek if already at 0 position. Use new is_special function to set endfile state. * io/open.c (test_endfile): Delete this function. * io/io.h: Delete prototype for test_endfile. Add prototype for is_special. * io/unix.c (is_special): New function. Fix whitespace. * io/transfer.c (next_record_r): Remove use of test_endfile. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/file_pos.c trunk/libgfortran/io/io.h trunk/libgfortran/io/open.c trunk/libgfortran/io/transfer.c trunk/libgfortran/io/unix.c
Subject: Bug 31052 Author: jvdelisle Date: Sun Mar 18 18:17:24 2007 New Revision: 123039 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123039 Log: 2007-03-18 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * gfortran.dg/namelist_27.f90: New test. Added: trunk/gcc/testsuite/gfortran.dg/namelist_27.f90 Modified: trunk/gcc/testsuite/ChangeLog
(In reply to comment #19) Jerry, I tried FX's snapshot from today (20070319) and the example above works. A somewhat more twisted example which I have not yet been able to reduce leads to an IOSTAT value of 5001 (BTW: what does that value mean?) in the namelist read, although the subroutine position_nml succeeds. The file libgfortran is not in that distribution. Could you please put here these IOSTAT values? Nor are these in the .info files. Thanks!
Subject: Re: Bad IOSTAT values when readings NAMELISTs past EOF On Mon, Mar 19, 2007 at 10:59:25PM -0000, anlauf at gmx dot de wrote: > > > ------- Comment #20 from anlauf at gmx dot de 2007-03-19 22:59 ------- > (In reply to comment #19) > > Jerry, > > I tried FX's snapshot from today (20070319) and the example > above works. A somewhat more twisted example which I have > not yet been able to reduce leads to an IOSTAT value of 5001 > (BTW: what does that value mean?) in the namelist read, although > the subroutine position_nml succeeds. > > The file libgfortran is not in that distribution. Could you > please put here these IOSTAT values? Nor are these in the > .info files. Thanks! typedef enum { ERROR_FIRST = -3, /* Marker for the first error. */ ERROR_EOR = -2, ERROR_END = -1, ERROR_OK = 0, /* Indicates success, must be zero. */ ERROR_OS = 5000, /* Operating system error, more info in errno. */ ERROR_OPTION_CONFLICT, ERROR_BAD_OPTION, ERROR_MISSING_OPTION, ERROR_ALREADY_OPEN, ERROR_BAD_UNIT, ERROR_FORMAT, ERROR_BAD_ACTION, ERROR_ENDFILE, ERROR_BAD_US, ERROR_READ_VALUE, ERROR_READ_OVERFLOW, ERROR_INTERNAL, ERROR_INTERNAL_UNIT, ERROR_ALLOCATION, ERROR_DIRECT_EOR, ERROR_SHORT_RECORD, ERROR_CORRUPT_FILE, ERROR_LAST /* Not a real error, the last error # + 1. */ }
Error 5001 would be ERROR_OPTION_CONFLICT. There are 18 different errors associated with that code. To find out which one, you must remove the iostat from the offending line so that gfortran will print the error message for you. Another option is to use iomsg and get the string. You can access the code base at gcc.gnu.org/viewcvs or better, download with anonymous svn. See instructions here: http://gcc.gnu.org/svn.html. If you do the svn approach, its very simple to get latest update and you are a few keystrokes from building it yourself. :)
(In reply to comment #22) > Error 5001 would be ERROR_OPTION_CONFLICT. There are 18 different errors > associated with that code. To find out which one, you must remove the iostat > from the offending line so that gfortran will print the error message for you. OK, this is what I get with iostat removed and compiled with -fbacktrace: At line 264 of file ../../../../oo-model/mo_thinning.f90 Fortran runtime error: Missing format for FORMATTED data transfer Backtrace for this error: + function data_transfer_init (0x89C9B92) at line 1806 of file transfer.c + function __mo_thinning__read_nml_thin (0x83570AC) at line 264 of file mo_thinning.f90 [...] The code in question does a: rewind (nnml) [...] call position_nml (nnml, 'THINNING', ...) [...] read (nnml, nml=THINNING) ! This is line 264 The read is only executed when the position_nml succeeds. > Another option is to use iomsg and get the string. Thanks, I did not know this F2003 feature. In that case I get for iomsg = "" read (nnml, nml=THINNING, iostat=ios, iomsg=iomsg) the following output: iostat = 5001 iomsg = Missing format for FORMATTED data transfer No crash, of course, at least in the read statement. > You can access the code base at gcc.gnu.org/viewcvs or better, I will bookmark that. > download with > anonymous svn. See instructions here: http://gcc.gnu.org/svn.html. If you do > the svn approach, its very simple to get latest update and you are a few > keystrokes from building it yourself. :) My next computer and internet connection will probably be powerful enough. ;-)
there is only one file on the web that matches: 'C READ IN HORIZONTAL AND VERTICAL MISALIGNMENT AND TILT' http://mad.home.cern.ch/frs/Source/DA/six.f
Created attachment 13255 [details] Untested patch to fix regression on sixtrack This patch reverts portions of the original patch and keeps a minimal set of changes to see if this clears the regression. The patch is against current trunk. If you could please test, I would appreciate. Regression tested on x86-64-Gnu/Linux I do not have a testcase here yet. If this fails, then I will try a further reduction.
Subject: Bug 31052 Author: jvdelisle Date: Fri Mar 23 00:13:19 2007 New Revision: 123139 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123139 Log: 2007-03-22 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * file_pos.c: Update Copyright year. * io/open.c (test_endfile): Restore test_endfile to fix SPEC regression. Update Copyright year. * io/io.h: Same. * io/unix.c (is_special): Add missing type for this function. Update Copyright year. * io/transfer.c (next_record_r): Restore test_endfile. (st_read): Fix whitespace. Update Copyright year Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/file_pos.c trunk/libgfortran/io/io.h trunk/libgfortran/io/open.c trunk/libgfortran/io/transfer.c trunk/libgfortran/io/unix.c
Created attachment 13273 [details] Another variant of the bug
Created attachment 13274 [details] Namelist for the modified demo
(In reply to comment #27) Jerry, with the latest variant of your patch I get a failing namelist read with the new reduced testcase gfcbug61a.f90 (uses gfcbug61a.nml). If I remove one of the entries in the namelist file, the second read succeeds. There must still be a problem with REWIND/BACKSPACE/READ.
Subject: Re: Bad IOSTAT values when readings NAMELISTs past EOF I will keep at it on this. It seems we have a conflict between the SPEC test that was failing and the namelist testcases. Unfortunately I don't have the SPEC test case to see to try to weed this out. What I will do is work on the patch i had before and send it to folks to test the SPEC portion. That may take a while, so please be patient. Sorry for the troubles.
Here is a new patch. I need someone to test on SPEC. It is very simple. Index: transfer.c =================================================================== *** transfer.c (revision 123205) --- transfer.c (working copy) *************** next_record_r (st_parameter_dt *dtp) *** 2228,2234 **** break; } ! if (dtp->u.p.current_unit->flags.access == ACCESS_SEQUENTIAL) test_endfile (dtp->u.p.current_unit); } --- 2228,2235 ---- break; } ! if (dtp->u.p.current_unit->flags.access == ACCESS_SEQUENTIAL ! && !dtp->u.p.namelist_mode) test_endfile (dtp->u.p.current_unit); }
> Unfortunately I don't > have the SPEC test case to see to try to weed this out. have a look a comment #24 the sources are actually available
(In reply to comment #31) > Here is a new patch. I need someone to test on SPEC. It is very simple. > > Index: transfer.c > =================================================================== > *** transfer.c (revision 123205) > --- transfer.c (working copy) > *************** next_record_r (st_parameter_dt *dtp) > *** 2228,2234 **** > break; > } > > ! if (dtp->u.p.current_unit->flags.access == ACCESS_SEQUENTIAL) > test_endfile (dtp->u.p.current_unit); > } > > --- 2228,2235 ---- > break; > } > > ! if (dtp->u.p.current_unit->flags.access == ACCESS_SEQUENTIAL > ! && !dtp->u.p.namelist_mode) > test_endfile (dtp->u.p.current_unit); > } This patch doesn't break 200.sixtrack (when added on top of the Jerry's previous patch above).
Regarding comment 24 and 32. I have that downloaded and tried to compile and there are duplications of subroutines and multiple main programs. As I get time, I intend to get it sorted out and add it to my testing here.
Subject: Bug 31052 Author: jvdelisle Date: Wed Mar 28 01:19:39 2007 New Revision: 123284 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123284 Log: 2007-03-27 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * io/transfer.c (next_record_r): Do not call test_endfile if in namelist mode. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/transfer.c
Closing, will not backport to 4.2 unless someone feels strongly about it.
(In reply to comment #36) > Closing, will not backport to 4.2 unless someone feels strongly about it. Jerry, I hate to bother you, but it is not really fixed. Add a comment line at the end of the namelist file gfcbug61a.nml, so that it reads: &REPORT type='report1' / &REPORT type='report2' / ! Now the last attempt to read this namelist fails again. Sorry, reopening this one...
Subject: Bug number PR31052 A patch for this bug has been added to the patch tracker. The mailing list url for the patch is http://gcc.gnu.org/ml/gcc-patches/2007-04/msg00014.html
Subject: Bug 31052 Author: jvdelisle Date: Sun Apr 1 18:32:20 2007 New Revision: 123403 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123403 Log: 2007-04-01 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * io/open.c (test_endfile): Delete this function. (edit_modes): Delete call to test_endfile. (new_unit): Likewise. * io/io.h: Delete prototype for test_endfile. * io/transfer.c (next_record_r): Remove use of test_endfile. (st_read): Add test for end file condition and adjust status. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/io.h trunk/libgfortran/io/open.c trunk/libgfortran/io/transfer.c
Subject: Bug 31052 Author: jvdelisle Date: Sun Apr 1 18:33:13 2007 New Revision: 123404 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123404 Log: 2007-04-01 Jerry DeLisle <jvdelisle@gcc.gnu.org> PR libgfortran/31052 * gfortran.dg/namelist_28.f90: New test. Added: trunk/gcc/testsuite/gfortran.dg/namelist_28.f90 Modified: trunk/gcc/testsuite/ChangeLog
(In reply to comment #40) > New Revision: 123404 > > URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=123404 > Log: > 2007-04-01 Jerry DeLisle <jvdelisle@gcc.gnu.org> Jerry, thanks so far, but now I am back to the situation described in comment #23. I will try to produce a new reduced testcase that leads to this crash in the next days. I'd like to ask you to keep this PR open for a few more days until we know whether the other problem is related or unrelated to this PR. Thanks, -ha
Will keep open until we get them all!. Looks like I need another test case. :)
(In reply to comment #42) In my attempts at narrowing down I found a (minor) problem with INQUIRE, see PR 31532. Will continue on this one.
sixtrack in SPEC CPU 2K started to fail on Mar. 19: http://people.redhat.com/dnovillo/spec2000.em64t/gcc/log/20070319/CFP2000.002.html and it still fails today. Is it caused by this bug fix?
(In reply to comment #44) > sixtrack in SPEC CPU 2K started to fail on Mar. 19: And then start passing and then again started to fail again on/around April 1st. HJL when will you please get your dates correct.
Subject: Re: [4.2 only] Bad IOSTAT values when readings NAMELISTs past EOF pinskia at gcc dot gnu dot org wrote: > ------- Comment #45 from pinskia at gcc dot gnu dot org 2007-04-13 05:32 ------- > (In reply to comment #44) >> sixtrack in SPEC CPU 2K started to fail on Mar. 19: > And then start passing and then again started to fail again on/around April > 1st. HJL when will you please get your dates correct. > > Now I am confused because I reverted and re-patched and it was tested by spark and sixtrack was then fine. I specifically had this tested before committing the last fix.
(In reply to comment #46) > Subject: Re: [4.2 only] Bad IOSTAT values when readings > NAMELISTs past EOF > > > Now I am confused because I reverted and re-patched and it was tested by spark > and sixtrack was then fine. I specifically had this tested before committing > the last fix. > I have verfied that this patch http://gcc.gnu.org/viewcvs?view=rev&revision=123403 caused sixtrack regression.
Created attachment 13365 [details] A possible fix for sixtrack, again and again .... hjl, Would you please try this patch and see if it resolves the problem. This is a hunt in the dark since I don't have this code to test. I have not been able to get the "publicly available" equivalent six.f to build. This diff is against current trunk.
(In reply to comment #48) > Created an attachment (id=13365) [edit] > A possible fix for sixtrack, again and again .... > > hjl, > > Would you please try this patch and see if it resolves the problem. This is a > hunt in the dark since I don't have this code to test. I have not been able to > get the "publicly available" equivalent six.f to build. This diff is against > current trunk. > Yes, it works. Thanks.
Patch committed to trunk.
Two testcases for 4.3 pass on 4.2. Fixed on 4.3. Because of the problems with regressions on SPEC, I would rather not backport this. Closing. Herald, if something else comes up, please file a new report. This one is too long.