Bug 25829 - [F2003] Asynchronous IO support
[F2003] Asynchronous IO support
Status: ASSIGNED
Product: gcc
Classification: Unclassified
Component: fortran
4.2.0
: P3 enhancement
: ---
Assigned To: Janne Blomqvist
:
Depends on: 34705 35339 25561
Blocks: 20585
  Show dependency treegraph
 
Reported: 2006-01-17 21:53 UTC by Janne Blomqvist
Modified: 2012-09-21 18:48 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2008-02-17 01:19:36


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Janne Blomqvist 2006-01-17 21:53:56 UTC
F2003 supports the ASYNCHRONOUS='YES' specifier in some IO statements, as well as the WAIT io-unit statement.
Comment 1 Andrew Pinski 2006-01-17 23:33:35 UTC
Confirmed.
Comment 2 Jerry DeLisle 2008-02-17 01:19:36 UTC
I will be working pn this and have a strat in my local experimental branch
Comment 3 Jerry DeLisle 2008-02-17 05:02:02 UTC
Keep this PR in mind while doing this. PR34705
Comment 4 Jerry DeLisle 2008-02-23 16:48:30 UTC
Status: local experimental trunk

Front-end has most if not all keywords compiling.

DECIMAL= is working all the way through to runtime.

WAIT compiles and executes a stub in the runtime with a new st_parameter_wait structure to pass parameters from the front end to the library.  The stub is _gfortran_st_wait.
Comment 5 Jerry DeLisle 2008-04-05 22:18:49 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Sat Apr  5 22:18:03 2008
New Revision: 133943

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133943
Log:
2008-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR fortran/25829 28655
	* gfortran.map: Add new symbol, _gfortran_st_wait.
	* libgfortran.h (st_paramter_common): Add new I/O parameters.
	* open.c (st_option decimal_opt[], st_option encoding_opt[],
	st_option round_opt[], st_option sign_opt[], st_option async_opt[]): New
	parameter option arrays. (edit_modes): Add checks for new parameters.
	(new_unit): Likewise. (st_open): Likewise.
	* list_read.c (CASE_SEPERATORS): Add ';' as a valid separator.
	(eat_separator): Handle deimal comma. (read_logical): Fix whitespace.
	(parse_real): Handle decimal comma. (read_real): Handle decimal comma.
	* read.c (read_a): Use decimal status flag to allow comma in place of a
	decimal point. (read_f): Allow comma as acceptable character in float.
	According to decimal flag, substitute a period for a comma.
	(read_x): If decimal status flag is comma, disable the read_comma flag,
	not allowing comma as a delimiter, an extension otherwise.
	* io.h: (unit_decimal, unit_encoding, unit_round, unit_sign,
	unit_async): New enumerators. Add all new I/O parameters.
	* unix.c (unix_stream, int_stream): Add io_mode	asychronous I/O	control.
	(move_pos_offset, fd_alloc_w_at): Fix some whitespace.
	(fd_sfree): Use new enumerator. (fd_read): Likewise.
	(fd_write): Likewise. (fd_close): Fix whitespace.
	(fd_open): Use new enumertors. (tempfile, regular_file,
	open_external): Fix whitespace. (output_stream, error_stream): Set
	method. (stream_offset): Fix whitespace.
	* transfer.c: (st_option decimal_opt[], sign_opt[], blank_opt[]): New
	option arrays. 	(formatted_transfer_scalar): Set sf_read_comma flag
	based on new decimal_status flag. (data_transfer_init): Initialize new
	parameters. Add checks for decimal, sign, and blank. (st_wait): New stub.
	* format.c: (format_lex): Add format specifiers DP, DC, and D.
	(parse_format_list): Parse the new specifiers.
	* write.c (write_decimal): Use new sign enumerators to set the sign.
	(write_complex): Handle decimal comma and semi-colon separator.
	(nml_write_obj): Likewise.
	* write_float.def: Revise sign enumerators. (calculate_sign): Use new
	sign enumerators. (output_float): Likewise. Use new decimal_status flag
	to set the decimal character to a point or a comma.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/gfortran.map
    trunk/libgfortran/io/format.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c
    trunk/libgfortran/io/write.c
    trunk/libgfortran/io/write_float.def
    trunk/libgfortran/libgfortran.h

Comment 6 Jerry DeLisle 2008-04-05 22:24:10 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Sat Apr  5 22:23:27 2008
New Revision: 133944

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133944
Log:
2008-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>
	    Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR fortran/25829 28655
	* dump-parse-tree.c (gfc_show_code_node): Show new I/O parameters.
	* gfortran.h (gfc_statement): Add ST_WAIT enumerator.
	(gfc_open): Add pointers for decimal, encoding, round, sign,
	asynchronous. (gfc_inquire): Add pointers for asynchronous, decimal,
	encoding, pending, round, sign, size, id.
	(gfc_wait): New typedef struct. (gfc_dt): Add pointers for id, pos,
	asynchronous, blank, decimal, delim, pad, round, sign.
	(gfc_exec_op): Add EXEC_WAIT enumerator. (gfc_code): Add pointer for
	wait. (gfc_free_wait), (gfc_resolve_wait): New function prototypes.
	* trans-stmt.h (gfc_trans_wait): New function prototype.
	* trans.c (gfc_trans_code): Add case for EXEC_WAIT.
	* io.c (io_tag): Add new tags for DECIMAL, ENCODING, ROUND, SIGN,
	ASYCHRONOUS, ID. (match_open_element): Add matchers for new tags.
	(gfc_free_open): Free new pointers. (gfc_resolve_open): Resolve new
	tags. (gfc_resolve_open): Remove comment around check for allowed
	values and ASYNCHRONOUS, update it.  Likewise for DECIMAL, ENCODING,
	ROUND, and SIGN. (match_dt_element): Add matching for new tags.
	(gfc_free_wait): New function. (gfc_resolve_wait): New function.
	(match_wait_element): New function. (gfc_match_wait): New function.
	* resolve.c (gfc_resolve_blocks): Add case for EXEC_WAIT.
	(resolve_code): Add case for EXEC_WAIT. 
	* st.c (gfc_free_statement): Add case for EXEC_WAIT.
	* trans-io.c (ioparam_type): Add IOPARM_ptype_wait. (gfc_st_parameter):
	Add "wait" entry. (iocall): Add IOCALL_WAIT enumerator.
	(gfc_build_io_library_fndecls): Add function declaration for st_wait.
	(gfc_trans_open): Add mask bits for new I/O tags.
	(gfc_trans_inquire): Add mask bits for new I/O tags.
	(gfc_trans_wait): New translation function.
	(build_dt): Add mask bits for new I/O tags.
	* match.c (gfc_match_if) Add matcher for "wait".
	* match.h (gfc_match_wait): Prototype for new function.
	* ioparm.def: Add new I/O parameter definitions.
	* parse.c (decode_statement): Add match for "wait" statement.
	(next_statement): Add case for ST_WAIT. (gfc_ascii_statement): Same.

Modified:
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/fortran/dump-parse-tree.c
    trunk/gcc/fortran/gfortran.h
    trunk/gcc/fortran/io.c
    trunk/gcc/fortran/ioparm.def
    trunk/gcc/fortran/match.c
    trunk/gcc/fortran/match.h
    trunk/gcc/fortran/parse.c
    trunk/gcc/fortran/resolve.c
    trunk/gcc/fortran/st.c
    trunk/gcc/fortran/trans-io.c
    trunk/gcc/fortran/trans-stmt.h
    trunk/gcc/fortran/trans.c

Comment 7 Jerry DeLisle 2008-04-05 22:34:08 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Sat Apr  5 22:33:14 2008
New Revision: 133945

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133945
Log:
2008-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR fortran/25829 28655
	* gfortran.dg/f2003_io_1.f03: New test.
	* gfortran.dg/f2003_io_2.f03: New test.
	* gfortran.dg/f2003_io_3.f03: New test.
	* gfortran.dg/f2003_io_4.f03: New test.
	* gfortran.dg/f2003_io_5.f03: New test.
	* gfortran.dg/f2003_io_6.f03: New test.
	* gfortran.dg/f2003_io_7.f03: New test.

Added:
    trunk/gcc/testsuite/gfortran.dg/f2003_io_1.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_2.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_3.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_4.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_5.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_6.f03
    trunk/gcc/testsuite/gfortran.dg/f2003_io_7.f03
Modified:
    trunk/gcc/testsuite/ChangeLog

Comment 8 Jerry DeLisle 2008-04-07 22:06:37 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Mon Apr  7 22:05:52 2008
New Revision: 133988

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133988
Log:
2008-04-07  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR fortran/25829 28655
	* io/open.c (edit_modes): Set flags.async. (new_unit) Set flags.async
	and flags.status. (st_open): Initialize flags.async.
	* io/list_read.c (read_charactor): Use delim_status instead of
	flags.delim.
	* io/read.c (read_x): Use pad_status instead of flags.pad.
	* io/inquire.c (inquire_via_unit): Add new checks.
	(inquire_via_filename): Likewise.
	* io/io.h (st_parameter_inquire): Add new flags.
	(st_parameter_dt): Likewise.
	* io/unit.c (get_internal_unit): Set flags.async. (init_units): Set
	flags.async.
	* io/transfer.c: Add delim and pad option arrays. (read_sf): Use
	pad_status instead of flags.pad. (read_block): Likewise.
	(data_transfer_init): Set flags.async and add checks.
	* io/write.c (write_character): Use delim_status.
	(list_formatted_write_scalar): Likewise. (nml_write_obj): Likewise.
	(namelist_write): Likewise.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/inquire.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/write.c

Comment 9 Jerry DeLisle 2008-04-07 22:08:31 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Mon Apr  7 22:07:44 2008
New Revision: 133989

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133989
Log:
2008-04-07  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR fortran/25829 28655
	* io.c (io_tag): Add new tags for decimal, encoding, asynchronous,
	round, sign, and id. (match_open_element): Match new tags.
	(gfc_resolve_open): Resolve new tags. (gfc_match_open): Enable encoding
	for DEFAULT only. Update error messages. (match_dt_element): Fix match
	tag for asynchronous. Update error messages. (gfc_free_inquire): Free
	new expressions. (match_inquire_element): Match new tags.
	(gfc_match_inquire): Add constraint for ID and PENDING.
	(gfc_resolve_inquire): Resolve new tags.
	* trans-io.c (gfc_trans_inquire): Clean up whitespace and fix setting of
	mask for ID parameter.
	* ioparm.def: Fix order of parameters for pending, round, and sign.
	NOTE: These must line up with the definitions in libgfortran/io/io.h. or
	things don't work.

Modified:
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/fortran/io.c
    trunk/gcc/fortran/ioparm.def
    trunk/gcc/fortran/trans-io.c

Comment 10 Jerry DeLisle 2008-04-07 22:11:26 UTC
Subject: Bug 25829

Author: jvdelisle
Date: Mon Apr  7 22:10:41 2008
New Revision: 133991

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133991
Log:
2008-04-07  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	PR fortran/25829 28655
	* gfortran.dg/write_check2.f90: Update dg-error.
	* gfortran.dg/io_constraints_1.f90: Udate dg-error.

Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gfortran.dg/io_constraints_1.f90
    trunk/gcc/testsuite/gfortran.dg/write_check2.f90

Comment 11 Daniel Franke 2009-04-10 21:23:03 UTC
Jerry, is this complete? If not, could you please summarize what's left?
Thanks.
Comment 12 Jerry DeLisle 2009-04-11 15:26:30 UTC
gfortran currently accepts asynchronous I/O syntax. The F2003 standard permits the implementation to actually do synchronous I/O.  gfortran run time library does not actually perform I/O asynchronously, but does so synchronously.

To implement this we need to develop the underlying scheme with the unit and DTP structures, develop config magic to identify targets that actually have support for asynchronous I/O, write the code that keeps track of the I/O calls, there status and completion, and address the threading concerns.
Comment 13 Janne Blomqvist 2009-10-28 22:03:37 UTC
Reassigning to myself, I have a plan how to proceed with supporting proper async I/O in the library.
Comment 14 Tobias Burnus 2009-12-11 22:28:03 UTC
Test case:
  http://users.erols.com/dnagle/pub/async_io.f03

Works with ifort, fails with gfortran. I think it needs real asynchronous I/O to work. Additionally, gfortran does not yet support the ASYNCHRONOUS attribute.
Comment 15 Tobias Burnus 2009-12-11 22:28:55 UTC
(In reply to comment #14)
> Test case:
>   http://users.erols.com/dnagle/pub/async_io.f03
which additionally needs the file/module:
  http://users.erols.com/dnagle/pub/f03def.f90
Comment 16 Tobias Burnus 2010-01-08 10:11:18 UTC
For completeness: I just committed support for the ASYNCHRONOUS attribute:
  http://gcc.gnu.org/ml/fortran/2010-01/msg00049.html
  http://gcc.gnu.org/ml/gcc-cvs/2010-01/msg00192.html
which is treated as no op.

One needs to think carefully whether one needs to take care of ASYNCHRONOUS for the middle end when real async I/O is implemented. For a simple:
  write(9, ..., asynchronous='yes') foo
  ...
  write(9, ...) ! force WAIT on unit 9
  foo = 8
presumably not using "&foo" in the first write call lets the address escape thus unless a function is known (to the ME) to not touch "foo" (indirectly), no problem should occur. But how about:

  ! Note: using "foo" in AIO implies the ASYNCHRONOUS attribute for foo
  write(9, ..., asynchronous='yes', id=waitid) foo
  call finish(foo, waitid)
contains
  subroutine finish(var, wait)
    asynchronous :: var
    wait(wait)
    var = 7
  end subroutine finish

Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus in this case we probably need to tell the ME about the ASYNCHRONOUS attribute.
Comment 17 Tobias Burnus 2010-01-08 10:42:17 UTC
(In reply to comment #16)
> Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus
> in this case we probably need to tell the ME about the ASYNCHRONOUS attribute.

Probably by dropping the "restricted" for the pointer argument (as one would to for TARGET). One should also check that it works for:
  subroutine finish(var, wait)
    BLOCK
      asynchronous :: var
      wait(wait)
      var = 7
    end BLOCK
  end subroutine finish
assuming that "asynchronous" applies in the block to the dummy argument. (Should work analogously for volatile.)
Comment 18 Tobias Burnus 2010-07-13 17:34:20 UTC
Reminder: The function decl ("fn spec") needs to be updated for asynchronous I/O; in particular WAIT needs to have the "X" (unused argument) removed. Possibly, other function names should be used to keep the nonclobber of the current transfer functions. Cf. also PR 43665.
Comment 19 Tobias Burnus 2011-03-04 18:08:14 UTC
(In reply to comment #16)
>   subroutine finish(var, wait)
>     asynchronous :: var
>     wait(wait)
>     var = 7
>   end subroutine finish
> 
> Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus
> in this case we probably need to tell the ME about the ASYNCHRONOUS attribute.

A brute-force method would be to add a __sync_synchronize (BUILT_IN_SYNCHRONIZE) call after the WAIT. But maybe one finds a better way which restricts the barrier to only "var" (and other asynchronous variables) without influencing unrelated variables.
Comment 20 Jakub Jelinek 2011-03-04 18:11:56 UTC
If ASYNCHRONOUS expands to volatile, no barrier should be necessary.
Comment 21 Tobias Burnus 2011-04-15 09:10:56 UTC
(In reply to comment #19)
> A brute-force method would be to add a __sync_synchronize

Actually, this idea does not work properly - neither for INQUIRE(...,PENDING=) nor for ASYNCHRONOUS with MPI 3. (Cf. link below)

(In reply to comment #20)
> If ASYNCHRONOUS expands to volatile, no barrier should be necessary.

Well, VOLATILE has the wrong semantics, i.e. it will only partially solve the problem. Additionally, you create huge missed-optimization issues.

I have now asked at GCC@ (and fortran@) for some suggestions:
http://gcc.gnu.org/ml/fortran/2011-04/msg00143.html

(There is currently also a vivid discussion on J3's interop and MPI Forum's MPI3-Fortran mailing lists about ASYNCHRONOUS and nonblocking MPI calls.)
Comment 22 Tobias Burnus 2011-07-13 09:24:40 UTC
Note: TR 29113 introduces a wider coverage of ASYNCHRONOUS, cf. almost-PDTR 29113 at ftp://ftp.nag.co.uk/sc22wg5/N1851-N1900/N1866.pdf.

I think the easiest is to not set the "restrict" of dummy arguments involved in ASYNCHRONOUS I/O. That way, one has:

  call user_write(id, var)
  ...
  call user_wait(id)
  var = ...

The address of the "var" escapes at "user_write" and is thus available to "user_wait" - hence, "var =" will not be moved across "user_wait". Ditto for gfortran's asynchronous I/O.
Comment 23 Tobias Burnus 2011-07-13 13:15:39 UTC
(In reply to comment #22)
> I think the easiest is to not set the "restrict" of dummy arguments involved in
> ASYNCHRONOUS I/O.

It seems that GCC understands something different than I by "restrict"; it seems to be much less aggressive than I thought - and seems to match exactly what we need, such that no action seems to be required.

Cf. http://gcc.gnu.org/ml/gcc/2011-07/msg00208.html and see PR 49733 for a missed-optimization PR, which is for adding an variable attribute which does optimize across function calls - which is allowed most of the time in Fortran (except for ASYNCHRONOUS and for coarrays).