|Summary:||[F03] Asynchronous IO support|
|Product:||gcc||Reporter:||Janne Blomqvist <jb>|
|Component:||fortran||Assignee:||Not yet assigned to anyone <unassigned>|
|Severity:||enhancement||CC:||dfranke, gcc-bugs, jakub|
|Build:||Known to work:|
|Known to fail:||Last reconfirmed:||2008-02-17 01:19:36|
|Bug Depends on:||34705, 35339, 25561|
Description Janne Blomqvist 2006-01-17 21:53:56 UTC
F2003 supports the ASYNCHRONOUS='YES' specifier in some IO statements, as well as the WAIT io-unit statement.
Comment 1 Andrew Pinski 2006-01-17 23:33:35 UTC
Comment 2 Jerry DeLisle 2008-02-17 01:19:36 UTC
I will be working pn this and have a strat in my local experimental branch
Comment 4 Jerry DeLisle 2008-02-23 16:48:30 UTC
Status: local experimental trunk Front-end has most if not all keywords compiling. DECIMAL= is working all the way through to runtime. WAIT compiles and executes a stub in the runtime with a new st_parameter_wait structure to pass parameters from the front end to the library. The stub is _gfortran_st_wait.
Comment 5 Jerry DeLisle 2008-04-05 22:18:49 UTC
Subject: Bug 25829 Author: jvdelisle Date: Sat Apr 5 22:18:03 2008 New Revision: 133943 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133943 Log: 2008-04-05 Jerry DeLisle <email@example.com> PR fortran/25829 28655 * gfortran.map: Add new symbol, _gfortran_st_wait. * libgfortran.h (st_paramter_common): Add new I/O parameters. * open.c (st_option decimal_opt, st_option encoding_opt, st_option round_opt, st_option sign_opt, st_option async_opt): New parameter option arrays. (edit_modes): Add checks for new parameters. (new_unit): Likewise. (st_open): Likewise. * list_read.c (CASE_SEPERATORS): Add ';' as a valid separator. (eat_separator): Handle deimal comma. (read_logical): Fix whitespace. (parse_real): Handle decimal comma. (read_real): Handle decimal comma. * read.c (read_a): Use decimal status flag to allow comma in place of a decimal point. (read_f): Allow comma as acceptable character in float. According to decimal flag, substitute a period for a comma. (read_x): If decimal status flag is comma, disable the read_comma flag, not allowing comma as a delimiter, an extension otherwise. * io.h: (unit_decimal, unit_encoding, unit_round, unit_sign, unit_async): New enumerators. Add all new I/O parameters. * unix.c (unix_stream, int_stream): Add io_mode asychronous I/O control. (move_pos_offset, fd_alloc_w_at): Fix some whitespace. (fd_sfree): Use new enumerator. (fd_read): Likewise. (fd_write): Likewise. (fd_close): Fix whitespace. (fd_open): Use new enumertors. (tempfile, regular_file, open_external): Fix whitespace. (output_stream, error_stream): Set method. (stream_offset): Fix whitespace. * transfer.c: (st_option decimal_opt, sign_opt, blank_opt): New option arrays. (formatted_transfer_scalar): Set sf_read_comma flag based on new decimal_status flag. (data_transfer_init): Initialize new parameters. Add checks for decimal, sign, and blank. (st_wait): New stub. * format.c: (format_lex): Add format specifiers DP, DC, and D. (parse_format_list): Parse the new specifiers. * write.c (write_decimal): Use new sign enumerators to set the sign. (write_complex): Handle decimal comma and semi-colon separator. (nml_write_obj): Likewise. * write_float.def: Revise sign enumerators. (calculate_sign): Use new sign enumerators. (output_float): Likewise. Use new decimal_status flag to set the decimal character to a point or a comma. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/gfortran.map trunk/libgfortran/io/format.c trunk/libgfortran/io/io.h trunk/libgfortran/io/list_read.c trunk/libgfortran/io/open.c trunk/libgfortran/io/read.c trunk/libgfortran/io/transfer.c trunk/libgfortran/io/unit.c trunk/libgfortran/io/unix.c trunk/libgfortran/io/write.c trunk/libgfortran/io/write_float.def trunk/libgfortran/libgfortran.h
Comment 6 Jerry DeLisle 2008-04-05 22:24:10 UTC
Subject: Bug 25829 Author: jvdelisle Date: Sat Apr 5 22:23:27 2008 New Revision: 133944 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133944 Log: 2008-04-05 Jerry DeLisle <firstname.lastname@example.org> Francois-Xavier Coudert <email@example.com> PR fortran/25829 28655 * dump-parse-tree.c (gfc_show_code_node): Show new I/O parameters. * gfortran.h (gfc_statement): Add ST_WAIT enumerator. (gfc_open): Add pointers for decimal, encoding, round, sign, asynchronous. (gfc_inquire): Add pointers for asynchronous, decimal, encoding, pending, round, sign, size, id. (gfc_wait): New typedef struct. (gfc_dt): Add pointers for id, pos, asynchronous, blank, decimal, delim, pad, round, sign. (gfc_exec_op): Add EXEC_WAIT enumerator. (gfc_code): Add pointer for wait. (gfc_free_wait), (gfc_resolve_wait): New function prototypes. * trans-stmt.h (gfc_trans_wait): New function prototype. * trans.c (gfc_trans_code): Add case for EXEC_WAIT. * io.c (io_tag): Add new tags for DECIMAL, ENCODING, ROUND, SIGN, ASYCHRONOUS, ID. (match_open_element): Add matchers for new tags. (gfc_free_open): Free new pointers. (gfc_resolve_open): Resolve new tags. (gfc_resolve_open): Remove comment around check for allowed values and ASYNCHRONOUS, update it. Likewise for DECIMAL, ENCODING, ROUND, and SIGN. (match_dt_element): Add matching for new tags. (gfc_free_wait): New function. (gfc_resolve_wait): New function. (match_wait_element): New function. (gfc_match_wait): New function. * resolve.c (gfc_resolve_blocks): Add case for EXEC_WAIT. (resolve_code): Add case for EXEC_WAIT. * st.c (gfc_free_statement): Add case for EXEC_WAIT. * trans-io.c (ioparam_type): Add IOPARM_ptype_wait. (gfc_st_parameter): Add "wait" entry. (iocall): Add IOCALL_WAIT enumerator. (gfc_build_io_library_fndecls): Add function declaration for st_wait. (gfc_trans_open): Add mask bits for new I/O tags. (gfc_trans_inquire): Add mask bits for new I/O tags. (gfc_trans_wait): New translation function. (build_dt): Add mask bits for new I/O tags. * match.c (gfc_match_if) Add matcher for "wait". * match.h (gfc_match_wait): Prototype for new function. * ioparm.def: Add new I/O parameter definitions. * parse.c (decode_statement): Add match for "wait" statement. (next_statement): Add case for ST_WAIT. (gfc_ascii_statement): Same. Modified: trunk/gcc/fortran/ChangeLog trunk/gcc/fortran/dump-parse-tree.c trunk/gcc/fortran/gfortran.h trunk/gcc/fortran/io.c trunk/gcc/fortran/ioparm.def trunk/gcc/fortran/match.c trunk/gcc/fortran/match.h trunk/gcc/fortran/parse.c trunk/gcc/fortran/resolve.c trunk/gcc/fortran/st.c trunk/gcc/fortran/trans-io.c trunk/gcc/fortran/trans-stmt.h trunk/gcc/fortran/trans.c
Comment 7 Jerry DeLisle 2008-04-05 22:34:08 UTC
Subject: Bug 25829 Author: jvdelisle Date: Sat Apr 5 22:33:14 2008 New Revision: 133945 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133945 Log: 2008-04-05 Jerry DeLisle <firstname.lastname@example.org> PR fortran/25829 28655 * gfortran.dg/f2003_io_1.f03: New test. * gfortran.dg/f2003_io_2.f03: New test. * gfortran.dg/f2003_io_3.f03: New test. * gfortran.dg/f2003_io_4.f03: New test. * gfortran.dg/f2003_io_5.f03: New test. * gfortran.dg/f2003_io_6.f03: New test. * gfortran.dg/f2003_io_7.f03: New test. Added: trunk/gcc/testsuite/gfortran.dg/f2003_io_1.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_2.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_3.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_4.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_5.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_6.f03 trunk/gcc/testsuite/gfortran.dg/f2003_io_7.f03 Modified: trunk/gcc/testsuite/ChangeLog
Comment 8 Jerry DeLisle 2008-04-07 22:06:37 UTC
Subject: Bug 25829 Author: jvdelisle Date: Mon Apr 7 22:05:52 2008 New Revision: 133988 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133988 Log: 2008-04-07 Jerry DeLisle <email@example.com> PR fortran/25829 28655 * io/open.c (edit_modes): Set flags.async. (new_unit) Set flags.async and flags.status. (st_open): Initialize flags.async. * io/list_read.c (read_charactor): Use delim_status instead of flags.delim. * io/read.c (read_x): Use pad_status instead of flags.pad. * io/inquire.c (inquire_via_unit): Add new checks. (inquire_via_filename): Likewise. * io/io.h (st_parameter_inquire): Add new flags. (st_parameter_dt): Likewise. * io/unit.c (get_internal_unit): Set flags.async. (init_units): Set flags.async. * io/transfer.c: Add delim and pad option arrays. (read_sf): Use pad_status instead of flags.pad. (read_block): Likewise. (data_transfer_init): Set flags.async and add checks. * io/write.c (write_character): Use delim_status. (list_formatted_write_scalar): Likewise. (nml_write_obj): Likewise. (namelist_write): Likewise. Modified: trunk/libgfortran/ChangeLog trunk/libgfortran/io/inquire.c trunk/libgfortran/io/io.h trunk/libgfortran/io/list_read.c trunk/libgfortran/io/open.c trunk/libgfortran/io/read.c trunk/libgfortran/io/transfer.c trunk/libgfortran/io/unit.c trunk/libgfortran/io/write.c
Comment 9 Jerry DeLisle 2008-04-07 22:08:31 UTC
Subject: Bug 25829 Author: jvdelisle Date: Mon Apr 7 22:07:44 2008 New Revision: 133989 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133989 Log: 2008-04-07 Jerry DeLisle <firstname.lastname@example.org> PR fortran/25829 28655 * io.c (io_tag): Add new tags for decimal, encoding, asynchronous, round, sign, and id. (match_open_element): Match new tags. (gfc_resolve_open): Resolve new tags. (gfc_match_open): Enable encoding for DEFAULT only. Update error messages. (match_dt_element): Fix match tag for asynchronous. Update error messages. (gfc_free_inquire): Free new expressions. (match_inquire_element): Match new tags. (gfc_match_inquire): Add constraint for ID and PENDING. (gfc_resolve_inquire): Resolve new tags. * trans-io.c (gfc_trans_inquire): Clean up whitespace and fix setting of mask for ID parameter. * ioparm.def: Fix order of parameters for pending, round, and sign. NOTE: These must line up with the definitions in libgfortran/io/io.h. or things don't work. Modified: trunk/gcc/fortran/ChangeLog trunk/gcc/fortran/io.c trunk/gcc/fortran/ioparm.def trunk/gcc/fortran/trans-io.c
Comment 10 Jerry DeLisle 2008-04-07 22:11:26 UTC
Subject: Bug 25829 Author: jvdelisle Date: Mon Apr 7 22:10:41 2008 New Revision: 133991 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133991 Log: 2008-04-07 Jerry DeLisle <email@example.com> PR fortran/25829 28655 * gfortran.dg/write_check2.f90: Update dg-error. * gfortran.dg/io_constraints_1.f90: Udate dg-error. Modified: trunk/gcc/testsuite/ChangeLog trunk/gcc/testsuite/gfortran.dg/io_constraints_1.f90 trunk/gcc/testsuite/gfortran.dg/write_check2.f90
Comment 11 Daniel Franke 2009-04-10 21:23:03 UTC
Jerry, is this complete? If not, could you please summarize what's left? Thanks.
Comment 12 Jerry DeLisle 2009-04-11 15:26:30 UTC
gfortran currently accepts asynchronous I/O syntax. The F2003 standard permits the implementation to actually do synchronous I/O. gfortran run time library does not actually perform I/O asynchronously, but does so synchronously. To implement this we need to develop the underlying scheme with the unit and DTP structures, develop config magic to identify targets that actually have support for asynchronous I/O, write the code that keeps track of the I/O calls, there status and completion, and address the threading concerns.
Comment 13 Janne Blomqvist 2009-10-28 22:03:37 UTC
Reassigning to myself, I have a plan how to proceed with supporting proper async I/O in the library.
Comment 14 Tobias Burnus 2009-12-11 22:28:03 UTC
Test case: http://users.erols.com/dnagle/pub/async_io.f03 Works with ifort, fails with gfortran. I think it needs real asynchronous I/O to work. Additionally, gfortran does not yet support the ASYNCHRONOUS attribute.
Comment 15 Tobias Burnus 2009-12-11 22:28:55 UTC
(In reply to comment #14) > Test case: > http://users.erols.com/dnagle/pub/async_io.f03 which additionally needs the file/module: http://users.erols.com/dnagle/pub/f03def.f90
Comment 16 Tobias Burnus 2010-01-08 10:11:18 UTC
For completeness: I just committed support for the ASYNCHRONOUS attribute: http://gcc.gnu.org/ml/fortran/2010-01/msg00049.html http://gcc.gnu.org/ml/gcc-cvs/2010-01/msg00192.html which is treated as no op. One needs to think carefully whether one needs to take care of ASYNCHRONOUS for the middle end when real async I/O is implemented. For a simple: write(9, ..., asynchronous='yes') foo ... write(9, ...) ! force WAIT on unit 9 foo = 8 presumably not using "&foo" in the first write call lets the address escape thus unless a function is known (to the ME) to not touch "foo" (indirectly), no problem should occur. But how about: ! Note: using "foo" in AIO implies the ASYNCHRONOUS attribute for foo write(9, ..., asynchronous='yes', id=waitid) foo call finish(foo, waitid) contains subroutine finish(var, wait) asynchronous :: var wait(wait) var = 7 end subroutine finish Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus in this case we probably need to tell the ME about the ASYNCHRONOUS attribute.
Comment 17 Tobias Burnus 2010-01-08 10:42:17 UTC
(In reply to comment #16) > Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus > in this case we probably need to tell the ME about the ASYNCHRONOUS attribute. Probably by dropping the "restricted" for the pointer argument (as one would to for TARGET). One should also check that it works for: subroutine finish(var, wait) BLOCK asynchronous :: var wait(wait) var = 7 end BLOCK end subroutine finish assuming that "asynchronous" applies in the block to the dummy argument. (Should work analogously for volatile.)
Comment 18 Tobias Burnus 2010-07-13 17:34:20 UTC
Reminder: The function decl ("fn spec") needs to be updated for asynchronous I/O; in particular WAIT needs to have the "X" (unused argument) removed. Possibly, other function names should be used to keep the nonclobber of the current transfer functions. Cf. also PR 43665.
Comment 19 Tobias Burnus 2011-03-04 18:08:14 UTC
(In reply to comment #16) > subroutine finish(var, wait) > asynchronous :: var > wait(wait) > var = 7 > end subroutine finish > > Here, the ME might be tempted to change the order of WAIT and "var = 7"; thus > in this case we probably need to tell the ME about the ASYNCHRONOUS attribute. A brute-force method would be to add a __sync_synchronize (BUILT_IN_SYNCHRONIZE) call after the WAIT. But maybe one finds a better way which restricts the barrier to only "var" (and other asynchronous variables) without influencing unrelated variables.
Comment 20 Jakub Jelinek 2011-03-04 18:11:56 UTC
If ASYNCHRONOUS expands to volatile, no barrier should be necessary.
Comment 21 Tobias Burnus 2011-04-15 09:10:56 UTC
(In reply to comment #19) > A brute-force method would be to add a __sync_synchronize Actually, this idea does not work properly - neither for INQUIRE(...,PENDING=) nor for ASYNCHRONOUS with MPI 3. (Cf. link below) (In reply to comment #20) > If ASYNCHRONOUS expands to volatile, no barrier should be necessary. Well, VOLATILE has the wrong semantics, i.e. it will only partially solve the problem. Additionally, you create huge missed-optimization issues. I have now asked at GCC@ (and fortran@) for some suggestions: http://gcc.gnu.org/ml/fortran/2011-04/msg00143.html (There is currently also a vivid discussion on J3's interop and MPI Forum's MPI3-Fortran mailing lists about ASYNCHRONOUS and nonblocking MPI calls.)
Comment 22 Tobias Burnus 2011-07-13 09:24:40 UTC
Note: TR 29113 introduces a wider coverage of ASYNCHRONOUS, cf. almost-PDTR 29113 at ftp://ftp.nag.co.uk/sc22wg5/N1851-N1900/N1866.pdf. I think the easiest is to not set the "restrict" of dummy arguments involved in ASYNCHRONOUS I/O. That way, one has: call user_write(id, var) ... call user_wait(id) var = ... The address of the "var" escapes at "user_write" and is thus available to "user_wait" - hence, "var =" will not be moved across "user_wait". Ditto for gfortran's asynchronous I/O.
Comment 23 Tobias Burnus 2011-07-13 13:15:39 UTC
(In reply to comment #22) > I think the easiest is to not set the "restrict" of dummy arguments involved in > ASYNCHRONOUS I/O. It seems that GCC understands something different than I by "restrict"; it seems to be much less aggressive than I thought - and seems to match exactly what we need, such that no action seems to be required. Cf. http://gcc.gnu.org/ml/gcc/2011-07/msg00208.html and see PR 49733 for a missed-optimization PR, which is for adding an variable attribute which does optimize across function calls - which is allowed most of the time in Fortran (except for ASYNCHRONOUS and for coarrays).
Comment 24 Janne Blomqvist 2016-10-09 18:17:20 UTC