Bug 25561

Summary: Eventually get rid of the Alloc Stream Facility
Product: gcc Reporter: Janne Blomqvist <jb>
Component: libfortranAssignee: Janne Blomqvist <jb>
Status: RESOLVED FIXED    
Severity: enhancement CC: gcc-bugs, tkoenig
Priority: P3    
Version: unknown   
Target Milestone: ---   
Host: Target:
Build: Known to work:
Known to fail: Last reconfirmed: 2006-01-29 23:25:44
Bug Depends on:    
Bug Blocks: 25829, 25830, 25828, 38668    

Description Janne Blomqvist 2005-12-25 18:23:06 UTC
Currently the gfortran IO library is based on a design called the "Alloc Stream Facility" (ASF), described in the paper 

 *  "Exploiting the advantages of mapped files for stream I/O",
 *  O. Krieger, M. Stumm and R. Umrau, "Proceedings of the 1992 Winter
 *  USENIX conference", p. 27-42.

The idea is to present an interface that avoids an extra copy when used together with mmap, and the application uses ASF directly instead of read/write. However, Fortran IO follows a traditional read/write model, and thus the buffer copy is still needed in the library.

Thus, ASF makes the IO library unnecessarily complex and doesn't improve performance. If ASF would provide any real benefit, it could be implemented in the C library or the kernel, and any program using read/write, including the gfortran IO library, would benefit.
Comment 1 Janne Blomqvist 2006-01-17 22:10:14 UTC
Switched dependencies to the correct order.
Comment 2 Thomas Koenig 2006-01-29 23:25:44 UTC
Good idea.

Confirmed.
Comment 3 Francois-Xavier Coudert 2007-09-14 11:33:22 UTC
Why marking this as blocking asynchronous I/O?
Comment 4 Janne Blomqvist 2007-09-16 14:12:50 UTC
(In reply to comment #3)
> Why marking this as blocking asynchronous I/O?

Well, no hard dependency. I just thought it would make sense to do this first, as it would reduce the work required to convert the library to a more traditional read/write design. I'm sure it's possible to shoehorn async i/o into the current library as well. Especially as there is no point in adding support for "real" async i/o at the moment, since e.g. Linux supports it only in conjunction with direct I/O. Frontend syntax support can be added before the actual backend takes advantage of it.



Comment 5 Janne Blomqvist 2008-05-02 15:37:19 UTC
Working on a patch.
Comment 6 Janne Blomqvist 2008-05-13 19:07:02 UTC
Patch for part 1 here:

http://gcc.gnu.org/ml/gcc-patches/2008-05/msg00769.html
Comment 7 Janne Blomqvist 2008-05-15 15:54:29 UTC
Subject: Bug 25561

Author: jb
Date: Thu May 15 15:53:34 2008
New Revision: 135373

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=135373
Log:
Part 1 of PR 25561.

2008-05-15  Janne Blomqvist  <jb@gcc.gnu.org>

	PR libfortran/25561
	* Makefile.am: Add fbuf.c to gfor_io_src.
	* Makefile.in: Regenerate.
	* io/io.h (read_block): Remove.
	(struct stream): Remove alloc_r_at function pointer.
	(salloc_r): Remove.
	(salloc_r_at): Remove.
	(salloc_w_at): Remove.
	(salloc_w): Remove offset argument.
	(struct fbuf): New struct for format buffer.
	(struct gfc_unit): Add fbuf.
	(read_block_form): New prototype.
	(fbuf_init): Likewise.
	(fbuf_destroy): Likewise.
	(fbuf_reset): Likewise.
	(fbuf_alloc): Likewise.
	(fbuf_flush): Likewise.
	(fbuf_seek): Likewise.
	* io/file_pos.c (formatted_backspace): Change to use sread.
	(unformatted_backspace): Likewise.
	(st_backspace): Flush format buffer.
	(st_rewind): Likewise.
	* io/list_read.c (next_char): Likewise.
	(nml_query): Tidying, flush format buffer.
	* io/open.c (new_unit): Init format buffer.
	* io/read.c (read_l): Change to use read_block_form.
	(read_a): Likewise.
	(read_decimal): Likewise.
	(read_radix): Likewise.
	(read_f): Likewise.
	(read_x): Empty reads also for stream I/O.
	* io/transfer.c (read_sf): Change to use sread.
	(read_block): Rename to read_block_form, change prototype, use sread.
	(read_block_direct): Don't seek stream files.
	(write_block): Change to use fbuf if external file, don't seek stream
	files.
	(write_buf): Don't seek stream files.
	(formatted_transfer_scalar): Use fbuf for external files.
	(us_read): Change to use sread.
	(pre_position): Do nothing for stream I/O.
	(data_transfer_init): Flush fbuf when switching from write to read, if
	POS is specified, seek stream file to correct offset.
	(skip_record): Change to use sread.
	(min_off): New function.
	(next_record_r): Change to use sread.
	(next_record_w): Change to use sset/sseek, flush fbuf.
	(finalize_transfer): Flush fbuf.
	* io/unit.c (init_units): Init fbuf for stdout, stderr.
	(close_unit_1): Destroy fbuf.
	(finish_last_advance_record): Flush fbuf, no need to seek.
	* io/unix.c (fd_alloc_r_at): Remove unused where argument.
	(fd_alloc_w_at): Likewise.
	(fd_read): Remove third argument to fd_alloc_r_at.
	(fd_write): Remove third argument to fd_alloc_w_at.
	(fd_sset): Likewise.
	(fd_open): Don't set alloc_r_at.
	(mem_alloc_r_at): Remove unused where argument.
	(mem_alloc_w_at): Likewise.
	(mem_read): Don't incorrectly return previous errno, remove unused
	third argument to alloc function.
	(mem_write): Likewise.
	(mem_set): Likewise.
	(open_internal): Don't set alloc_r_at pointer.
	* io/fbuf.c: New file.

Modified:
    trunk/libgfortran/Makefile.am
    trunk/libgfortran/Makefile.in
    trunk/libgfortran/io/file_pos.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c

Comment 8 Janne Blomqvist 2008-05-15 18:32:29 UTC
Subject: Bug 25561

Author: jb
Date: Thu May 15 18:30:29 2008
New Revision: 135388

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=135388
Log:
Remove XFAIL from gfortran.dg/fmt_t_7.f

2008-05-15  Janne Blomqvist  <jb@gcc.gnu.org>

	PR libfortran/34974
	* gfortran.dg/fmt_t_7.f: Remove XFAIL, patch for PR 25561/Part 1 fixes
	this.

Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gfortran.dg/fmt_t_7.f

Comment 9 Janne Blomqvist 2008-05-16 08:12:39 UTC
Patch with some improvements for part 1 here:

http://gcc.gnu.org/ml/gcc-patches/2008-05/msg00924.html
Comment 10 Janne Blomqvist 2008-05-16 17:43:28 UTC
Subject: Bug 25561

Author: jb
Date: Fri May 16 17:42:44 2008
New Revision: 135433

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=135433
Log:
Part 1.1 of PR25561.

2008-05-15  Janne Blomqvist  <jb@gcc.gnu.org>

	PR libfortran/25561
	* io/io.h (struct fbuf): Change pointer to position offset.
	* io/fbuf.c (fbuf_init): Reduce default size of buffer, ptr=>pos
	changes.
	(fbuf_reset): ptr=>pos changes.
	(fbuf_alloc): If the request doesn't fit, don't waste memory by
	keeping flushed bytes. ptr=>pos changes.
	(fbuf_flush): ptr=>pos changes.
	(fbuf_seek): Don't seek past the left tab limit, don't update active
	byte count.
	* io/open.c (new_unit): If RECL has been specified, used that as
	initial buffer size.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/fbuf.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/open.c

Comment 11 Janne Blomqvist 2009-01-05 22:15:20 UTC
Patch getting rid of ASF for external files:

http://gcc.gnu.org/ml/gcc-patches/2009-01/msg00222.html
Comment 12 Janne Blomqvist 2009-01-18 09:30:22 UTC
Small patch on top of the big patch in comment #11 in order to fix a couple minor issues:

http://gcc.gnu.org/ml/fortran/2009-01/msg00152.html
Comment 13 Janne Blomqvist 2009-03-22 10:51:19 UTC
Subject: Bug 25561

Author: jb
Date: Sun Mar 22 10:51:05 2009
New Revision: 144993

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=144993
Log:
2009-03-22  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/25561 libfortran/37754
	* io/io.h (struct stream): Define new stream interface function
	pointers, and inline functions for accessing it.
	(struct fbuf): Use int instead of size_t, remove flushed element.
	(mem_alloc_w): New prototype.
	(mem_alloc_r): New prototype.
	(stream_at_bof): Remove prototype.
	(stream_at_eof): Remove prototype.
	(file_position): Remove prototype.
	(flush): Remove prototype.
	(stream_offset): Remove prototype.
	(unit_truncate): New prototype.
	(read_block_form): Change to return pointer, int* argument.
	(hit_eof): New prototype.
	(fbuf_init): Change prototype.
	(fbuf_reset): Change prototype.
	(fbuf_alloc): Change prototype.
	(fbuf_flush): Change prototype.
	(fbuf_seek): Change prototype.
	(fbuf_read): New prototype.
	(fbuf_getc_refill): New prototype.
	(fbuf_getc): New inline function.
        * io/fbuf.c (fbuf_init): Use int, get rid of flushed.
	(fbuf_debug): New function.
	(fbuf_reset): Flush, and return position offset.
	(fbuf_alloc): Simplify, don't flush, just realloc.
	(fbuf_flush): Make usable for read mode, salvage remaining bytes.
	(fbuf_seek): New whence argument.
	(fbuf_read): New function.
	(fbuf_getc_refill): New function.
	* io/file_pos.c (formatted_backspace): Use new stream interface.
	(unformatted_backspace): Likewise.
	(st_backspace): Make sure format buffer is reset, use new stream
	interface, use unit_truncate.
	(st_endfile): Likewise.
	(st_rewind): Likewise.
	* io/intrinsics.c: Use new stream interface.
	* io/list_read.c (push_char): Don't use u.p.scratch, use realloc
	to resize.
	(free_saved): Don't check u.p.scratch.
	(next_char): Use new stream interface, use fbuf_getc() for external files.
	(finish_list_read): flush format buffer.
	(nml_query): Update to use modified interface:s
	* io/open.c (test_endfile): Use new stream interface.
	(edit_modes): Likewise.
	(new_unit): Likewise, set bytes_left to 1 for stream files.
	* io/read.c (read_l): Use new read_block_form interface.
	(read_utf8): Likewise.
	(read_utf8_char1): Likewise.
	(read_default_char1): Likewise.
	(read_utf8_char4): Likewise.
	(read_default_char4): Likewise.
	(read_a): Likewise.
	(read_a_char4): Likewise.
	(read_decimal): Likewise.
	(read_radix): Likewise.
	(read_f): Likewise.
	* io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
	usage of u.p.line_buffer.
	(read_block_form): Update interface to return pointer, use
	fbuf_read for direct access.
	(read_block_direct): Update to new stream interface.
	(write_block): Use mem_alloc_w for internal I/O.
	(write_buf): Update to new stream interface.
	(formatted_transfer_scalar): Don't use u.p.line_buffer, use
	fbuf_seek for external files.
	(us_read): Update to new stream interface.
	(us_write): Likewise.
	(data_transfer_init): Always check if we switch modes and flush.
	(skip_record): Use new stream interface, fix comparison.
	(next_record_r): Check for and reset u.p.at_eof, use new stream
	interface, use fbuf_getc for spacing.
	(write_us_marker): Update to new stream interface, don't inline.
	(next_record_w_unf): Likewise.
	(sset): New function.
	(next_record_w): Use new stream interface, use fbuf for printing
	newline.
	(next_record): Use new stream interface.
	(finalize_transfer): Remove sfree call, use new stream interface.
	(st_iolength_done): Don't use u.p.scratch.
	(st_read): Don't check for end of file.
	(st_read_done): Don't use u.p.scratch, use unit_truncate.
	(hit_eof): New function.
	* io/unit.c (init_units): Always init fbuf for formatted units.
	(update_position): Use new stream interface.
	(unit_truncate): New function.
	(finish_last_advance_record): Use fbuf to print newline.
	* io/unix.c: Remove unused SSIZE_MAX macro.
	(BUFFER_SIZE): Make static const variable rather than macro.
	(struct unix_stream): Remove dirty_offset, len, method,
	small_buffer. Order elements by decreasing size.
	(struct int_stream): Remove.
	(move_pos_offset): Remove usage of dirty_offset.
	(reset_stream): Remove.
	(do_read): Rename to raw_read, update to match new stream
	interface.
	(do_write): Rename to raw_write, update to new stream interface.
	(raw_seek): New function.
	(raw_tell): New function.
	(raw_truncate): New function.
	(raw_close): New function.
	(raw_flush): New function.
	(raw_init): New function.
	(fd_alloc): Remove.
	(fd_alloc_r_at): Remove.
	(fd_alloc_w_at): Remove.
	(fd_sfree): Remove.
	(fd_seek): Remove.
	(fd_truncate): Remove.
	(fd_sset): Remove.
	(fd_read): Remove.
	(fd_write): Remove.
	(fd_close): Remove.
	(fd_open): Remove.
	(fd_flush): Rename to buf_flush, update to new stream interface
	and unix_stream.
	(buf_read): New function.
	(buf_write): New function.
	(buf_seek): New function.
	(buf_tell): New function.
	(buf_truncate): New function.
	(buf_close): New function.
	(buf_init): New function.
	(mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
	(mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
	(mem_read): Change to match new stream interface.
	(mem_write): Likewise.
	(mem_seek): Likewise.
	(mem_tell): Likewise.
	(mem_truncate): Likewise.
	(mem_close): Likewise.
	(mem_flush): New function.
	(mem_sfree): Remove.
	(empty_internal_buffer): Cast to correct type.
	(open_internal): Use correct type, init function pointers.
	(fd_to_stream): Test whether to open file as buffered or raw.
	(output_stream): Remove mode set.
	(error_stream): Likewise.
	(flush_all_units_1): Use new stream interface.
	(flush_all_units): Likewise.
	(stream_at_bof): Remove.
	(stream_at_eof): Remove.
	(file_position): Remove.
	(file_length): Update logic to use stream interface.
	(flush): Remove.
	(stream_offset): Remove.
	* io/write.c (write_utf8_char4): Use int instead of size_t.
	(write_x): Extra safety check.
	(namelist_write_newline): Use new stream interface.

Added:
    branches/fortran-dev/libgfortran/ChangeLog.dev
Modified:
    branches/fortran-dev/libgfortran/io/fbuf.c
    branches/fortran-dev/libgfortran/io/file_pos.c
    branches/fortran-dev/libgfortran/io/intrinsics.c
    branches/fortran-dev/libgfortran/io/open.c
    branches/fortran-dev/libgfortran/io/read.c
    branches/fortran-dev/libgfortran/io/unix.c
    branches/fortran-dev/libgfortran/io/write.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c

Comment 14 Jerry DeLisle 2009-04-05 20:14:22 UTC
Subject: Bug 25561

Author: jvdelisle
Date: Sun Apr  5 20:13:56 2009
New Revision: 145571

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=145571
Log:
2009-04-05  Daniel Kraft  <d@domob.eu>

	PR fortran/38654
	* io/read.c (read_f): Reworked to speed up floating point parsing.
	(convert_real): Use pointer-casting instead of memcpy and temporaries.

2009-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/37754
	* io/io.h (format_hash_entry): New structure for hash table.
	(format_hash_table): The hash table itself.
	(free_format_data): Revise function prototype.
	(free_format_hash_table, init_format_hash,
	free_format_hash): New function prototypes.
	* io/unit.c (close_unit_1): Use free_format_hash_table.
	* io/transfer.c (st_read_done, st_write_done): Free format data if
	internal unit.
	* io/format.c (free_format_hash_table): New function that frees any
	memory allocated previously for cached format data.
	(reset_node): New static helper function to reset the format counters
	for a format node.
	(reset_fnode_counters): New static function recursively calls reset_node
	to traverse the	fnode tree.
	(format_hash): New simple hash function based on XOR, probabalistic,
	tosses collisions.
	(save_parsed_format): New static function to save the parsed format
	data to use again.
	(find_parsed_format): New static function searches the hash table
	looking for a match.
	(free_format_data): Revised to accept pointer to format data rather than
	the dtp pointer so that the function can be used in more places.
	(format_lex): Editorial.
	(parse_format_list): Set flag used to determine of format data hashing
	is to be used.  Internal units are not persistent enough for this.
	(revert): Move to ne location in file.
	(parse_format): Use new functions to look for previously parsed
	format strings and use them rather than re-parse.  If not found, saves
	the parsed format data for later use.
	
2009-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/37754
	* io/transfer.c (formatted_transfer_scalar): Remove this function by
	factoring it into two new functions, one for read and one for write,
	eliminating all the conditionals for read or write mode.
	(formatted transfer_scalar_read): New function.
	(formatted transfer_scalar_write): New function.
	(formatted_transfer): Use new functions.

2009-04-05  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/25561 libfortran/37754
	* io/io.h (struct stream): Define new stream interface function
	pointers, and inline functions for accessing it.
	(struct fbuf): Use int instead of size_t, remove flushed element.
	(mem_alloc_w): New prototype.
	(mem_alloc_r): New prototype.
	(stream_at_bof): Remove prototype.
	(stream_at_eof): Remove prototype.
	(file_position): Remove prototype.
	(flush): Remove prototype.
	(stream_offset): Remove prototype.
	(unit_truncate): New prototype.
	(read_block_form): Change to return pointer, int* argument.
	(hit_eof): New prototype.
	(fbuf_init): Change prototype.
	(fbuf_reset): Change prototype.
	(fbuf_alloc): Change prototype.
	(fbuf_flush): Change prototype.
	(fbuf_seek): Change prototype.
	(fbuf_read): New prototype.
	(fbuf_getc_refill): New prototype.
	(fbuf_getc): New inline function.
        * io/fbuf.c (fbuf_init): Use int, get rid of flushed.
	(fbuf_debug): New function.
	(fbuf_reset): Flush, and return position offset.
	(fbuf_alloc): Simplify, don't flush, just realloc.
	(fbuf_flush): Make usable for read mode, salvage remaining bytes.
	(fbuf_seek): New whence argument.
	(fbuf_read): New function.
	(fbuf_getc_refill): New function.
	* io/file_pos.c (formatted_backspace): Use new stream interface.
	(unformatted_backspace): Likewise.
	(st_backspace): Make sure format buffer is reset, use new stream
	interface, use unit_truncate.
	(st_endfile): Likewise.
	(st_rewind): Likewise.
	* io/intrinsics.c: Use new stream interface.
	* io/list_read.c (push_char): Don't use u.p.scratch, use realloc
	to resize.
	(free_saved): Don't check u.p.scratch.
	(next_char): Use new stream interface, use fbuf_getc() for external files.
	(finish_list_read): flush format buffer.
	(nml_query): Update to use modified interface:s
	* io/open.c (test_endfile): Use new stream interface.
	(edit_modes): Likewise.
	(new_unit): Likewise, set bytes_left to 1 for stream files.
	* io/read.c (read_l): Use new read_block_form interface.
	(read_utf8): Likewise.
	(read_utf8_char1): Likewise.
	(read_default_char1): Likewise.
	(read_utf8_char4): Likewise.
	(read_default_char4): Likewise.
	(read_a): Likewise.
	(read_a_char4): Likewise.
	(read_decimal): Likewise.
	(read_radix): Likewise.
	(read_f): Likewise.
	* io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
	usage of u.p.line_buffer.
	(read_block_form): Update interface to return pointer, use
	fbuf_read for direct access.
	(read_block_direct): Update to new stream interface.
	(write_block): Use mem_alloc_w for internal I/O.
	(write_buf): Update to new stream interface.
	(formatted_transfer_scalar): Don't use u.p.line_buffer, use
	fbuf_seek for external files.
	(us_read): Update to new stream interface.
	(us_write): Likewise.
	(data_transfer_init): Always check if we switch modes and flush.
	(skip_record): Use new stream interface, fix comparison.
	(next_record_r): Check for and reset u.p.at_eof, use new stream
	interface, use fbuf_getc for spacing.
	(write_us_marker): Update to new stream interface, don't inline.
	(next_record_w_unf): Likewise.
	(sset): New function.
	(next_record_w): Use new stream interface, use fbuf for printing
	newline.
	(next_record): Use new stream interface.
	(finalize_transfer): Remove sfree call, use new stream interface.
	(st_iolength_done): Don't use u.p.scratch.
	(st_read): Don't check for end of file.
	(st_read_done): Don't use u.p.scratch, use unit_truncate.
	(hit_eof): New function.
	* io/unit.c (init_units): Always init fbuf for formatted units.
	(update_position): Use new stream interface.
	(unit_truncate): New function.
	(finish_last_advance_record): Use fbuf to print newline.
	* io/unix.c: Remove unused SSIZE_MAX macro.
	(BUFFER_SIZE): Make static const variable rather than macro.
	(struct unix_stream): Remove dirty_offset, len, method,
	small_buffer. Order elements by decreasing size.
	(struct int_stream): Remove.
	(move_pos_offset): Remove usage of dirty_offset.
	(reset_stream): Remove.
	(do_read): Rename to raw_read, update to match new stream
	interface.
	(do_write): Rename to raw_write, update to new stream interface.
	(raw_seek): New function.
	(raw_tell): New function.
	(raw_truncate): New function.
	(raw_close): New function.
	(raw_flush): New function.
	(raw_init): New function.
	(fd_alloc): Remove.
	(fd_alloc_r_at): Remove.
	(fd_alloc_w_at): Remove.
	(fd_sfree): Remove.
	(fd_seek): Remove.
	(fd_truncate): Remove.
	(fd_sset): Remove.
	(fd_read): Remove.
	(fd_write): Remove.
	(fd_close): Remove.
	(fd_open): Remove.
	(fd_flush): Rename to buf_flush, update to new stream interface
	and unix_stream.
	(buf_read): New function.
	(buf_write): New function.
	(buf_seek): New function.
	(buf_tell): New function.
	(buf_truncate): New function.
	(buf_close): New function.
	(buf_init): New function.
	(mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
	(mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
	(mem_read): Change to match new stream interface.
	(mem_write): Likewise.
	(mem_seek): Likewise.
	(mem_tell): Likewise.
	(mem_truncate): Likewise.
	(mem_close): Likewise.
	(mem_flush): New function.
	(mem_sfree): Remove.
	(empty_internal_buffer): Cast to correct type.
	(open_internal): Use correct type, init function pointers.
	(fd_to_stream): Test whether to open file as buffered or raw.
	(output_stream): Remove mode set.
	(error_stream): Likewise.
	(flush_all_units_1): Use new stream interface.
	(flush_all_units): Likewise.
	(stream_at_bof): Remove.
	(stream_at_eof): Remove.
	(file_position): Remove.
	(file_length): Update logic to use stream interface.
	(flush): Remove.
	(stream_offset): Remove.
	* io/write.c (write_utf8_char4): Use int instead of size_t.
	(write_x): Extra safety check.
	(namelist_write_newline): Use new stream interface.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/fbuf.c
    trunk/libgfortran/io/file_pos.c
    trunk/libgfortran/io/format.c
    trunk/libgfortran/io/intrinsics.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c
    trunk/libgfortran/io/write.c

Comment 15 Jerry DeLisle 2009-05-27 01:22:19 UTC
Subject: Bug 25561

Author: jvdelisle
Date: Wed May 27 01:21:22 2009
New Revision: 147887

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=147887
Log:
2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/37754
	* io/write_float.def: Simplify format calculation.
	
2009-05-23  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	Backport from mainline:
	PR fortran/22423
	* io/transfer.c (read_block_direct): Avoid warning.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/39667
	* io/file_pos.c (st_rewind): Don't truncate or flush.
	* io/intrinsics.c (fgetc): Flush if switching mode.
	(fputc): Likewise.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/39782
	* io/transfer.c (data_transfer_init): Don't flush before seek.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	* io/io.h (is_preconnected): Remove prototype.
	* io/unix.c (is_preconnected): Remove function.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/38668
	* io/transfer.c (finalize_transfer): Don't flush for advance='no'.

2009-05-23 Danny Smith  <dannysmith@clear.net.nz>

	Backport from mainline:
	* io/write.c (itoa) : Rename back to gfc_itoa.
	(write_i): Adjust call to write_decimal.
	(write_integer):  Use gfc_itoa.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	* io/io.h (move_pos_offset): Remove prototype.
	* io/transfer.c (formatted_transfer_scalar_read): Use sseek
	instead of move_pos_offset.
	* io/unix.c (move_pos_offset): Remove.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/39665 libfortran/39702 libfortran/39709
	* io/io.h (st_parameter_dt): Revert aligned attribute from u.p.value.
	* io/list_read.c (read_complex): Read directly into user pointer.
	(read_real): Likewise.
	(list_formatted_read_scalar): Update read_complex and read_real calls.
	(nml_read_obj): Read directly into user pointer.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/39665
	* io/io.h (st_parameter_dt): Add aligned attribute to u.p.value.
	* io/read.c (convert_real): Add note about alignment requirements.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	* io/open.c (already_open): Test for POSIX close return value.
	* io/unit.c (close_unit_1): Likewise.
	* io/unix.c (raw_close): Return 0 for success for preconnected units.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	* runtime/error.c (gfc_itoa): Move to io/write.c
	(xtoa): Rename to gfc_xtoa.
	* runtime/backtrace.c (show_backtrace): Call gfc_xtoa.
	* libgfortran.h (gfc_itoa): Remove prototype.
	(xtoa): Rename prototype to gfc_xtoa.
	* io/list_read.c (nml_read_obj): Use size_t for string length.
	* io/transfer.c (read_block_direct): Change nbytes arg from
	pointer to value.
	(unformatted_read): Minor cleanup, call read_block_directly properly.
	(skip_record): Use ssize_t.
	(next_record_w_unf): Avoid stell() call by calling sseek with SEEK_CUR.
	(iolength_transfer): Make sure to multiply before cast.
	* io/intrinsics.c (fgetc): Remove unnecessary variable.
	* io/format.c (format_hash): Use gfc_charlen_type.
	* io/write.c (itoa): Move from runtime/error.c:gfc_itoa, rename,
	make static.
	(write_i): Call with pointer to itoa.
	(write_z): Call with pointer to gfc_xtoa.
	(write_integer): Pointer to itoa.
	(nml_write_obj): Type cleanup, don't call strlen in loop.
	
2009-05-23  H.J. Lu  <hongjiu.lu@intel.com>

	Backport from mainline:
	PR libgfortran/39664
	* io/unix.c (raw_close): Don't close STDOUT_FILENO,
	STDERR_FILENO nor STDIN_FILENO.

2009-05-23  David Edelsohn  <edelsohn@gnu.org>
	
	Backport from mainline:
	* io/io.h (struct stream): Rename truncate to trunc.
	(struncate): Same.
	* io/unix.c (raw_init): Rename truncate to trunc.
	(buf_init): Same.
	(open_internal): Same.
	
2009-05-23  Daniel Kraft  <d@domob.eu>

	Backport from mainline:
	PR fortran/38654
	* io/read.c (read_f): Reworked to speed up floating point parsing.
	(convert_real): Use pointer-casting instead of memcpy and temporaries.

2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/37754
	* io/io.h (format_hash_entry): New structure for hash table.
	(format_hash_table): The hash table itself.
	(free_format_data): Revise function prototype.
	(free_format_hash_table, init_format_hash,
	free_format_hash): New function prototypes.
	* io/unit.c (close_unit_1): Use free_format_hash_table.
	* io/transfer.c (st_read_done, st_write_done): Free format data if
	internal unit.
	* io/format.c (free_format_hash_table): New function that frees any
	memory allocated previously for cached format data.
	(reset_node): New static helper function to reset the format counters
	for a format node.
	(reset_fnode_counters): New static function recursively calls reset_node
	to traverse the	fnode tree.
	(format_hash): New simple hash function based on XOR, probabalistic,
	tosses collisions.
	(save_parsed_format): New static function to save the parsed format
	data to use again.
	(find_parsed_format): New static function searches the hash table
	looking for a match.
	(free_format_data): Revised to accept pointer to format data rather than
	the dtp pointer so that the function can be used in more places.
	(format_lex): Editorial.
	(parse_format_list): Set flag used to determine of format data hashing
	is to be used.  Internal units are not persistent enough for this.
	(revert): Move to ne location in file.
	(parse_format): Use new functions to look for previously parsed
	format strings and use them rather than re-parse.  If not found, saves
	the parsed format data for later use.
	
2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/37754
	* io/transfer.c (formatted_transfer_scalar): Remove this function by
	factoring it into two new functions, one for read and one for write,
	eliminating all the conditionals for read or write mode.
	(formatted transfer_scalar_read): New function.
	(formatted transfer_scalar_write): New function.
	(formatted_transfer): Use new functions.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

	Backport from mainline:
	PR libfortran/25561 libfortran/37754
	* io/io.h (struct stream): Define new stream interface function
	pointers, and inline functions for accessing it.
	(struct fbuf): Use int instead of size_t, remove flushed element.
	(mem_alloc_w): New prototype.
	(mem_alloc_r): New prototype.
	(stream_at_bof): Remove prototype.
	(stream_at_eof): Remove prototype.
	(file_position): Remove prototype.
	(flush): Remove prototype.
	(stream_offset): Remove prototype.
	(unit_truncate): New prototype.
	(read_block_form): Change to return pointer, int* argument.
	(hit_eof): New prototype.
	(fbuf_init): Change prototype.
	(fbuf_reset): Change prototype.
	(fbuf_alloc): Change prototype.
	(fbuf_flush): Change prototype.
	(fbuf_seek): Change prototype.
	(fbuf_read): New prototype.
	(fbuf_getc_refill): New prototype.
	(fbuf_getc): New inline function.
	* io/fbuf.c (fbuf_init): Use int, get rid of flushed.
	(fbuf_debug): New function.
	(fbuf_reset): Flush, and return position offset.
	(fbuf_alloc): Simplify, don't flush, just realloc.
	(fbuf_flush): Make usable for read mode, salvage remaining bytes.
	(fbuf_seek): New whence argument.
	(fbuf_read): New function.
	(fbuf_getc_refill): New function.
	* io/file_pos.c (formatted_backspace): Use new stream interface.
	(unformatted_backspace): Likewise.
	(st_backspace): Make sure format buffer is reset, use new stream
	interface, use unit_truncate.
	(st_endfile): Likewise.
	(st_rewind): Likewise.
	* io/intrinsics.c: Use new stream interface.
	* io/list_read.c (push_char): Don't use u.p.scratch, use realloc
	to resize.
	(free_saved): Don't check u.p.scratch.
	(next_char): Use new stream interface, use fbuf_getc() for external files.
	(finish_list_read): flush format buffer.
	(nml_query): Update to use modified interface:s
	* io/open.c (test_endfile): Use new stream interface.
	(edit_modes): Likewise.
	(new_unit): Likewise, set bytes_left to 1 for stream files.
	* io/read.c (read_l): Use new read_block_form interface.
	(read_utf8): Likewise.
	(read_utf8_char1): Likewise.
	(read_default_char1): Likewise.
	(read_utf8_char4): Likewise.
	(read_default_char4): Likewise.
	(read_a): Likewise.
	(read_a_char4): Likewise.
	(read_decimal): Likewise.
	(read_radix): Likewise.
	(read_f): Likewise.
	* io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
	usage of u.p.line_buffer.
	(read_block_form): Update interface to return pointer, use
	fbuf_read for direct access.
	(read_block_direct): Update to new stream interface.
	(write_block): Use mem_alloc_w for internal I/O.
	(write_buf): Update to new stream interface.
	(formatted_transfer_scalar): Don't use u.p.line_buffer, use
	fbuf_seek for external files.
	(us_read): Update to new stream interface.
	(us_write): Likewise.
	(data_transfer_init): Always check if we switch modes and flush.
	(skip_record): Use new stream interface, fix comparison.
	(next_record_r): Check for and reset u.p.at_eof, use new stream
	interface, use fbuf_getc for spacing.
	(write_us_marker): Update to new stream interface, don't inline.
	(next_record_w_unf): Likewise.
	(sset): New function.
	(next_record_w): Use new stream interface, use fbuf for printing
	newline.
	(next_record): Use new stream interface.
	(finalize_transfer): Remove sfree call, use new stream interface.
	(st_iolength_done): Don't use u.p.scratch.
	(st_read): Don't check for end of file.
	(st_read_done): Don't use u.p.scratch, use unit_truncate.
	(hit_eof): New function.
	* io/unit.c (init_units): Always init fbuf for formatted units.
	(update_position): Use new stream interface.
	(unit_truncate): New function.
	(finish_last_advance_record): Use fbuf to print newline.
	* io/unix.c: Remove unused SSIZE_MAX macro.
	(BUFFER_SIZE): Make static const variable rather than macro.
	(struct unix_stream): Remove dirty_offset, len, method,
	small_buffer. Order elements by decreasing size.
	(struct int_stream): Remove.
	(move_pos_offset): Remove usage of dirty_offset.
	(reset_stream): Remove.
	(do_read): Rename to raw_read, update to match new stream
	interface.
	(do_write): Rename to raw_write, update to new stream interface.
	(raw_seek): New function.
	(raw_tell): New function.
	(raw_truncate): New function.
	(raw_close): New function.
	(raw_flush): New function.
	(raw_init): New function.
	(fd_alloc): Remove.
	(fd_alloc_r_at): Remove.
	(fd_alloc_w_at): Remove.
	(fd_sfree): Remove.
	(fd_seek): Remove.
	(fd_truncate): Remove.
	(fd_sset): Remove.
	(fd_read): Remove.
	(fd_write): Remove.
	(fd_close): Remove.
	(fd_open): Remove.
	(fd_flush): Rename to buf_flush, update to new stream interface
	and unix_stream.
	(buf_read): New function.
	(buf_write): New function.
	(buf_seek): New function.
	(buf_tell): New function.
	(buf_truncate): New function.
	(buf_close): New function.
	(buf_init): New function.
	(mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
	(mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
	(mem_read): Change to match new stream interface.
	(mem_write): Likewise.
	(mem_seek): Likewise.
	(mem_tell): Likewise.
	(mem_truncate): Likewise.
	(mem_close): Likewise.
	(mem_flush): New function.
	(mem_sfree): Remove.
	(empty_internal_buffer): Cast to correct type.
	(open_internal): Use correct type, init function pointers.
	(fd_to_stream): Test whether to open file as buffered or raw.
	(output_stream): Remove mode set.
	(error_stream): Likewise.
	(flush_all_units_1): Use new stream interface.
	(flush_all_units): Likewise.
	(stream_at_bof): Remove.
	(stream_at_eof): Remove.
	(file_position): Remove.
	(file_length): Update logic to use stream interface.
	(flush): Remove.
	(stream_offset): Remove.
	* io/write.c (write_utf8_char4): Use int instead of size_t.
	(write_x): Extra safety check.
	(namelist_write_newline): Use new stream interface.


Modified:
    branches/gcc-4_4-branch/libgfortran/ChangeLog
    branches/gcc-4_4-branch/libgfortran/io/fbuf.c
    branches/gcc-4_4-branch/libgfortran/io/file_pos.c
    branches/gcc-4_4-branch/libgfortran/io/format.c
    branches/gcc-4_4-branch/libgfortran/io/intrinsics.c
    branches/gcc-4_4-branch/libgfortran/io/io.h
    branches/gcc-4_4-branch/libgfortran/io/list_read.c
    branches/gcc-4_4-branch/libgfortran/io/open.c
    branches/gcc-4_4-branch/libgfortran/io/read.c
    branches/gcc-4_4-branch/libgfortran/io/transfer.c
    branches/gcc-4_4-branch/libgfortran/io/unit.c
    branches/gcc-4_4-branch/libgfortran/io/unix.c
    branches/gcc-4_4-branch/libgfortran/io/write.c
    branches/gcc-4_4-branch/libgfortran/io/write_float.def
    branches/gcc-4_4-branch/libgfortran/libgfortran.h
    branches/gcc-4_4-branch/libgfortran/runtime/backtrace.c
    branches/gcc-4_4-branch/libgfortran/runtime/error.c

Comment 16 Tobias Burnus 2009-06-04 06:43:15 UTC
Can this PR now be closed?
Comment 17 Janne Blomqvist 2009-06-04 07:40:40 UTC
(In reply to comment #16)
> Can this PR now be closed?

Yes, I think so. There are still some remnants of ASF in internal I/O, but I don't think it hurts.