Re: [patch, libgfortran] PR37754 [4.4 Regression] READ I/O Performance regression from 4.3 to 4.4/4.5

[Paul Richard Thomas <paul dot richard dot thomas at gmail dot com>]

Hi Jerry,

Well done to you and, apparently, many of the other gfortran maintainers :-)

Are you going to backport to 4.3 as well?

Paul

On Sat, May 23, 2009 at 1:59 PM, Jerry DeLisle <jvdelisle@verizon.net> wrote:
> The attached patch is a backport of the I/O performance patches and
> associated fixes from trunk.
>
> Regression tested on x86-64.
>
> OK for 4.4. ?This will close out the PR.
>
> Regards,
>
> Jerry
>
>
> 2009-05-23 ?Jerry DeLisle ?<jvdelisle@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/37754
> ? ? ? ?* io/write_float.def: Simplify format calculation.
>
> 2009-05-23 ?Francois-Xavier Coudert ?<fxcoudert@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR fortran/22423
> ? ? ? ?* io/transfer.c (read_block_direct): Avoid warning.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/39667
> ? ? ? ?* io/file_pos.c (st_rewind): Don't truncate or flush.
> ? ? ? ?* io/intrinsics.c (fgetc): Flush if switching mode.
> ? ? ? ?(fputc): Likewise.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/39782
> ? ? ? ?* io/transfer.c (data_transfer_init): Don't flush before seek.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/io.h (is_preconnected): Remove prototype.
> ? ? ? ?* io/unix.c (is_preconnected): Remove function.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/38668
> ? ? ? ?* io/transfer.c (finalize_transfer): Don't flush for advance='no'.
>
> 2009-05-23 Danny Smith ?<dannysmith@clear.net.nz>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/write.c (itoa) : Rename back to gfc_itoa.
> ? ? ? ?(write_i): Adjust call to write_decimal.
> ? ? ? ?(write_integer): ?Use gfc_itoa.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/io.h (move_pos_offset): Remove prototype.
> ? ? ? ?* io/transfer.c (formatted_transfer_scalar_read): Use sseek
> ? ? ? ?instead of move_pos_offset.
> ? ? ? ?* io/unix.c (move_pos_offset): Remove.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/39665 libfortran/39702 libfortran/39709
> ? ? ? ?* io/io.h (st_parameter_dt): Revert aligned attribute from u.p.value.
> ? ? ? ?* io/list_read.c (read_complex): Read directly into user pointer.
> ? ? ? ?(read_real): Likewise.
> ? ? ? ?(list_formatted_read_scalar): Update read_complex and read_real
> calls.
> ? ? ? ?(nml_read_obj): Read directly into user pointer.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/39665
> ? ? ? ?* io/io.h (st_parameter_dt): Add aligned attribute to u.p.value.
> ? ? ? ?* io/read.c (convert_real): Add note about alignment requirements.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/open.c (already_open): Test for POSIX close return value.
> ? ? ? ?* io/unit.c (close_unit_1): Likewise.
> ? ? ? ?* io/unix.c (raw_close): Return 0 for success for preconnected units.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/list_read.c (nml_read_obj): Use size_t for string length.
> ? ? ? ?* io/transfer.c (read_block_direct): Change nbytes arg from
> ? ? ? ?pointer to value.
> ? ? ? ?(unformatted_read): Minor cleanup, call read_block_directly properly.
> ? ? ? ?(skip_record): Use ssize_t.
> ? ? ? ?(next_record_w_unf): Avoid stell() call by calling sseek with
> SEEK_CUR.
> ? ? ? ?(iolength_transfer): Make sure to multiply before cast.
> ? ? ? ?* io/intrinsics.c (fgetc): Remove unnecessary variable.
> ? ? ? ?* io/format.c (format_hash): Use gfc_charlen_type.
> ? ? ? ?* io/write.c (itoa): Move from runtime/error.c:gfc_itoa, rename,
> ? ? ? ?make static.
> ? ? ? ?(write_i): Call with pointer to itoa.
> ? ? ? ?(write_z): Call with pointer to gfc_xtoa.
> ? ? ? ?(write_integer): Pointer to itoa.
> ? ? ? ?(nml_write_obj): Type cleanup, don't call strlen in loop.
>
> 2009-05-23 ?H.J. Lu ?<hongjiu.lu@intel.com>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libgfortran/39664
> ? ? ? ?* io/unix.c (raw_close): Don't close STDOUT_FILENO,
> ? ? ? ?STDERR_FILENO nor STDIN_FILENO.
>
> 2009-05-23 ?David Edelsohn ?<edelsohn@gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?* io/io.h (struct stream): Rename truncate to trunc.
> ? ? ? ?(struncate): Same.
> ? ? ? ?* io/unix.c (raw_init): Rename truncate to trunc.
> ? ? ? ?(buf_init): Same.
> ? ? ? ?(open_internal): Same.
>
> 2009-05-23 ?Daniel Kraft ?<d@domob.eu>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR fortran/38654
> ? ? ? ?* io/read.c (read_f): Reworked to speed up floating point parsing.
> ? ? ? ?(convert_real): Use pointer-casting instead of memcpy and
> temporaries.
>
> 2009-05-23 ?Jerry DeLisle ?<jvdelisle@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/37754
> ? ? ? ?* io/io.h (format_hash_entry): New structure for hash table.
> ? ? ? ?(format_hash_table): The hash table itself.
> ? ? ? ?(free_format_data): Revise function prototype.
> ? ? ? ?(free_format_hash_table, init_format_hash,
> ? ? ? ?free_format_hash): New function prototypes.
> ? ? ? ?* io/unit.c (close_unit_1): Use free_format_hash_table.
> ? ? ? ?* io/transfer.c (st_read_done, st_write_done): Free format data if
> ? ? ? ?internal unit.
> ? ? ? ?* io/format.c (free_format_hash_table): New function that frees any
> ? ? ? ?memory allocated previously for cached format data.
> ? ? ? ?(reset_node): New static helper function to reset the format counters
> ? ? ? ?for a format node.
> ? ? ? ?(reset_fnode_counters): New static function recursively calls
> reset_node
> ? ? ? ?to traverse the fnode tree.
> ? ? ? ?(format_hash): New simple hash function based on XOR, probabalistic,
> ? ? ? ?tosses collisions.
> ? ? ? ?(save_parsed_format): New static function to save the parsed format
> ? ? ? ?data to use again.
> ? ? ? ?(find_parsed_format): New static function searches the hash table
> ? ? ? ?looking for a match.
> ? ? ? ?(free_format_data): Revised to accept pointer to format data rather
> than
> ? ? ? ?the dtp pointer so that the function can be used in more places.
> ? ? ? ?(format_lex): Editorial.
> ? ? ? ?(parse_format_list): Set flag used to determine of format data
> hashing
> ? ? ? ?is to be used. ?Internal units are not persistent enough for this.
> ? ? ? ?(revert): Move to ne location in file.
> ? ? ? ?(parse_format): Use new functions to look for previously parsed
> ? ? ? ?format strings and use them rather than re-parse. ?If not found,
> saves
> ? ? ? ?the parsed format data for later use.
>
> 2009-05-23 ?Jerry DeLisle ?<jvdelisle@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/37754
> ? ? ? ?* io/transfer.c (formatted_transfer_scalar): Remove this function by
> ? ? ? ?factoring it into two new functions, one for read and one for write,
> ? ? ? ?eliminating all the conditionals for read or write mode.
> ? ? ? ?(formatted transfer_scalar_read): New function.
> ? ? ? ?(formatted transfer_scalar_write): New function.
> ? ? ? ?(formatted_transfer): Use new functions.
>
> 2009-05-23 ?Janne Blomqvist ?<jb@gcc.gnu.org>
>
> ? ? ? ?Backport from mainline:
> ? ? ? ?PR libfortran/25561 libfortran/37754
> ? ? ? ?* io/io.h (struct stream): Define new stream interface function
> ? ? ? ?pointers, and inline functions for accessing it.
> ? ? ? ?(struct fbuf): Use int instead of size_t, remove flushed element.
> ? ? ? ?(mem_alloc_w): New prototype.
> ? ? ? ?(mem_alloc_r): New prototype.
> ? ? ? ?(stream_at_bof): Remove prototype.
> ? ? ? ?(stream_at_eof): Remove prototype.
> ? ? ? ?(file_position): Remove prototype.
> ? ? ? ?(flush): Remove prototype.
> ? ? ? ?(stream_offset): Remove prototype.
> ? ? ? ?(unit_truncate): New prototype.
> ? ? ? ?(read_block_form): Change to return pointer, int* argument.
> ? ? ? ?(hit_eof): New prototype.
> ? ? ? ?(fbuf_init): Change prototype.
> ? ? ? ?(fbuf_reset): Change prototype.
> ? ? ? ?(fbuf_alloc): Change prototype.
> ? ? ? ?(fbuf_flush): Change prototype.
> ? ? ? ?(fbuf_seek): Change prototype.
> ? ? ? ?(fbuf_read): New prototype.
> ? ? ? ?(fbuf_getc_refill): New prototype.
> ? ? ? ?(fbuf_getc): New inline function.
> ? ? ? ?* io/fbuf.c (fbuf_init): Use int, get rid of flushed.
> ? ? ? ?(fbuf_debug): New function.
> ? ? ? ?(fbuf_reset): Flush, and return position offset.
> ? ? ? ?(fbuf_alloc): Simplify, don't flush, just realloc.
> ? ? ? ?(fbuf_flush): Make usable for read mode, salvage remaining bytes.
> ? ? ? ?(fbuf_seek): New whence argument.
> ? ? ? ?(fbuf_read): New function.
> ? ? ? ?(fbuf_getc_refill): New function.
> ? ? ? ?* io/file_pos.c (formatted_backspace): Use new stream interface.
> ? ? ? ?(unformatted_backspace): Likewise.
> ? ? ? ?(st_backspace): Make sure format buffer is reset, use new stream
> ? ? ? ?interface, use unit_truncate.
> ? ? ? ?(st_endfile): Likewise.
> ? ? ? ?(st_rewind): Likewise.
> ? ? ? ?* io/intrinsics.c: Use new stream interface.
> ? ? ? ?* io/list_read.c (push_char): Don't use u.p.scratch, use realloc
> ? ? ? ?to resize.
> ? ? ? ?(free_saved): Don't check u.p.scratch.
> ? ? ? ?(next_char): Use new stream interface, use fbuf_getc() for external
> files.
> ? ? ? ?(finish_list_read): flush format buffer.
> ? ? ? ?(nml_query): Update to use modified interface:s
> ? ? ? ?* io/open.c (test_endfile): Use new stream interface.
> ? ? ? ?(edit_modes): Likewise.
> ? ? ? ?(new_unit): Likewise, set bytes_left to 1 for stream files.
> ? ? ? ?* io/read.c (read_l): Use new read_block_form interface.
> ? ? ? ?(read_utf8): Likewise.
> ? ? ? ?(read_utf8_char1): Likewise.
> ? ? ? ?(read_default_char1): Likewise.
> ? ? ? ?(read_utf8_char4): Likewise.
> ? ? ? ?(read_default_char4): Likewise.
> ? ? ? ?(read_a): Likewise.
> ? ? ? ?(read_a_char4): Likewise.
> ? ? ? ?(read_decimal): Likewise.
> ? ? ? ?(read_radix): Likewise.
> ? ? ? ?(read_f): Likewise.
> ? ? ? ?* io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
> ? ? ? ?usage of u.p.line_buffer.
> ? ? ? ?(read_block_form): Update interface to return pointer, use
> ? ? ? ?fbuf_read for direct access.
> ? ? ? ?(read_block_direct): Update to new stream interface.
> ? ? ? ?(write_block): Use mem_alloc_w for internal I/O.
> ? ? ? ?(write_buf): Update to new stream interface.
> ? ? ? ?(formatted_transfer_scalar): Don't use u.p.line_buffer, use
> ? ? ? ?fbuf_seek for external files.
> ? ? ? ?(us_read): Update to new stream interface.
> ? ? ? ?(us_write): Likewise.
> ? ? ? ?(data_transfer_init): Always check if we switch modes and flush.
> ? ? ? ?(skip_record): Use new stream interface, fix comparison.
> ? ? ? ?(next_record_r): Check for and reset u.p.at_eof, use new stream
> ? ? ? ?interface, use fbuf_getc for spacing.
> ? ? ? ?(write_us_marker): Update to new stream interface, don't inline.
> ? ? ? ?(next_record_w_unf): Likewise.
> ? ? ? ?(sset): New function.
> ? ? ? ?(next_record_w): Use new stream interface, use fbuf for printing
> ? ? ? ?newline.
> ? ? ? ?(next_record): Use new stream interface.
> ? ? ? ?(finalize_transfer): Remove sfree call, use new stream interface.
> ? ? ? ?(st_iolength_done): Don't use u.p.scratch.
> ? ? ? ?(st_read): Don't check for end of file.
> ? ? ? ?(st_read_done): Don't use u.p.scratch, use unit_truncate.
> ? ? ? ?(hit_eof): New function.
> ? ? ? ?* io/unit.c (init_units): Always init fbuf for formatted units.
> ? ? ? ?(update_position): Use new stream interface.
> ? ? ? ?(unit_truncate): New function.
> ? ? ? ?(finish_last_advance_record): Use fbuf to print newline.
> ? ? ? ?* io/unix.c: Remove unused SSIZE_MAX macro.
> ? ? ? ?(BUFFER_SIZE): Make static const variable rather than macro.
> ? ? ? ?(struct unix_stream): Remove dirty_offset, len, method,
> ? ? ? ?small_buffer. Order elements by decreasing size.
> ? ? ? ?(struct int_stream): Remove.
> ? ? ? ?(move_pos_offset): Remove usage of dirty_offset.
> ? ? ? ?(reset_stream): Remove.
> ? ? ? ?(do_read): Rename to raw_read, update to match new stream
> ? ? ? ?interface.
> ? ? ? ?(do_write): Rename to raw_write, update to new stream interface.
> ? ? ? ?(raw_seek): New function.
> ? ? ? ?(raw_tell): New function.
> ? ? ? ?(raw_truncate): New function.
> ? ? ? ?(raw_close): New function.
> ? ? ? ?(raw_flush): New function.
> ? ? ? ?(raw_init): New function.
> ? ? ? ?(fd_alloc): Remove.
> ? ? ? ?(fd_alloc_r_at): Remove.
> ? ? ? ?(fd_alloc_w_at): Remove.
> ? ? ? ?(fd_sfree): Remove.
> ? ? ? ?(fd_seek): Remove.
> ? ? ? ?(fd_truncate): Remove.
> ? ? ? ?(fd_sset): Remove.
> ? ? ? ?(fd_read): Remove.
> ? ? ? ?(fd_write): Remove.
> ? ? ? ?(fd_close): Remove.
> ? ? ? ?(fd_open): Remove.
> ? ? ? ?(fd_flush): Rename to buf_flush, update to new stream interface
> ? ? ? ?and unix_stream.
> ? ? ? ?(buf_read): New function.
> ? ? ? ?(buf_write): New function.
> ? ? ? ?(buf_seek): New function.
> ? ? ? ?(buf_tell): New function.
> ? ? ? ?(buf_truncate): New function.
> ? ? ? ?(buf_close): New function.
> ? ? ? ?(buf_init): New function.
> ? ? ? ?(mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
> ? ? ? ?(mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
> ? ? ? ?(mem_read): Change to match new stream interface.
> ? ? ? ?(mem_write): Likewise.
> ? ? ? ?(mem_seek): Likewise.
> ? ? ? ?(mem_tell): Likewise.
> ? ? ? ?(mem_truncate): Likewise.
> ? ? ? ?(mem_close): Likewise.
> ? ? ? ?(mem_flush): New function.
> ? ? ? ?(mem_sfree): Remove.
> ? ? ? ?(empty_internal_buffer): Cast to correct type.
> ? ? ? ?(open_internal): Use correct type, init function pointers.
> ? ? ? ?(fd_to_stream): Test whether to open file as buffered or raw.
> ? ? ? ?(output_stream): Remove mode set.
> ? ? ? ?(error_stream): Likewise.
> ? ? ? ?(flush_all_units_1): Use new stream interface.
> ? ? ? ?(flush_all_units): Likewise.
> ? ? ? ?(stream_at_bof): Remove.
> ? ? ? ?(stream_at_eof): Remove.
> ? ? ? ?(file_position): Remove.
> ? ? ? ?(file_length): Update logic to use stream interface.
> ? ? ? ?(flush): Remove.
> ? ? ? ?(stream_offset): Remove.
> ? ? ? ?* io/write.c (write_utf8_char4): Use int instead of size_t.
> ? ? ? ?(write_x): Extra safety check.
> ? ? ? ?(namelist_write_newline): Use new stream interface.
>
>



-- 
The knack of flying is learning how to throw yourself at the ground and miss.
       --Hitchhikers Guide to the Galaxy