GCC 4.7.0 Status Report (2011-10-11)

[Janne Blomqvist]

On Fri, Oct 14, 2011 at 13:15, Tobias Burnus <burnus@net-b.de> wrote:
> On 10/14/2011 10:53 AM, Janne Blomqvist wrote:
>>
>> I think one could add 50016 as a regression as well.
>> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50016
>>
>> It seems IO performance in some cases on Windows has tanked since the
>> 4.6 release (at least by 2 orders of magnitude according to the bug
>> reporter). I don't have a windows system to test on, so I can't
>> confirm whether my suspicion that I mentioned in the PR is correct.
>> But it seems reasonable.
>>
>> I think in general, we ought to think what inquiring for the size of
>> an open file means (the standard doesn't really help here, AFAICS):
>>
>> - Is it the size of the file as reported by the OS?
>>
>> - Is it the size of the file as seen by libgfortran? (which can be
>> different than the previous option, due to buffering)
>>
>> - Or should we flush the buffer in order to make the OS and
>> libgfortran sizes consistent at the moment of the inquire execution?
>> This is what we do now, but for some strange reason it seems to
>> require the equivalent of fsync() on windows (WTF?), so performance is
>> bad.

Sorry, it seems I misspoke. Looking at the implementation
(libgfortran/io/inquire.c) it seems that what we actually do is
something like the first of the three options above (i.e. we always
figure out the size of the file, regardless if it's opened or not, by
calling stat("path/to/file",...)).

In the testcase inquire_size.f90 we then execute the FLUSH statement
in order to make sure that the libgfortran and OS views of the file
size matches. FLUSH then, in turn, causes buf_flush() to be called
which is the function that calls _commit() on win32.

> How about the following:
>
> By default, e.g. for raw_flush and friends, do not call Windows' _commit.
> That's equivalent to the current Linux/Unix behaviour and undoes the patch
> for PR 44698.
>
> If the user calls INQUIRE on an open file or calls the FLUSH() statement or
> the FLUSH() subroutine, an explicit _commit is done on Windows. That way,
> the system buffer is flushed and an access with a different file descriptor
> to the same file sees the current version. That's the same as one already
> has on Unix/Linux/Darwin, except that there it happens automatically and
> that FLUSH does not cause data written to the filesystem (only fsync()
> does), but that's OK.
>
> I think that should solve the problem of PR 44698 and PR 50016 and give a
> consistent view of the file system without imposing too much performance
> problems.
>
> Janne, what do you think? What do others think?

I'd prefer to avoid _commit() at all costs, if at all possible; as we
have seen, it's really slow (since disks are very very slow compared
to RAM). I just find it hard to believe that there is no way to flush
user-space buffers to the kernel on windows so that the file data and
metadata are updated without having to also force the kernel to flush
dirty data to disk.

If nothing else, maybe we should change inquire for size in
inquire_via_unit() to use file_length() instead, which would use the
libgfortran info about the file size instead of the stat() syscall?

-- 
Janne Blomqvist