Bug 95749 - std::filesystem::file_size returns wrong size for large files on Windows
Summary: std::filesystem::file_size returns wrong size for large files on Windows
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libstdc++ (show other bugs)
Version: 10.1.0
: P3 normal
Target Milestone: 9.4
Assignee: Jonathan Wakely
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-18 18:27 UTC by Christoph Reiter
Modified: 2020-08-10 11:07 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2020-06-18 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Reiter 2020-06-18 18:27:03 UTC
Downstream report: https://github.com/msys2/MINGW-packages/issues/6585

----

#include <filesystem>
#include <fstream>
#include <iostream>
#include <stdint.h>
#include <stdio.h>

int main(int argc, char** argv)
{

    uint64_t std_filesystem_file_size;
    uint64_t std_fstream_file_size;
    std::filesystem::path sourceFile = std::filesystem::current_path() / "example_3GiB.bin";

    std::ofstream ofs(sourceFile);
    ofs << "this is some text in the new file\n";
    ofs.close();
    
    std::filesystem::resize_file(sourceFile,3221225472);
    std_filesystem_file_size = std::filesystem::file_size(sourceFile);

    std::ifstream in(sourceFile.string(), std::ifstream::ate | std::ifstream::binary);
    std_fstream_file_size = in.tellg();

    std::cout << "3 GiB file:" << std::endl;
    std::cout << "std_filesystem_file_size: " << std_filesystem_file_size << std::endl;
    std::cout << "std_fstream_file_size: " << std_fstream_file_size << std::endl;
    return 0;
}

----

$ g++ -std=c++17 a.cpp
$ ./a.exe
3 GiB file:
std_filesystem_file_size: 18446744072635809792
std_fstream_file_size: 3221225472

----

The first value is wrong. My uneducated guess would be that https://github.com/gcc-mirror/gcc/blob/41d6b10e96a1de98e90a7c0378437c3255814b16/libstdc%2B%2B-v3/src/filesystem/ops-common.h#L66 should use _wstat64() instead of _wstat and stat_type should be _stat64 instead of _stat, otherwise things are limited to 32bit.

See https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/stat-functions?view=vs-2019 for details.
Comment 1 GCC Commits 2020-08-10 11:04:32 UTC
The releases/gcc-10 branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:b5cc5c95664347082100a504710f5ca0467306a5

commit r10-8600-gb5cc5c95664347082100a504710f5ca0467306a5
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Mon Aug 10 12:04:27 2020 +0100

    libstdc++: Use _wstat64 for Windows [PR 95749]
    
    In order to handle large files on Windows we need to use stat API with
    64-bit st_size member.
    
    libstdc++-v3/ChangeLog:
    
            PR libstdc++/95749
            * src/filesystem/ops-common.h [_GLIBCXX_FILESYSTEM_IS_WINDOWS]
            (stat_type): Change to __stat64.
            (stat): Use _wstat64.
    
    (cherry picked from commit 9939be5758b52ed2fe1a7e56b94ce6d0f4d81580)
Comment 2 GCC Commits 2020-08-10 11:06:03 UTC
The releases/gcc-9 branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:caac3ee7008286404323c4aa93ee0e1c4753c4c2

commit r9-8800-gcaac3ee7008286404323c4aa93ee0e1c4753c4c2
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Mon Aug 10 11:10:26 2020 +0100

    libstdc++: Use _wstat64 for Windows [PR 95749]
    
    In order to handle large files on Windows we need to use stat API with
    64-bit st_size member.
    
    libstdc++-v3/ChangeLog:
    
            PR libstdc++/95749
            * src/filesystem/ops-common.h [_GLIBCXX_FILESYSTEM_IS_WINDOWS]
            (stat_type): Change to __stat64.
            (stat): Use _wstat64.
    
    (cherry picked from commit 9939be5758b52ed2fe1a7e56b94ce6d0f4d81580)
Comment 3 Jonathan Wakely 2020-08-10 11:07:21 UTC
Fixed for 9.4 and 10.3, thanks for the report.
Comment 4 Jonathan Wakely 2020-08-10 11:07:41 UTC
Also fixed on master by g:9939be5758b52ed2fe1a7e56b94ce6d0f4d81580 but I'm not sure why that didn't get added here.