Summary: | Support std::filesystem on Windows | ||
---|---|---|---|
Product: | gcc | Reporter: | Jan Niklas Hasse <jhasse> |
Component: | libstdc++ | Assignee: | Jonathan Wakely <redi> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | daniel.kruegler, i.nixman, webrown.cpp |
Priority: | P3 | Keywords: | patch |
Version: | unknown | ||
Target Milestone: | 9.0 | ||
See Also: | https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66203 | ||
Host: | Target: | *-*-mingw32* | |
Build: | Known to work: | ||
Known to fail: | Last reconfirmed: | 2017-02-16 00:00:00 | |
Attachments: |
test results for trunk
test results for trunk |
Description
Jan Niklas Hasse
2016-12-20 09:43:25 UTC
Because nobody who uses Windows has contributed any help to make it work. Nothing will change until somebody does that. I've written 100% of the filesystem code so far, but I'm not going to do the Windows implementation too. I'm willing to help. Is there an existing starting point for Windows? Can boost::filesystem's implementation be used? (In reply to Jan Niklas Hasse from comment #2) > I'm willing to help. Great! Please read https://gcc.gnu.org/onlinedocs/libstdc++/manual/appendix_contributing.html especially the part about legal paperwork. > Is there an existing starting point for Windows? Can > boost::filesystem's implementation be used? No. Just look at the existing code and figure out what the equivalent code would be for Windows, and write an alternative implementation e.g. replace calls to POSIX stat() with whatever Windows uses to query file properties, and replace calls to POSIX chmod with whatever Windows uses to change file permissions. > Great! Please read https://gcc.gnu.org/onlinedocs/libstdc++/manual/appendix_contributing.html especially the part about legal paperwork. Do you mean the part about legal paperwork in https://gcc.gnu.org/contribute.html ? > No. Just look at the existing code and figure out what the equivalent code would be for Windows, and write an alternative implementation e.g. replace calls to POSIX stat() with whatever Windows uses to query file properties, and replace calls to POSIX chmod with whatever Windows uses to change file permissions. And can I look at how boost::filesystem does this? (In reply to Jan Niklas Hasse from comment #4) > Do you mean the part about legal paperwork in > https://gcc.gnu.org/contribute.html ? Yes. > And can I look at how boost::filesystem does this? You can't copy any code, that would be a copyright violation. It's OK to see which Windows API functions Boost uses for a particular filesystem operation, but if you're not sure what is covered by copyright law then it's safer not to use another project at all. I was hoping that somebody who actually knows the Windows API would contribute the code, so they already know what the appropriate functions are. > Yes. Okay thanks, I'll look into it. > You can't copy any code, that would be a copyright violation. It's OK to see > which Windows API functions Boost uses for a particular filesystem > operation, but if you're not sure what is covered by copyright law then it's > safer not to use another project at all. According to https://www.gnu.org/licenses/license-list.html the Boost License is GPL compatible. So when I mark the code I copied with the source, it should work, shouldn't it? > I was hoping that somebody who actually knows the Windows API would > contribute the code, so they already know what the appropriate functions are. I know the Windows API, but the details are hard to get right. Especially getting everything to be bug-free. Many man hours have already been spend on boost::filesystem and the API is similar to std::filesystem. Why should I re-invent the wheel? (In reply to Jan Niklas Hasse from comment #6) > According to https://www.gnu.org/licenses/license-list.html the Boost > License is GPL compatible. So when I mark the code I copied with the source, > it should work, shouldn't it? See the legal prerequisites. As it says, the FSF prefers to hold the copyright on all code contributed to GCC. If you copy somebody else's code you can't assign copyright to the FSF because you don't own the copyright. We could request all the authors of Boost.Filesystem to assign copyright, but I don't want to do that, I want an independent implementation (the Boost implementation doesn't follow exactly the same specification anyway). I'm not interested in simply copying the Boost.Filesystem code into GCC. If you want to use the Boost implementation then just use that, with appropriate preprocessor hackery and namespace aliases. I'm not interested in doing all this work again from scratch, especially since working with the Windows API is a pain. There's a patch at https://gcc.gnu.org/ml/libstdc++/2017-02/msg00041.html I haven't reviewed or tested it yet. (In reply to Jonathan Wakely from comment #9) > There's a patch at https://gcc.gnu.org/ml/libstdc++/2017-02/msg00041.html > > I haven't reviewed or tested it yet. https://gcc.gnu.org/ml/gcc-patches/2017-03/msg00772.html Tested on i686/x86_64-MinGW-W64 and x86_64-linux-gnu. (In reply to niXman from comment #10) > (In reply to Jonathan Wakely from comment #9) > > There's a patch at https://gcc.gnu.org/ml/libstdc++/2017-02/msg00041.html > > > > I haven't reviewed or tested it yet. > > https://gcc.gnu.org/ml/gcc-patches/2017-03/msg00772.html > > Tested on i686/x86_64-MinGW-W64 and x86_64-linux-gnu. Adding "patch" keyword *** Bug 85670 has been marked as a duplicate of this bug. *** Author: redi Date: Thu May 31 19:20:24 2018 New Revision: 261034 URL: https://gcc.gnu.org/viewcvs?rev=261034&root=gcc&view=rev Log: PR libstdc++/78870 support std::filesystem on Windows PR libstdc++/78870 support std::filesystem on Windows * config.h.in: Regenerate. * configure: Regenerate. * configure.ac: Check for link, readlink and symlink. * include/bits/fs_path.h (path::operator/=(const path&)): Move definition out of class body. (path::is_absolute(), path::_M_append(path)): Likewise. (operator<<(basic_ostream, const path&)): Use std::quoted directly. (operator>>(basic_istream, path&)): Likewise. (u8path): Reorder definitions and fix Windows implementation. (path::is_absolute()): Define inline and fix for Windows. [!_GLIBCXX_FILESYSTEM_IS_WINDOWS] (path::operator/=(const path&)): Define POSIX version inline. (path::_M_append(path)): Define inline. * include/experimental/bits/fs_path.h (path::is_absolute()): Move definition out of class body. (operator<<(basic_ostream, const path&)): Fix type of delimiter and escape characters. (operator>>(basic_istream, path&)): Likewise. (path::is_absolute()): Define inline and fix for Windows. * src/filesystem/dir-common.h (__gnu_posix): New namespace. (__gnu_posix::char_type, __gnu_posix::DIR, __gnu_posix::dirent) (__gnu_posix::opendir, __gnu_posix::readdir, __gnu_posix::closedir): Define as adaptors for Windows functions/types or as using-declarations for POSIX functions/types. (_Dir_base, get_file_type): Qualify names to use declarations from __gnu_posix namespace. (_Dir_base::is_dor_or_dotdot): New helper functions. * src/filesystem/dir.cc (_Dir, recursive_directory_iterator): Qualify names to use declarations from __gnu_posix namespace. * src/filesystem/ops-common.h (__gnu_posix): New nested namespace. (__gnu_posix::open, __gnu_posix::close, __gnu_posix::stat_type) (__gnu_posix::stat, __gnu_posix::lstat, __gnu_posix::mode_t) (__gnu_posix::chmod, __gnu_posix::mkdir, __gnu_posix::getcwd) (__gnu_posix::chdir, __gnu_posix::utimbuf, __gnu_posix::utime) (__gnu_posix::rename, __gnu_posix::truncate, __gnu_posix::char_type): Define as adaptors for Windows functions/types or as using-declarations for POSIX functions/types. (stat_type, do_copy_file): Qualify names to use declarations from __gnu_posix namespace. (do_space): Declare new function. (make_file_type): Only use S_ISLNK if defined. * src/filesystem/ops.cc (char_ptr, filesystem::canonical): Use path::value_type not char. (filesystem::copy, create_dir, filesystem::create_directory): Qualify names to use declarations from __gnu_posix namespace. (filesystem::create_hard_link): Check HAVE_LINK autoconf macro and add implementation for Windows. (filesystem::create_symlink): Check HAVE_SYMLINK autoconf macro. (filesystem::current_path(error_code&)): Use __gnu_posix::getcwd. [!_PC_PATH_MAX]: Don't use pathconf. [PATH_MAX]: Use if defined. (filesystem::current_path(const path&, error_code&)) (filesystem::equivalent, do_stat, filesystem::hard_link_count) (filesystem::last_write_time, filesystem::permissions): Use names from __gnu_posix. (filesystem::read_symlink): Check HAVE_READLINK autoconf macro. (filesystem::remove) [_GLIBCXX_FILESYSTEM_IS_WINDOWS]: Add implementation for Windows. (filesystem::rename, filesystem::resize_file): Use names from __gnu_posix. (filesystem::space): Use do_space. [_GLIBCXX_FILESYSTEM_IS_WINDOWS]: Get absolute path to directory. (filesystem::status, filesystem::symlink_status): Use names from __gnu_posix. (filesystem::temp_directory_path): Add implementation for Windows. * src/filesystem/path.cc (dot): Define constant. (path::replace_extension): Use dot. (path::_M_find_extension): Likewise. Use path::string_type not std::string. (path::_M_split_cmpts): Use dot. (filesystem_error::_M_get_what): Use u8string() not native(). * src/filesystem/std-dir.cc (_Dir, recursive_directory_iterator): Qualify names to use declarations from __gnu_posix namespace. * src/filesystem/std-ops.cc (filesystem::absolute(const path&)): Use correct error_code. (filesystem::absolute(const path&, error_code&)): Add implementation for Windows. (char_ptr, filesystem::canonical): Use path::value_type not char. (do_copy_file): Use names from __gnu_posix. [_GLIBCXX_FILESYSTEM_IS_WINDOWS]: Do not use fchmod, fchmodat or sendfile. (filesystem::copy, create_dir, filesystem::create_directory): Qualify names to use declarations from __gnu_posix namespace. (filesystem::create_hard_link): Check HAVE_LINK autoconf macro and add implementation for Windows. (filesystem::create_symlink): Check HAVE_SYMLINK autoconf macro. (filesystem::current_path(error_code&)): Use __gnu_posix::getcwd. [!_PC_PATH_MAX]: Don't use pathconf. [PATH_MAX]: Use if defined. (filesystem::current_path(const path&, error_code&)) (filesystem::equivalent, do_stat, filesystem::hard_link_count) (filesystem::last_write_time, filesystem::permissions): Use names from __gnu_posix. (filesystem::read_symlink): Check HAVE_READLINK autoconf macro. (filesystem::remove) [_GLIBCXX_FILESYSTEM_IS_WINDOWS]: Add implementation for Windows. (filesystem::rename, filesystem::resize_file): Use names from __gnu_posix. (do_space): Define. (filesystem::space): Use do_space. (filesystem::status, filesystem::symlink_status): Use names from __gnu_posix. (filesystem::temp_directory_path): Add implementation for Windows. * src/filesystem/std-path.cc [_GLIBCXX_FILESYSTEM_IS_WINDOWS] (path::operator/=(const path&)): Define for Windows. (dot): Define constant. (path::replace_extension, is_dot): Use dot. (path::lexically_normal): Check _M_type instead of calling non-existent function. (path::_M_find_extension): Use dot. Use path::string_type not std::string. (path::_M_split_cmpts): Use dot. (filesystem_error::_M_get_what): Use u8string() not native(). * testsuite/27_io/filesystem/iterators/directory_iterator.cc: Do not use symlinks. * testsuite/27_io/filesystem/iterators/recursive_directory_iterator.cc: Likewise. * testsuite/27_io/filesystem/operations/absolute.cc: Use __gnu_test::root_path() instead of "/" and add Windows-specific tests. * testsuite/27_io/filesystem/operations/canonical.cc: Use path::string() to get narrow string, not path::native(). * testsuite/27_io/filesystem/operations/copy.cc: Construct fstreams with std::filesystem::path not std::basic_string. * testsuite/27_io/filesystem/operations/copy_file.cc: Likewise. * testsuite/27_io/filesystem/operations/exists.cc: Use __gnu_test::root_path() instead of "/". * testsuite/27_io/filesystem/operations/is_empty.cc: Construct fstreams with std::filesystem::path not std::basic_string. * testsuite/27_io/filesystem/operations/last_write_time.cc: Use path::string() to get narrow string. * testsuite/27_io/filesystem/operations/space.cc: Check results for errors, expect sensible values otherwise. * testsuite/27_io/filesystem/operations/temp_directory_path.cc: Add helpers for adjusting the environment on Windows. * testsuite/27_io/filesystem/path/append/path.cc: Test Windows-specific behaviour. * testsuite/27_io/filesystem/path/construct/format.cc: Fix creation of path::string_type objects. * testsuite/27_io/filesystem/path/construct/locale.cc: Compare native string to wide string on Windows. * testsuite/27_io/filesystem/path/decompose/root_directory.cc: Allow for backslash as root-directory. * testsuite/27_io/filesystem/path/decompose/stem.cc: Use path::string() to get narrow string. * testsuite/27_io/filesystem/path/itr/traversal.cc: Test Windows-style paths. * testsuite/27_io/filesystem/path/native/string.cc: Use string_type not std::string. * testsuite/27_io/filesystem/path/query/is_absolute.cc: Adjust for different definintion of absolute paths on Windows. * testsuite/experimental/filesystem/iterators/directory_iterator.cc: Do not use symlinks. * testsuite/experimental/filesystem/operations/absolute.cc: Test Windows behaviour. * testsuite/experimental/filesystem/operations/copy.cc: Construct fstreams with NTCTS not std::basic_string. * testsuite/experimental/filesystem/operations/copy_file.cc: Likewise. * testsuite/experimental/filesystem/operations/exists.cc: Use __gnu_test::root_path() instead of "/". * testsuite/experimental/filesystem/operations/is_empty.cc: Construct fstreams with NTCTS not std::basic_string. * testsuite/experimental/filesystem/operations/last_write_time.cc: Use path::string() to get narrow string. * testsuite/experimental/filesystem/operations/space.cc: Use __gnu_test::root_path() instead of "/". * testsuite/experimental/filesystem/operations/temp_directory_path.cc: Add helpers for adjusting the environment on Windows. * testsuite/experimental/filesystem/path/append/path.cc: Use path::string() to get narrow strings for comparisons. * testsuite/experimental/filesystem/path/concat/path.cc: Likewise. * testsuite/experimental/filesystem/path/decompose/root_directory.cc: Likewise. * testsuite/experimental/filesystem/path/decompose/stem.cc: Likewise. * testsuite/experimental/filesystem/path/native/string.cc: Use string_type not std::string. * testsuite/experimental/filesystem/path/query/is_absolute.cc: Adjust for different definintion of absolute paths on Windows. * testsuite/util/testsuite_fs.h (__gnu_test::root_path()): New function. (__gnu_test::scoped_file): Construct fstreams with NTCTS not std::basic_string. Modified: trunk/libstdc++-v3/ChangeLog trunk/libstdc++-v3/config.h.in trunk/libstdc++-v3/configure trunk/libstdc++-v3/configure.ac trunk/libstdc++-v3/include/bits/fs_path.h trunk/libstdc++-v3/include/experimental/bits/fs_path.h trunk/libstdc++-v3/src/filesystem/dir-common.h trunk/libstdc++-v3/src/filesystem/dir.cc trunk/libstdc++-v3/src/filesystem/ops-common.h trunk/libstdc++-v3/src/filesystem/ops.cc trunk/libstdc++-v3/src/filesystem/path.cc trunk/libstdc++-v3/src/filesystem/std-dir.cc trunk/libstdc++-v3/src/filesystem/std-ops.cc trunk/libstdc++-v3/src/filesystem/std-path.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/iterators/directory_iterator.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/iterators/recursive_directory_iterator.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/absolute.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/canonical.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/copy.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/copy_file.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/exists.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/is_empty.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/last_write_time.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/space.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/operations/temp_directory_path.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/append/path.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/construct/format.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/construct/locale.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/decompose/root_directory.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/decompose/stem.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/itr/traversal.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/native/string.cc trunk/libstdc++-v3/testsuite/27_io/filesystem/path/query/is_absolute.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/iterators/directory_iterator.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/absolute.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/copy.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/copy_file.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/exists.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/is_empty.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/last_write_time.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/space.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/operations/temp_directory_path.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/append/path.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/concat/path.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/decompose/root_directory.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/decompose/stem.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/native/string.cc trunk/libstdc++-v3/testsuite/experimental/filesystem/path/query/is_absolute.cc trunk/libstdc++-v3/testsuite/util/testsuite_fs.h The Filesystem library now builds and mostly works on MinGW (for both the TS and C++17 versions). The library is not enabled by default for mingw targets, so you need to build GCC with --enable-libstdcxx-filesystem-ts If people can test and confirm it works fine for both mingw and mingw-w64 then we can enable it by default. P.S. thanks to niXman for the patch that inspired the approach used. Hi Jonathan, After this patch, the aarch64_be-none-elf and aarch64-none-elf configurations fail to configure libstdc++ because of https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66203 configure:80055: checking for link configure:80055: error: Link tests are not allowed after GCC_NO_EXECUTABLES. I think the patch is not including the required changes in <fstream> in order to properly handle path arguments. Specifically, both constructors and methods open() need an additional overload taking const wchar_t*, as required per specs: 30.9.1 Header <fstream> synopsis [...] 3 In this subclause, member functions taking arguments of const filesystem::path::value_type* are only be provided on systems where filesystem::path::value_type (30.10.7) is not char. [ Note: These functions enable class path support for systems with a wide native path character type, such as wchar_t. —end note ] This involves methods: basic_filebuf::open(const filesystem::path::value_type* s, ios_base::openmode mode); explicit basic_ifstream::basic_ifstream(const filesystem::path::value_type* s, ios_base::openmode mode = ios_base::in); void basic_ifstream::open(const filesystem::path::value_type* s, ios_base::openmode mode = ios_base::in); explicit basic_ofstream::basic_ofstream(const filesystem::path::value_type* s, ios_base::openmode mode = ios_base::out); void basic_ofstream::open(const filesystem::path::value_type* s, ios_base::openmode mode = ios_base::out); explicit basic_fstream::basic_fstream(const std::filesystem::path::value_type* s, ios_base::openmode mode = ios_base::in|ios_base::out); void basic_fstream::open(const std::filesystem::path::value_type* s, ios_base::openmode mode = ios_base::in|ios_base::out); I hope this helps That was already done two weeks ago by r260479 (In reply to Jonathan Wakely from comment #18) > That was already done two weeks ago by r260479 Thank you! And a last question: Do you plan to merge both patches into the gcc 8 branch? Probably not. On current branch master [1] and gcc-8-branch the libstdc++fs DLL is not created for x86_64-w64-mingw32. Please re-open. [1] remotes/origin/master 84b3cf31c30 PR target/85832 PR target/86036 * config/i386/sse.md (<avx512>_eq<mode>3<mask_scalar_me ``` lh_mouse@lhmouse-pc /e/GitHub/MINGW-packages/mingw-w64-gcc-git $ gcc --version gcc.exe (master HEAD with MCF thread model, built by LH_Mouse.) 9.0.0 20180604 (experimental) Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ```` This is the list of DLLs installed with `make install`: ``` libatomic-1.dll libgcc_s_seh-1.dll libgomp-1.dll libquadmath-0.dll libssp-0.dll libstdc++-6.dll ``` Nothing has been committed for gcc-8-branch so of course it isn't created. For trunk, did you read comment 14? (In reply to Jonathan Wakely from comment #23) > Nothing has been committed for gcc-8-branch so of course it isn't created. > > For trunk, did you read comment 14? Sorry didn't notice it. I started a new build just now. Created attachment 44234 [details]
test results for trunk
Created attachment 44235 [details]
test results for trunk
Some tests failed, but despite the failure, all tests compiled now. We might want to mark some tests as XFAIL for *-*-mingw* targets. *** Bug 86507 has been marked as a duplicate of this bug. *** |