GCC-version: 7.2.0 System: KDE neon (based on Ubuntu 16.04 LTS) x64, x86_64-linux-gnu GCC-configuration-options: --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --enable-checking=release --enable-languages=c,c++ --program-suffix=-7.2 --disable-multilib --prefix=/opt/gcc-7.2.0 compile line: g++ -std=c++17 *.cpp *.h -lstdc++fs thrown exception: filesystem error: cannot copy file: No such file or directory [/path/to/sourcefile] [/path/to/destinationfile] preprocessed file: see attachment detailed description: The function std::experimental::filesystem::copy_file is not able to copy any file larger than 2.0GiB (2147483647 Bytes). If you try this, the mentioned exception will be thrown after 2.0GiB of an >2.0GiB-file were copied - so that the copy process is canceled and the copied file is incomplete. I could reproduce this behavior in more than one program by simply using std::experimental::filesystem::copy. However, when I compiled it with GCC 5.4.0 (g++ -std=c++14 *.cpp *.h -lstdc++fs) everything works fine and I can copy files with a size of over 2.0GiB. I am sorry for any English mistakes. Please contact me if there are any open questions.
(In reply to T B from comment #0) > However, when I compiled it with GCC 5.4.0 (g++ -std=c++14 *.cpp *.h > -lstdc++fs) everything works fine and I can copy files with a size of over > 2.0GiB. That's strange, because the code for copy_file is identical.
(In reply to Jonathan Wakely from comment #1) > (In reply to T B from comment #0) > > However, when I compiled it with GCC 5.4.0 (g++ -std=c++14 *.cpp *.h > > -lstdc++fs) everything works fine and I can copy files with a size of over > > 2.0GiB. > > That's strange, because the code for copy_file is identical. Then it is really strange. A repetition of my test results in exactly the same however - with g++ 5.4 it works still fine. Perhaps it is the fact that I use a prebuild GCC 5.4 and a selfcompiled GCC 7.2. Could someone else please try it out? Otherwise I will try to rebuild my GCC 7.2. It seems that the preprocessed file was too large. But I can not find anything important inside. If you think there could be anything inside, I will hand them in later.
I recompiled the GCC 7.2.0 with this configuration: --enable-threads=posix --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --enable-checking=release --enable-languages=c,c++ --program-suffix=-7.2 --disable-multilib --enable-c99 --enable-libstdcxx-filesystem-ts But it does not have any effect. It seems that the libstdc++fs is corrupted. If I replace the libstdc++fs.a file with the lib-file of GCC 5.4 it works fine. Any ideas?
What does this output, for both gcc-5 and gcc-7? #include <iostream> int main() { std::cout << "Large file support: " #ifdef _GLIBCXX_USE_LFS "enabled" #else "disabled" #endif "\n"; }
Both seem to have large file support enabled, it outputs both times enabled.
OK I'll try to reproduce it as soon as I can.
Have you already tried it out?
I can reproduce this. It stops copying at 2G and reports an error.
"sendfile() will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)" Oops.
Author: redi Date: Thu Dec 14 21:49:03 2017 New Revision: 255666 URL: https://gcc.gnu.org/viewcvs?rev=255666&root=gcc&view=rev Log: PR libstdc++/83279 handle sendfile not copying entire file PR libstdc++/83279 * src/filesystem/std-ops.cc (do_copy_file): Handle sendfile not copying entire file. Modified: trunk/libstdc++-v3/ChangeLog trunk/libstdc++-v3/src/filesystem/std-ops.cc
Fixed on trunk so far.
Thanks! However, if I understand your code right, you simply fixed the error code, right? So that it is still not possible to copy files larger than 2GiB. Wouldn't it be possible to create a loop in which in each run 2GiB will be copied until (count == n)? To be honest, streams are not my core competence. How is this issue solved in GCC 5.4?
No, I didn't just fix the error code, the code will copy files larger than 2G now. I assume your 5.4 build doesn't use sendfile, so doesn't have the 2G limitation of sendfile.
Then I have to reread the code, thank you. I will mark this topic as resolved.
Issue fixed.
Please don't close it, I'm still going to fix it on the branches.
Okay, sorry. Thanks for your efforts!
(In reply to Jonathan Wakely from comment #10) > Author: redi > Date: Thu Dec 14 21:49:03 2017 > New Revision: 255666 #ifdef _GLIBCXX_USE_SENDFILE - off_t offset = 0; - const auto n = ::sendfile(out.fd, in.fd, &offset, from_st->st_size); - if (n < 0 && (errno == ENOSYS || errno == EINVAL)) + n = ::sendfile(out.fd, in.fd, nullptr, count); This change will trigger PR 70975. Older (?) Solaris 12 targets require non-null pointer for the offset argument.
I forgot about that. I'll need to put the offset value back and then add a seek to the input streambuf.
Author: redi Date: Fri Jan 5 21:43:56 2018 New Revision: 256289 URL: https://gcc.gnu.org/viewcvs?rev=256289&root=gcc&view=rev Log: PR libstdc++/83279 Use non-null offset argument for sendfile PR libstdc++/83279 * src/filesystem/std-ops.cc (do_copy_file): Use non-null offset with sendfile. Modified: trunk/libstdc++-v3/ChangeLog trunk/libstdc++-v3/src/filesystem/std-ops.cc
Author: redi Date: Fri Jan 5 22:22:12 2018 New Revision: 256290 URL: https://gcc.gnu.org/viewcvs?rev=256290&root=gcc&view=rev Log: PR libstdc++/83279 handle sendfile not copying entire file Backport from mainline 2017-12-14 Jonathan Wakely <jwakely@redhat.com> PR libstdc++/83279 * src/filesystem/std-ops.cc (do_copy_file): Handle sendfile not copying entire file. Modified: branches/gcc-7-branch/libstdc++-v3/ChangeLog branches/gcc-7-branch/libstdc++-v3/src/filesystem/ops.cc
Author: redi Date: Fri Jan 5 22:47:50 2018 New Revision: 256296 URL: https://gcc.gnu.org/viewcvs?rev=256296&root=gcc&view=rev Log: PR libstdc++/83279 handle sendfile not copying entire file Backport from mainline 2017-12-14 Jonathan Wakely <jwakely@redhat.com> PR libstdc++/83279 * src/filesystem/std-ops.cc (do_copy_file): Handle sendfile not copying entire file. Modified: branches/gcc-6-branch/libstdc++-v3/ChangeLog branches/gcc-6-branch/libstdc++-v3/src/filesystem/ops.cc
fixed for 6.5 and 7.3