Consider the following test code: #include <vector> #include <array> #include <future> #include <iostream> using std::vector; using std::array; using std::cout; using std::endl; typedef unsigned int uint; double TestFutures() { return 1; } void DoWork() { const uint nPoints=50; const uint nThreads=100; vector<double> results(nThreads,0); // For each data point... for (uint k=0; k<nPoints; ++k) { // ... launch the threads vector<std::future<double> > values; for (uint w=0; w<nThreads; ++w) { values.push_back(std::async(TestFutures)); } // ... and now read the results for (uint w=0; w<nThreads; ++w) { results[w]+= values[w].get(); } } } int main() { const uint nTimes=50; for (uint k=0; k<nTimes; ++k) { cout << "Cycle: " << k << endl; DoWork(); } } I compile it with gcc 4.7.2 (MinGW 4.7.2) in Qt 5.0.1, Qt Creator 2.6.2. Then I run it, either in the debugger, or stand alone, release or debug version. If I monitor the memory usage, it increases monotonically during the execution of the loops. By increasing the variable nTimes in the main function, I can make the program allocate an arbitrary amount of memory (I have tried up to 1Gb). I monitor the memory usage with task manager or procexp. If I then take the same code and compile it and run it under MS Visual Studio Express 2012, the memory usage remains stable during the loop. This, I think, is what you would expect, as the vector of futures should be destroyed after each k-iteration. I reported this in the MinGw bug forum and was redirected here. Just to let you know, I still have the same issue if I substitute the vector of futures with an array, i.e. this DoWork() routine: void DoWork() { const uint nPoints=50; const uint nThreads=100; vector<double> results(nThreads,0); // For each data point... for (uint k=0; k<nPoints; ++k) { // ... launch the threads array<std::future<double>,nThreads> values; for (uint w=0; w<nThreads; ++w) { values[w]=std::async(TestFutures); } // ... and now read the results for (uint w=0; w<nThreads; ++w) { results[w]+= values[w].get(); } } }
(In reply to DrD from comment #0) > // ... launch the threads > vector<std::future<double> > values; > for (uint w=0; w<nThreads; ++w) { > values.push_back(std::async(TestFutures)); > } One quick comment, without analysing the issue: No threads are launched in your program. Currently GCC's std::async always behaves as std::async(std::launch::deferred, ...) so to run in a new thread you need to explicitly call std::async(std::launch::async, TestFutures)
(In reply to Jonathan Wakely from comment #1) > (In reply to DrD from comment #0) > > // ... launch the threads > > vector<std::future<double> > values; > > for (uint w=0; w<nThreads; ++w) { > > values.push_back(std::async(TestFutures)); > > } > > One quick comment, without analysing the issue: > > No threads are launched in your program. Currently GCC's std::async always > behaves as std::async(std::launch::deferred, ...) so to run in a new thread > you need to explicitly call std::async(std::launch::async, TestFutures) Oh! thank you very much for the tip!
Also I can't reproduce this on GNU/Linux, the memory usage is bounded as expected. I'm surprised this even compiles with Mingw, are you using Mingw-w64 with libpthread-win32? Since it seems to be platform-specific it could be a resource leak of mutexes or condition variables, so my wild guess is that libpthread-win32 requires pthread_mutex_destroy or pthread_cond_destroy to be used. Does it make any difference if you add these to the very top of the source file, before including any headers? #define _GTHREAD_USE_MUTEX_INIT_FUNC #define _GTHREAD_USE_COND_INIT_FUNC
If my guess is right you should be able to reproduce the unbounded memory usage with this: #include <future> int f() { return 0; } int main() { for (int i=0; i < 100000; ++i) std::async(f).get(); } And you should not see it happen with this: #define _GTHREAD_USE_MUTEX_INIT_FUNC #define _GTHREAD_USE_COND_INIT_FUNC #include <future> int f() { return 0; } int main() { for (int i=0; i < 100000; ++i) std::async(f).get(); } Please check and update the bug report, thanks.
(In reply to Jonathan Wakely from comment #3) > Also I can't reproduce this on GNU/Linux, the memory usage is bounded as > expected. > > I'm surprised this even compiles with Mingw, are you using Mingw-w64 with > libpthread-win32? Since it seems to be platform-specific it could be a > resource leak of mutexes or condition variables, so my wild guess is that > libpthread-win32 requires pthread_mutex_destroy or pthread_cond_destroy to > be used. > > Does it make any difference if you add these to the very top of the source > file, before including any headers? > > #define _GTHREAD_USE_MUTEX_INIT_FUNC > #define _GTHREAD_USE_COND_INIT_FUNC It does compile in MinGw. Unfortunately the two #define lines don't help either. I forgot to mention that I had already been warned that the test code works OK in Linux, see: http://stackoverflow.com/questions/16489389/memory-usage-with-future-and-containers-in-qt-mingw
(In reply to Jonathan Wakely from comment #4) > If my guess is right you should be able to reproduce the unbounded memory > usage with this: > > #include <future> > > int f() { return 0; } > > int main() > { > for (int i=0; i < 100000; ++i) > std::async(f).get(); > } > > > And you should not see it happen with this: > > #define _GTHREAD_USE_MUTEX_INIT_FUNC > #define _GTHREAD_USE_COND_INIT_FUNC > > #include <future> > > int f() { return 0; } > > int main() > { > for (int i=0; i < 100000; ++i) > std::async(f).get(); > } > > Please check and update the bug report, thanks. Hi Jonathan, Thanks for your help. I just tried this code and the memory problem is there with and without the #define lines. I haven't managed to get this bug tested on Windows by anyone else yet. I have tried in two different computers, though (not enough, I think). Any suggestions?
You didn't answer the question about which mingw compiler you are using, the standard mingw gcc does not support std::async.
(In reply to Jonathan Wakely from comment #7) > You didn't answer the question about which mingw compiler you are using, the > standard mingw gcc does not support std::async. From my first post: "I compile it with gcc 4.7.2 (MinGW 4.7.2) in Qt 5.0.1, Qt Creator 2.6.2." Is that what you need or are you after some other detail I have omitted?
Some other detail. I don't care about Qt Creator, it's not a compiler, and the version of Qt is compeltely irrelevant because you're not using Qt. I don't believe you can be using Mingw 4.7.2, because that doesn't support std::async(). What does 'gcc -v' show?
(In reply to Jonathan Wakely from comment #7) > You didn't answer the question about which mingw compiler you are using, the > standard mingw gcc does not support std::async. From my first post: "I compile it with gcc 4.7.2 (MinGW 4.7.2) in Qt 5.0.1, Qt Creator 2.6.2." Is that what you need or are you after some other detail I have omitted?(In reply to Jonathan Wakely from comment #9) > Some other detail. I don't care about Qt Creator, it's not a compiler, and > the version of Qt is compeltely irrelevant because you're not using Qt. > > I don't believe you can be using Mingw 4.7.2, because that doesn't support > std::async(). What does 'gcc -v' show? I am using the MinGW copy that comes with Qt. If I go to the bin folder I obtain: C:\Qt\Qt5.0.1\Tools\MinGW\bin>gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=c:/qt/qt5.0.1/tools/mingw/bin/../libexec/gcc/i686-w64-mingw32/4.7.2/lto-wrapper.exe Target: i686-w64-mingw32 Configured with: ../../../src/gcc-4.7.2/configure --host=i686-w64-mingw32 --build=i686-w64-mingw32 --target=i686-w64-mingw32 --prefix=/temp/x32-4.7.2-posix-sjlj-r8/prefix --with-sysroot=/temp/x32-4.7.2-posix-sj lj-r8/prefix --enable-shared --enable-static --enable-targets=all --enable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-lto --enable -graphite --enable-cloog-backend=isl --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --enable-sjlj-exceptions --disable-ppl-version-check --disable-cloog-version-c heck --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch-32=i6 86 --with-arch-64=nocona --with-tune-32=core2 --with-tune-64=core2 --with-host-libstdcxx='-static -lstdc++' --with-libiconv --with-system-zlib --with-gmp=/temp/mingw-prereq/i686-w64-mingw32-static --with-mpfr=/ temp/mingw-prereq/i686-w64-mingw32-static --with-mpc=/temp/mingw-prereq/i686-w64-mingw32-static --with-ppl=/temp/mingw-prereq/i686-w64-mingw32-static --with-cloog=/temp/mingw-prereq/i686-w64-mingw32-static --wi th-pkgversion='Built by MinGW-builds project' --with-bugurl=http://sourceforge.net/projects/mingwbuilds/ CFLAGS='-O2 -pipe -fomit-frame-pointer -I/temp/x32-4.7.2-posix-sjlj-r8/libs/include -I/temp/mingw-prereq/ x32-zlib/include -I/temp/mingw-prereq/i686-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -fomit-frame-pointer -I/temp/x32-4.7.2-posix-sjlj-r8/libs/include -I/temp/mingw-prereq/x32-zlib/include -I/temp/mingw-p rereq/i686-w64-mingw32-static/include' CPPFLAGS= LDFLAGS='-pipe -L/temp/x32-4.7.2-posix-sjlj-r8/libs/lib -L/temp/mingw-prereq/x32-zlib/lib -L/temp/mingw-prereq/i686-w64-mingw32-static/lib -L/temp/x32-4.7.2-posi x-sjlj-r8/prefix/opt/lib' Thread model: posix gcc version 4.7.2 (Built by MinGW-builds project) I am relatively new to MinGw (it shows), so I am not too familiar with the particularities of Qt's copy.
(In reply to DrD from comment #10) > Target: i686-w64-mingw32 > Configured with: ../../../src/gcc-4.7.2/configure --host=i686-w64-mingw32 > --build=i686-w64-mingw32 --target=i686-w64-mingw32 > --prefix=/temp/x32-4.7.2-posix-sjlj-r8/prefix > --with-sysroot=/temp/x32-4.7.2-posix-sj > lj-r8/prefix --enable-shared --enable-static --enable-targets=all > --enable-multilib --enable-languages=c,c++,fortran,lto > --enable-libstdcxx-time=yes --enable-threads=posix Thanks, that's what I needed, it confirms it's mingw-w64 with pthread support. Kai, do you have any ideas for how to track down the "leak" here? My knowledge of mingw-w64 and Windows in general is close to zero.
Using Qt Creator I have confirmed the leak, and can reproduce it with this #include <pthread.h> #include <unistd.h> int main() { for (int i=0; i < 100000; ++i) { pthread_mutex_t mx = PTHREAD_MUTEX_INITIALIZER; pthread_mutex_lock(&mx); pthread_mutex_unlock(&mx); usleep(10000); } } This is a simplified version of the code in libstdc++, here's another leaky program using libstdc++: //#define _GTHREAD_USE_MUTEX_INIT_FUNC #include <mutex> #include <unistd.h> int main() { for (int i=0; i < 100000; ++i) { std::mutex mx; mx.lock(); mx.unlock(); usleep(10000); } } Uncommenting the first line fixes it, so we should define that macro in libstdc++-v3/config/os/mingw32-w64/os_defines.h
dup of PR 59807
*** Bug 59807 has been marked as a duplicate of this bug. ***
Posted patch to ML for defining _GTHREAD_USE_MUTEX_INIT_FUNC in mingw-w64, and mingw32 case. It is true that posix-threading support for Windows is right now only provided by mingw-w64 based toolchains, nevertheless is the issue a general one on native Windows platforms using posix-threading.
Author: ktietz Date: Tue Oct 14 16:58:37 2014 New Revision: 216210 URL: https://gcc.gnu.org/viewcvs?rev=216210&root=gcc&view=rev Log: 2014-10-14 Kai Tietz <ktietz@redhat.com> PR libstdc++/57440 * config/os/mingw32/os_defines.h (_GTHREAD_USE_MUTEX_INIT_FUNC): Define to avoid leak. * config/os/mingw32-w64/os_defines.h: Likewise. Modified: trunk/libstdc++-v3/ChangeLog trunk/libstdc++-v3/config/os/mingw32-w64/os_defines.h trunk/libstdc++-v3/config/os/mingw32/os_defines.h
Author: ktietz Date: Tue Oct 14 17:05:04 2014 New Revision: 216212 URL: https://gcc.gnu.org/viewcvs?rev=216212&root=gcc&view=rev Log: PR libstdc++/57440 * config/os/mingw32/os_defines.h (_GTHREAD_USE_MUTEX_INIT_FUNC): Define to avoid leak. * config/os/mingw32-w64/os_defines.h: Likewise. Modified: branches/gcc-4_9-branch/libstdc++-v3/ChangeLog branches/gcc-4_9-branch/libstdc++-v3/config/os/mingw32-w64/os_defines.h branches/gcc-4_9-branch/libstdc++-v3/config/os/mingw32/os_defines.h
Author: ktietz Date: Tue Oct 14 17:06:27 2014 New Revision: 216213 URL: https://gcc.gnu.org/viewcvs?rev=216213&root=gcc&view=rev Log: PR libstdc++/57440 * config/os/mingw32/os_defines.h (_GTHREAD_USE_MUTEX_INIT_FUNC): Define to avoid leak. * config/os/mingw32-w64/os_defines.h: Likewise. Modified: branches/gcc-4_8-branch/libstdc++-v3/ChangeLog branches/gcc-4_8-branch/libstdc++-v3/config/os/mingw32-w64/os_defines.h branches/gcc-4_8-branch/libstdc++-v3/config/os/mingw32/os_defines.h
Applied patch to trunk, and all open branches (4.9, 4.8). Closed as fixed.