Re: The size of thread level storage under OpenMP.

[Jakub Jelinek <jakub at redhat dot com>]

On Fri, May 23, 2008 at 10:54:28PM +0200, Toon Moene wrote:
> Tobias Burnus wrote:
> 
> >Toon Moene wrote:
> >>I have an OpenMP enabled Fortran program that dies a quick death when 
> >>compiled with -fopenmp
> 
> >A classical problem with OpenMP is that one runs out of stack memory. 
> >The current stack setting can be queried using the sh/bash using "ulimit 
> >-s" and using the (t)csh using "limit stacksize". Try to set it to 
> >"unlimited". Often systems limit the available stack memory; A soft 
> >limit of 8192 kbytes is a typical value under Linux.
> 
> We already set ulimit -s to unlimited (since times premordial) because 
> all of our local arrays are automatic arrays).
> 
> This cannot be the problem.  The code runs fine if I replace -fopenmp 
> with -frecursive.

That is a problem.  If ulimit -s is unlimited, then in threaded apps
each thread has the default limit (on i386 or x86_64 that is 2MB).
If that wasn't the case, then ulimit -s unlimited would need to imply
single threaded apps would be allowed only.
The thing is that with the exception of initial threads, all other stacks
need to have fixed size, allocated when the thread is created, and you can't
allocate a bunch of unlimited sized stacks for each thread.
If you are on a 64-bit arch, you can set ulimit -s to something large,
say ulimit -s 2097152 for 2GB stacks.  On 32-bit arches you should use
ulimit -s not much larger than what you actually need for each threads,
because otherwise you'll run out of virtual address space.

	Jakub