RE: Is malloc allocating memory on the virtual memory space ?

["Guerra-Salcedo, Cesar M" <cesar dot guerra-salcedo at mnsu dot edu>]

Hi Tom,

Thanks for your reply. I forgot to mention that the same program works seamless in my PC (1 processor 1.7G memory) so when I moved to UNIX I thought...It should work (considering that is C).
these are the numbers when I ran ulimit -a

core file size          (blocks, -c) 2097151
data seg size           (kbytes, -d) 1048576
file size               (blocks, -f) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 2048
pipe size            (512 bytes, -p) 16
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 9019
virtual memory          (kbytes, -v) unlimited

Any more clues on how to overcome the problem ?

Thanks a lot
Best regards

Csr




________________________________________
From: Tom St Denis [tstdenis@ellipticsemi.com]
Sent: Thursday, September 06, 2007 7:36 AM
To: Guerra-Salcedo, Cesar M
Cc: gcc-help@gcc.gnu.org
Subject: Re: Is malloc allocating memory on the virtual memory space ?

Guerra-Salcedo, Cesar M wrote:
> Hello,
>
>
>
> I have been trying to understand some performance and limitations issues of a program written in C for
>
> Hp-UX 11. I am using Malloc to allocate memory for two arrays (huge arrays). If I try to go beyond 1G
>
> the program crashes. when I look at the stats using glance RSS and VSS look pretty much the same (RSS 1047364 and VSS 1048808) my understanding (I guess wrong) was that Malloc allocated on Virtual memory.
>
> Is there a way in C to allocate on VM only so the process address space (1G) is not touched (or barely touched). Thanks a lot for your help
>

Process memory is virtual memory, well normally.  The usual way heap
allocations goes is that the kernel will create page table entries for
your allocated memory, but not actually [or always] map it to physical
memory until it's touched (e.g. creates a page fault).

The program shouldn't crash just because you malloc beyond 1G, more
likely it's returning NULL and you're dereferencing the pointer.  Could
be you have a ulimit in place, or a platform specific hard limit.  At
anyrate it has to map the memory into your process'es address space.
For example, if you have a hard limit of 1GB of process memory, and your
code is 512MB, then you can only allocate 512MB of data before running
out of address space.

Well strictly speaking this all depends on the platform... but usually
code/data are mapped to the same address space, which allows for things
like self-modifying code and the like.

Tom