Update on shared memory coarrays

[Iain Sandoe]

Hi Nicolas,

Nicolas König <koenigni@student.ethz.ch> wrote:
>
> On 05/01/2021 14:11, Iain Sandoe wrote:
>> Hi folks,
>> Nicolas König <koenigni@student.ethz.ch> wrote:
>>> On 04/01/2021 03:34, Iain Sandoe wrote:
>>>> Thomas Koenig via Fortran <fortran@gcc.gnu.org> wrote:
>>>>> Am 03.01.21 um 16:42 schrieb Iain Sandoe:
>>>> It seems shm is not a favoured IPC mechanism for Darwin (it’s  
>>>> possibly  disallowed for GUI apps, according to search results - I  
>>>> didn’t check  that)
>>>>>> (we can file a radar, and maybe it gets fixed at some point - but   
>>>>>> that’s no help to all the existing system versions)
>>>>>
>>>>> Makes sense to file a bug report with Apple.  And yes, we'll have to
>>>>> deal with this some other way.
>>>> [for command line apps] Either mechanism for accessing shm works;
>>>> [1] shm_open / fd / mmap / munmap
>>>> [2] shmget / shmat / shmdt <= allocation is rounded up to the nearest   
>>>> page boundary.
>>>> Both have the constraint [on Darwin] that the size of the allocation   
>>>> cannot be increased after it is set (by ftruncate() [1] or shmget()  
>>>> [2]).
>>>> It can be decreased (by [2] only), but it’s not clear if that  
>>>> actually  has any physical effect.
>>>
>>> That's a problem, since coarrays can be allocated at runtime, and we   
>>> can't possibly precompute their size.
>> OK.
>>> To get coarrays to work, we would probably need to create new shared   
>>> memory objects (with predictable names, so they can be opened from  
>>> other  images), but that would be a significant time investment.
>> How does it work between different images with shared memory unless the
>> name to shm_open () is predictable between images?
>
> Currently, the images are launched by one "master"-process, that also  
> handles all the management. There, we can get some shared memory via  
> mmap(MAP_ANON | MAP_SHARED), so the master could communicate the base  
> name of the image. At the moment, it just opens a shared_memory object  
> and passes the file descriptor to its children.

Ah, I see - so the name can in that case be arbitrary
OTOH, what if one runs multiple Fortran codes (as the same user)?
(some simple hashing with the the master’s PID would fix that, I guess)

>>> It also doesn't bring my hopes up that other things like  
>>> pthread_{mutex,cond}attr_setpshared() will work.
>> Darwin is Posix SUSv3 (actually, AFAICT the majority of the certified
>> platforms are; there only seems to be one certified to UNIX7).
>> - so pthreads stuff that’s mandated in SUSv3 is definitely present on  
>> Darwin.
>> (in practice some of the optional stuff is also present - you can look at
>> what libstdc++ does to get an idea of what works in a cross-platform  
>> manner)
>
> Process shared mutexes/condition variables are optional :) Attached is a  
> test case that should check whether the two work.

> <shared_cond_test.c>

This will work from macOS 10.7+ (Darwin11+)

earlier OS versions fail with
pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED)failed:  
Undefined error: 0

(the compile succeeds, but some functionality is missing).

Iain