Re: [RFC PATCH] Coalesce host to device transfers in libgomp

[Alexander Monakov <amonakov at ispras dot ru>]

On Tue, 24 Oct 2017, Jakub Jelinek wrote:
> > Why did you chose the 32KB and 4KB limits? I wonder if that would have
> > any impact on firstprivate_int values. If this proves to be effective,
> > it seems like we should be able to eliminate GOMP_MAP_FIRSTPRIVATE_INT
> > altogether.
> 
> The thing is that this is a generic code, so it is hard to come up with
> reasonable limits.  We could even have some limits e.g. in *devicep
> if we get different needs for different offloading targets.
> 
> The 32KB and 4KB just come from some discussions with Alexander on IRC
> that larger copies saturate the PCI and the overhead isn't significant, so
> in that case copying e.g. megabyte into another memory and then to the
> device would likely not be beneficial.

Hm, I guess some miscommunication happened here.  On IRC I said,

>> from my tests, at 32+MB it approaches bus bandwidth (10GB/s for gen3 pcie),
>> at few kilobytes I'd expect aggregation to pay off

Note I really meant 32+ megabytes, not kilobytes, but of course I'm not
suggesting that libgomp allocates a multi-megabyte staging buffer and
memcpy's everything into it all the time.

Generally speaking, for optimal transfers one should use permanently allocated
locked ("pinned") memory and/or asynchronous transfers, but unfortunately at the
moment I don't have a good understanding of existing design and OpenMP spec
constraints to know what libgomp can/should do.

Alexander