This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH,nvptx] Use CUDA driver API to select default runtime launch, geometry


On 08/01/2018 04:01 PM, Cesar Philippidis wrote:
> On 08/01/2018 03:18 AM, Tom de Vries wrote:
>> On 07/31/2018 04:58 PM, Cesar Philippidis wrote:
>>> The attached patch teaches libgomp how to use the CUDA thread occupancy
>>> calculator built into the CUDA driver. Despite both being based off the
>>> CUDA thread occupancy spreadsheet distributed with CUDA, the built in
>>> occupancy calculator differs from the occupancy calculator in og8 in two
>>> key ways. First, og8 launches twice the number of gangs as the driver
>>> thread occupancy calculator. This was my attempt at preventing threads
>>> from idling, and it operating on a similar principle of running 'make
>>> -jN', where N is twice the number of CPU threads.
>>
>> You're saying the two methods are different, and that the difference
>> between the two methods is a factor two, which is a heuristic you added
>> yourself on top of one of the methods, which implies that in fact the
>> two methods are identical. Is my understanding correct here?
> 
> With the exception being that og8 multiples num_gangs by a factor of
> two, those two algorithms are identical, at least with respect to gangs.
> 
>>> Second, whereas og8
>>> always attempts to maximize the CUDA block size, the driver may select a
>>> smaller block, which effectively decreases num_workers.
>>>
>>
>> So, do I understand it correctly that using the function
>> cuOccupancyMaxPotentialBlockSize gives us "minimum block size that can
>> achieve the maximum occupancy" or some such and og8 gives us "maximum
>> block size"?
> 
> Correct.
> 
>>> In terms of performance, there really isn't that much of a difference
>>> between the CUDA driver's occupancy calculator and og8's. However, on
>>> the tests that are impacted, they are generally within a factor of two
>>> from one another, with some tests running faster with the driver
>>> occupancy calculator and others with og8's.
>>>
>>
>> Ack. Well, until we understand that in more detail, going with the
>> driver's occupancy calculator seems the right thing to do.
>>
>>> Unfortunately, support for the CUDA driver API isn't universal; it's
>>> only available in CUDA version 6.5 (or 6050) and newer. In this patch,
>>> I'm exploiting the fact that init_cuda_lib only checks for errors on the
>>> last library function initialized.
>>
>> That sounds incorrect to me. In init_cuda_lib I see:
>> ...
>> # define CUDA_ONE_CALL(call) CUDA_ONE_CALL_1 (call)
>> # define CUDA_ONE_CALL_1(call) \
>>   cuda_lib.call = dlsym (h, #call);     \
>>   if (cuda_lib.call == NULL)            \
>>     return false;
>>   CUDA_CALLS
>> ...
>> so in fact every library function is checked. Have you tested this with
>> pre 6-5 cuda?
> 
> I misread that. You're correct. So far, I've only tested this out with
> CUDA 9.
> 
>> I think we need to add and handle:
>> ...
>>   CUDA_ONE_CALL_MAYBE_NULL (cuOccupancyMaxPotentialBlockSize)
>> ...
>>
>>> Therefore it guards the usage of
>>>
>>>   cuOccupancyMaxPotentialBlockSizeWithFlags
>>>
>>> by checking driver_version.
>>
>> If we allow the cuOccupancyMaxPotentialBlockSize field to be NULL, we
>> can test for NULL, which seems a simpler solution than testing the version.
>>
>>> If the driver occupancy calculator isn't
>>> available, it falls back to the existing defaults. Maybe the og8 thread
>>> occupancy would make a better default for older versions of CUDA, but
>>> that's a patch for another day.
>>>
>>
>> Agreed.
>>
>>> Is this patch OK for trunk?
>>
>> The patch doesn't build in a setup with
>> --enable-offload-targets=nvptx-none and without cuda, that enables usage
>> of plugin/cuda/cuda.h:
>> ...
>> /data/offload-nvptx/src/libgomp/plugin/plugin-nvptx.c:98:16: error:
>> ‘cuOccupancyMaxPotentialBlockSize’ undeclared here (not in a function);
>> did you mean ‘cuOccupancyMaxPotentialBlockSizeWithFlags’?
>>  CUDA_ONE_CALL (cuOccupancyMaxPotentialBlockSize) \
>> ...
>>
>>> @@ -1220,11 +1227,39 @@ nvptx_exec (void (*fn), size_t mapnum, void **hostaddrs, void **devaddrs,
>>>  
>>>        {
>>>  	bool default_dim_p[GOMP_DIM_MAX];
>>> +	int vectors = nvthd->ptx_dev->default_dims[GOMP_DIM_VECTOR];
>>> +	int workers = nvthd->ptx_dev->default_dims[GOMP_DIM_WORKER];
>>> +	int gangs = nvthd->ptx_dev->default_dims[GOMP_DIM_GANG];
>>> +
>>> +	/* The CUDA driver occupancy calculator is only available on
>>> +	   CUDA version 6.5 (6050) and newer.  */
>>> +	if (nvthd->ptx_dev->driver_version > 6050)
>>> +	  {
>>> +	    int grids, blocks;
>>> +	    CUDA_CALL_ASSERT (cuOccupancyMaxPotentialBlockSize, &grids,
>>> +			      &blocks, function, NULL, 0,
>>> +			      dims[GOMP_DIM_WORKER] * dims[GOMP_DIM_VECTOR]);
>>> +	    GOMP_PLUGIN_debug (0, "cuOccupancyMaxPotentialBlockSize: "
>>> +			       "grid = %d, block = %d\n", grids, blocks);
>>> +
>>
>>
>>> +	    if (GOMP_PLUGIN_acc_default_dim (GOMP_DIM_GANG) == 0)
>>
>> You should use gomp_openacc_dims[0].
>>
>>> +	      gangs = grids * (blocks / warp_size);
>>
>> So, we launch with gangs == grids * workers ? Is that intentional?
> 
> Yes. At least that's what I've been using in og8. Setting num_gangs =
> grids alone caused significant slow downs.
> 

Well, what you're saying here is: increasing num_gangs increases
performance.

You don't explain why you multiply with workers specifically.

>>> +
>>> +	    if (GOMP_PLUGIN_acc_default_dim (GOMP_DIM_WORKER) == 0)
>>> +	      workers = blocks / vectors;
>>
>> Also, the new default calculation is not nicely separated from the
>> fallback default calculation.  I've updated the patch with a cleaner
>> separation, attached and build without cuda but untested.
> 
> That looks nicer. Did you want to take over this patch?

No.

Thanks,
- Tom


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]