This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [patch] adjust default nvptx launch geometry for OpenACC offloaded regions
- From: Cesar Philippidis <cesar at codesourcery dot com>
- To: Tom de Vries <tdevries at suse dot de>
- Cc: "gcc-patches at gcc dot gnu dot org" <gcc-patches at gcc dot gnu dot org>, Jakub Jelinek <jakub at redhat dot com>, Thomas Schwinge <thomas at codesourcery dot com>
- Date: Thu, 26 Jul 2018 07:27:29 -0700
- Subject: Re: [patch] adjust default nvptx launch geometry for OpenACC offloaded regions
- References: <c36981ca-04bc-08df-0af7-c2e1a8201640@codesourcery.com> <7864b0d0-b39f-713b-9d5c-13e91c11bd55@suse.de> <0034e0fb-402e-8681-0f62-3fd274d00a99@codesourcery.com> <130dc2ce-1db1-1fdd-a4a3-63e479466beb@suse.de> <5c0b9b2d-bbe3-dfd9-1ae5-a164a4079e0c@codesourcery.com>
Hi Tom,
I see that you're reviewing the libgomp changes. Please disregard the
following hunk:
On 07/11/2018 12:13 PM, Cesar Philippidis wrote:
> @@ -1199,12 +1202,59 @@ nvptx_exec (void (*fn), size_t mapnum, void **hostaddrs, void **devaddrs,
> default_dims[GOMP_DIM_VECTOR]);
> }
> pthread_mutex_unlock (&ptx_dev_lock);
> + int vectors = default_dims[GOMP_DIM_VECTOR];
> + int workers = default_dims[GOMP_DIM_WORKER];
> + int gangs = default_dims[GOMP_DIM_GANG];
> +
> + if (nvptx_thread()->ptx_dev->driver_version > 6050)
> + {
> + int grids, blocks;
> + CUDA_CALL_ASSERT (cuOccupancyMaxPotentialBlockSize, &grids,
> + &blocks, function, NULL, 0,
> + dims[GOMP_DIM_WORKER] * dims[GOMP_DIM_VECTOR]);
> + GOMP_PLUGIN_debug (0, "cuOccupancyMaxPotentialBlockSize: "
> + "grid = %d, block = %d\n", grids, blocks);
> +
> + gangs = grids * dev_size;
> + workers = blocks / vectors;
> + }
I revisited this change yesterday and I noticed it was setting gangs
incorrectly. Basically, gangs should be set as follows
gangs = grids * (blocks / warp_size);
or to be more closer to og8 as
gangs = 2 * grids * (blocks / warp_size);
The use of that magic constant 2 is to prevent thread starvation. That's
a similar concept behind make -j<2*#threads>.
Anyway, I'm still experimenting with that change. There are still some
discrepancies between the way that I select num_workers and how the
driver does. The driver appears to be a little bit more conservative,
but according to the thread occupancy calculator, that should yield
greater performance on GPUs.
I just wanted to give you a heads up because you seem to be working on this.
Thanks for all of your reviews!
By the way, are you now maintainer of the libgomp nvptx plugin?
Cesar