This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] liboffloadmic emulation mode: make it asynchronous


On Mon, Sep 28, 2015 at 05:53:42PM +0300, Ilya Verbin wrote:
> Currently the COI emulator is single-threaded, i.e. it is able to run only one
> target function at a time, e.g. the following testcase:
> 
>   #pragma omp parallel sections num_threads(2)
>     {
>       #pragma omp section
>       #pragma omp target
>       while (1)
> 	putchar ('.');
> 
>       #pragma omp section
>       #pragma omp target
>       while (1)
> 	putchar ('o');
>     }
> 
> prints only dots using emul, while using real libcoi it prints:
> ...o.ooooo.o.o...o...o....oooo.oo.o.....o.ooo.oooooo...o.ooooooooo.o...o.ooooooo
> Of course, it's not possible to test new OpenMP 4.1's async features using such
> an emulator.
> 
> The patch bellow makes it asynchronous, it creates an auxiliary thread for each
> COIPipeline in host and in target processes.  In general, a new COIPipeline is
> created by liboffloadmic for each host thread with offload, i.e. the example
> above has:
> 4 threads in the host process (2 OpenMP threads + 2 auxiliary threads) and
> 3 threads in the target process (1 main thread + 2 auxiliary threads).
> An auxiliary host thread runs a target function in the new thread in target
> process and waits for its completion.  When the function is finished, the host
> thread signals an event and can run a callback, if it is registered.
> liboffloadmic waits for signalled events by calling COIEventWait.
> This is identical to how real libcoi works.
> 
> make check-target-libgomp and some internal tests did not show any regression.
> TSan report is clean.  Is it OK for trunk?

For now ok.  Though, I'd say I'd prefer if there were no auxiliary threads
on the host side, just whatever thread is asked to send something to/from
the device, wait for something and/or poll for something just polling the
pipes.  Are there auxiliary host threads also for the case when using
the real COI, offloading to hw?
> 
> 
> liboffloadmic/
> 	* plugin/libgomp-plugin-intelmic.cpp (OFFLOAD_ACTIVE_WAIT_ENV): New
> 	define.
> 	(init): Set OFFLOAD_ACTIVE_WAIT env var to 0, if it is not set.
> 	* runtime/emulator/coi_common.h (PIPE_HOST_PATH): Replace with ...
> 	(PIPE_HOST2TGT_NAME): ... this.
> 	(PIPE_TARGET_PATH): Replace with ...
> 	(PIPE_TGT2HOST_NAME): ... this.
> 	(MALLOCN): New define.
> 	(READN): Likewise.
> 	(WRITEN): Likewise.
> 	(enum cmd_t): Replace CMD_RUN_FUNCTION with CMD_PIPELINE_RUN_FUNCTION.
> 	Add CMD_PIPELINE_CREATE, CMD_PIPELINE_DESTROY.
> 	* runtime/emulator/coi_device.cpp (engine_dir): New static variable.
> 	(pipeline_thread_routine): New static function.
> 	(COIProcessWaitForShutdown): Use global engine_dir instead of mic_dir.
> 	Rename pipe_host and pipe_target to pipe_host2tgt and pipe_tgt2host.
> 	If cmd is CMD_PIPELINE_CREATE, create a new thread for the pipeline.
> 	Remove cmd == CMD_RUN_FUNCTION case.
> 	* runtime/emulator/coi_device.h (COIERRORN): New define.
> 	* runtime/emulator/coi_host.cpp: Include set, map, queue.
> 	Replace typedefs with enums and structs.
> 	(struct Function): Remove name, add num_buffers, bufs_size,
> 	bufs_data_target, misc_data_len, misc_data, return_value_len,
> 	return_value, completion_event.
> 	(struct Callback): New.
> 	(struct Process): Remove pipeline.  Add pipe_host2tgt and pipe_tgt2host.
> 	(struct Pipeline): Remove pipe_host and pipe_target.  Add thread,
> 	destroy, is_destroyed, pipe_host2tgt_path, pipe_tgt2host_path,
> 	pipe_host2tgt, pipe_tgt2host, queue, process.
> 	(max_pipeline_num): New static variable.
> 	(pipelines): Likewise.
> 	(max_event_num): Likewise.
> 	(non_signalled_events): Likewise.
> 	(errored_events): Likewise.
> 	(callbacks): Likewise.
> 	(cleanup): Do not check tmp_dirs before free.
> 	(start_critical_section): New static function.
> 	(finish_critical_section): Likewise.
> 	(pipeline_is_destroyed): Likewise.
> 	(maybe_invoke_callback): Likewise.
> 	(signal_event): Likewise.
> 	(get_event_result): Likewise.
> 	(COIBufferCopy): Rename arguments according to headers.  Add asserts.
> 	Use process' main pipes, instead of pipeline's pipes.  Signal completion
> 	event.
> 	(COIBufferCreate): Rename arguments according to headers.  Add asserts.
> 	Use process' main pipes, instead of pipeline's pipes.
> 	(COIBufferCreateFromMemory): Rename arguments according to headers.
> 	Add asserts.
> 	(COIBufferDestroy): Rename arguments according to headers.  Add asserts.
> 	Use process' main pipes, instead of pipeline's pipes.
> 	(COIBufferGetSinkAddress): Rename arguments according to headers.
> 	Add asserts.
> 	(COIBufferMap): Rename arguments according to headers.  Add asserts.
> 	Signal completion event.
> 	(COIBufferRead): Likewise.
> 	(COIBufferSetState): Likewise.
> 	(COIBufferUnmap): Likewise.
> 	(COIBufferWrite): Likewise.
> 	(COIEngineGetCount): Add assert.
> 	(COIEngineGetHandle): Rename arguments according to headers.
> 	Add assert.
> 	(COIEventWait): Rename arguments according to headers.  Add asserts.
> 	Implement waiting for events with zero or infinite timeout.
> 	(COIEventRegisterCallback): New function.
> 	(pipeline_thread_routine): New static function.
> 	(COIPipelineCreate): Create a new thread for the pipeline.
> 	(COIPipelineDestroy): Exit pipeline thread.
> 	(COIPipelineRunFunction): Add the function into pipeline's queue,
> 	instead running it here.  Wait for it's completion in case of
> 	synchronous execution.
> 	(COIProcessCreateFromMemory): Rename arguments according to headers.
> 	Add asserts.  Create process' main pipes, instead of pipeline's pipes.
> 	(COIProcessDestroy): Rename arguments according to headers.
> 	Add asserts.  Destroy all undestroyed pipelines.
> 	(COIProcessGetFunctionHandles): Rename arguments according to headers.
> 	Add asserts.  Use process' main pipes, instead of pipeline's pipes.
> 	Remove useless function names.
> 	(COIProcessLoadLibraryFromMemory): Add asserts.  Use process' main
> 	pipes, instead of pipeline's pipes.
> 	(COIProcessUnloadLibrary): Likewise.
> 	(COIEngineGetInfo): Add assert.
> 	* runtime/emulator/coi_host.h (COIERRORN): New define.

	Jakub


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]