학술논문

Automatic aggregation of subtask accesses for nested OpenMP-style tasks
Document Type
Conference
Source
2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) SBAC-PAD Computer Architecture and High Performance Computing (SBAC-PAD), 2022 IEEE 34th International Symposium on. :315-325 Nov, 2022
Subject
Computing and Processing
Runtime
Codes
Annotations
Distributed databases
Manuals
Benchmark testing
Parallel processing
HPC
Runtime Systems
OpenMP
OmpSs-2
Task Parallelism
Distributed Computing
Language
ISSN
2643-3001
Abstract
Task-based programming is a high performance and productive model to express parallelism. Tasks encapsulate work to be executed across multiple cores or offloaded to GPUs, FPGAs, other accelerators or other nodes. In order to maintain parallelism and afford maximum freedom to the scheduler, the task dependency graph should be created in parallel and well in advance of task execution. A key limitation with OpenMP and OmpSs-2 tasking is that a task cannot be created until all its accesses and its descendents' accesses are known. Current approaches to work around this limitation either stop task creation and execution using a taskwait or they substitute “fake” accesses known as sentinels. This paper proposes the auto clause, which indicates that the task may create subtasks that access unspecified memory regions or it may allocate and return memory at addresses that are of course not yet known. Unlike approaches using taskwaits, there is no interruption to the concurrent creation and execution of tasks, maintaining parallelism and the scheduler's ability to optimize load balance and data locality. Unlike existing approaches using sentinels, all tasks can be given a precise specification of their own data accesses, so that a single mechanism is used to control task ordering, program data transfers on distributed memory and optimize data locality, e.g. on NUMA systems. The auto clause also provides an incremental path to develop programs with nested tasks, by removing the need for every parent task to have a complete specification of the accesses of its descendent tasks. This is redundant information that can be time consuming and error-prone to describe. We present a straightforward runtime implementation that achieves a 1.4 times speedup for n-body with OmpSs-2@Cluster task offloading to 32 nodes and