학술논문

Mapping Out the HPC Dependency Chaos
Document Type
Conference
Source
SC22: International Conference for High Performance Computing, Networking, Storage and Analysis SC High Performance Computing, Networking, Storage and Analysis, SC22: International Conference for. :1-12 Nov, 2022
Subject
Communication, Networking and Broadcast Technologies
Chaos
Biological system modeling
High performance computing
Taxonomy
Semantics
Full stack
Packaging
toolchains
package management
operating systems
filesystem hierarchy
Language
ISSN
2167-4337
Abstract
High Performance Computing (HPC) software stacks have become complex, with the dependencies of some applications numbering in the hundreds. Packaging, distributing, and administering software stacks of that scale is a complex undertaking anywhere. HPC systems deal with esoteric compilers, hardware, and a panoply of uncommon combinations. In this paper, we explore the mechanisms available for packaging software to find its own dependencies in the context of a taxonomy of software distribution, and discuss their benefits and pitfalls. We discuss workarounds for some common problems caused by using these composed stacks and introduce Shrinkwrap: A solution to producing binaries that directly load their dependencies from precise locations and in a precise order. Beyond simplifying the use of the binaries, this approach also speeds up loading as much as 7× for a large dynamically-linked MPI application in our evaluation.