학술논문

Integration of Burst Buffer in High-level Parallel I/O Library for Exa-scale Computing Era
Document Type
Conference
Source
2018 IEEE/ACM 3rd International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS) PDSW-DISCS Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS), 2018 IEEE/ACM 3rd International Workshop on. :1-12 Nov, 2018
Subject
Computing and Processing
Libraries
Buffer storage
File systems
Servers
Supercomputers
Metadata
Hard disks
burst-buffer;-Log;-Exa-scale
Language
Abstract
While the computing power of supercomputers continues to improve at an astonishing rate, companion I/O systems are struggling to keep up in performance. To mitigate the performance gap, several supercomputing systems have been configured to incorporate burst buffers into their I/O stack; the exact role of which, however, still remains unclear. In this paper, we examine the features of burst buffers and study their impact on application I/O performance. Our goal is to demonstrate that burst buffers can be utilized by parallel I/O libraries to significantly improve performance. To this end, we developed an I/O driver in PnetCDF that uses a log-based format to store individual I/O requests on the burst buffer – later to be flushed to the parallel file system as one request. We evaluated our implementation by running standard I/O benchmarks on Cori, a Cray XC40 supercomputer at NERSC with a centralized burst buffer system, and Theta, a Cray XC40 supercomputer at ALCF with locally available SSDs. Our results show that IO aggregation is a promising role for burst buffers in high-level I/O libraries.