학술논문

I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis
Document Type
Conference
Source
2021 IEEE/ACM Sixth International Parallel Data Systems Workshop (PDSW) PDSW Parallel Data Systems Workshop (PDSW), 2021 IEEE/ACM Sixth International. :15-22 Nov, 2021
Subject
Computing and Processing
File systems
Data visualization
Software
Libraries
Data systems
Performance analysis
Kernel
Language
Abstract
Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.