학술논문

Towards Increasing the Error Handling Time Window in Large-Scale Distributed Systems Using Console and Resource Usage Logs
Document Type
Conference
Source
2015 IEEE Trustcom/BigDataSE/ISPA Trustcom/BigDataSE/ISPA, 2015 IEEE. 3:61-68 Aug, 2015
Subject
Communication, Networking and Broadcast Technologies
Computing and Processing
Radiation detectors
Correlation
Anomaly detection
Resource usage data
Error logs
PCA
Root-cause analysis
large-scale HPC systems
Language
Abstract
Resource-intensive applications such as scientific applications require the architecture or system on which they execute to display a very high level of dependability to reduce the impact of faults. Typically, the state of the underlying system is captured through messages that are recorded in a log file, which has been proven useful to system administrators in understanding the root-causes of system failures (and for their subsequent debugging). However, the time window between when the first error message is detected in the log file and time of the ensuing failure may not be large enough to allow the administrators to save the state of the running application, which will result in lost execution time. We thus address this fundamental question: Is it possible to extend this time window? The answer is positive: We show that, by using (i) resource usage logs to track anomalous resource usage and (ii) error logs to identify root-causes of system failures, it is possible to increase the time window, on average, by 50 minutes. These files were those obtained for the Ranger Supercomputer from TACC. We achieve this by applying anomaly detection techniques on resource usage data and conducting a root-cause analysis on error log files.