학술논문

LogFold: Enhancing Log Anomaly Detection Through Sequence Folding and Reconstruction
Document Type
Conference
Source
2023 30th Asia-Pacific Software Engineering Conference (APSEC) APSEC Software Engineering Conference (APSEC), 2023 30th Asia-Pacific. :181-190 Dec, 2023
Subject
Computing and Processing
Industries
Soft sensors
Quality of service
Throughput
Transformers
Reliability engineering
Software reliability
anomaly detection
log analysis
deep learning
system reliability
Language
ISSN
2640-0715
Abstract
Modern large-scale systems and networks necessitate automated anomaly detection to support the high availability and quality of services. Since logs are an essential data source that can accurately reflect the state of a system, log anomaly detection has attracted a lot of attention from researchers in both academia and industry. As the technology of artificial intelligence advances, plenty of work has adopted deep learning to detect log anomalies and achieved promising results. Nevertheless, it usually suffers from a lack of labels, excessive log sequence length, and low throughput problems when deploying to real-world systems. To address these challenges, we propose Log-Fold, an unsupervised Transformer-based log anomaly detection approach. In LogFold, we propose fold embedding, which can compress long log sequences to enhance the efficiency of anomaly detection. And we design a sequence reconstruction technique to enhance the effectiveness of anomaly detection. Our evaluation shows LogFold achieves 90.55% and 99.90% Fl-score on HDFS and BGL datasets, respectively, outperforming state-of-the-art methods. Besides, the fold embedding layer achieves compression rates of 36.55% and 64.86% on HDFS and BGL datasets, respectively, which helps to improve the throughput of LogFold.