학술논문

The Sub-Sequence Summary Method for Detecting Anomalies in Logs
Document Type
Periodical
Source
IEEE Access Access, IEEE. 11:37412-37423 2023
Subject
Aerospace
Bioengineering
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Engineered Materials, Dielectrics and Plasmas
Engineering Profession
Fields, Waves and Electromagnetics
General Topics for Engineers
Geoscience
Nuclear Engineering
Photonics and Electrooptics
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Transportation
Feature extraction
Training data
Anomaly detection
Transformers
Numerical models
Biological system modeling
Semantics
Log analytics
anomaly detection
sequence anomaly
attribute anomaly
auto-encoder
Language
ISSN
2169-3536
Abstract
This paper introduces a novel method for detecting log anomalies using deep learning. In contrast to state-of-the-art methods that rely on sequence models such as LSTMs or Transformers, our approach does not require the subsequent log lines to be fed directly into the model. Instead, we extract specific features from the log sequence, and derive anomaly scores from the reconstruction loss of an ordinary auto-encoder. These features are easy to obtain and contain sequential information. The presented method can detect both sequence and attribute anomalies using a single integrated model. We present two variants: a template-based method and a fully semantic-based method.