Survey of Research Issues and Proposed Solutions for Detecting Parameter Anomalies in System Logs

Abstract

In the ever-evolving field of software development, the demand for automation of fault analysis that is time-consuming and expertise-requiring is growing. One solution to this challenge is the study of anomaly detection using text logs, which has seen numerous research efforts. However, despite the variety of patterns that system anomalies can exhibit, many studies have predominantly focused on sequence anomalies. This is largely attributed to the limited availability of datasets, with the commonly used Loghub data being oriented towards sequence anomalies. This research addresses the current challenges in anomaly detection models and proposes several new methods for detecting parameter anomalies. Initially, due to the lack of datasets of parameter anomalies, we prepared common parameter anomaly scenarios and compared them with existing sequence anomaly detection models (including DNN models for sequence anomalies and DNN models using semantic information), and with a variety of proposed methods. The prepared parameter anomaly patterns include four Integer types and three String types. For instance, a parameter within a certain range (-100 to 100) is considered normal, while parameters outside this range are deemed anomalies. Our proposed method begins by extracting parameters using LogParser and determining whether they are of Int or String type. For Int types, we use Z-Score, IQR, K-NN and DBSCAN for evaluation, while for String types, we use a Bert-based positive-negative classifier. The experimental results showed that the DNN model for sequence anomaly had an F1 Score of less than 0.5 for all patterns. In contrast, our proposed methods achieved F1 Scores exceeding 0.9 or 0.8 for almost all methods, except for one anomaly pattern. It was found that the proposed methods are effective for common parameter anomaly problems. Furthermore, since our methods do not require prior training, they are particularly advantageous for ad-hoc learning in the context of continuously updated software development.

Keywords: Anomaly Detection, Software Log, Log Analysis, Deep Learning, Log Generator, Prameter Anomaly Detection

DOI: 10.54941/ahfe1005479