Enhancements to Language Modeling Techniques for Adaptable Log Message Classification

Abstract

Minimizing the resolution time of service-impacting incidents is a fundamental objective of Information Technology (IT) operations. Efficient root cause analysis, adaptable to diverse service environments, is key to meeting this objective. One method that provides additional insight into an incident, and hence allows enhanced root cause analysis, is categorisation of the events and log messages that characterize an incident into pre-defined operational groups. Well established natural language processing techniques that utilize pre-trained language models and word embeddings can be leveraged for this task. The adaptability of pre-trained models to classify log messages, containing large quantities of domain-specific language, remains unknown. The current contribution investigates multiple ways of addressing this deficiency. We demonstrate increased granularity of word embeddings by using character decompositions and sub-word level representations, and also explore the augmentation of word embeddings using features derived from convolutional operations. After observing that the performance of high-specificity models decreases as the number of previously unseen words increases, we explore the circumstances in which we can use a model trained with a low-specificity corpus to correctly classify log messages. Through the application of fine-tuning techniques, we can adapt our pre-trained classifier to classify log messages from service environments not encountered during pre-training in a time, and memory efficient manner. We conclude that we can effectively adapt pre-trained classifiers for impromptu service environments.