Network log analysis based on the topic word mover's distance

Abstract

Telecommunication networks continuously generate various system logs which include plentiful information of system status. So these logs can be used to detect whether a network is under a fault scenario or not. In this paper, we propose an improved word mover's distance (WMD) called Topic Word Mover's Distance (T-WMD) to measure the distance between two log samples and then classify different fault logs to determine the fault root cause. Compared with original WMD, T-WMD takes topic information into consideration and provides more latent semantic information of log corpus. Experiments of k-nearest neighbor (k-nn) fault logs classification show that our T-WMD metric outperforms the original WMD.