Anomaly Detection and Attribution in Network

Abstract

In this article, we address the problem of not only id entifying phenomena, but also attributing the phenomenon to the movement that induces it. This causes to a combinatorial optimisation problem, which is prohibitively expensive. Instead we design two anomaly detection algorithms that are small in complexity. The first is based on the system for cross-entropy (CE), which identifies flow anomalies and labels flow anomalies. The second algorithm detects anomalies through GLRT on aggregated flow transformation a compact low-dimensional representation of raw traffic flows. The two algorithms complement each other and allow the network operator to use the algorithm for flow aggregation first so that device irregularities can be identified easily. After discovery of an exception, the user Can analyse further that individual flows are anomalous using CE-based algorithm. We perform extensive performance tests and trials on synthetic and semi-synthetic data with our algorithms, as well as real Internet traffic data gathered from the MAWI database, and finally make recommendations as to their usability.