BackStreamDB: A stream processing engine for backbone traffic monitoring with anomaly detection

Abstract

AbstractBackStreamDB is distributed traffic monitoring system based on a stream processing engine (SPE) designed to monitor the traffic of wide area backbones. BackStreamDB provides arbitrary metrics about the traffic in real time, taking into account the backbone as a whole. The system was developed for and successfully deployed on the Brazilian National Academic Network (RNP). In this work, we describe the functionality for the detection of traffic anomalies. A large number of Internet attacks are continuously reported, and several types of attacks result in anomalous traffic. In the proposed strategy for anomaly detection, the traffic is sampled by monitors that are distributed across the backbone, which are accessed and processed by the SPE. BackStreamDB was extended with stream processing modules for computing traffic entropy and principal component analysis, which are the employed to detect traffic anomalies. Experimental results are reported which were obtained to validate the effectiveness of the proposed strategy for different types of attacks.