Deep Reinforcement Learning-Based Traffic Sampling for Multiple Traffic Analyzers on Software-Defined Networks

Abstract

Intrusion detection system (IDS) and deep packet inspection (DPI) are widely used to detect network attacks and anomalies, thereby enhancing cyber-security. Conventional traffic analyzers such as IDS have fixed locations and a limited capacity to perform DPI on large volumes of network traffic. Nowadays, software-defined networking (SDN) technology, which provides flexibility, elasticity, and programmability by decoupling the network control and data planes, makes it possible to capture entire or a certain portion of data traffic flows on SDN-capable switches and steer the captured network traffic to one of the traffic analyzers on the network. Therefore, how to sample network traffic and where to steer the sampled traffic among multiple traffic analyzers are critical problems facing cyber-security. Since there is a possibility that potentially useful information will be lost in not-captured traffic, deciding the sampling points and sampling rates of network traffic remains important. Additionally, after determining the sampling points and rates, sampled traffic must be sent to one of the multiple traffic analyzers for traffic inspection, which may incur additional network delivery overheads. We propose a less-intrusive traffic sampling mechanism for multiple traffic analyzers on an SDN-capable network using a deep deterministic policy gradient (DDPG), which is a representative deep reinforcement learning (DRL) algorithm for continuous action control. The proposed system learns sampling resource allocation policy under the uncertainty of flow distribution according to sampled traffic inspection results obtained from multiple traffic analyzers. Through extensive simulations and the SDN-based testbed experiments, we demonstrate that the proposed approach has a high probability of capturing malicious flows while maintaining a balanced load of multiple traffic analyzers and reducing flow monitoring overheads.