DDoS Traffic Control Using Transfer Learning DQN With Structure Information

Abstract

A DDoS attack is one of the most serious threats to the current Internet. The Router throttling is a popular method to response against DDoS attacks. Currently, coordinated team learning (CTL) has adopted tile coding for continuous state representation and strategy learning. It is suitable for this distributed challenge but lacks robustness. Our first contribution is that we adapt deep network as function approximation for continuous state representation, as a deep reinforcement learning approach is robust in many different Atari games with a little modification of the learning architecture. Furthermore, current multiagent router throttling methods only consider traffic-reading information. Therefore, for a homogeneous team scenario, all agents can share parameters with the same deep network. However, for heterogeneous team scenarios, if all agents still share one deep network, the learning policy may not be sufficiently ideal. Our second contribution is that we add team structure information so that all agents can still share one deep network. However, deep reinforcement learning is a considerably time-consuming task. Transfer learning is an appropriate method because learning policy in a simple scenario allows us to transfer the policy to other different and even complex scenarios. For transfer learning regarding the DDoS control problem, we propose a progressive transfer learning approach, which is our third contribution. Therefore, we can learn a better policy with less time consumption. Moreover, with progressive transfer learning, we can promote our method in a more complex environment. The experimental results validate that our three contributions truly achieve better performance than the existing methods.