A two-phase sequential pattern mining framework to detect stealthy P2P botnets

Abstract

The botnet has been one of the most common threats to network security. Among all emerged botnet, Peer to Peer (P2P) botnets are more perilous and resistant due to their distributed nature. In addition to their resiliency against takedown strategies, modern P2P botnets are stealthier in the way they perform fraudulent activities. One of the main challenges to detect P2P bots/botnets is the presence of benign P2P traffic. The botnet traffic can blend in with legitimate P2P traffic, and it makes the P2P bots stealthier. However, the problem of detecting P2P botnets in the presence of legitimate P2P traffic has received little attention from the research community.In this paper, a novel P2P botnet detection framework resilient to the presence of legitimate P2P traffic is proposed based on a two-phase Sequential Pattern Mining (SPM) approach. The proposed framework is evaluated in many different cases of the coexistence of P2P malicious and legitimate traffics, using real-world network traffic. Our experimental results show that the proposed framework is capable of detecting P2P bots in the presence of legitimate P2P traffic with a detection rate of 99.2%. Besides its accurate detection, our proposed framework is highly scalable and can detect even one bot in the network or different bots from different bot families.