Enhancing Botnet Detection in Network Security Using Profile Hidden Markov Models

Abstract

A botnet is a network of compromised computer systems, or bots, remotely controlled by an attacker through bot controllers. This covert network poses a threat through large-scale cyber attacks, including phishing, distributed denial of service (DDoS), data theft, and server crashes. Botnets often camouflage their activity by utilizing common internet protocols, such as HTTP and IRC, making their detection challenging. This paper addresses this threat by proposing a method to identify botnets based on distinctive communication patterns between command and control servers and bots. Recognizable traits in botnet behavior, such as coordinated attacks, heartbeat signals, and periodic command distribution, are analyzed. Probabilistic models, specifically Hidden Markov Models (HMMs) and Profile Hidden Markov Models (PHMMs), are employed to learn and identify these activity patterns in network traffic data. This work utilizes publicly available datasets containing a combination of botnet, normal, and background traffic to train and test these models. The comparative analysis reveals that both HMMs and PHMMs are effective in detecting botnets, with PHMMs exhibiting superior accuracy in botnet detection compared to HMMs.