Detecting Botnets Using Hidden Markov Model, Profile Hidden Markov Model and Network Flow Analysis

Abstract

Botnet is a network of infected computer systems called bots managed remotely by an attacker using bot controllers. Using distributed systems, botnets can be used for large-scale cyber attacks to execute unauthorized actions on the targeted system like phishing, distributed denial of service (DDoS), data theft, and crashing of servers. Common internet protocols used by normal systems for regular communication like hypertext transfer (HTTP) and internet relay chat (IRC) are also used by botnets. Thus, distinguishing botnet activity from normal activity can be challenging. To address this issue, this project proposes an approach to detect botnets using peculiar traits in the communication between command and control servers and bots. Patterns can be observed in botnet behavior like orchestrated attacks, heartbeat signals, or periodic distribution of commands. Hidden Markov Models (HMM) and Profile Hidden Markov Model (PHMM) are probabilistic models that can be trained on network traffic data to identify activity patterns that suggest botnet activity. In this project, HMM and PHMM are used to detect and classify botnets using publicly available datasets for real network data consisting of botnet traffic mixed with normal and background traffic. A comparative analysis of performance of HMM and PHMM is conducted in this project and the results show that HMM and PHMM can be useful in detecting botnets. PHMM outperforms HMM in terms of accuracy of botnet detection.