CCGA: Clustering and Capturing Group Activities for DGA-Based Botnets Detection

Abstract

Botnet is a part of the most destructive threats to network security and is often used in malicious activities. DGA-based botnet, which uses Domain Generation Algorithm (DGA) to evade detection, has become the main channel to carry out online crimes. In the past, many detection mechanisms focusing on domain features are proposed, but the potential problem is that the features extracting only from the domain names are insufficient and the enemies could easily forge them to disturb detection. In this paper, we propose a novel approach named CCGA to detect DGA-based botnet by leveraging the concerted group behaviors of infected hosts on DNS traffic. The analysis of group behaviors enhances the robustness of our system irrespective of various evasion techniques, such as fake-querying, packet encryption and noise generated by normal users. The proposed scheme associates hosts together in an unsupervised way and then uses supervised learning to distinguish whether it's a botnet. Our system is evaluated in a large ISP over two days and compared with the state of art FANCI. Experimental results show that CCGA can accurately and effectively detect DGA-based botnet in a real-world network. Our system also catches 5 unknown botnet groups and provides a novel method to verify them. Therefore, the system will provide an unique perspective on the current state of globally distributed malware, particularly the ones that use DNS.