Hierarchical Anomaly-Based Detection of Distributed DNS Attacks on Enterprise Networks

Abstract

Domain Name System (DNS) is a critical service for enterprise operations, and is often made openly accessible across firewalls. Malicious actors use this fact to attack organizational DNS servers, or use them as reflectors to attack other victims. Further, attackers can operate with little resources, can hide behind open recursive resolvers, and can amplify their attack volume manifold. The rising frequency and effectiveness of DNS-based DDoS attacks make this a growing concern for organizations. Solutions available today, such as firewalls and intrusion detection systems, use combinations of black-lists of malicious sources and thresholds on DNS traffic volumes to detect and defend against volumetric attacks, which are not robust to attack sources that morph their identity or adapt their rates to evade detection. We propose a method for detecting distributed DNS attacks that uses a hierarchical graph structure to track DNS traffic at three levels of host, subnet, and autonomous system (AS), combined with machine learning that identifies anomalous behaviors at various levels of the hierarchy. Our method can detect distributed attacks even with low rates and stealthy patterns. Our contributions are three-fold: (1) We analyze real DNS traffic over a week (nearly 400M packets) from the edges of two large enterprise networks to highlight various types of incoming DNS queries and the behavior of malicious entities generating query scans and floods; (2) We develop a hierarchical graph structure to monitor DNS activity, identify key attributes, and train/tune/evaluate anomaly detection models for various levels of the hierarchy, yielding more than 99% accuracy at each level; and (3) We apply our scheme to a month’s worth of DNS data from the two enterprises and compare the results against blacklists and firewall logs to demonstrate its ability in detecting distributed attacks that might be missed by legacy methods while maintaining a decent real-time performance.