Real time detection of malicious DoH traffic using statistical analysis

Abstract

The DNS protocol plays a fundamental role in the operation of ubiquitous networks. All devices connected to these networks need DNS to work, both for traditional domain name to IP address translation, and for more advanced services such as resource discovery. DNS over HTTPS (DoH) solves certain security problems present in the DNS protocol. However, malicious DNS tunnels, a covert way of encapsulating malicious traffic in a DNS connection, are difficult to detect because the encrypted data prevents performing an analysis of the content of the DNS traffic.In this study, we introduce a real-time system for detecting malicious DoH tunnels, which is based on analyzing DoH traffic using statistical methods. Our research demonstrates that it is feasible to identify in real-time malicious traffic by analyzing specific parameters extracted from DoH traffic. In addition, we conducted statistical analysis to identify the most significant features that distinguish malicious traffic from benign traffic. Using the selected features, we achieved satisfactory results in classifying DoH traffic as either benign or malicious.