MVDet: Encrypted malware traffic detection via multi-view analysis

Abstract

Detecting encrypted malware traffic promptly to halt the further propagation of an attack is critical. Currently, machine learning becomes a key technique for extracting encrypted malware traffic patterns. However, due to the dynamic nature of network environments and the frequent updates of malware, current methods face the challenges of detecting unknown malware traffic in open-world environment. To address the issue, we introduce MVDet, a novel method that employs machine learning to mine the behavioral features of malware traffic based on multi-view analysis. Unlike traditional methods, MVDet innovatively characterizes the behavioral features of malware traffic at 4-tuple flows from four views: statistical view, DNS view, TLS view, and business view, which is a more stable feature representation capable of handling complex network environments and malware updates. Additionally, we achieve a short-time behavioral features construction, significantly reducing the time cost for feature extraction and malware detection. As a result, we can detect malware behavior at an early stage promptly. Our evaluation demonstrates that MVDet can detect a wide variety of known malware traffic and exhibits efficient and robust detection in both open-world and unknown malware scenarios. MVDet outperforms state-of-the-art methods in closed-world known malware detection, open-world known malware detection, and open-world unknown malware detection.