An Efficient Deep Unsupervised Domain Adaptation for Unknown Malware Detection

Abstract

As an innovative way of communicating information, the Internet has become an indispensable part of our lives. However, it also facilitates a more widespread attack of malware. With the assistance of modern cryptanalysis, emerging malware having symmetric properties, such as encryption and decryption, pack and unpack, presents new challenges to effective malware detection. Currently, numerous malware detection approaches are based on supervised learning. The biggest challenge is that the existing systems rely on a large amount of labeled data, which is usually difficult to gain. Moreover, since the newly emerging malware has a different data distribution from the original training samples, the detection performance of these systems will degrade along with the emergence of new malware. To solve these problems, we propose an Unsupervised Domain Adaptation (UDA)-based malware detection method by jointly aligning the distribution of known and unknown malware. Firstly, the distribution divergence between the source and target domain is minimized with the help of symmetric adversarial learning to learn shared feature representations. Secondly, to further obtain semantic information of unlabeled target domain data, this paper reduces the class-level distribution divergence by aligning the class center of labeled source and pseudo-labeled target domain data. Finally, we mainly use a residual network with a self-attention mechanism to extract more accurate feature information. A series of experiments are performed on two public datasets. Experimental results illustrate that the proposed approach outperforms the existing detection methods with an accuracy of 95.63% and 95.04% in detecting unknown malware on two datasets, respectively.