Adversarial attacks against image-based malware detection using autoencoders

Abstract

Over the past decade, deep learning approaches have been applied to the detection of malicious software, otherwise known as malware. Despite their improved performance compared to conventional detection methods such as static and dynamic analysis, however, deep learning-based malware detection systems have been shown to be vulnerable to adversarial attacks. Few image-based malware detection systems have been proposed, especially those that evaluate their performance against adversarial attacks. Furthermore, little research has been done beyond the classification of malware targeted at Windows (PE) or Android systems, leaving entire realms such as Mac (Mach-O), Linux (ELF), and embedded software unexplored and unprotected. These realms, specifically embedded software, are used in critical technology such as avionic systems and special care must be taken to ensure their safety. In this paper, we present an image-based malware detection system on PE, ELF, Mach- O, and embedded C code files. The system’s architecture incorporates layers of encoders that are taken from independently-trained autoencoders and multi-layer perceptron that returns the output of the network. We evaluate the performance of the system against adversarial attacks, or the misclassification of a malware file as a benign, by adding gradient based perturbations to unused sections of the malware often referred to as the slack bits. The network achieves an accuracy of 96.51% on non-adversarial PE and ELF files, 95.45% on transfer learned non-adversarial Mach-O files, and 99.2% on transfer learned non-adversarial synthetic plane files. For the classification of adversarial examples, the network achieved a 81% success rate of misclassification on adversarial PE and ELF files and a 99% success rate of misclassification on adversarial synthetic plane files.