Abstract
Deep learning methods are being increasingly widely used in static malware detection field because they can summarize the feature of malware and its variants that have never appeared before. But similar to the picture recognition model, the static malware detection model based on deep learning is also vulnerable to the interference of adversarial samples. When the input feature vectors of the malware detection model is based on static features of Windows PE (Portable Executable, PE) file, the model is vulnerable to gradient-based attacks. Regarding the issue above, a method of adversarial sample generation is proposed, which can summarize the blind spots of the original detection model. However, the existing malware adversarial sample generation method is not universal and low in generation efficiency due to the need for human control and difficulty in maintaining a normal file format. In response to these problems, this paper proposes a novel method of automatic adversarial samples generation based on deep reinforcement learning. Firstly, a static PE malware detection model based on deep learning called DeepDetectNet is constructed, the original AUC of which can reach 0.989. Then, an adversarial sample generation model based on reinforcement learning called RLAttackNet is implemented, which generates malware samples that can bypass DeepDetectNet. Finally, when we re-input the adversarial samples into the previously trained DeepDetectNet, the original defects of DeepDetectNet can be reinforced. Experimental results show that the RLAttackNet proposed in this paper can generate about 19.13% of malware samples bypass DeepDetectNet. When DeepDetectNet is retrained with these adversarial samples, the AUC value improves from 0.989 to 0.996 and attack success rate has a significant drop, from 19.13% to 3.1%, compared with the original model.
Highlights
With human’s increasing dependence on computer system, the detection of malware has become a crucial problem in cyberspace security
We will use ROC, in which x-axis is FPR (defined by Eq (18)) and y-axis is TPR (defined by Eq (19)), and AUC, which denotes the area under ROC curve, to verify the effectiveness of the original DeepDetectNet, the result of which will be used as the baseline for later experiments
We construct an adversarial sample generation model based on deep reinforcement learning, called RLAttackNet, to improve static PE malware detection
Summary
With human’s increasing dependence on computer system, the detection of malware has become a crucial problem in cyberspace security. For static PE malware detection based on deep learning, there are two main framework of feature extraction. Taking MalConv as the target of attack, Suciu O et al [21] studied several specific methods of generating adversarial samples, the results of which reveal the defects of the second feature extraction framework represented by MalConv. The static PE malware detection model based on deep learning called DeepDetectNet is constructed in this paper, which uses the traditional feature extraction method based. An adversarial method to improve deep learning-based static malware detection model. We can discover the defects that are difficult to be found directly in DeepDetectNet. The existing attack methods against machine learning and deep learning detection model are mainly divided into black box attack and white box attack [22]. The performance of retrained detection model is extremely good
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
