A multi-level deep learning system for malware detection

Abstract

To defend against an increasing number of sophisticated malware attacks, deep-learning based Malware Detection Systems (MDSs) have become a vital component of our economic and national security. Traditionally, researchers build the single deep learning model using the entire dataset. However, the single deep learning model may not handle the increasingly complex malware data distributions effectively since different sample subspaces representing a group of similar malware may have unique data distribution. In order to further improve the performance of deep learning based MDSs, we propose a Multi-Level Deep Learning System (MLDLS) that organizes multiple deep learning models using the tree structure. Each model in the tree structure of MLDLS was not built on the whole dataset. Instead, each deep learning model focuses on learning a specific data distribution for a particular group of malware and all deep learning models in the tree work together to make a final decision. Consequently, the learning effectiveness of each deep learning model built for one cluster can be improved. Experimental results show that our proposed system performs better than the traditional approach.