A two‐stage deep learning framework for image‐based android malware detection and variant classification

Abstract

AbstractWith the popularity of the internet and smartphones, malware on smartphones has increased dramatically. In addition, the ubiquity and openness of the Android operating system have made it a lucrative platform for cybercriminals to develop malware. Traditional malware detection techniques require a lot of time and manual effort to classify malware accurately. Recently, deep learning (DL) based malware detection and classification techniques have been developed to solve this issue. This article proposes a DL‐based two‐stage framework that detects Android malware and classifies its variants using image‐based malware representations of the Android DEX files. The framework uses the EfficientNetB0 convolutional neural network (CNN) to extracts relevant features from the malware color images. The extracted features are then passed through a global average pooling layer and fed into a stacking classifier. The stacking classifier employs linear support vector machine (SVM) and random forest (RF) algorithms as base‐level classifiers and logistic regression as the meta‐level classifier. This method obtained an accuracy of 100% in the binary classification of Android malware images and a 92.9% accuracy in 5‐class (Adsware, Adware + Adware, Clicker + Trojan, Spyware, and Benign) classification, and an 88.6% accuracy in 4‐class (Adsware, Adware + Adware, Clicker + Trojan, and Spyware) classification. We compared our method with 26 state‐of‐the‐art pretrained CNN models (including the original EfficientNetB0) and large‐scale learning classifiers such as EfficientNetB0‐SVM and EfficientNetB0‐RF. The proposed framework outperformed the compared methods in all performance metrics. Experiments also demonstrate that substituting the softmax layer of CNNs with a large‐scale learning classifier or stacking classifier results in an enhanced performance over the original network.