Multi-Class Malware Classification Using Deep Residual Network with Non-SoftMax Classifier

Abstract

The traditional malware detection approaches rely heavily on feature extraction procedure, in our paper we convert the raw bytecodes of malware samples to 3-channel RGB images and applied an 18-layers deep residual network to classify the malware. Instead of using the SoftMax function as the only classification layer in the deep residual model, we replaced SoftMax layer with non-SoftMax classifiers: Support Vector Machine (SVM), Random Forest (RF), and K-nearest Neighbor as the new classification layer to classify the feature tensors extracted from the residual network and compared the overall classification performance of the proposed model on the Malimg dataset. Our result shows that the 18-layer deep residual network model has comparable average testing classification accuracies of 85.73% by the SoftMax function, 86.80% by the Support Vector Machine, 87.47% by Random Forest, 86.27% by K-nearest Neighbor algorithms.