Malicious Software Family Classification using Machine Learning Multi-class Classifiers

Abstract

Due to the rapid growth of targeted malware attacks, malware analysis and family classification are important for all types of users such as personal, enterprise, and government. Traditional signature-based malware detection and anti-virus systems fail to classify the new variants of unknown malware into their corresponding families. Therefore, we propose malware family classification system for 11 malicious families by extracting their prominent API features from the reports of enhanced and scalable version of cuckoo sandbox. Moreover, the proposed system contributes feature extraction algorithm, feature reduction and representation procedure for identifying and representing the extracted feature attributes. To classify the different types of malicious software Random Forest (RF), K-Nearest Neighbor (KNN), and Decision Table (DT) machine learning multi-class classifiers have been used in this system and RF and KNN classifiers provide 95.8% high accuracy in malware family classification.