Feature Reduction and Optimization of Malware Detection System Using Ant Colony Optimization and Rough Sets

Abstract

Malware is a malicious program that can cause a security breach of a system. Malware detection and classification is one of the burning topics of research in information security. Executable files are the major source of input for static malware detection. Machine learning techniques are very efficient in behavioral-based malware detection and need a dataset of malware with different features. In windows, malware can be detected by analyzing the portable executable (PE) files. This work contributes to identifying the minimum feature set for malware detection employing a rough set dependent feature significance combined with Ant Colony Optimization (ACO) as the heuristic-search technique. A malware dataset named claMP with both integrated features and raw features was considered as the benchmark dataset for this work. The analytical results prove that 97.15% and 92.8% data size optimization has been achieved with a minimum loss of accuracy for claMP integrated and raw datasets, respectively.