A machine learning based approach to detect malicious android apps using discriminant system calls

Abstract

The openness of Android framework and the enhancement of users trust have gained the attention of malware writers. The momentum of downloaded applications (app for short) from numerous app stores has stimulated the proliferation of mobile malware. Now the threat is due to the sophistication in malware being written to bypass signature-based detectors. In this paper, we investigate system calls to tackle mobile malware on Android operating system. To do so, we first employed machine learning to extract system calls. We then performed the empirical estimation of system calls derived from diverse datasets employing human interaction and random inputs. After accomplishing intensive experiments on synthesized system calls with two feature selection approach, namely Absolute Difference of Weighted System Calls (ADWSC) and Ranked System Calls using Large Population Test (RSLPT), we validated the results on five datasets. All classifiers generated in Area Under Curve of 1.0 with an accuracy exceeding 99.9% suggest the appropriateness and efficacy of the proposed approach. Finally, we evaluated the effectiveness of classifier against adversarial attacks and found that the classifiers are vulnerable to data poisoning and label flipping attacks. Adversarial examples created by poisoning malware samples resulted in the significant drop of classifier performance on perturbing 12–18 prominent attributes. Moreover, we implemented class label poisoning attacks which brought down the classification accuracy by 50% on altering labels of 50 malicious training instances.