Enhancing android application security: A novel approach using DroidXGB for malware detection based on permission analysis

Abstract

AbstractThe prevalence of malicious Android applications targeting the platform has introduced significant challenges in the realm of security testing. Traditional solutions have proven insufficient in handling the growing number of malicious apps, resulting in persistent exposure of Android smartphones to evolving forms of malware. This study investigates the potential of extreme gradient boosting (XGB) in identifying complex and high‐dimensional malicious permissions. By leveraging attribute combination and selection techniques, XGBoost demonstrates promising capabilities in this area. However, enhancing the XGBoost model presents a formidable challenge. To overcome this, This research employs adaptive grey wolf optimization (AGWO) for hyper‐parameter tuning. AGWO utilizes continuous values to represent the position and movement of the grey wolf, enabling XGBoost to search for optimal hyper‐parameter values in a continuous space. The proposed approach, DroidXGB, utilizes XGBoost and AGWO to analyze permissions and identify malware Android applications. It aims to address security vulnerabilities and compares its performance with baseline algorithms and state‐of‐the‐art methods using four benchmark datasets. The results showcase DroidXGB's impressive accuracy of 98.39%, outperforming other existing methods and significantly enhancing Android malware detection and security testing capabilities.