Using software metrics for predicting vulnerable classes and methods in Java projects: A machine learning approach

Abstract

Abstract[Context]A software vulnerability becomes harmful for software when an attacker successfully exploits the insecure code and reveals the vulnerability. A single vulnerability in code can put the entire software at risk. Therefore, maintaining software security throughout the software life cycle is an important and at the same time challenging task for development teams. This can also leave the door open for vulnerable code being evolved during successive releases. In recent years, researchers have used software metrics‐based vulnerability prediction approaches to detect vulnerable code early and ensure secure code releases. Software metrics have been employed to predict vulnerability specifically in C/C++ and Java‐based systems. However, the prediction performance of metrics at different granularity levels (class level or method level) has not been analyzed. In this paper, we focused on metrics that are specific to lower granularity levels (Java classes and methods). Based on statistical analysis, we first identified a set of class‐level metrics and a set of method‐level metrics and then employed them as features in machine learning techniques to predict vulnerable classes and methods, respectively. This paper describes a comparative study on how our selected metrics perform at different granularity levels. Such a comparative study can help the developers in choosing the appropriate metrics (at the desired level of granularity). [Objective] The goal of this research is to propose a set of metrics at two lower granularity levels and provide evidence for their usefulness during vulnerability prediction (which will help in maintaining secure code and ensure secure software evolution). [Method] For four Java‐based open source systems (including two releases of Apache Tomcat), we designed and conducted experiments based on statistical tests to propose a set of software metrics that can be used for predicting vulnerable code components (i.e., vulnerable classes and methods). Next, we used our identified metrics as features to train supervised machine learning algorithms to classify Java code as vulnerable or non‐vulnerable. [Result] Our study has successfully identified a set of class‐level metrics and a second set of method‐level metrics that can be useful from a vulnerability prediction standpoint. We achieved recall higher than 70% and precision higher than 75% in vulnerability prediction using our identified class‐level metrics as features of machine learning. Furthermore, method‐level metrics showed recall higher than 65% and precision higher than 80%.