Identifying threshold values of an open source software using Receiver Operating Characteristics curve (ROC)

Abstract

Software metrics are widely used to predict the classes in a software that are comparatively more change prone than others. Developers construct various prediction models which are used for the prediction of change prone classes. Thus, focussed attention can be laid on such classes leading to saving of lots of resources in terms of money, manpower and cost. But construction and usage of the prediction models using metrics is not always efficient and practical for developers. Thus, the alternative approach can be to identify certain alarming values of metrics above which a class is said to be at high risk. These alarming values are known as thresholds of the metrics. The main focus of this study is to identify metric threshold values using receiver operating characteristic curves. We have used threshold values to identify change prone classes (or classes that may lead to some risk in future) so that developers can carefully examine and pay attention on those classes. The study has used five releases of an open source, widely used operating system, Android. To predict change prone classes in any release of software, the threshold values of its immediate predecessor release are used. For example, to predict change prone classes in Android 4.0, the threshold values of Android 2.3 are used. In addition to this, the threshold methodology based on receiver operating characteristics curve is compared with the traditional statistical methodology based on logistic regression. We conclude that researchers may use the methodology based on receiver operating characteristics curve to calculate the thresholds.