Fault-proneness Prediction Models Research Articles

Categorisation‐based approach for predicting the fault‐proneness of object‐oriented classes in software post‐releases

Subsequent releases of a system have common development environments and characteristics. However, prediction models based on within-project data potentially suffer from being based on fault data reported within relatively short maintenance time intervals, which potentially decreases their prediction abilities. In this study, the authors propose an approach that improves the classification performance of models based on within-project data that are applied to predict the fault-proneness of the classes in a software post-release (PR). The proposed approach involves selecting a set of immediate pre-releases and constructing a prediction model based on each pre-release. The PR classes are categorised based on whether they are newly developed or they are reused, with or without modification, from one or more of the selected pre-releases. The prediction models are applied to the PR classes reused from selected pre-releases, and the results are used to construct a fault-proneness prediction model. After applying this prediction model to all PR classes, the fault-proneness results are adjusted by considering the relationship between the prediction results of the individual pre-release models and the actual fault data. They reported an empirical study that shows that the classification performance of the categorisation-based fault-proneness prediction models is considerably better than those constructed using existing approaches.

An approach for fault prediction in SOA-based systems using machine learning techniques

PurposeSoftware fault prediction is an important concept that can be applied at an early stage of the software life cycle. Effective prediction of faults may improve the reliability and testability of software systems. As service-oriented architecture (SOA)-based systems become more and more complex, the interaction between participating services increases frequently. The component services may generate enormous reports and fault information. Although considerable research has stressed on developing fault-proneness prediction models in service-oriented systems (SOS) using machine learning (ML) techniques, there has been little work on assessing how effective the source code metrics are for fault prediction. The paper aims to discuss this issue.Design/methodology/approachIn this paper, the authors have proposed a fault prediction framework to investigate fault prediction in SOS using metrics of web services. The effectiveness of the model has been explored by applying six ML techniques, namely, Naïve Bayes, Artificial Networks (ANN), Adaptive Boosting (AdaBoost), decision tree, Random Forests and Support Vector Machine (SVM), along with five feature selection techniques to extract the essential metrics. The authors have explored accuracy, precision, recall, f-measure and receiver operating characteristic curves of the area under curve values as performance measures.FindingsThe experimental results show that the proposed system can classify the fault-proneness of web services, whether the service is faulty or non-faulty, as a binary-valued output automatically and effectively.Research limitations/implicationsOne possible threat to internal validity in the study is the unknown effects of undiscovered faults. Specifically, the authors have injected possible faults into the classes using Java C3.0 tool and only fixed faults are injected into the classes. However, considering the Java C3.0 community of development, testing and use, the authors can generalize that the undiscovered faults should be few and have less impact on the results presented in this study, and that the results may be limited to the investigated complexity metrics and the used ML techniques.Originality/valueIn the literature, only few studies have been observed to directly concentrate on metrics-based fault-proneness prediction of SOS using ML techniques. However, most of the contributions are regarding the fault prediction of the general systems rather than SOS. A majority of them have considered reliability, changeability, maintainability using a logging/history-based approach and mathematical modeling rather than fault prediction in SOS using metrics. Thus, the authors have extended the above contributions further by applying supervised ML techniques over web services metrics and measured their capability by employing fault injection methods.

Prediction of software fault-prone classes using an unsupervised hybrid SOM algorithm

In software engineering fault proneness prediction is one of the important fields for quality measurement using multiple code metrics. The metrics thresholds are very practical in measuring the code quality for fault proneness prediction. It helps to improvise the software quality in short time with very low cost. Many researchers are in the race to develop a measuring attribute for the software quality using various methodologies. Currently so many fault proneness prediction models are available. Among that most of the methods are used to identify the faults either by data history or by special supervising algorithms. In most of the real time cases the fault data bases may not be available so that the process becomes tedious. This article proposes a hybrid model for identifying the faults in the software models and also we proposed coupling model along with the algorithm so that the metrics are used to identify the faults and the coupling model couples the metrics and the faults for the developed system software.

Predicting Fault-Proneness of Reused Object-Oriented Classes in Software Post-Releases

The performance and applicability of a fault-proneness prediction model built based on internal quality measures computed on classes of one or more software systems and applied to classes from a different software system may be heavily influenced by several factors. These factors are related to the specific characteristics of the software system and its development environment. The issues related to these factors may be alleviated by building a model on a release of a software system and then applying it to following releases, in which product and process characteristics are close to the initial release. In this paper, we investigate the ability of the fault-proneness prediction models built by using fault data available up to a release of a system to predict faulty classes among those that are potentially reusable and those that are actually reused in the subsequent releases. We adapt an approach to building and using prediction models for classes reused, with or without modification, from preceding releases of a system. Our results show that relying on up-to-date fault data for the classes of a software system may significantly improve the overall accuracy in predicting fault-proneness of the post-release classes reused from an initial release. A model based on data from a system may be applied to other systems with a lesser degree of accuracy than a model built on up-to-date data from the same system.

Software metrics thresholds calculation techniques to predict fault-proneness: An empirical comparison

Context: Nowadays, fault-proneness prediction is an important field of software engineering. It can be used by developers and testers to prioritize tests. This would allow a better allocation of resources, reducing testing time and costs, and improving the effectiveness of software testing. Non-supervised fault-proneness prediction models, especially thresholds-based models, can easily be automated and give valuable insights to developers and testers on the classification performed.Objective: In this paper, we investigated three thresholds calculation techniques that can be used for fault-proneness prediction: ROC Curves, VARL (Value of an Acceptable Risk Level) and Alves rankings. We compared the performance of these techniques with the performance of four machine learning and two clustering based models.Method: Threshold values were calculated on a total of twelve different public datasets: eleven from the PROMISE Repository and another based on the Eclipse project. Thresholds-based models were then constructed using each thresholds calculation technique investigated. For comparison, results were also computed for supervised machine learning and clustering based models. Inter-dataset experimentation between different systems and versions of a same system was performed.Results: Results show that ROC Curves is the best performing method among the three thresholds calculation methods investigated, closely followed by Alves Rankings. VARL method didn’t give valuable results for most of the datasets investigated and was easily outperformed by the two other methods. Results also show that thresholds-based models using ROC Curves outperformed machine learning and clustering based models.Conclusion: The best of the three thresholds calculation techniques for fault-proneness prediction is ROC Curves, but Alves Rankings is a good choice too. In fact, the advantage of Alves Rankings over ROC Curves technique is that it is completely unsupervised and can therefore give pertinent threshold values when fault data is not available.

Investigating the impact of fault data completeness over time on predicting class fault-proneness

ContextThe adequacy of fault-proneness prediction models in representing the relationship between the internal quality of classes and their fault-proneness relies on several factors. One of these factors is the completeness of the fault data. A fault-proneness prediction model that is built using fault data collected during testing or after a relatively short period of time after release may be inadequate and may not be reliable enough in predicting faulty classes. ObjectiveWe empirically study the relationship between the time interval since a system is released and the performance of the fault-proneness prediction models constructed based on the fault data reported within the time interval. MethodWe construct prediction models using fault data collected at several time intervals since a system has been released and study the performance of the models in representing the relationship between the internal quality of classes and their fault-proneness. In addition, we empirically explore the relationship between the performance of a prediction model and the percentage of increase in the number of classes detected faulty (PIF) over time. ResultsOur results show evidence in favor of the expectation that predictions models that use more complete fault data, to a certain extent, more adequately represent the expected relationship between the internal quality of classes and their fault-proneness and have better performance. A threshold based on the PIF value can be used as an indicator for deciding when to stop collecting fault data. When reaching this threshold, collecting additional fault data will not significantly improve the prediction ability of the constructed model. ConclusionWhen constructing fault-proneness prediction models, researchers and software engineers are advised to rely on systems that have relatively long maintenance histories. Researchers and software engineers can use the PIF value as an indicator for deciding when to stop collecting fault data.

An in-depth study of the potentially confounding effect of class size in fault prediction

Background. The extent of the potentially confounding effect of class size in the fault prediction context is not clear, nor is the method to remove the potentially confounding effect, or the influence of this removal on the performance of fault-proneness prediction models.Objective. We aim to provide an in-depth understanding of the effect of class size on the true associations between object-oriented metrics and fault-proneness.Method. We first employ statistical methods to examine the extent of the potentially confounding effect of class size in the fault prediction context. After that, we propose a linear regression-based method to remove the potentially confounding effect. Finally, we empirically investigate whether this removal could improve the prediction performance of fault-proneness prediction models.Results. Based on open-source software systems, we found: (a) the confounding effect of class size on the associations between object-oriented metrics and fault-proneness in general exists; (b) the proposed linear regression-based method can effectively remove the confounding effect; and (c) after removing the confounding effect, the prediction performance of fault prediction models with respect to both ranking and classification can in general be significantly improved.Conclusion. We should remove the confounding effect of class size when building fault prediction models.

A systematic and comprehensive investigation of methods to build and evaluate fault prediction models

This paper describes a study performed in an industrial setting that attempts to build predictive models to identify parts of a Java system with a high fault probability. The system under consideration is constantly evolving as several releases a year are shipped to customers. Developers usually have limited resources for their testing and would like to devote extra resources to faulty system parts. The main research focus of this paper is to systematically assess three aspects on how to build and evaluate fault-proneness models in the context of this large Java legacy system development project: (1) compare many data mining and machine learning techniques to build fault-proneness models, (2) assess the impact of using different metric sets such as source code structural measures and change/fault history (process measures), and (3) compare several alternative ways of assessing the performance of the models, in terms of (i) confusion matrix criteria such as accuracy and precision/recall, (ii) ranking ability, using the receiver operating characteristic area (ROC), and (iii) our proposed cost-effectiveness measure (CE). The results of the study indicate that the choice of fault-proneness modeling technique has limited impact on the resulting classification accuracy or cost-effectiveness. There is however large differences between the individual metric sets in terms of cost-effectiveness, and although the process measures are among the most expensive ones to collect, including them as candidate measures significantly improves the prediction models compared with models that only include structural measures and/or their deltas between releases – both in terms of ROC area and in terms of CE. Further, we observe that what is considered the best model is highly dependent on the criteria that are used to evaluate and compare the models. And the regular confusion matrix criteria, although popular, are not clearly related to the problem at hand, namely the cost-effectiveness of using fault-proneness prediction models to focus verification efforts to deliver software with less faults at less cost.