Fault-proneness Prediction Research Articles

Effort-Aware Fault-Proneness Prediction Using Non-API-Based Package-Modularization Metrics

Source code complexity of legacy object-oriented (OO) software has a trickle-down effect over the key activities of software development and maintenance. Package-based OO design is widely believed to be an effective modularization. Recently, theories and methodologies have been proposed to assess the complementary aspects of legacy OO systems through package-modularization metrics. These package-modularization metrics basically address non-API-based object-oriented principles, like encapsulation, commonality-of-goal, changeability, maintainability, and analyzability. Despite their ability to characterize package organization, their application towards cost-effective fault-proneness prediction is yet to be determined. In this paper, we present theoretical illustration and empirical perspective of non-API-based package-modularization metrics towards effort-aware fault-proneness prediction. First, we employ correlation analysis to evaluate the relationship between faults and package-level metrics. Second, we use multivariate logistic regression with effort-aware performance indicators (ranking and classification) to investigate the practical application of proposed metrics. Our experimental analysis over open-source Java software systems provides statistical evidence for fault-proneness prediction and relatively better explanatory power than traditional metrics. Consequently, these results guide developers for reliable and modular package-based software design.

Risky Dynamic Typing-related Practices in Python: An Empirical Study

Python’s dynamic typing nature provides developers with powerful programming abstractions. However, many type-related bugs are accumulated in code bases of Python due to the misuse of dynamic typing. The goal of this article is to aid in the understanding of developers’ high-risk practices toward dynamic typing and the early detection of type-related bugs. We first formulate the rules of six types of risky dynamic typing-related practices (type smells for short) in Python. We then develop a rule-based tool named RUPOR, which builds an accurate type base to detect type smells. Our evaluation shows that RUPOR outperforms the existing type smell detection techniques (including the Large Language Models–based approaches, Mypy, and PYDYPE) on a benchmark of 900 Python methods. Based on RUPOR, we conduct an empirical study on 25 real-world projects. We find that type smells are significantly related to the occurrence of post-release faults. The fault-proneness prediction model built with type smell features slightly outperforms the model built without them. We also summarize the common patterns, including inserting type check to fix type smell bugs. These findings provide valuable insights for preventing and fixing type-related bugs in the programs written in dynamic-typed languages.

Error-Type—A Novel Set of Software Metrics for Software Fault Prediction

In software development, identifying software faults is an important task. The presence of faults not only reduces the quality of the software, but also increases the cost of development life cycle. Fault identification can be performed by analysing the characteristics of the buggy source codes from the past and predict the present ones based on the same characteristics using statistical or machine learning models. Many studies have been conducted to predict the fault proneness of software systems. However, most of them provide either inadequate or insufficient information and thus make the fault prediction task difficult. In this paper, we present a novel set of software metrics called Error-type software metrics, which provides prediction models with information about patterns of different types of Java runtime error. Particular, in this study, the ESM values consist of information of three common Java runtime errors which are Index Out Of Bounds Exception, Null Pointer Exception, and Class Cast Exception. Also, we proposed a methodology for modelling, extracting, and evaluating error patterns from software modules using Stream X-Machine (a formal modelling method) and machine learning techniques. The experimental results showed that the proposed Error-type software metrics could significantly improve the performances of machine learning models in fault-proneness prediction.

Impact of Parameter Tuning for Optimizing Deep Neural Network Models for Predicting Software Faults

Deep neural network models built by the appropriate design decisions are crucial to obtain the desired classifier performance. This is especially desired when predicting fault proneness of software modules. When correctly identified, this could help in reducing the testing cost by directing the efforts more towards the modules identified to be fault prone. To be able to build an efficient deep neural network model, it is important that the parameters such as number of hidden layers, number of nodes in each layer, and training details such as learning rate and regularization methods be investigated in detail. The objective of this paper is to show the importance of hyperparameter tuning in developing efficient deep neural network models for predicting fault proneness of software modules and to compare the results with other machine learning algorithms. It is shown that the proposed model outperforms the other algorithms in most cases.

Retraction: Analogous Approach towards Performance Analysis for Software Defect Prediction and Prioritization (IOP Conf. Ser.: Mater. Sci. Eng. 981 022078)

This article has been retracted by IOP Publishing following an allegation that this article contains tortured phrases [1], masking overlap of other work [2, 3] without citation.IOP Publishing has investigated in line with the COPE guidelines, and agree this article should be retracted.IOP Publishing wishes to credit PubPeer commenters and the Problematic Paper Screener for bringing the issue to our attention.The authors agree to this retraction.1. Cabanac G, Labbe C, Magazinov A, 2021, arXiv:2107.06751v12. Li, L, Lessmann S, Baesens B, 2019, arXiv:1901.017263. L. Guo, Y. Ma, B. Cukic and Harshinder Singh, (2005), ‘Robust prediction of fault-proneness by random forests', International Symposium on Software Reliability Engineering (ISSRE), IEEE, 10.1109/ISSRE.2004.35Retraction published: 23 February 2022

Categorisation‐based approach for predicting the fault‐proneness of object‐oriented classes in software post‐releases

Subsequent releases of a system have common development environments and characteristics. However, prediction models based on within-project data potentially suffer from being based on fault data reported within relatively short maintenance time intervals, which potentially decreases their prediction abilities. In this study, the authors propose an approach that improves the classification performance of models based on within-project data that are applied to predict the fault-proneness of the classes in a software post-release (PR). The proposed approach involves selecting a set of immediate pre-releases and constructing a prediction model based on each pre-release. The PR classes are categorised based on whether they are newly developed or they are reused, with or without modification, from one or more of the selected pre-releases. The prediction models are applied to the PR classes reused from selected pre-releases, and the results are used to construct a fault-proneness prediction model. After applying this prediction model to all PR classes, the fault-proneness results are adjusted by considering the relationship between the prediction results of the individual pre-release models and the actual fault data. They reported an empirical study that shows that the classification performance of the categorisation-based fault-proneness prediction models is considerably better than those constructed using existing approaches.

An approach for fault prediction in SOA-based systems using machine learning techniques

PurposeSoftware fault prediction is an important concept that can be applied at an early stage of the software life cycle. Effective prediction of faults may improve the reliability and testability of software systems. As service-oriented architecture (SOA)-based systems become more and more complex, the interaction between participating services increases frequently. The component services may generate enormous reports and fault information. Although considerable research has stressed on developing fault-proneness prediction models in service-oriented systems (SOS) using machine learning (ML) techniques, there has been little work on assessing how effective the source code metrics are for fault prediction. The paper aims to discuss this issue.Design/methodology/approachIn this paper, the authors have proposed a fault prediction framework to investigate fault prediction in SOS using metrics of web services. The effectiveness of the model has been explored by applying six ML techniques, namely, Naïve Bayes, Artificial Networks (ANN), Adaptive Boosting (AdaBoost), decision tree, Random Forests and Support Vector Machine (SVM), along with five feature selection techniques to extract the essential metrics. The authors have explored accuracy, precision, recall, f-measure and receiver operating characteristic curves of the area under curve values as performance measures.FindingsThe experimental results show that the proposed system can classify the fault-proneness of web services, whether the service is faulty or non-faulty, as a binary-valued output automatically and effectively.Research limitations/implicationsOne possible threat to internal validity in the study is the unknown effects of undiscovered faults. Specifically, the authors have injected possible faults into the classes using Java C3.0 tool and only fixed faults are injected into the classes. However, considering the Java C3.0 community of development, testing and use, the authors can generalize that the undiscovered faults should be few and have less impact on the results presented in this study, and that the results may be limited to the investigated complexity metrics and the used ML techniques.Originality/valueIn the literature, only few studies have been observed to directly concentrate on metrics-based fault-proneness prediction of SOS using ML techniques. However, most of the contributions are regarding the fault prediction of the general systems rather than SOS. A majority of them have considered reliability, changeability, maintainability using a logging/history-based approach and mathematical modeling rather than fault prediction in SOS using metrics. Thus, the authors have extended the above contributions further by applying supervised ML techniques over web services metrics and measured their capability by employing fault injection methods.

Threshold-based empirical validation of object-oriented metrics on different severity levels

Software metrics has become desideratum for the fault-proneness, reusability and effort prediction. To enhance and intensify the sufficiency of object-oriented (OO) metrics, it is crucial to perceive the relationship between OO metrics and fault-proneness at distinct severity levels. This paper characterise on the investigation of the software parts with higher probability of occurrence of faults. We examined the effect of thresholds on the OO metrics and build the predictive model based on those threshold values. This paper also instanced on the empirical validation of threshold values calculated for the OO metrics for predicting faults at different severity levels and builds the statistical model using logistic regression. This paper depicts the detection of fault-proneness by extracting the relevant OO metrics and focus on those projects that falls outside the specified risk level for allocating the more resources to them. We presented the effects of threshold values at different risk levels and also validated results on the KC1 dataset using machine learning and different classifiers.

Using Tri-Relation Networks for Effective Software Fault-Proneness Prediction

Software modules and developers are two core elements during the process of software development. Previous studies have shown that analyzing relations between 1) software modules; (2) developers; and (3) modules and developers, is critical to understand how they interact with each other, which ultimately affects software quality. Specifically, relations such as developer contribution relation, module dependency relation, and developer collaboration relation have been used independently or in pairs to build networks for software fault-proneness prediction. However, none of them investigate the joint effort of these three relations. Bearing this in mind, in this paper, we propose a tri-relation network, a weighted network that integrates developer contribution, module dependency, and developer collaboration relations to study their combined impact on software quality. Four network node centrality metrics are further derived from the proposed network to predict the fault-proneness of a given software module at the file level. Moreover, we have explored a mechanism to refine current networks in order to further improve the effectiveness of software fault-proneness prediction.

Prediction of software fault-prone classes using an unsupervised hybrid SOM algorithm

In software engineering fault proneness prediction is one of the important fields for quality measurement using multiple code metrics. The metrics thresholds are very practical in measuring the code quality for fault proneness prediction. It helps to improvise the software quality in short time with very low cost. Many researchers are in the race to develop a measuring attribute for the software quality using various methodologies. Currently so many fault proneness prediction models are available. Among that most of the methods are used to identify the faults either by data history or by special supervising algorithms. In most of the real time cases the fault data bases may not be available so that the process becomes tedious. This article proposes a hybrid model for identifying the faults in the software models and also we proposed coupling model along with the algorithm so that the metrics are used to identify the faults and the coupling model couples the metrics and the faults for the developed system software.

Taxonomy of machine learning algorithms in software fault prediction using object oriented metrics

Prediction of Fault proneness of a software component is the compelling field of investigations in software testing arena. Software coupling plays a vital role in assessing the software quality through fault prediction and complexity measures. Various fault prediction models, have used the object oriented metrics for the predicting and localizing the faults. Many of these metrics have direct influence on the quality of software. More over prior knowledge of the fault proneness of a component may significantly reduce the testing effort and time. The measures of object oriented features like inheritance, polymorphism and encapsulation etc may be used to estimate fault proneness. Many researchers have investigated the usage of object oriented metrics in the software fault prediction. In this study we present taxonomy of usage these metrics in the fault prediction. We also present the analysis of machine learning techniques in fault prediction.

Predicting Fault-Proneness of Reused Object-Oriented Classes in Software Post-Releases

The performance and applicability of a fault-proneness prediction model built based on internal quality measures computed on classes of one or more software systems and applied to classes from a different software system may be heavily influenced by several factors. These factors are related to the specific characteristics of the software system and its development environment. The issues related to these factors may be alleviated by building a model on a release of a software system and then applying it to following releases, in which product and process characteristics are close to the initial release. In this paper, we investigate the ability of the fault-proneness prediction models built by using fault data available up to a release of a system to predict faulty classes among those that are potentially reusable and those that are actually reused in the subsequent releases. We adapt an approach to building and using prediction models for classes reused, with or without modification, from preceding releases of a system. Our results show that relying on up-to-date fault data for the classes of a software system may significantly improve the overall accuracy in predicting fault-proneness of the post-release classes reused from an initial release. A model based on data from a system may be applied to other systems with a lesser degree of accuracy than a model built on up-to-date data from the same system.

Software metrics thresholds calculation techniques to predict fault-proneness: An empirical comparison

Context: Nowadays, fault-proneness prediction is an important field of software engineering. It can be used by developers and testers to prioritize tests. This would allow a better allocation of resources, reducing testing time and costs, and improving the effectiveness of software testing. Non-supervised fault-proneness prediction models, especially thresholds-based models, can easily be automated and give valuable insights to developers and testers on the classification performed.Objective: In this paper, we investigated three thresholds calculation techniques that can be used for fault-proneness prediction: ROC Curves, VARL (Value of an Acceptable Risk Level) and Alves rankings. We compared the performance of these techniques with the performance of four machine learning and two clustering based models.Method: Threshold values were calculated on a total of twelve different public datasets: eleven from the PROMISE Repository and another based on the Eclipse project. Thresholds-based models were then constructed using each thresholds calculation technique investigated. For comparison, results were also computed for supervised machine learning and clustering based models. Inter-dataset experimentation between different systems and versions of a same system was performed.Results: Results show that ROC Curves is the best performing method among the three thresholds calculation methods investigated, closely followed by Alves Rankings. VARL method didn’t give valuable results for most of the datasets investigated and was easily outperformed by the two other methods. Results also show that thresholds-based models using ROC Curves outperformed machine learning and clustering based models.Conclusion: The best of the three thresholds calculation techniques for fault-proneness prediction is ROC Curves, but Alves Rankings is a good choice too. In fact, the advantage of Alves Rankings over ROC Curves technique is that it is completely unsupervised and can therefore give pertinent threshold values when fault data is not available.

Investigating the impact of fault data completeness over time on predicting class fault-proneness

ContextThe adequacy of fault-proneness prediction models in representing the relationship between the internal quality of classes and their fault-proneness relies on several factors. One of these factors is the completeness of the fault data. A fault-proneness prediction model that is built using fault data collected during testing or after a relatively short period of time after release may be inadequate and may not be reliable enough in predicting faulty classes. ObjectiveWe empirically study the relationship between the time interval since a system is released and the performance of the fault-proneness prediction models constructed based on the fault data reported within the time interval. MethodWe construct prediction models using fault data collected at several time intervals since a system has been released and study the performance of the models in representing the relationship between the internal quality of classes and their fault-proneness. In addition, we empirically explore the relationship between the performance of a prediction model and the percentage of increase in the number of classes detected faulty (PIF) over time. ResultsOur results show evidence in favor of the expectation that predictions models that use more complete fault data, to a certain extent, more adequately represent the expected relationship between the internal quality of classes and their fault-proneness and have better performance. A threshold based on the PIF value can be used as an indicator for deciding when to stop collecting fault data. When reaching this threshold, collecting additional fault data will not significantly improve the prediction ability of the constructed model. ConclusionWhen constructing fault-proneness prediction models, researchers and software engineers are advised to rely on systems that have relatively long maintenance histories. Researchers and software engineers can use the PIF value as an indicator for deciding when to stop collecting fault data.

An empirical analysis of the effectiveness of software metrics and fault prediction model for identifying faulty classes

Software fault prediction models are used to predict faulty modules at the very early stage of software development life cycle. Predicting fault proneness using source code metrics is an area that has attracted several researchers' attention. The performance of a model to assess fault proneness depends on the source code metrics which are considered as the input for the model. In this work, we have proposed a framework to validate the source code metrics and identify a suitable set of source code metrics with the aim to reduce irrelevant features and improve the performance of the fault prediction model. Initially, we applied a t-test analysis and univariate logistic regression analysis to each source code metric to evaluate their potential for predicting fault proneness. Next, we performed a correlation analysis and multivariate linear regression stepwise forward selection to find the right set of source code metrics for fault prediction. The obtained set of source code metrics are considered as the input to develop a fault prediction model using a neural network with five different training algorithms and three different ensemble methods. The effectiveness of the developed fault prediction models are evaluated using a proposed cost evaluation framework. We performed experiments on fifty six Open Source Java projects. The experimental results reveal that the model developed by considering the selected set of source code metrics using the suggested source code metrics validation framework as the input achieves better results compared to all other metrics. The experimental results also demonstrate that the fault prediction model is best suitable for projects with faulty classes less than the threshold value depending on fault identification efficiency (low – 48.89%, median- 39.26%, and high – 27.86%).

Software fault proneness prediction: a comparative study between bagging, boosting, and stacking ensemble and base learner methods

Modules with defects might be the prime reason for decreasing the software quality and increasing the cost of maintenance. Therefore, the prediction of faulty modules of systems under test at early stages contributes to the overall quality of software products. In this research three symmetric ensemble methods: bagging, boosting and stacking are used to predict faulty modules based on evaluating the performance of 11 base learners. The results reveal that the defect prediction performance of the base learner classifier and ensemble learner classifiers is the same for naive Bayes, Bayes net, PART, random forest, IB1, VFI, decision table, and NB tree base learners, the case was different for boosted SMO, bagged J48 and boosted and bagged random tree. In addition the results showed that the random forest classifier is one of the most significant classifiers that should be stacked with other classifiers to gain the better fault prediction.

Empirical analysis of network measures for predicting high severity software faults

Network measures are useful for predicting fault-prone modules. However, existing work has not distinguished faults according to their severity. In practice, high severity faults cause serious problems and require further attention. In this study, we explored the utility of network measures in high severity faultproneness prediction. We constructed software source code networks for four open-source projects by extracting the dependencies between modules. We then used univariate logistic regression to investigate the associations between each network measure and fault-proneness at a high severity level. We built multivariate prediction models to examine their explanatory ability for fault-proneness, as well as evaluated their predictive effectiveness compared to code metrics under forward-release and cross-project predictions. The results revealed the following: (1) most network measures are significantly related to high severity fault-proneness; (2) network measures generally have comparable explanatory abilities and predictive powers to those of code metrics; and (3) network measures are very unstable for cross-project predictions. These results indicate that network measures are of practical value in high severity fault-proneness prediction.

Understanding the value of considering client usage context in package cohesion for fault-proneness prediction

By far, many package cohesion metrics have been proposed from internal structure view and external usage view. Based on whether client usage context (i.e., the way packages are used by their clients) is exploited, we group these metrics into two categories: non-context-based and context-based. Currently, there is no comprehensive empirical research devoted to understanding the actual value of client usage context for fault-proneness prediction. In this study, we conduct a thorough empirical study to investigate the value of considering client usage context in package cohesion for fault-proneness prediction. First, we use principal component analysis to examine the relationships between context-based and non-context-based cohesion metrics. Second, we employ univariate logistic regression analysis to investigate the correlation between context-based cohesion metrics and fault-proneness. Then, we build multivariate prediction models to analyze the ability of context-based cohesion metrics for fault-proneness prediction when used alone or used together with non-context-based cohesion metrics. To obtain comprehensive evaluations, we evaluate the effectiveness of these multivariate models in the ranking and classification scenarios from both cross-validation and across-version perspectives. The experimental results show that (1) context-based cohesion metrics are complementary to non-context-based cohesion metrics; (2) most of context-based cohesion metrics have a significantly negative association with fault-proneness; (3) when used alone or used together with non-context-based cohesion metrics, context-based cohesion metrics can substantially improve the effectiveness of fault-proneness prediction in most studied systems under both cross-validation and across-version evaluation. Client usage context has an important value in package cohesion for fault-proneness prediction.

State-of-the-Art in Empirical Validation of Software Metrics for Fault Proneness Prediction: Systematic Review

With the sharp rise in software dependability and failure cost, high quality has been in great demand. However, guaranteeing high quality in software systems which have grown in size and complexity coupled with the constraints imposed on their development has become increasingly difficult, time and resource consuming activity. Consequently, it becomes inevitable to deliver software that have no serious faults. In this case, object-oriented (OO) products being the de facto standard of software development with their unique features could have some faults that are hard to find or pinpoint the impacts of changes. The earlier faults are identified, found and fixed, the lesser the costs and the higher the quality. To assess product quality, software metrics are used. Many OO metrics have been proposed and developed. Furthermore, many empirical studies have validated metrics and class fault proneness (FP) relationship. The challenge is which metrics are related to class FP and what activities are performed. Therefore, this study bring together the state-of-the-art in fault prediction of FP that utilizes CK and size metrics. We conducted a systematic literature review over relevant published empirical validation articles. The results obtained are analysed and presented. It indicates that 29 relevant empirical studies exist and measures such as complexity, coupling and size were found to be strongly related to FP.

Empirical analysis of network measures for effort-aware fault-proneness prediction

ContextRecently, network measures have been proposed to predict fault-prone modules. Leveraging the dependency relationships between software entities, network measures describe the structural features of software systems. However, there is no consensus about their effectiveness for fault-proneness prediction. Specifically, the predictive ability of network measures in effort-aware context has not been addressed. ObjectiveWe aim to provide a comprehensive evaluation on the predictive effectiveness of network measures with the effort needed to inspect the code taken into consideration. MethodWe first constructed software source code networks of 11 open-source projects by extracting the data and call dependencies between modules. We then employed univariate logistic regression to investigate how each single network measure was correlated with fault-proneness. Finally, we built multivariate prediction models to examine the usefulness of network measures under three prediction settings: cross-validation, across-release, and inter-project predictions. In particular, we used the effort-aware performance indicators to compare their predictive ability against the commonly used code metrics in both ranking and classification scenarios. ResultsBased on the 11 open-source software systems, our results show that: (1) most network measures are significantly positively related to fault-proneness; (2) the performance of network measures varies under different prediction settings; (3) network measures have inconsistent effects on various projects. ConclusionNetwork measures are of practical value in the context of effort-aware fault-proneness prediction, but researchers and practitioners should be careful of choosing whether and when to use network measures in practice.