Software Quality Prediction Models Research Articles

A systematic review of hyperparameter tuning techniques for software quality prediction models

BACKGROUND: Software quality prediction models play a crucial role in identifying vulnerable software components during early stages of development, and thereby optimizing the resource allocation and enhancing the overall software quality. While various classification algorithms have been employed for developing these prediction models, most studies have relied on default hyperparameter settings, leading to significant variability in model performance. Tuning the hyperparameters of classification algorithms can enhance the predictive capability of quality models by identifying optimal settings for improved accuracy and effectiveness. METHOD: This systematic review examines studies that have utilized hyperparameter tuning techniques to develop prediction models in software quality domain. The review focused on diverse areas such as defect prediction, maintenance estimation, change impact prediction, reliability prediction, and effort estimation, as these domains demonstrate the wide applicability of common learning algorithms. RESULTS: This review identified 31 primary studies on hyperparameter tuning for software quality prediction models. The results demonstrate that tuning the parameters of classification algorithms enhances the performance of prediction models. Additionally, the study found that certain classification algorithms exhibit high sensitivity to their parameter settings, achieving optimal performance when tuned appropriately. Conversely, certain classification algorithms exhibit low sensitivity to their parameter settings, making tuning unnecessary in such instances. CONCLUSION: Based on the findings of this review, the study conclude that the predictive capability of software quality prediction models can be significantly improved by tuning their hyperparameters. To facilitate effective hyperparameter tuning, we provide practical guidelines derived from the insights obtained through this study.

A Software Quality Prediction Model for Aspect Oriented System using Neuro-Fuzzy Approach

Nowadays, the usage of software has increased exponentially in various fields like education systems, industries, health systems, and many others. Various software architectures are already available in the market e.g. modular oriented, component-based, object-oriented, aspect-oriented, etc. Aspect-Oriented (AO) system software has gained much attention due to its superior features to the aforementioned systems. However, AO systems face the challenges of being complex and hard testing environment, a quality assessment of these systems is necessary. In this paper, a software quality estimation model for aspect oriented system using neuro-fuzzy approach has been developed. For which, a detailed study on aspect oriented systems has been accomplished in terms of various attributes affecting the quality of these software. In this paper, a framework of software quality prediction model has been designed using the adaptive neuro-fuzzy inference engine (ANFIS) approach. Data of 200 software pieces have been collected in this study where 150 software data is used to train the ANFIS model whereas 50 software data is used for testing purposes. The quality estimated by the proposed ANFIS model is compared with the actual quality of these software data and quantitative analysis is performed in terms of error measures. Finally, it was found that the proposed ANFIS model worked better in terms of MSE, MRE, MARE, MBRE, and MIBRE error measures.

Comparison of Machine Learning Techniques for Software Quality Prediction

Software quality prediction is one the most challenging tasks in the development and maintenance of software. Machine learning (ML) is widely being incorporated for the prediction of the quality of a final product in the early development stages of the software development life cycle (SDLC). An ML prediction model uses software metrics and faulty data from previous projects to detect high-risk modules for future projects, so that the testing efforts can be targeted to those specific ‘risky' modules. Hence, ML-based predictors contribute to the detection of development anomalies early and inexpensively and ensure the timely delivery of a successful, failure-free and supreme quality software product within budget. This article has a comparison of 30 software quality prediction models (5 technique * 6 dataset) built on five ML techniques: artificial neural network (ANN); support vector machine (SVMs); Decision Tree (DTs); k-Nearest Neighbor (KNN); and Naïve Bayes Classifiers (NBC), using six datasets: CM1, KC1, KC2, PC1, JM1, and a combined one. These models exploit the predictive power of static code metrics, McCabe complexity metrics, for quality prediction. All thirty predictors are compared using a receiver operator curve (ROC), area under the curve (AUC), and accuracy as performance evaluation criteria. The results show that the ANN technique for software quality prediction is promising for accurate quality prediction irrespective of the dataset used.

Data Mining Techniques for Software Quality Prediction in Open Source Software

Software quality monitoring and analysis are among the most productive topics in software engineering research. Their results may be effectively employed by engineers during software development life cycle. Open source software constitutes a valid test case for the assessment of software characteristics. The data mining approach has been proposed in literature to extract software characteristics from software engineering data. This paper aims at comparing diverse data mining techniques (e.g., derived from machine learning) for developing effective software quality prediction models. To achieve this goal, we tackled various issues, such as the collection of software metrics from open source repositories, the assessment of prediction models to detect software issues and the adoption of statistical methods to evaluate data mining techniques. The results of this study aspire to identify the data mining techniques that perform better amongst all the ones used in this paper for software quality prediction models.

An extensive analysis of search-based techniques for predicting defective classes

In spite of constant planning, effective documentation and proper implementation of a software during its life cycle, many defects still occur. Various empirical studies have found that prediction models developed using software metrics can be used to predict these defects. Researchers have advocated the use of search-based techniques and their hybridized versions in literature for developing software quality prediction models. This study conducts an extensive comparison of 20 search-based techniques, 16 hybridized techniques and 17 machine-learning techniques amongst each other, to develop software defect prediction models using 17 data sets. The comparison framework used in the study is efficient as it (i) deals with the stochastic nature of the techniques (ii) provides a fair comparison amongst the techniques (iii) promotes repeatability of the study and (iv) statistically validates the results. The results of the study indicate promising ability of search-based techniques and their hybridized versions for predicting defective classes.

Software Quality Prediction Model with the Aid of Advanced Neural Network with HCS

Software quality is regarded as the highly important factors for assessing the global competitive position of any software product. To assure quality, and to assess the reliability of software products, many software quality prediction models have been proposed in the past decades. In this proposed method we have utilized a hybrid method for quality prediction. The prediction is done with the help of the Advanced Neural network which is incorporated with Hybrid Cuckoo search (HCS) optimization algorithm for better prediction accuracy. The application software is first subjected to test case generation and once the test cases are generated they are applied to advanced neural network for the prediction of quality. The neural network is improved by utilizing HCS which optimizes the weight factor for improving the prediction. The quality metrics like maintainability and reliability are estimated for predicting the software quality and the results are compared with other existing techniques to verify the effectiveness of our proposed method.

A Novel Feature Subset Selection Algorithm for Software Defect Prediction

Software Defect Prediction has been an area of growing importance. It is always required to maintain high reliability and high quality for any software being developed. A software quality prediction model is built using software metrics and defect data collected from a previously developed system release or similar software project. Upon validation of such a model, it could be used for predicting the fault-proneness of program modules that are currently under development. A low quality or fault prone prediction can justify the application of available quality improvement resources to those programs. In contrast, a non fault prone prediction can justify non- application of the limited resources to these already high- quality programs. And finally high software reliability and quality are maintained with an effective use of the available resources. A feature extraction algorithm based on graph clustering is applied over the historical software data collected for defect prediction purpose and its impact on different data sets are analyzed in this paper.

THE USE OF UNDER- AND OVERSAMPLING WITHIN ENSEMBLE FEATURE SELECTION AND CLASSIFICATION FOR SOFTWARE QUALITY PREDICTION

Software quality prediction models are useful tools for creating high quality software products. The general process is that practitioners use software metrics and defect data along with various data mining techniques to build classification models for identifying potentially faulty program modules, thereby enabling effective project resource allocation. The predictive accuracy of these classification models is often affected by the quality of input data. Two main problems which can affect the quality of input data are high dimensionality (too many independent attributes in a dataset) and class imbalance (many more members of one class than the other class in a binary classification problem). To resolve both of these problems, we present an iterative feature selection approach which repeatedly applies data sampling (to overcome class imbalance) followed by feature selection (to overcome high dimensionality), and finally combines the ranked feature lists from the separate iterations of sampling. After feature selection, models are built either using a plain learner or by using a boosting algorithm which incorporates sampling. In order to assess the impact of various balancing, filter, and learning techniques in the feature selection and model-building process on software quality prediction, we employ two sampling techniques, random undersampling (RUS) and synthetic minority oversampling technique (SMOTE), and two ensemble boosting approaches, RUSBoost and SMOTEBoost (in which RUS and SMOTE, respectively, are integrated into a boosting technique), as well as six feature ranking techniques. We apply the proposed techniques to several groups of datasets from two real-world software systems and use two learners to build classification models. The experimental results demonstrate that RUS results in better prediction than SMOTE, and also that boosting is more effective in improving classification performance than not using boosting. In addition, some feature ranking techniques, like chi-squared and information gain, exhibit better and more stable classification behavior than other rankers.

A Model for Software Product Quality Prediction

When the expression “Software Quality” is used, we usually think in terms of an excellent software product that fulfills our expectations. These expectations are based on the intended use. Number of models has been proposed for evaluation of software quality based on various characteristics. In this paper quality of software product is defined in terms of basic components as constituent part of any program or software and proposed a software quality prediction model based on basic components. It has been justified with example that if any software quality model uses the tacit knowledge that will be better than any other model in terms of quality.

The use of decision trees for cost‐sensitive classification: an empirical study in software quality prediction

AbstractThis empirical study investigates two commonly used decision tree classification algorithms in the context of cost‐sensitive learning. A review of the literature shows that the cost‐based performance of a software quality prediction model is usually determined after the model‐training process has been completed. In contrast, we incorporate cost‐sensitive learning during the model‐training process. The C4.5 and Random Forest decision tree algorithms are used to build defect predictors either with, or without, any cost‐sensitive learning technique. The paper investigates six different cost‐sensitive learning techniques: AdaCost, Adc2, Csb2, MetaCost, Weighting, and Random Undersampling (RUS). The data come from case study include 15 software measurement datasets obtained from several high‐assurance systems. In addition, to a unique insight into the cost‐based performance of defection prediction models, this study is one of the first to use misclassification cost as a parameter during the model‐training process. The practical appeal of this research is that it provides a software quality practitioner with a clear process for how to consider (during model training) and analyze (during model evaluation) the cost‐based performance of a defect prediction model. RUS is ranked as the best cost‐sensitive technique among those considered in this study. © 2011 John Wiley & Sons, Inc. WIREs Data Mining Knowl Discov 2011 1 448–459 DOI: 10.1002/widm.38This article is categorized under: Algorithmic Development > Hierarchies and Trees Technologies > Classification Technologies > Prediction

Software Quality Prediction Method with Hybrid Applying Principal Components Analysis and Wavelet Neural Network and Genetic Algorithm

The principal components analysis method is used to select the training and testing data from software metrics to eliminate the multicollinearitiy in the original data and reduce the dimensions of the data, the genetic algorithm is applied to optimize the parameters of the constructed wavelet neural network, and then a novel software quality prediction model is proposed and its performance is evaluated. The experimental results with six error parameters show that the presented model combining principal components analysis with wavelet neural network and genetic algorithm can obtain better prediction accuracy than the models using the neural network of back propagation and generalized regression neural network.

Choosing software metrics for defect prediction: an investigation on feature selection techniques

AbstractThe selection of software metrics for building software quality prediction models is a search‐based software engineering problem. An exhaustive search for such metrics is usually not feasible due to limited project resources, especially if the number of available metrics is large. Defect prediction models are necessary in aiding project managers for better utilizing valuable project resources for software quality improvement. The efficacy and usefulness of a fault‐proneness prediction model is only as good as the quality of the software measurement data. This study focuses on the problem of attribute selection in the context of software quality estimation. A comparative investigation is presented for evaluating our proposed hybrid attribute selection approach, in which feature ranking is first used to reduce the search space, followed by a feature subset selection. A total of seven different feature ranking techniques are evaluated, while four different feature subset selection approaches are considered. The models are trained using five commonly used classification algorithms. The case study is based on software metrics and defect data collected from multiple releases of a large real‐world software system. The results demonstrate that while some feature ranking techniques performed similarly, the automatic hybrid search algorithm performed the best among the feature subset selection methods. Moreover, performances of the defect prediction models either improved or remained unchanged when over 85were eliminated. Copyright © 2011 John Wiley & Sons, Ltd.

Nomenclature unification of software product measures

A large number of software quality prediction models are based on software product measures (SPdMs). There are different interpretations and representations of these measures which generate inconsistencies in their naming conventions. These inconsistencies affect the efforts to develop a generic approach to predict software quality. This study identifies two types of such inconsistencies and categorises them into Type I and Type II. Type I inconsistency emerges when different labels are suggested for the same software product measure. Type II inconsistency appears when same label is used for different measures. This study suggests a unification and categorisation framework to remove Type I and Type II inconsistencies. The proposed framework categorises SPdMs with respect to three dimensions: usage frequency, software development paradigm and software lifecycle phase. The framework is applied to 140 SPdMs and a searchable unified measures database (UMD) is developed. Overall, 48.5% of the measures are found inconsistent. Out of the total measures studied 34.28% measures are frequently used. It has been found that 30.71% measures are used in object oriented paradigm and 31.43% measures are used in conventional paradigm. There is an overlap of 37.86% measures between the two paradigms. The UMD reveals that the percentages of measures used in design and implementation phases are 52.86 and 35%, respectively.

An ant colony optimization algorithm to improve software quality prediction models: Case of class stability

ContextAssessing software quality at the early stages of the design and development process is very difficult since most of the software quality characteristics are not directly measurable. Nonetheless, they can be derived from other measurable attributes. For this purpose, software quality prediction models have been extensively used. However, building accurate prediction models is hard due to the lack of data in the domain of software engineering. As a result, the prediction models built on one data set show a significant deterioration of their accuracy when they are used to classify new, unseen data. ObjectiveThe objective of this paper is to present an approach that optimizes the accuracy of software quality predictive models when used to classify new data. MethodThis paper presents an adaptive approach that takes already built predictive models and adapts them (one at a time) to new data. We use an ant colony optimization algorithm in the adaptation process. The approach is validated on stability of classes in object-oriented software systems and can easily be used for any other software quality characteristic. It can also be easily extended to work with software quality predictive problems involving more than two classification labels. ResultsResults show that our approach out-performs the machine learning algorithm C4.5 as well as random guessing. It also preserves the expressiveness of the models which provide not only the classification label but also guidelines to attain it. ConclusionOur approach is an adaptive one that can be seen as taking predictive models that have already been built from common domain data and adapting them to context-specific data. This is suitable for the domain of software quality since the data is very scarce and hence predictive models built from one data set is hard to generalize and reuse on new data.

소프트웨어 품질 예측 모델을 위한 분류 프레임워크

본 논문에서는 위험도라는 품질 인자를 예로 들어 메트릭 기반 소프트웨어 품질 예측 모델들을 네가지 타입으로 분류하는 프레임워크를 제안한다. 모델들은 다음과 같은 두가지 기준에 의해 분류된다: 모델 입력 메트릭 형태, 과거 프로젝트 데이터의 필요 유무. 분류된 타입들은 각각의 특성을 가지며 새롭게 정의된 몇가지 기준들에 의해 타 타입들과 장단점이 비교되었다. 이러한 정성적인 평가를 거쳐 품질 예측 모델을 이용하고자하는 개발 집단은 어떤 품질 예측 모델이 자신들에게 적합한지를 판단할 수 있게 된다. 또한 각 타입에 속하는 위험도 예측 모델들을 구현해 예측 성능을 측정한 선행 연구 데이터를 분석하여 예측 성능에 못지않게 모델이 속한 타입의 특성이 모델 선정의 중요한 관건이 됨을 보였다. This paper proposes a framework for classifying metric-based software quality prediction models, especially case of software criticality, into four types. Models are classified along two vectors: input metric forms and the necessity of past project data. Each type has its own characteristics and its strength and weakness are compared with those of other types using newly defined criteria. Through this qualitative evaluation each organization can choose a proper model to suit its environment. My earlier studies of criticality prediction model implemented specific models in each type and evaluated their prediction performances. In this paper I analyze the experimental results and show that the characteristics of a model type is the another key of successful model selection.

Quality prediction modelling for software customisation in the absence of defect data

Given the rapid development of computer techniques and the growing demand from the customer side, system customisation is becoming a necessity for software development, and this area of specialisation is attracting increasing attention from scholars. The innovative functions or applications brought by customisation are challenging the traditional software quality prediction approaches, many of which rest on the premise that the defect data from current system testing or similar systems should be available. This paper proposes a software quality prediction model which focuses on software quality prediction when defect data is unavailable. In addition, this paper improves a quality prediction model presented through substituting the traditional clustering algorithm with a less noise-sensitive algorithm. Through an empirical study on a real ERP system under customisation, the paper concludes that the quality prediction model is feasible in practice and the improved model performs better than the original one in prediction accuracy and effectiveness.

AN IMPLEMENTATION OF METRICS EXTRACTION AND ANALYZER TOOL

Nowadays, developers use N-tier application architecture which is a flexible and reusable structure for distribution to any number of client interfaces. In this study, we have implemented metrics extraction and analyzer tool for Business Tier and Data Access Tier source codes. We used existing object-oriented metrics and complexity software metrics as proposed for Business Tier code1,3. We use a set of metrics for the Data Access Tier that has been proposed in our previous study2. The primary aim of the implementation of the metric extraction and analyzer tool is to facilitate generation of software metrics which are used as independent variables in software quality prediction models. A secondary goal is to gather and present information on the attributes of source code to assist the programmers in managing the software projects. This tool is able to generate the principal components, correlation analysis report and descriptive statistics report. These analysis reports are essential reports for the study of software measurement and software quality management.

Empirical evaluation of a fuzzy logic-based software quality prediction model

Software inspection, due to its repeated success on industrial applications, has now become an industry standard practice. Recently, researchers began analyzing inspection data to obtain insights on how software processes can be improved. For example, project managers need to identify potentially error-prone software components so that limited project resource may be optimally allocated. This paper proposes an automated and fuzzy logic-based approach to satisfy such a need. Fuzzy logic offers significant advantages over other approaches due to its ability to naturally represent qualitative aspect of inspection data and apply flexible inference rules. In order to empirically evaluate the effectiveness of our approach, we have analyzed published inspection data and the ones collected from two separate inspection experiments which we had conducted. χ 2 analysis is applied to statistically demonstrate validity of the proposed quality prediction model.

CASE-BASED SOFTWARE QUALITY PREDICTION

Highly reliable software is becoming an essential ingredient in many systems. However, assuring reliability often entails time-consuming costly development processes. One cost-effective strategy is to target reliability-enhancement activities to those modules that are likely to have the most problems. Software quality prediction models can predict the number of faults expected in each module early enough for reliability enhancement to be effective. This paper introduces a case-based reasoning technique for the prediction of software quality factors. Case-based reasoning is a technique that seeks to answer new problems by identifying similar "cases" from the past. A case-based reasoning system can function as a software quality prediction model. To our knowledge, this study is the first to use case-based reasoning systems for predicting quantitative measures of software quality. A case study applied case-based reasoning to software quality modeling of a family of full-scale industrial software systems. The case-based reasoning system's accuracy was much better than a corresponding multiple linear regression model in predicting the number of design faults. When predicting faults in code, its accuracy was significantly better than a corresponding multiple linear regression model for two of three test data sets and statistically equivalent for the third.

Software quality control and prediction model for maintenance

We develop a quality control and prediction model for improving the quality of software delivered by development to maintenance. This model identifies modules that require priority attention during...