Software Quality Prediction Research Articles

베이지안 분류기를 이용한 소프트웨어 품질 분류

Many metric-based classification models have been proposed to predict fault-proneness of software module. This paper presents two prediction models using Bayesian classifier which is one of the most popular modern classification algorithms. Bayesian model based on Bayesian probability theory can be a promising technique for software quality prediction. This is due to the ability to represent uncertainty using probabilities and the ability to partly incorporate expert's knowledge into training data. The two models, Na<TEX>$\ddot{i}$</TEX>veBayes(NB) and Bayesian Belief Network(BBN), are constructed and dimensionality reduction of training data and test data are performed before model evaluation. Prediction accuracy of the model is evaluated using two prediction error measures, Type I error and Type II error, and compared with well-known prediction models, backpropagation neural network model and support vector machine model. The results show that the prediction performance of BBN model is slightly better than that of NB. For the data set with ambiguity, although the BBN model's prediction accuracy is not as good as the compared models, it achieves better performance than the compared models for the data set without ambiguity.

A Model for Software Product Quality Prediction

When the expression “Software Quality” is used, we usually think in terms of an excellent software product that fulfills our expectations. These expectations are based on the intended use. Number of models has been proposed for evaluation of software quality based on various characteristics. In this paper quality of software product is defined in terms of basic components as constituent part of any program or software and proposed a software quality prediction model based on basic components. It has been justified with example that if any software quality model uses the tacit knowledge that will be better than any other model in terms of quality.

The use of decision trees for cost‐sensitive classification: an empirical study in software quality prediction

AbstractThis empirical study investigates two commonly used decision tree classification algorithms in the context of cost‐sensitive learning. A review of the literature shows that the cost‐based performance of a software quality prediction model is usually determined after the model‐training process has been completed. In contrast, we incorporate cost‐sensitive learning during the model‐training process. The C4.5 and Random Forest decision tree algorithms are used to build defect predictors either with, or without, any cost‐sensitive learning technique. The paper investigates six different cost‐sensitive learning techniques: AdaCost, Adc2, Csb2, MetaCost, Weighting, and Random Undersampling (RUS). The data come from case study include 15 software measurement datasets obtained from several high‐assurance systems. In addition, to a unique insight into the cost‐based performance of defection prediction models, this study is one of the first to use misclassification cost as a parameter during the model‐training process. The practical appeal of this research is that it provides a software quality practitioner with a clear process for how to consider (during model training) and analyze (during model evaluation) the cost‐based performance of a defect prediction model. RUS is ranked as the best cost‐sensitive technique among those considered in this study. © 2011 John Wiley &amp; Sons, Inc. WIREs Data Mining Knowl Discov 2011 1 448–459 DOI: 10.1002/widm.38This article is categorized under: Algorithmic Development &gt; Hierarchies and Trees Technologies &gt; Classification Technologies &gt; Prediction

Software Quality Prediction Method with Hybrid Applying Principal Components Analysis and Wavelet Neural Network and Genetic Algorithm

The principal components analysis method is used to select the training and testing data from software metrics to eliminate the multicollinearitiy in the original data and reduce the dimensions of the data, the genetic algorithm is applied to optimize the parameters of the constructed wavelet neural network, and then a novel software quality prediction model is proposed and its performance is evaluated. The experimental results with six error parameters show that the presented model combining principal components analysis with wavelet neural network and genetic algorithm can obtain better prediction accuracy than the models using the neural network of back propagation and generalized regression neural network.

Choosing software metrics for defect prediction: an investigation on feature selection techniques

AbstractThe selection of software metrics for building software quality prediction models is a search‐based software engineering problem. An exhaustive search for such metrics is usually not feasible due to limited project resources, especially if the number of available metrics is large. Defect prediction models are necessary in aiding project managers for better utilizing valuable project resources for software quality improvement. The efficacy and usefulness of a fault‐proneness prediction model is only as good as the quality of the software measurement data. This study focuses on the problem of attribute selection in the context of software quality estimation. A comparative investigation is presented for evaluating our proposed hybrid attribute selection approach, in which feature ranking is first used to reduce the search space, followed by a feature subset selection. A total of seven different feature ranking techniques are evaluated, while four different feature subset selection approaches are considered. The models are trained using five commonly used classification algorithms. The case study is based on software metrics and defect data collected from multiple releases of a large real‐world software system. The results demonstrate that while some feature ranking techniques performed similarly, the automatic hybrid search algorithm performed the best among the feature subset selection methods. Moreover, performances of the defect prediction models either improved or remained unchanged when over 85were eliminated. Copyright © 2011 John Wiley &amp; Sons, Ltd.

Software Defect Prediction Using Non-Negative Matrix Factorization

Quality is considered as an important issue in the fields of software engineering. However, building quality software is very expensive, in order to raise the effectiveness and efficiency of quality assurance and testing, software defect prediction is used to identify defect-prone modules in an upcoming version of a software system and help to allow the effort on those modules. Although many models have been proposed, this problem has not resolved thoroughly. For overcoming these limits, recent results show that researcher should pay more attention to improve the quality of the data. Aimed at this purpose, in this paper, we propose a novel approach to resolve the problem of software defect prediction. The method is classification using Non-Negative Matrix Factorization (NMF). In this paper, NMF algorithm is not only used for extracting external features but also as a powerful way for classification of software defect data. Experiments demonstrating the efficiency of the proposed approach are performed for software defect data classification. And the results show that it outperforms the state of the art techniques tested for this experiment. Finally, we suggest that it can be a useful and practical way addition to the framework of software quality prediction.

A conceptual Bayesian net model for integrated software quality prediction

Software quality can be described by a set of features, such as functionality, reliability, usability, efficiency, maintainability, portability and others. There are various models for software quality prediction developed in the past. Unfortunately, they typically focus on a single quality feature. The main goal of this study is to develop a predictive model that integrates several features of software quality, including relationships between them. This model is an expert-driven Bayesian net, which can be used in diverse analyses and simulations. The paper discusses model structure, behaviour, calibration and enhancement options as well as possible use in fields other than software engineering.

Predicting Software Quality by Optimized BP Network Based on PSO

The prediction model of software quality is the key technology in the software quality evaluation system, which can be used to evaluate software quality characteristics that users care about. Prediction models are often used to find the nonlinear relationship between metric data and quality factors. The paper predicted the relationship between metric data and quality factors with historical data by using the optimized BP network based on PSO. According to the algorithm, 28 groups of data are adopted in the experiment, and compared with the results by using BP network. Experiments show that the algorithm has a better performance than the BP network algorithm and perfectly solve the problem of slow convergence and easily getting into local minimum.

Nomenclature unification of software product measures

A large number of software quality prediction models are based on software product measures (SPdMs). There are different interpretations and representations of these measures which generate inconsistencies in their naming conventions. These inconsistencies affect the efforts to develop a generic approach to predict software quality. This study identifies two types of such inconsistencies and categorises them into Type I and Type II. Type I inconsistency emerges when different labels are suggested for the same software product measure. Type II inconsistency appears when same label is used for different measures. This study suggests a unification and categorisation framework to remove Type I and Type II inconsistencies. The proposed framework categorises SPdMs with respect to three dimensions: usage frequency, software development paradigm and software lifecycle phase. The framework is applied to 140 SPdMs and a searchable unified measures database (UMD) is developed. Overall, 48.5% of the measures are found inconsistent. Out of the total measures studied 34.28% measures are frequently used. It has been found that 30.71% measures are used in object oriented paradigm and 31.43% measures are used in conventional paradigm. There is an overlap of 37.86% measures between the two paradigms. The UMD reveals that the percentages of measures used in design and implementation phases are 52.86 and 35%, respectively.

An ant colony optimization algorithm to improve software quality prediction models: Case of class stability

ContextAssessing software quality at the early stages of the design and development process is very difficult since most of the software quality characteristics are not directly measurable. Nonetheless, they can be derived from other measurable attributes. For this purpose, software quality prediction models have been extensively used. However, building accurate prediction models is hard due to the lack of data in the domain of software engineering. As a result, the prediction models built on one data set show a significant deterioration of their accuracy when they are used to classify new, unseen data. ObjectiveThe objective of this paper is to present an approach that optimizes the accuracy of software quality predictive models when used to classify new data. MethodThis paper presents an adaptive approach that takes already built predictive models and adapts them (one at a time) to new data. We use an ant colony optimization algorithm in the adaptation process. The approach is validated on stability of classes in object-oriented software systems and can easily be used for any other software quality characteristic. It can also be easily extended to work with software quality predictive problems involving more than two classification labels. ResultsResults show that our approach out-performs the machine learning algorithm C4.5 as well as random guessing. It also preserves the expressiveness of the models which provide not only the classification label but also guidelines to attain it. ConclusionOur approach is an adaptive one that can be seen as taking predictive models that have already been built from common domain data and adapting them to context-specific data. This is suitable for the domain of software quality since the data is very scarce and hence predictive models built from one data set is hard to generalize and reuse on new data.

Evolutionary Optimization of Software Quality Modeling with Multiple Repositories

A novel search-based approach to software quality modeling with multiple software project repositories is presented. Training a software quality model with only one software measurement and defect data set may not effectively encapsulate quality trends of the development organization. The inclusion of additional software projects during the training process can provide a cross-project perspective on software quality modeling and prediction. The genetic-programming-based approach includes three strategies for modeling with multiple software projects: Baseline Classifier, Validation Classifier, and Validation-and-Voting Classifier. The latter is shown to provide better generalization and more robust software quality models. This is based on a case study of software metrics and defect data from seven real-world systems. A second case study considers 17 different (nonevolutionary) machine learners for modeling with multiple software data sets. Both case studies use a similar majority-voting approach for predicting fault-proneness class of program modules. It is shown that the total cost of misclassification of the search-based software quality models is consistently lower than those of the non-search-based models. This study provides clear guidance to practitioners interested in exploiting their organization's software measurement data repositories for improved software quality modeling.

A survey on search-based software design

This survey investigates search-based approaches to software design. The basics of the most popular meta-heuristic algorithms are presented as background to the search-based viewpoint. Software design is considered from a wide viewpoint, including topics that can also be categorized as software maintenance or re-engineering. Search-based approaches have been used in research from the high architecture design level to software clustering and finally software refactoring. Enhancing and predicting software quality with search-based methods is also taken into account as a part of the design process. The background for the underlying software engineering problems is discussed, after which search-based approaches are presented. Summarizing remarks and tables collecting the fundamental issues of approaches for each type of problem are given. The choices regarding critical decisions, such as representation and fitness function, when used in meta-heuristic search algorithms, are emphasized and discussed in detail. Ideas for future research directions are also given.

A novel composite model approach to improve software quality prediction

Context: How can quality of software systems be predicted before deployment? In attempting to answer this question, prediction models are advocated in several studies. The performance of such models drops dramatically, with very low accuracy, when they are used in new software development environments or in new circumstances. Objective The main objective of this work is to circumvent the model generalizability problem. We propose a new approach that substitutes traditional ways of building prediction models which use historical data and machine learning techniques. Method In this paper, existing models are decision trees built to predict module fault-proneness within the NASA Critical Mission Software. A genetic algorithm is developed to combine and adapt expertise extracted from existing models in order to derive a “composite” model that performs accurately in a given context of software development. Experimental evaluation of the approach is carried out in three different software development circumstances. Results The results show that derived prediction models work more accurately not only for a particular state of a software organization but also for evolving and modified ones. Conclusion Our approach is considered suitable for software data nature and at the same time superior to model selection and data combination approaches. It is then concluded that learning from existing software models (i.e., software expertise) has two immediate advantages; circumventing model generalizability and alleviating the lack of data in software-engineering.

A Bayesian network approach to assess and predict software quality using activity-based quality models

Context Software quality is a complex concept. Therefore, assessing and predicting it is still challenging in practice as well as in research. Activity-based quality models break down this complex concept into concrete definitions, more precisely facts about the system, process, and environment as well as their impact on activities performed on and with the system. However, these models lack an operationalisation that would allow them to be used in assessment and prediction of quality. Bayesian networks have been shown to be a viable means for this task incorporating variables with uncertainty. Objective The qualitative knowledge contained in activity-based quality models are an abundant basis for building Bayesian networks for quality assessment. This paper describes a four-step approach for deriving systematically a Bayesian network from an assessment goal and a quality model. Method The four steps of the approach are explained in detail and with running examples. Furthermore, an initial evaluation is performed, in which data from NASA projects and an open source system is obtained. The approach is applied to this data and its applicability is analysed. Results The approach is applicable to the data from the NASA projects and the open source system. However, the predictive results vary depending on the availability and quality of the data, especially the underlying general distributions. Conclusion The approach is viable in a realistic context but needs further investigation in case studies in order to analyse its predictive validity.

소프트웨어 품질 예측 모델을 위한 분류 프레임워크

본 논문에서는 위험도라는 품질 인자를 예로 들어 메트릭 기반 소프트웨어 품질 예측 모델들을 네가지 타입으로 분류하는 프레임워크를 제안한다. 모델들은 다음과 같은 두가지 기준에 의해 분류된다: 모델 입력 메트릭 형태, 과거 프로젝트 데이터의 필요 유무. 분류된 타입들은 각각의 특성을 가지며 새롭게 정의된 몇가지 기준들에 의해 타 타입들과 장단점이 비교되었다. 이러한 정성적인 평가를 거쳐 품질 예측 모델을 이용하고자하는 개발 집단은 어떤 품질 예측 모델이 자신들에게 적합한지를 판단할 수 있게 된다. 또한 각 타입에 속하는 위험도 예측 모델들을 구현해 예측 성능을 측정한 선행 연구 데이터를 분석하여 예측 성능에 못지않게 모델이 속한 타입의 특성이 모델 선정의 중요한 관건이 됨을 보였다. This paper proposes a framework for classifying metric-based software quality prediction models, especially case of software criticality, into four types. Models are classified along two vectors: input metric forms and the necessity of past project data. Each type has its own characteristics and its strength and weakness are compared with those of other types using newly defined criteria. Through this qualitative evaluation each organization can choose a proper model to suit its environment. My earlier studies of criticality prediction model implemented specific models in each type and evaluated their prediction performances. In this paper I analyze the experimental results and show that the characteristics of a model type is the another key of successful model selection.

Quality prediction modelling for software customisation in the absence of defect data

Given the rapid development of computer techniques and the growing demand from the customer side, system customisation is becoming a necessity for software development, and this area of specialisation is attracting increasing attention from scholars. The innovative functions or applications brought by customisation are challenging the traditional software quality prediction approaches, many of which rest on the premise that the defect data from current system testing or similar systems should be available. This paper proposes a software quality prediction model which focuses on software quality prediction when defect data is unavailable. In addition, this paper improves a quality prediction model presented through substituting the traditional clustering algorithm with a less noise-sensitive algorithm. Through an empirical study on a real ERP system under customisation, the paper concludes that the quality prediction model is feasible in practice and the improved model performs better than the original one in prediction accuracy and effectiveness.

Empirical validation of object-oriented metrics for predicting fault proneness models

Empirical validation of software metrics used to predict software quality attributes is important to ensure their practical relevance in software organizations. The aim of this work is to find the relation of object-oriented (OO) metrics with fault proneness at different severity levels of faults. For this purpose, different prediction models have been developed using regression and machine learning methods. We evaluate and compare the performance of these methods to find which method performs better at different severity levels of faults and empirically validate OO metrics given by Chidamber and Kemerer. The results of the empirical study are based on public domain NASA data set. The performance of the predicted models was evaluated using Receiver Operating Characteristic (ROC) analysis. The results show that the area under the curve (measured from the ROC analysis) of models predicted using high severity faults is low as compared with the area under the curve of the model predicted with respect to medium and low severity faults. However, the number of faults in the classes correctly classified by predicted models with respect to high severity faults is not low. This study also shows that the performance of machine learning methods is better than logistic regression method with respect to all the severities of faults. Based on the results, it is reasonable to claim that models targeted at different severity levels of faults could help for planning and executing testing by focusing resources on fault-prone parts of the design and code that are likely to cause serious failures.

Software quality prediction using fuzzy integration: a case study

Given the complexity of many contemporary software systems, it is often difficult to gauge the overall quality of their underlying software components. A potential technique to automatically evaluate such qualitative attributes is to use software metrics as quantitative predictors. In this case study, an aggregation technique based on fuzzy integration is presented that combines the predicted qualitative assessments from multiple classifiers. Multiple linear classifiers are presented with randomly selected subsets of automatically generated software metrics describing components from a sophisticated biomedical data analysis system. The external reference test is a software developer’s thorough assessment of complexity, maintainability, and usability, which is used to assign corresponding quality class labels to each system component. The aggregated qualitative predictions using fuzzy integration are shown to be superior to the predictions from the respective best single classifiers.

A Multi-Objective Software Quality Classification Model Using Genetic Programming

A key factor in the success of a software project is achieving the best-possible software reliability within the allotted time & budget. Classification models which provide a risk-based software quality prediction, such as fault-prone & not fault-prone, are effective in providing a focused software quality assurance endeavor. However, their usefulness largely depends on whether all the predicted fault-prone modules can be inspected or improved by the allocated software quality-improvement resources, and on the project-specific costs of misclassifications. Therefore, a practical goal of calibrating classification models is to lower the expected cost of misclassification while providing a cost-effective use of the available software quality-improvement resources. This paper presents a genetic programming-based decision tree model which facilitates a multi-objective optimization in the context of the software quality classification problem. The first objective is to minimize the "Modified Expected Cost of Misclassification", which is our recently proposed goal-oriented measure for selecting & evaluating classification models. The second objective is to optimize the number of predicted fault-prone modules such that it is equal to the number of modules which can be inspected by the allocated resources. Some commonly used classification techniques, such as logistic regression, decision trees, and analogy-based reasoning, are not suited for directly optimizing multi-objective criteria. In contrast, genetic programming is particularly suited for the multi-objective optimization problem. An empirical case study of a real-world industrial software system demonstrates the promising results, and the usefulness of the proposed model

A Comprehensive Empirical Study of Count Models for Software Fault Prediction

Count models, such as the Poisson regression model, and the negative binomial regression model, can be used to obtain software fault predictions. With the aid of such predictions, the development team can improve the quality of operational software. The zero-inflated, and hurdle count models may be more appropriate when, for a given software system, the number of modules with faults are very few. Related literature lacks quantitative guidance regarding the application of count models for software quality prediction. This study presents a comprehensive empirical investigation of eight count models in the context of software fault prediction. It includes comparative hypothesis testing, model selection, and performance evaluation for the count models with respect to different criteria. The case study presented is that of a full-scale industrial software system. It is observed that the information obtained from hypothesis testing, and model selection techniques was not consistent with the predictive performances of the count models. Moreover, the comparative analysis based on one criterion did not match that of another criterion. However, with respect to a given criterion, the performance of a count model is consistent for both the fit, and test data sets. This ensures that, if a fitted model is considered good based on a given criterion, then the model will yield a good prediction based on the same criterion. The relative performances of the eight models are evaluated based on a one-way anova model, and Tukey's multiple comparison technique. The comparative study is useful in selecting the best count model for estimating the quality of a given software system