Software Quality Estimation Research Articles

The Approach to Estimating Software Quality Based on Cognitive Model

The article studies generally accepted estimation of software quality, which shows its compliance with users’ expectations. The research deals with investigation of software quality and requirements to it and focuses on the fact that software quality depends on its correspondence to GOST R ISO/MEK 25010-2015. It is proposed to use a common model of quality in order to estimate software by analyzing expert appraisals and their aggregation through the mathematic tool of cognitive maps. The use of cognitive maps gives an opportunity to estimate the quality of software product. At the same time we can get a possibility to identify controlled factors of software product quality, affecting which we can increase its target factor of quality. Findings of the research can be used in practice to develop and estimate software quality on different levels of difficulty and specialization.

Many-objective optimization of non-functional attributes based on refactoring of software models

Software quality estimation is a challenging and time-consuming activity, and models are crucial to face the complexity of such activity on modern software applications. In this context, software refactoring is a crucial activity within development life-cycles where requirements and functionalities rapidly evolve. One main challenge is that the improvement of distinctive quality attributes may require contrasting refactoring actions on software, as for trade-off between performance and reliability (or other non-functional attributes). In such cases, multi-objective optimization can provide the designer with a wider view on these trade-offs and, consequently, can lead to identify suitable refactoring actions that take into account independent or even competing objectives. In this paper, we present an approach that exploits the NSGA-II as the genetic algorithm to search optimal Pareto frontiers for software refactoring while considering many objectives. We consider performance and reliability variations of a model alternative with respect to an initial model, the amount of performance antipatterns detected on the model alternative, and the architectural distance, which quantifies the effort to obtain a model alternative from the initial one. We applied our approach on two case studies: a Train Ticket Booking Service, and CoCoME. We observed that our approach is able to improve performance (by up to 42%) while preserving or even improving the reliability (by up to 32%) of generated model alternatives. We also observed that there exists an order of preference of refactoring actions among model alternatives. Based on our analysis, we can state that performance antipatterns confirmed their ability to improve performance of a subject model in the context of many-objective optimization. In addition, the metric that we adopted for the architectural distance seems to be suitable for estimating the refactoring effort.

A Software Quality Prediction Model for Aspect Oriented System using Neuro-Fuzzy Approach

Nowadays, the usage of software has increased exponentially in various fields like education systems, industries, health systems, and many others. Various software architectures are already available in the market e.g. modular oriented, component-based, object-oriented, aspect-oriented, etc. Aspect-Oriented (AO) system software has gained much attention due to its superior features to the aforementioned systems. However, AO systems face the challenges of being complex and hard testing environment, a quality assessment of these systems is necessary. In this paper, a software quality estimation model for aspect oriented system using neuro-fuzzy approach has been developed. For which, a detailed study on aspect oriented systems has been accomplished in terms of various attributes affecting the quality of these software. In this paper, a framework of software quality prediction model has been designed using the adaptive neuro-fuzzy inference engine (ANFIS) approach. Data of 200 software pieces have been collected in this study where 150 software data is used to train the ANFIS model whereas 50 software data is used for testing purposes. The quality estimated by the proposed ANFIS model is compared with the actual quality of these software data and quantitative analysis is performed in terms of error measures. Finally, it was found that the proposed ANFIS model worked better in terms of MSE, MRE, MARE, MBRE, and MIBRE error measures.

Software quality analysis based on cost and error using fuzzy combined COCOMO model

Software quality analysis and estimation is essential in developing a software to avoid faults and increase the reliability. Software quality model (SQM) is highly concerned with standard metrics to qualify the software modules to classify bug or no bug. By using these models, it is easy to identify the hurdles called as errors or faults Apriori to the development cycle. More likely the metrics will not follow the standard protocol in terms of size, performance, technology and the complexity involved. It will vary across the projects. Surprisingly there is no model-based architecture driven tool is available to intact the baseline estimates of the project based on the previous knowledge resource. In earlier research works, various quality assurance metrics are used for analysing the SQ. Also, there is no existing approaches can do earlier prediction of the faults/errors or reduced misclassification rates. But, the COCOMO (COnstructive COst MOdel) gives an approximate estimate in terms of the month constant will not be same for simulating the study. Hence By combining more than one model estimates COCOMO and Gaussian Membership Function software estimate relative error will be the best suite. A fuzzy-based analogy is obtained in the present study to select the nearest path from the history available to meet the project cost and time. Small or standard training sets were considered to deploy the estimate, and compare the performance with different estimators. From the experiment, it is concluded that the proposed fuzzy-COCOMO model outperforms than the existing approaches in terms of relative error.

Object oriented quality prediction through artificial intelligence and machine learning: a survey

Software quality estimation is an important aspect as it eliminates design and code defects. Object- oriented quality metrics prediction can help in the estimation of software quality of any defects and the chances of errors. In this paper a survey and the case analytics have been presented for the object-oriented quality prediction. It shows the analytical and experimental aspects of previous methodologies. This survey also elaborates different object-oriented parameters which is useful for the same problem. It also elaborates the problem aspects as well the limitations for the future directions. Machine learning and artificial intelligence methods have been considered mostly for this survey. The parameters considered are inheritance, dynamic behavior, encapsulation, objects etc.

The application of the theory of dynamic systems to software quality estimation

Hardware and software are considered from the point of view of the theory of dynamic systems. It has been shown that a program may be considered as dynamic system placed into the phase space of the program variables. The theory of dynamic systems as applied to programming allows to classify programs revealing the important quality indices of the program products which are related to the basic concepts of the ergodic theory as well as to that of information and informatics. As a program is run, possibilities (complexity) of the computer may be evaluated and important quality indices of the hardware may be appreciated from the point of view of the programming itself. This approach lets also separately evaluate the quality of both hardware and software.

Incremental Feature Selection Method for Software Defect Prediction

Software defect prediction models are essential for understanding quality attributes relevant for software organization to deliver better software reliability. This paper focuses mainly based on the selection of attributes in the perspective of software quality estimation for incremental database. A new dimensionality reduction method Wilk’s Lambda Average Threshold (WLAT) is presented for selection of optimal features which are used for classifying modules as fault prone or not. This paper uses software metrics and defect data collected from benchmark data sets. The comparative results confirm that the statistical search algorithm (WLAT) outperforms the other relevant feature selection methods for most classifiers. The main advantage of the proposed WLAT method is: The selected features can be reused when there is increase or decrease in database size, without the need of extracting features afresh. In addition, performances of the defect prediction models either remains unchanged or improved even after eliminating 85% of the software metrics.

Software Quality Estimation of Component Based Software System by Using Fuzzy MOORA Approach

Software Quality Estimation of Component Based Software System by Using Fuzzy MOORA Approach

CSE database: extended annotations and new recommendations for ECG software testing.

Nowadays, cardiovascular diseases represent the most common cause of death in western countries. Among various examination techniques, electrocardiography (ECG) is still a highly valuable tool used for the diagnosis of many cardiovascular disorders. In order to diagnose a person based on ECG, cardiologists can use automatic diagnostic algorithms. Research in this area is still necessary. In order to compare various algorithms correctly, it is necessary to test them on standard annotated databases, such as the Common Standards for Quantitative Electrocardiography (CSE) database. According to Scopus, the CSE database is the second most cited standard database. There were two main objectives in this work. First, new diagnoses were added to the CSE database, which extended its original annotations. Second, new recommendations for diagnostic software quality estimation were established. The ECG recordings were diagnosed by five new cardiologists independently, and in total, 59 different diagnoses were found. Such a large number of diagnoses is unique, even in terms of standard databases. Based on the cardiologists' diagnoses, a four-round consensus (4R consensus) was established. Such a 4R consensus means a correct final diagnosis, which should ideally be the output of any tested classification software. The accuracy of the cardiologists' diagnoses compared with the 4R consensus was the basis for the establishment of accuracy recommendations. The accuracy was determined in terms of sensitivity=79.20-86.81%, positivepredictive value=79.10-87.11%, and the Jaccardcoefficient=72.21-81.14%, respectively. Within these ranges, the accuracy of the software is comparable with the accuracy of cardiologists. The accuracy quantification of the correct classification is unique. Diagnostic software developers can objectively evaluate the success of their algorithm and promote its further development. The annotations and recommendations proposed in this work will allow for faster development and testing of classification software. As a result, this might facilitate cardiologists' work and lead to faster diagnoses and earlier treatment.

Software Reliability – A Review

Software reliability is about to define the stability or the life of software system with different properties. These properties include the trustfulness of software system, software cost, execution time, software stability etc. The aspects related to these software system includes the probability of software faults, frequency of fault occurrence, criticality of fault, associated module with respective fault etc. In a software development process, the pre estimation of software reliability is required to deliver the software product. According to the required level of software quality estimation of software cost, development time is also estimated. There are number of quality measure that approves the software reliability. Each stage of software life cycle itself takes some time quantum to deal with software reliability. Higher the software quality, lesser the software maintainability. This paper presents a review on the software reliability models. Keywords— Software Reliability, Reliability Metrics, MTBF, MTTR.

Quantitative Estimation of Software Quality in Hospital Information System

Quantitative Estimation of Software Quality in Hospital Information System

Aggregating Data Sampling with Feature Subset Selection to Address Skewed Software Defect Data

Defect prediction is an important process activity frequently used for improving the quality and reliability of software products. Defect prediction results provide a list of fault-prone modules which are necessary in helping project managers better utilize valuable project resources. In the software quality modeling process, high dimensionality and class imbalance are the two potential problems that may exist in data repositories. In this study, we investigate three data preprocessing approaches, in which feature selection is combined with data sampling, to overcome these problems in the context of software quality estimation. These three approaches are: Approach 1 — sampling performed prior to feature selection, but retaining the unsampled data instances; Approach 2 — sampling performed prior to feature selection, retaining the sampled data instances; and Approach 3 — sampling performed after feature selection. A comparative investigation is presented for evaluating the three approaches. In the experiments, we employed three sampling methods (random undersampling, random oversampling, and synthetic minority oversampling), each combined with a filter-based feature subset selection technique called correlation-based feature selection. We built the defect prediction models using five common classification algorithms. The case study was based on software metrics and defect data collected from multiple releases of a real-world software system. The results demonstrated that the type of sampling methods used in data preprocessing significantly affected the performance of the combination approaches. It was found that when the random undersampling technique was used, Approach 1 performed better than the other two approaches. However, when the feature selection technique was used in conjunction with an oversampling method (random oversampling or synthetic minority oversampling), we strongly recommended Approach 3.

Comparing Feature Selection Techniques for Software Quality Estimation Using Data-Sampling-Based Boosting Algorithms

Software defect prediction is a classification technique that utilizes software metrics and fault data collected during the software development process to identify fault-prone modules before the testing phase. It aims to optimize project resource allocation and eventually improve the quality of software products. However, two factors, high dimensionality and class imbalance, may cause low quality training data and subsequently degrade classification models. Feature (software metric) selection and data sampling are frequently used to overcome these problems. Feature selection (FS) is a process of choosing a subset of relevant features so that the quality of prediction models can be maintained or improved. Data sampling alters the dataset to change its balance level, therefore alleviating the problem of traditional classification models that are biased toward the overrepresented (majority) class. A recent study shows that another method, called boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models), is also effective for addressing the class imbalance problem. In this paper, we present a technique that uses FS followed by a boosting algorithm in the context of software quality estimation. We investigate four FS approaches: individual FS, repetitive sampled FS, sampled ensemble FS, and repetitive sampled ensemble FS, and study the impact of the four approaches on the quality of the prediction models. Ten base feature ranking techniques are examined in the case study. We also employ the boosting algorithm to construct classification models with no FS and use the results as the baseline for further comparison. The empirical results demonstrate that (1) FS is important and necessary prior to the learning process; (2) the repetitive sampled FS method generally has similar performance to the individual FS technique; and (3) the ensemble filter (including sampled ensemble filter and repetitive sampled ensemble filter) performs better than or similarly to the average of the corresponding individual base rankers.

Investigating Two Approaches for Adding Feature Ranking to Sampled Ensemble Learning for Software Quality Estimation

Defect prediction is very challenging in software development practice. Classification models are useful tools that can help for such prediction. Classification models can classify program modules into quality-based classes, e.g. fault-prone (fp) or not-fault-prone (nfp). This facilitates the allocation of limited project resources. For example, more resources are assigned to program modules that are of poor quality or likely to have a high number of faults based on the classification. However, two main problems, high dimensionality and class imbalance, affect the quality of training datasets and therefore classification models. Feature selection and data sampling are often used to overcome these problems. Feature selection is a process of choosing the most important attributes from the original dataset. Data sampling alters the dataset to change its balance level. Another technique, called boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models), is found to also be effective for resolving the class imbalance problem. In this study, we investigate an approach for combining feature selection with this ensemble learning (boosting) process. We focus on two different scenarios: feature selection performed prior to the boosting process and feature selection performed inside the boosting process. Ten individual base feature ranking techniques, as well as an ensemble ranker based on the ten, are examined and compared over the two scenarios. We also employ the boosting algorithm to construct classification models without performing feature selection and use the results as the baseline for further comparison. The experimental results demonstrate that feature selection is important and needed prior to the learning process. In addition, the ensemble feature ranking method generally has better or similar performance than the average of the base ranking techniques, and more importantly, the ensemble method exhibits better robustness than most base ranking techniques. As for the two scenarios, the results show that applying feature selection inside boosting performs better than using feature selection prior to boosting.

Software Quality Estimation through Analytic Hierarchy Process Approach

Software Quality Estimation through Analytic Hierarchy Process Approach

V Bhattacharya Analogy-Based Software Quality Prediction with Project Feature Weights

This paper presents analogy-based software quality estimation with project feature weights. The objective of this research is to predict the quality of project accurately and use the results in future predictions. The focus includes identifying parameters on which the quality of software depends. Estimation of rate of improvement of software quality chiefly depends on the development time. Assigning weights to these parameters to improve upon the results is also in the area of interest. In this paper two different similarity measures namely, Euclidian and Manhattan were the measures used for retrieving the matching cases from the knowledgebase to increases estimation accuracy & reliability. Expert judgment, weights and rating levels were used to assign weights and quality rating levels. The results show that assigning weights to software metrics increases the prediction performance considerably. In order to obtain the results, we have used indigenous tools.

Software Quality Estimation using Machine Learning: Case-based Reasoning Technique

Software quality estimation is one of the most interesting research areas in the domain of software engineering for last few decades. Large numbers of techniques and models have already been worked out in the area of error estimation. The aim of software quality estimation is to identify error prone tasks as the cost can be minimized with advance knowledge about the errors and this early treatment of error will enhance the software quality. In this paper we have explored a set of data in university setting. This paper advocates the use of case-based reasoning (i.e., CBR) to make a software quality estimation system by the help of human experts. CBR relies on historical information from similar past projects, whereby similarities are determined by comparing the projects, and key attributes. We have used different similarity measures to find the best method which increases estimation accuracy & reliability. This paper presents a work in which we have expanded our previous work [24]. The software is a console based application and thus does not use the GUI functions of the Operating System, which makes it very fast in execution. In order to obtain results we have used an indigenous tool for software quality estimation, run in c++ compiler. General Terms Software Engineering: Quality

AN EMPIRICAL STUDY OF FEATURE RANKING TECHNIQUES FOR SOFTWARE QUALITY PREDICTION

The primary goal of software quality engineering is to produce a high quality software product through the use of some specific techniques and processes. One strategy is applying data mining techniques to software metric and defect data collected during the software development process to identify potential low-quality program modules. In this paper, we investigate the use of feature selection in the context of software quality estimation (also referred to as software defect prediction), where a classification model is used to predict whether program modules (instances) are fault-prone or not-fault-prone. Seven filter-based feature ranking techniques are examined. Among them, six are commonly used, and the other one, named signal to noise ratio (SNR), is rarely employed. The objective of the paper is to compare these seven techniques for various software data sets and assess their effectiveness for software quality modeling. A case study is performed on 16 software data sets, and classification models are built with five different learners and evaluated with two performance metrics. Our experimental results are summarized based on statistical tests for significance. The main conclusion is that the SNR technique performs as well as the best performer of the six commonly used techniques.

Choosing software metrics for defect prediction: an investigation on feature selection techniques

AbstractThe selection of software metrics for building software quality prediction models is a search‐based software engineering problem. An exhaustive search for such metrics is usually not feasible due to limited project resources, especially if the number of available metrics is large. Defect prediction models are necessary in aiding project managers for better utilizing valuable project resources for software quality improvement. The efficacy and usefulness of a fault‐proneness prediction model is only as good as the quality of the software measurement data. This study focuses on the problem of attribute selection in the context of software quality estimation. A comparative investigation is presented for evaluating our proposed hybrid attribute selection approach, in which feature ranking is first used to reduce the search space, followed by a feature subset selection. A total of seven different feature ranking techniques are evaluated, while four different feature subset selection approaches are considered. The models are trained using five commonly used classification algorithms. The case study is based on software metrics and defect data collected from multiple releases of a large real‐world software system. The results demonstrate that while some feature ranking techniques performed similarly, the automatic hybrid search algorithm performed the best among the feature subset selection methods. Moreover, performances of the defect prediction models either improved or remained unchanged when over 85were eliminated. Copyright © 2011 John Wiley & Sons, Ltd.

Pattern Trees for Fault-Proneness Detection in Object-Oriented Software

Problem statement: This study introduced an application of pattern tree based classification technique in the area of object-oriented software quality estimation. This application explored the fault prediction accuracy of pattern trees. Approach: Similarity measures and fuzzy aggregations employed in the pattern tree technique had been used to generate tree models for fault detection in software modules. Experiments had been performed on datasets namely, KC1 and KC3 obtained from NASA's metric data program. Pattern tree models were built using metrics from the object-oriented software datasets. Results: AND/OR, OWA and WA had been selected for pattern tree induction. Pattern tree models build using RMSE similarity measure produced higher accuracy as compared to other similarity measures. Conclusion: The proposed application succeeded in improving the quality of the object-oriented software in terms of prediction accuracy. Pattern trees models were found to be less structural complex as compared to fuzzy decision trees.