Software Product Metrics Research Articles

A machine and deep learning analysis among SonarQube rules, product, and process metrics for fault prediction

BackgroundDevelopers spend more time fixing bugs refactoring the code to increase the maintainability than developing new features. Researchers investigated the code quality impact on fault-proneness, focusing on code smells and code metrics.ObjectiveWe aim at advancing fault-inducing commit prediction using different variables, such as SonarQube rules, product, process metrics, and adopting different techniques.MethodWe designed and conducted an empirical study among 29 Java projects analyzed with SonarQube and SZZ algorithm to identify fault-inducing and fault-fixing commits, computing different product and process metrics. Moreover, we investigated fault-proneness using different Machine and Deep Learning models.ResultsWe analyzed 58,125 commits containing 33,865 faults and infected by more than 174 SonarQube rules violated 1.8M times, on which 48 software product and process metrics were calculated. Results clearly identified a set of features that provided a highly accurate fault prediction (more than 95% AUC). Regarding the performance of the classifiers, Deep Learning provided a higher accuracy compared with Machine Learning models.ConclusionFuture works might investigate whether other static analysis tools, such as FindBugs or Checkstyle, can provide similar or different results. Moreover, researchers might consider the adoption of time series analysis and anomaly detection techniques.

Hybrid model with optimization tactics for software defect prediction

Defects are frequent in software systems, and they can cause a lot of issues for users. Despite the fact that many studies have been conducted on employing software product metrics to determine defect-prone modules, defect prediction techniques are still worth investigating. Hence, the aim of this work is to provide a unique Software Defect Prediction (SDP) approach that includes four steps like “(a) pre-processing, (b) feature extraction, (c) feature selection and (d) detection.” At first, the input data are given to the pre-processing step, as well as in the feature extraction step; the “statistical features, raw features, higher-order statistical features as well as proposed entropy features” are extracted from the pre-processed data. In addition, the retrieved features are sent into a feature selection step, wherein the appropriate features are selected utilizing a modified chi-square scheme. In the detection step, a hybrid Long Short-Term Memory (LSTM) and Deep Belief Network (DBN) classifiers are used to predict the defects. To provide a more accurate detection, the weights of both DBN and LSTM are optimally tuned via a Self Improved Social Ski-Driver Optimization (SISSDO) algorithm. This proposed SDP model is a beneficial practice for enhancing software quality and reliability. Moreover, the results of the adopted technique are assessed to traditional techniques on the basis of various measures. In particular, the accuracy of the suggested approach for dataset 3 is 5.80%, 6.52%, 5.07%, 7.97%, 5.80%, 9.42%, 9.42%, 10.15%, 2.17%, and 3.62% better than the extant HC[Formula: see text]+[Formula: see text]ALO, HC[Formula: see text]+[Formula: see text]SMO, HC[Formula: see text]+[Formula: see text]CMBO, HC[Formula: see text]+[Formula: see text]SSD, RNN, CNN, NN, Bi-LSTM, HC+SPFCNN, and HC[Formula: see text]+[Formula: see text]CWAR approaches, correspondingly. Moreover, the computation time of the suggested approach is 17.05%, 5.78%, 1.31%, and 50.53% better than the existing HC[Formula: see text]+[Formula: see text]ALO, HC[Formula: see text]+[Formula: see text]SMO, HC[Formula: see text]+[Formula: see text]CMBO, and HC[Formula: see text]+[Formula: see text]SSD approaches, correspondingly.

Product metrics for spreadsheets—A systematic review

Software product metrics allow practitioners to improve their products and to optimize development processes based on quantifiable characteristics of source code. To facilitate similar benefits for spreadsheet programs, researchers proposed various product metrics for spreadsheets over the last decades. However, to our knowledge, no comprehensive overview of those efforts is currently available. In this paper, we close this gap by conducting a literature review of research works that either inherently or explicitly define product metrics for spreadsheets. We scanned five major digital libraries for scientific papers that define or use spreadsheet product metrics. Based on the identified 37 papers, we created a novel catalog of product metrics for spreadsheets. The catalog can be used by practitioners and researchers as a central reference for spreadsheet product metrics. In the paper, we (i) describe the proposed metrics in detail, (ii) report how often and for what purposes the metrics are used, (iii) identify significant discrepancies in the naming and definition of the metrics, and (iv) investigate how the appropriateness of the metrics was evaluated.

A novel software defect prediction based on atomic class-association rule mining

To ensure the rational allocation of software testing resources and reduce costs, software defect prediction has drawn notable attention to many “white-box” and “black-box” classification algorithms. Although there have been lots of studies on using software product metrics to identify defect-prone modules, defect prediction algorithms are still worth exploring. For instance, it is not easy to directly implement the Apriori algorithm to classify defect-prone modules across a skewed dataset. Therefore, we propose a novel supervised approach for software defect prediction based on atomic class-association rule mining (ACAR). It holds the characteristics of only one feature of the antecedent and a unique class label of the consequent, which is a specific kind of association rules that explores the relationship between attributes and categories. It holds the characteristics of only one feature of the antecedent and a unique class label of the consequent, which is a specific kind of association rules that explores the relationship between attributes and categories. Such association patterns can provide meaningful knowledge that can be easily understood by software engineers. A new software defect prediction model infrastructure based on association rules is employed to improve the prediction of defect-prone modules, which is divided into data preprocessing, rule model building and performance evaluation. Moreover, ACAR can achieve a satisfactory classification performance compared with other seven benchmark learners (the extension of classification based on associations (CBA2), Support Vector Machine, Naive Bayesian, Decision Tree, OneR, K-nearest Neighbors and RIPPER) on NASA MDP and PROMISE datasets. In light of software defect associative prediction, a comparative experiment between ACAR and CBA2 is discussed in details. It is demonstrated that ACAR is better than CBA2 in terms of AUC, G-mean, Balance, and understandability. In addition, the average AUC of ACAR is increased by 2.9% compared with CBA2, which can reach 81.1%.

Performance and cost-effectiveness of change burst metrics in predicting software faults

The purpose of this study is to determine a type of software metric at file level exhibiting the best prediction performance. Studies have shown that software process metrics are better predictors of software faults than software product metrics. However, there is need for a specific software process metric which can guarantee the best fault prediction performances consistently across different experimental contexts. We collected software metrics data from Open Source Software projects. We used logistic regression and linear regression algorithms to predict bug status and number of bugs corresponding to a file, respectively. The prediction performance of these models was evaluated against numerical and graphical prediction model performance measures. We found that change burst metrics exhibit the best numerical performance measures and have the highest fault detection probability and least cost of misclassification of software components.

SPMDL

Software metrics are becoming more acceptable measures for software quality assessment. However, there is no standard form to represent metric definitions, which would be useful for metrics exchange and customization. In this article, we propose the Software Product Metrics Definition Language (SPMDL). We develop an XML-based description language to define software metrics in a precise and reusable form. Metric definitions in SPMDL are based on meta-models extracted from either source code or design artifacts, such as the Dagstuhl Middle Meta-model, with support for various abstraction levels. The language defines several flexible computation mechanisms, such as extended Object Constraint Language queries and predefined graph operations on the meta-model. SPMDL provides an unambiguous description of the metric definition; it is also easy to use and is extensible.

Automatic Calculation of Process Metrics and their Bug Prediction Capabilities

Identifying fault-prone code parts is useful for the developers to help reduce the time required for locating bugs. It is usually done by characterizing the already known bugs with certain kinds of metrics and building a predictive model from the data. For the characterization of bugs, software product and process metrics are the most popular ones. The calculation of product metrics is supported by many free and commercial software products. However, tools that are capable of computing process metrics are quite rare. In this study, we present a method of computing software process metrics in a graph database. We describe the schema of the database created and we present a way to readily get the process metrics from it. With this technique, process metrics can be calculated at the file, class and method levels. We used GitHub as the source of the change history and we selected 5 open-source Java projects for processing. To retrieve positional information about the classes and methods, we used SourceMeter, a static source code analyzer tool. We used Neo4j as the graph database engine, and its query language - cypher - to get the process metrics. We published the tools we created as open-source projects on GitHub. To demonstrate the utility of our tools, we selected 25 release versions of the 5 Java projects and calculated the process metrics for all of the source code elements (files, classes and methods) in these versions. Using our previous published bug database, we built bug databases for the selected projects that contain the computed process metrics and the corresponding bug numbers for files and classes. (We published these databases as an online appendix.) Then we applied 13 machine learning algorithms on the database we created to find out if it is feasible for bug prediction purposes. We achieved F-measure values on average of around 0.7 at the class level, and slightly better values of between 0.7 and 0.75 at the file level. The best performing algorithm was the RandomForest method for both cases.

Software Defect Prediction in Class Level Metric Aggregation Using Data Mining Techniques

Aim of study software defect is a flaw, miscalculation, or failure, in a computer program or framework delivering an inappropriate or surprising result, or making it perform in an unintended way. Software Defect Prediction (SDP) finds defective modules in software. The final product ought to have as few defects as possible to create top notch software. Early software defects discovery prompts diminished development costs and rework effort and better software. Software metrics guarantee quantitative methods to survey software quality. Software metrics are helpful to software process and product metrics. Thus, a defect prediction study is critical to guarantee quality software and software metric aggregation. In this study, the efficiency of classifier for SDP is assessed. Diverse classifiers like Naïve Bayes, K Nearest Neighbor (KNN), C4.5 and Multilayer Perceptrons Neural Network (MLPNN) are assessed for SDP.

Improving Recall of software defect prediction models using association mining

Use of software product metrics in defect prediction studies highlights the utility of these metrics. Public availability of software defect data based on the product metrics has resulted in the development of defect prediction models. These models experience a limitation in learning Defect-prone (D) modules because the available datasets are imbalanced. Most of the datasets are dominated by Not Defect-prone (ND) modules as compared to D modules. This affects the ability of classification models to learn the D modules more accurately. This paper presents an association mining based approach that allows the defect prediction models to learn D modules in imbalanced datasets. The proposed algorithm preprocesses data by setting specific metric values as missing and improves the prediction of D modules. The proposed algorithm has been evaluated using 5 public datasets. A Naive Bayes (NB) classifier has been developed before and after the proposed preprocessing. It has been shown that Recall of the classifier after the proposed preprocessing has improved. Stability of the approach has been tested by experimenting the algorithm with different number of bins. The results show that the algorithm has resulted in up to 40% performance gain.

New Generation of Software Metrics

Software measurement has always been an issue in software engineering. On one hand, engineering is about designing and building things and modeling, characterizing, monitoring, evaluating, defining, predicting, “prescripting”, controlling, and changing processes and their artifacts; additionally, measurement is essential to verify that the built artifacts comply with their requirements, to validate the product built versus its requirements and design and to keep the production process in control; the old management adage “you can’t manage what you don’t measure” is valid also in software development. On the other hand, because software is nonmaterial, it is a very difficult beast to measure. The compliance with requirements, which can vary greatly from system to system even in the same domain, is something difficult to “measure”, and it is almost impossible to devise metrics able to support such a measure that is repeatable across different systems. The development effort is perhaps easier to estimate and define across different software technologies and domains—after all it is always a matter of money—but it is still very difficult to measure in the ever changing languages, environments, and technologies. The difficulty of measuring software is even more critical in present times due to the increasing size and criticality of software within all technological systems, and the frequent changes in architectural paradigms and process models. New development methods, such as agile and lean approaches, the large diffusion of Open Source Software (OSS) products, service oriented architectures, cloud computing and softwareas-a-service business models, offshore development, ubiquitous computing, and the increasing integration of software and systems engineering, are examples of how software engineering is coping with the growing complexity of software and its applications. In this scenario, we need even more to be able to measure software products, processes, and competencies. To this purpose, radically new approaches, methods, and tools are being proposed in the software metrics field. Whereas it is too early to talk of breakthrough innovations and of definitive or also acceptable solutions for the abovecited issues, progress is being made in some interesting directions. Among these, we may mention software product and process metrics derived from complex network theory and in general from theoretical physics studies; social network analysis applied to software products and to developers’ relationships; measurement of size, effort, and quality of software developed in innovative ways, such as using agile and lean methodologies and open source projects; new metrics for service-oriented architectures and software-as-a service. This special issue presents new and relevant research work along these directions.

Misleading Metrics and Unsound Analyses

The authors demonstrate that the recommendations for analyzing productivity in the appendix to the ISO/IEC 15939 standard are inappropriate. They also show that problems with the ISO/IEC advice can be compounded if software engineers attempt to apply statistical process-control techniques to software productivity metrics. They recommend using small meaningful data sets as the basis for productivity analysis and using effort-estimation models to assess productivity rather than productivity metrics. This article is part of a special focus section of software metrics

Measuring High Performance Computing Productivity

One key to improving high performance computing (HPC) productivity is to find better ways to measure it. We define productivity in terms of mission goals, i.e. greater productivity means that more science is accomplished with less cost and effort. Traditional software productivity metrics and computing benchmarks have proven inadequate for assessing or predicting such end-to-end productivity. In this paper we introduce a new approach to measuring productivity in HPC applications that addresses both development time and execution time. Our goal is to develop a public repository of effective productivity benchmarks that anyone in the HPC community can apply to assess or predict productivity.

Few important considerations for deriving interface complexity metric for component-based systems

Component-based software engineering (CBSE) represents an exciting and promising paradigm for software development. Software components are one of the key issues in CBSE. The software development community is continuously seeking new methods for improving software quality and enhancing development productivity. There is an increasing need for component-based metrics to help manage and foster quality in component-based software development. The traditional software product and process metrics are neither suitable nor sufficient in measuring the complexity of software components, which ultimately is necessary for quality and productivity improvement within organisations adopting CBSE. In this paper, we propose an interface complexity metric (ICM) aimed at measuring the complexity of a software component based on the interface characterisation model of a software component that mainly include such as interface signature, interface constraints, interface packaging and configurations. Based on the value of this metric, the complexity of the software component could be managed within reasonable complexity limits. In this way, the software components could be kept simple which in turn help in enhancing the quality and productivity.

Ordering Fault-Prone Software Modules

Software developers apply various techniques early in development to improve software reliability, such as extra reviews, additional testing, and strategic assignment of personnel. Due to limited resources and time, it is often not practical to enhance the reliability of all modules. Our goal is to target reliability enhancement activities to those modules that would otherwise have problems later. Prior research has shown that a software quality model based on software product and process metrics can predict which modules are likely to have faults. A module-order model is a quantitative software quality model that is used to predict the rank-order of modules according to a quality factor, such as the number of faults. The contribution of this paper is definition of module-order models and a method for their evaluation and use. Two empirical case studies of full-scale industrial software systems provide empirical evidence of the usefulness of module-order models for targeting reliability enhancement.

Uncertain Classification of Fault-Prone Software Modules

Many development organizations try to minimize faults in software as a means for improving customer satisfaction. Assuring high software quality often entails time-consuming and costly development processes. A software quality model based on software metrics can be used to guide enhancement efforts by predicting which modules are fault-prone. This paper presents statistical techniques to determine which predictions by a classification tree should be considered uncertain. We conducted a case study of a large legacy telecommunications system. One release was the basis for the training dataset, and the subsequent release was the basis for the evaluation dataset. We built a classification tree using the TREEDISC algorithm, which is based on χ2 tests of contingency tables. The model predicted whether a module was likely to have faults discovered by customers, or not, based on software product, process, and execution metrics. We simulated practical use of the model by classifying the modules in the evaluation dataset. The model achieved useful accuracy, in spite of the very small proportion of fault-prone modules in the system. We assessed whether the classes assigned to the leaves were appropriate by statistical tests, and found sizable subsets of modules with uncertain classification. Discovering which modules have uncertain classifications allows sophisticated enhancement strategies to resolve uncertainties. Moreover, TREEDISC is especially well suited to identifying uncertain classifications.

An object-oriented metrics suite for Ada 95

Ada 95 added object-oriented programming capabilities to the existing Ada standard. The object-oriented paradigm results in new relationships within and among software modules. Traditional software product metrics, i.e., those metrics developed for functionally oriented software, do not account for these new relationships and may be insufficient for use in object-oriented development. This paper addresses this deficiency by 1) defining a set of object-oriented metrics tailored to Ada 95 software, and 2) providing empirical evidence of their validity. This set of metrics includes product measures quantifying the class-related software attributes of size, coupling, cohesion, and complexity. The results indicate that many of the metrics examined do correlate to the number of revisions made to the classes during development.

Cost-Benefit Analysis of Software Quality Models

Software reliability is increasingly important in today's marketplace. When traditional software development processes fail to deliver the level of reliability demanded by customers, radical changes in software development processes may be needed. Business process reengineering (BPR) is the popular term for comprehensive redesign of business processes. This paper focuses on the business processes that produce commercial software, and illustrates the central role that models have in implementation of BPR. Software metrics and software-quality modeling technology enable reengineering of software development processes, moving from a static process model to a dynamic one that adapts to the expected quality of each module. We present a method for cost-benefit analysis of BPR of software development processes as a function of model accuracy. The paper defines costs, benefits, profit, and return on investment from both short-term and long-term perspectives. The long-term perspective explicitly accounts for software maintenance efforts. A case study of a very large legacy telecommunications system illustrates the method. The dependent variable of the software-quality model was whether a module will have faults discovered by customers. The independent variables were software product and process metrics. In an example, the costs and benefits of using the model are compared to using random selection of modules for reliability enhancement. Such a cost-benefit analysis clarifies the implications of following model recommendations.

Classification-tree models of software-quality over multiple releases

This paper presents an empirical study that evaluates software-quality models over several releases, to address the question, How long will a model yield useful predictions? The classification and regression trees (CART) algorithm is introduced, CART can achieve a preferred balance between the two types of misclassification rates. This is desirable because misclassification of fault-prone modules often has much more severe consequences than misclassification of those that are not fault-prone. The case-study developed 2 classification-tree models based on 4 consecutive releases of a very large legacy telecommunication system. Forty-two software product, process and execution metrics were candidate predictors. Model 1 used measurements of the first release as the training data set; this model had 11 important predictors. Model 2 used measurements of the second release as the training data set; this model had 15 important predictors. Measurements of subsequent releases were evaluation data sets. Analysis of the models' predictors yielded insights into various software development practices. Both models had accuracy that would be useful to developers. One might suppose that software-quality models lose their value very quickly over successive releases due to evolution of the product and the underlying development processes. The authors found the models remained useful over all the releases studied.

Software product metrics

Software metrics are measures of software. Their primary use is to help us plan and predict software development. We can better control software quality and development effort if we can quantify the software. This is especially true in the early stages of development. Research has been done in the area of predicting software maintenance effort from software metrics. In general, software metrics can be classified into two categories: software product metrics and software process metrics. Software product metrics are measures of software products such as source code and design documents. Software process metrics are measures of software development process. For example, the size of the software is a measure of the software product itself, thus a product metrics. The effort required to design a software system may be influenced by how the software is designed, thus a process metrics. In this article, only software product metrics are addressed.

Balancing Misclassification Rates in Classification-TreeModels of Software Quality

Software product and process metrics can be useful predictors of which modules are likely to have faults during operations. Developers and managers can use such predictions by software quality mode...