Source Code Metrics Research Articles

Development of partial least squares regression with discriminant analysis for software bug prediction

Many prediction models and approaches have been introduced during the past decades that try to forecast bugged code elements based on static source code metrics, change and history metrics, or both. However, there is still no universal best solution to this problem, as most suitable features and models vary from dataset to dataset and depend on the context in which we use them. Therefore, novel approaches and further studies on this topic are highly necessary.In this paper, we employ a chemometric approach – Partial Least Squares with Discriminant Analysis (PLS-DA) – for predicting bug prone Classes in Java programs using static source code metrics. PLS-DA is successfully applied within the field of chemometrics, but to our best knowledge, it has never been used before in the software maintenance domain for predicting bugs. In addition, we have used rigorous statistical treatments and evaluation for representing the software engineering results.We show that our PLS-DA based prediction model achieves superior performances compared to the state-of-the-art approaches (i.e. F-measure of 0.44–0.47 at 90% confidence level) when no data re-sampling applied and comparable to others when applying up-sampling on the largest open bug dataset, while training the model is significantly faster, thus finding optimal parameters is much easier. In terms of completeness, which measures the amount of bugs contained in the Java Classes predicted to be defective, PLS-DA outperforms every other algorithm: it found 69.3% and 79.4% of the total bugs with no re-sampling and up-sampling, respectively.

Testability-driven development: An improvement to the TDD efficiency

Test-first development (TFD) is a software development approach involving automated tests before writing the actual code. TFD offers many benefits, such as improving code quality, reducing debugging time, and enabling easier refactoring. However, TFD also poses challenges and limitations, requiring more effort and time to write and maintain test cases, especially for large and complex projects. Refactoring for testability is improving the internal structure of source code to make it easier to test. Refactoring for testability can reduce the cost and complexity of software testing and speed up the test-first life cycle. However, measuring testability is a vital step before refactoring for testability, as it provides a baseline for evaluating the current state of the software and identifying the areas that need improvement. This paper proposes a mathematical model for calculating class testability based on test effectiveness and effort and a machine-learning regression model that predicts testability using source code metrics. It also introduces a testability-driven development (TsDD) method that conducts the TFD process toward developing testable code. TsDD focuses on improving testability and reducing testing costs by measuring testability frequently and refactoring to increase testability without running the program. Our testability prediction model has a mean squared error of 0.0311 and an R2 score of 0.6285. We illustrate the usefulness of TsDD by applying it to 50 Java classes from three open-source projects. TsDD achieves an average of 77.81 % improvement in the testability of these classes. Experts’ manual evaluation confirms the potential of TsDD in accelerating the TDD process.

Interpretable Software Defect Prediction from Project Effort and Static Code Metrics

Software defect prediction models enable test managers to predict defect-prone modules and assist with delivering quality products. A test manager would be willing to identify the attributes that can influence defect prediction and should be able to trust the model outcomes. The objective of this research is to create software defect prediction models with a focus on interpretability. Additionally, it aims to investigate the impact of size, complexity, and other source code metrics on the prediction of software defects. This research also assesses the reliability of cross-project defect prediction. Well-known machine learning techniques, such as support vector machines, k-nearest neighbors, random forest classifiers, and artificial neural networks, were applied to publicly available PROMISE datasets. The interpretability of this approach was demonstrated by SHapley Additive exPlanations (SHAP) and local interpretable model-agnostic explanations (LIME) techniques. The developed interpretable software defect prediction models showed reliability on independent and cross-project data. Finally, the results demonstrate that static code metrics can contribute to the defect prediction models, and the inclusion of explainability assists in establishing trust in the developed models.

A tertiary study on links between source code metrics and external quality attributes

A tertiary study on links between source code metrics and external quality attributes

Automatic detection of Feature Envy and Data Class code smells using machine learning

Code smells in software indicate poor design and implementation choices. Detecting and removing them is critical for sustainable software development. Machine learning (ML) can automate code smell detection. Most ML solutions train models from scratch on code smell datasets, using handcrafted source code metrics as features. Pretrained language models, like BERT, fueled a paradigm shift in natural language processing: from handcrafted features to automatically inferred features and from training models from scratch to using pretrained models. Code embeddings offer the potential to bring a similar paradigm shift to code analysis. Nevertheless, the potential of using pretrained neural code embeddings for code smell detection has yet to be fully explored. To this end, we evaluated ML models trained using different code representations: code metrics and state-of-the-art neural code embeddings (CodeT5 and CuBERT). We experimented with CodeT5 variants (base and small) and explored multiple ways of embedding code snippets (by combining line-level embeddings or passing the entire code snippet as input). We tested our approaches on the tasks of detecting Data Class and Feature Envy on the MLCQ dataset. Considering the results of this study and our previous research, performance-wise, there is no clear winner between using code metrics or code embeddings for different code smell types and programming languages. However, given that, in contrast to code metrics, code embeddings can automatically adapt to new programming constructs and are expected to scale better with dataset size, these models are likely to become the future state-of-the-art feature generation technique for code smell detection.

Graph-Based Keyphrase Extraction for Software Traceability in Source Code and Documentation Mapping

Natural Language Processing (NLP) forms the basis of several computational tasks. However, when applied to the software system’s, NLP provides several irrelevant features and the noise gets mixed up while extracting features. As the scale of software system’s increases, different metrics are needed to assess these systems. Diagrammatic and visual representation of the SE projects code forms an essential component of Source Code Analysis (SCA). These SE projects cannot be analyzed by traditional source code analysis methods nor can they be analyzed by traditional diagrammatic representation. Hence, there is a need to modify the traditional approaches in lieu of changing environments to reduce learning gap for the developers and traceability engineers. The traditional approaches fall short in addressing specific metrics in terms of document similarity and graph dependency approaches. In terms of source code analysis, the graph dependency graph can be used for finding the relevant key-terms and keyphrases as they occur not just intra-document but also inter-document. In this work, a similarity measure based on context is proposed which can be employed to find a traceability link between the source code metrics and API documents present in a package. Probabilistic graph-based keyphrase extraction approach is used for searching across the different project files.

Comparing the understandability of iteration mechanisms over Collections in Java

Abstract Source code understandability is a desirable quality factor affecting long-term code maintenance. Understandability of source code can be assessed in a variety of ways, including subjective evaluation of code fragments (perceived understandability), correctness, and response time to tasks performed. It can also be assessed using various source code metrics, such as cyclomatic complexity or cognitive complexity. Programming languages are evolving, giving programmers new ways to do the same things, e.g., iterating over collections. Functional solutions (lambda expressions and streams) are added to typical imperative constructs like iterators or for-each statements. This research aims to check if there is a correlation between perceived understandability, understandability measured by task correctness, and predicted by source code metrics for typical tasks that require iteration over collections implemented in Java. The answer is based on the results of an experiment. The experiment involved 99 participants of varying ages, declared Java knowledge and seniority measured in years. Functional code was perceived as the most understandable, but only in one case, the subjective assessment was confirmed by the correctness of answers. In two examples with the highest perceived understandability, streams received the worst correctness scores. Cognitive complexity and McCabe’s complexity had the lowest values in all tasks for the functional approach, but – unfortunately – they did not correlate with answer correctness. The main finding is that the functional approach to collection manipulation is the best choice for the filter-map-reduce idiom and its alternatives (e.g., filter-only). It should not be used in more complex tasks, especially those with higher complexity metrics.

A security vulnerability predictor based on source code metrics

Detecting security vulnerabilities in the source code of software systems is one of the most important challenges in the field of software security. We need an effective solution to discover and patch vulnerabilities before our valuable information is compromised. Security testing is a type of software testing that checks whether software is vulnerable to cyber attacks. This study aimed to pursue three main objectives: (1) The first goal is to identify the vulnerable functions of a C/C++ software program based on code metrics. This can reduce the cost of software security testing and also redirect the related activities to identified vulnerable functions rather than to the entire software, (2) The second goal is to identify the type of attack related to the vulnerability function, and (3) Finally, the ultimate goal is to analyze the relationship between code metrics and the vulnerabilities. This goal can help us understand which code structure is most likely to contain vulnerable code. This paper first aimed to create a comprehensive view of the source code of the target software using graph concepts. Second, a set of source code metrics and calculated by crawling on the related graph using the static analysis approach. Finally, the vulnerability prediction model presented in this paper is based on machine learning technique applied on metrics extracted from program source code. Compared to previous work, new achievements have been made in this paper. One of the most important ones is the very high accuracy detection of the proposed model in detecting the type of vulnerability. Moreover, 15 code metrics are used to predict vulnerabilities. Our analysis on feature importance indicates that what structure the software program code has, most likely, it will be vulnerable. Experimental results in 10 real projects (OpenSSL, SQLite, FreeType, LibTiff, Libxslt, Binutils, FFmpeg, ImageMagick, OpenSC, and rdesktop) indicated that the security testing predictor proposed in this paper could predict on average 89% of the really vulnerable functions of the source code and 86% of the vulnerability type of the detected functions correctly.

On the relationship between source-code metrics and cognitive load: A systematic tertiary review

The difficulty of software development tasks depends on several factors including the characteristics of the underlying source-code. These characteristics can be captured and measured using source-code metrics, which, in turn, can provide indications about the difficulty of the source-code. From a cognitive perspective, this difficulty is due to an increase in developers’ cognitive load, which can be estimated using psycho-physiological measures. Based on these measures, a handful of studies investigated the relationship between source-code metrics and cognitive load. For most of the metrics, such a relationship could not be established. While these studies used a small subset of metrics, the literature comprises hundreds of other metrics. Despite the existing reviews surveying these metrics, a consolidated overview is still needed to understand their properties and leverage their potential to align with cognitive load. This need is addressed in this paper through a Systematic Tertiary Review (STR) covering the full spectrum of source-code metrics, studying their properties and investigating their potential relationship to cognitive load. The outcome of this STR is intended to guide practitioners in choosing appropriate metrics, set the grounds for conceptualizing the relationship between source-code metrics and cognitive load and raise new research challenges for the future.

What really changes when developers intend to improve their source code: a commit-level study of static metric value and static analysis warning changes

Many software metrics are designed to measure aspects that are believed to be related to software quality. Static software metrics, e.g., size, complexity and coupling are used in defect prediction research as well as software quality models to evaluate software quality. Static analysis tools also include boundary values for complexity and size that generate warnings for developers. While this indicates a relationship between quality and software metrics, the extent of it is not well understood. Moreover, recent studies found that complexity metrics may be unreliable indicators for understandability of the source code. To explore this relationship, we leverage the intent of developers about what constitutes a quality improvement in their own code base. We manually classify a randomized sample of 2,533 commits from 54 Java open source projects as quality improving depending on the intent of the developer by inspecting the commit message. We distinguish between perfective and corrective maintenance via predefined guidelines and use this data as ground truth for the fine-tuning of a state-of-the art deep learning model for natural language processing. The benchmark we provide with our ground truth indicates that the deep learning model can be confidently used for commit intent classification. We use the model to increase our data set to 125,482 commits. Based on the resulting data set, we investigate the differences in size and 14 static source code metrics between changes that increase quality, as indicated by the developer, and changes unrelated to quality. In addition, we investigate which files are targets of quality improvements. We find that quality improving commits are smaller than non-quality improving commits. Perfective changes have a positive impact on static source code metrics while corrective changes do tend to add complexity. Furthermore, we find that files which are the target of perfective maintenance already have a lower median complexity than files which are the target of non-pervective changes. Our study results provide empirical evidence for which static source code metrics capture quality improvement from the developers point of view. This has implications for program understanding as well as code smell detection and recommender systems.

Method name recommendation based on source code metrics

Method naming is a critical factor in program comprehension, affecting software quality. State-of-the-art naming techniques use deep learning to compute the methods’ similarity considering their textual contents. They highly depend on identifiers’ names and do not compute semantical interrelations among methods’ instructions. Source code metrics compute such semantical interrelations. This article proposes using source code metrics to measure semantical and structural cross-project similarities. The metrics constitute features of a KNN model, determining k-most similar methods to a given method. Experiments with 4000000 Java methods on the proposed model demonstrate improvements in precision and recall of state-of-the-arts with 4.25 and 12.08%.

Software Defect Prediction using Deep Learning by Correlation Clustering of Testing Metrics

The software industry has made significant efforts in recent years to enhance software quality in businesses. The use of proactively defect prediction in the software will assist programmers and white box testing in detecting issues early, saving time and money. Conventional software defect prediction methods focus on traditional source code metrics such as code complexities, lines of code, and so on. These capabilities, unfortunately, are unable to retrieve the semantics of source code. In this paper, we have presented a novel Correlation Clustering fine-tuned CNN (CCFT-CNN) model based on testing Metrics. CCFT-CNN can predict the regions of source code that contain faults, errors, and bugs. Abstract Syntax Tree (AST) tokens are extracted as testing Metrics vectors from the source code. The correlation among AST testing Metrics is performed and clustered as a more relevant feature vector and fed into Convolutional Neural Network (CNN). Then, to enhance the accuracy of defect prediction, fine-tuning of the CNN model is performed by applying hyperparameters. The result analysis is performed on the PROMISE dataset that contains samples of open-source Java applications such as Camel Dataset, Jedit dataset, Poi dataset, Synapse dataset, Xerces dataset, and Xalan dataset. The result findings show that the CCFT- CNN model increases the average F-measure by 2% when compared to the baseline model.

SOFTWARE QUALITY PREDICTION USING MACHINE LEARNING TECHNIQUES AND SOURCE CODE METRICS: A REVIEW

Software quality prediction is the Machine Learning (ML) based technique in which ML models are trained using historical data. Output from these quality models can be used by software experts in the early phase of software development for improving the quality of software by controlling the various quality attributes like maintainability, reliability, security issues of software etc. In this study a systematic review of studies from 2005 to 2021 is performed. Studies that use ML techniques and source code metrics for Software Quality Prediction (SQP) are included for review. Study assesses the commonly used machine learning techniques and source code metric for SQP. Commonly used datasets, feature selection techniques and commonly used performance measures in software quality prediction are also assessed. In this paper 53 primary studies are selected for systematic review. Results of this study prove that Bayesian Learning (BL), Regression, Ensemble Learning (EL), Decision Tree (DT) and Support Vector Machine (SVM) are most commonly ML techniques used for quality prediction which comprises 58%, 52%, 41%, 32%, and 32% of the overall studies respectively. It is also assessed that NASA, PROMISE, Apache, Mozilla Firefox and Eclipse are the most commonly used datasets for training and testing the SQP models. LOC, CC, CBO, RFC, WMC, LCOM, DIT and NOC are among the most commonly used source code metrics in SQP. Based on the results from the selected studies it is concluded that ML techniques and source code metrics have the ability to improve the overall quality of the software.

Class-Level Refactoring Prediction by Ensemble Learning with Various Feature Selection Techniques

Background: Refactoring is changing a software system without affecting the software functionality. The current researchers aim i to identify the appropriate method(s) or class(s) that needs to be refactored in object-oriented software. Ensemble learning helps to reduce prediction errors by amalgamating different classifiers and their respective performances over the original feature data. Other motives are added in this paper regarding several ensemble learners, errors, sampling techniques, and feature selection techniques for refactoring prediction at the class level. Objective: This work aims to develop an ensemble-based refactoring prediction model with structural identification of source code metrics using different feature selection techniques and data sampling techniques to distribute the data uniformly. Our model finds the best classifier after achieving fewer errors during refactoring prediction at the class level. Methodology: At first, our proposed model extracts a total of 125 software metrics computed from object-oriented software systems processed for a robust multi-phased feature selection method encompassing Wilcoxon significant text, Pearson correlation test, and principal component analysis (PCA). The proposed multi-phased feature selection method retains the optimal features characterizing inheritance, size, coupling, cohesion, and complexity. After obtaining the optimal set of software metrics, a novel heterogeneous ensemble classifier is developed using techniques such as ANN-Gradient Descent, ANN-Levenberg Marquardt, ANN-GDX, ANN-Radial Basis Function; support vector machine with different kernel functions such as LSSVM-Linear, LSSVM-Polynomial, LSSVM-RBF, Decision Tree algorithm, Logistic Regression algorithm and extreme learning machine (ELM) model are used as the base classifier. In our paper, we have calculated four different errors i.e., Mean Absolute Error (MAE), Mean magnitude of Relative Error (MORE), Root Mean Square Error (RMSE), and Standard Error of Mean (SEM). Result: In our proposed model, the maximum voting ensemble (MVE) achieves better accuracy, recall, precision, and F-measure values (99.76, 99.93, 98.96, 98.44) as compared to the base trained ensemble (BTE) and it experiences less errors (MAE = 0.0057, MORE = 0.0701, RMSE = 0.0068, and SEM = 0.0107) during its implementation to develop the refactoring model. Conclusions: Our experimental result recommends that MVE with upsampling can be implemented to improve the performance of the refactoring prediction model at the class level. Furthermore, the performance of our model with different data sampling techniques and feature selection techniques has been shown in the form boxplot diagram of accuracy, F-measure, precision, recall, and area under the curve (AUC) parameters.

Towards Predicting Architectural Design Patterns: A Machine Learning Approach

Software architecture plays an important role in software development, especially in software quality and maintenance. Understanding the impact of certain architectural patterns on software quality and verification of software requirements has become increasingly difficult with the increasing complexity of codebases in recent years. Researchers over the years have proposed automated approaches based on machine learning. However, there is a lack of benchmark datasets and more accurate machine learning (ML) approaches. This paper presents an ML-based approach for software architecture detection, namely, MVP (Model–View–Presenter) and MVVM (Model–View–ViewModel). Firstly, we present a labeled dataset that consists of 5973 data points retrieved from GitHub. Nine ML methods are applied for detection of software architecture from source code metrics. Using precision, recall, accuracy, and F1 score, the outstanding ML model performance is 83%, 83%, 83%, and 83%, respectively. The ML model’s performance is validated using k-fold validation (k = 5). Our approach outperforms when compared with the state-of-the-art.

Are Source Code Metrics “Good Enough” in Predicting Security Vulnerabilities?

Modern systems produce and handle a large volume of sensitive enterprise data. Therefore, security vulnerabilities in the software systems must be identified and resolved early to prevent security breaches and failures. Predicting security vulnerabilities is an alternative to identifying them as developers write code. In this study, we studied the ability of several machine learning algorithms to predict security vulnerabilities. We created two datasets containing security vulnerability information from two open-source systems: (1) Apache Tomcat (versions 4.x and five 2.5.x minor versions). We also computed source code metrics for these versions of both systems. We examined four classifiers, including Naive Bayes, Decision Tree, XGBoost Classifier, and Logistic Regression, to show their ability to predict security vulnerabilities. Moreover, an ensemble learner was introduced using a stacking classifier to see whether the prediction performance could be improved. We performed cross-version and cross-project predictions to assess the effectiveness of the best-performing model. Our results showed that the XGBoost classifier performed best compared to other learners, i.e., with an average accuracy of 97% in both datasets. The stacking classifier performed with an average accuracy of 92% in Struts and 71% in Tomcat. Our best-performing model—XGBoost—could predict with an average accuracy of 87% in Tomcat and 99% in Struts in a cross-version setup.

An ensemble meta-estimator to predict source code testability

Unlike most other software quality attributes, testability cannot be evaluated solely based on the characteristics of the source code. The effectiveness of the test suite and the budget assigned to the test highly impact the testability of the code under test. The size of a test suite determines the test effort and cost, while the coverage measure indicates the test effectiveness. Therefore, testability can be measured based on the coverage and number of test cases provided by a test suite, considering the test budget. This paper offers a new equation to estimate testability regarding the size and coverage of a given test suite. The equation has been used to label 23,000 classes belonging to 110 Java projects with their testability measure. The labeled classes were vectorized using 262 metrics. The labeled vectors were fed into a family of supervised machine learning algorithms, regression, to predict testability in terms of the source code metrics. Regression models predicted testability with an R2 of 0.68 and a mean squared error of 0.03, suitable in practice. Fifteen software metrics highly affecting testability prediction were identified using a feature importance analysis technique on the learned model. The proposed models have improved mean absolute error by 38% due to utilizing new criteria, metrics, and data compared with the relevant study on predicting branch coverage as a test criterion. As an application of testability prediction, it is demonstrated that automated refactoring of 42 smelly Java classes targeted at improving the 15 influential software metrics could elevate their testability by an average of 86.87%.

Revisiting the debate: Are code metrics useful for measuring maintenance effort?

Evaluating and predicting software maintenance effort using source code metrics is one of the holy grails of software engineering. Unfortunately, previous research has provided contradictory evidence in this regard. The debate is still open: as a community we are not certain about the relationship between code metrics and maintenance impact. In this study we investigate whether source code metrics can indeed establish maintenance effort at the previously unexplored method level granularity. We consider $\sim $ 730K Java methods originating from 47 popular open source projects. After considering seven popular method level code metrics and using change proneness as a maintenance effort indicator, we demonstrate why past studies contradict one another while examining the same data. We also show that evaluation context is king. Therefore, future research should step away from trying to devise generic maintenance models and should develop models that account for the maintenance indicator being used and the size of the methods being analyzed. Ultimately, we show that future source code metrics can be applied reliably and that these metrics can provide insight into maintenance effort when they are applied in a judiciously context-sensitive manner.

The Impact of Ensemble Techniques on Software Maintenance Change Prediction: An Empirical Study

Various prediction models have been proposed by researchers to predict the change-proneness of classes based on source code metrics. However, some of these models suffer from low prediction accuracy because datasets exhibit high dimensionality or imbalanced classes. Recent studies suggest that using ensembles to integrate several models, select features, or perform sampling has the potential to resolve issues in the datasets and improve the prediction accuracy. This study aims to empirically evaluate the effectiveness of the ensemble models, feature selection, and sampling techniques on predicting change-proneness using different metrics. We conduct an empirical study to compare the performance of four machine learning models (naive Bayes, support vector machines, k-nearest neighbors, and random forests) on seven datasets for predicting change-proneness. We use two types of feature selection (relief and Pearson’s correlation coefficient) and three types of ensemble sampling techniques, which integrate different types of sampling techniques (SMOTE, spread sub-sample, and randomize). The results of this study reveal that the ensemble feature selection and sampling techniques yield improved prediction accuracy over most of the investigated models, and using sampling techniques increased the prediction accuracy of all models. Random forests provide a significant improvement over other prediction models and obtained the highest value of the average of the area under curve in all scenarios. The proposed ensemble feature selection and sampling techniques, along with the ensemble model (random forests), were found beneficial in improving the prediction accuracy of change-proneness.

Software Fault Severity Prediction Using Git History Metrics and Commits

In this paper, we propose new software agnostic metrics extracted from Git history. We compared the proposed metrics to many traditional code-based metrics in terms of fault severity prediction. We used three Machine Learning Algorithms (Random Forest, SVM and Multilayer Perceptron) to build the prediction models. We used data (source code, source code metrics, fault severity information) collected from three different data sources. Results show that the proposed software agnostic metrics perform better in terms of fault severity prediction compared to traditional code-based metrics. They were able to achieve 84% of accuracy in fault severity prediction. We also introduced some terms extracted from commits and showed their effectiveness for fault severity classification.