Code Churn Research Articles

Improving effort-aware just-in-time defect prediction with weighted code churn and multi-objective slime mold algorithm

Effort-aware just-in-time software defect prediction (JIT-SDP) aims to effectively utilize the limited resources of software quality assurance (SQA) to detect more software defects. This improves the efficiency of SQA work and the quality of the software. However, there is disagreement regarding the representation of the key feature variable, SQA effort, in the field of effort-aware JIT-SDP. Additionally, the most recent metaheuristic optimization algorithms (MOAs) have not yet been effectively integrated with multi-objective effort-aware JIT-SDP tasks. These deficiencies, in both feature representation and model optimization (MO), result in a significant disparity between the performance of effort-aware JIT-SDP techniques and the expectations of the industry. In this study, we present a novel method called weighted code churn (CC) and improved multi-objective slime mold algorithm (SMA) (WCMS) for effort-aware JIT-SDP. It comprises two stages: feature improvement (FI) and MO. In the FI phase, we normalize the two feature variables: number of modified files (NF) and distribution of modified code across each file (Entropy). We then use an exponential function to quantify the level of difficulty of the change. The equation is as follows: DD = NFEntropy, where DD is an acronym for the degree of difficulty, NF denotes the base number, and Entropy denotes the index. We define change effort as the product of the difficulty degree in implementing the change and CC, with weighted CC representing the change effort. During the MO stage, we improve the SMA by incorporating multi-objective handling capabilities and devising mechanisms for multi-objective synchronization and conflict resolution. We develop a multi-objective optimization algorithm for hyperparameter optimization (HPO) of the JIT-SDP model in WCMS. To evaluate the performance of our method, we conducted experiments using six well-known open-source projects and employed two effort-aware performance evaluation metrics. We evaluated our method based on three scenarios: cross-validation, time-wise cross-validation, and across-project prediction. The experimental results indicate that the proposed method outperforms the benchmark method. Furthermore, the proposed method demonstrates greater scalability and generalization capabilities.

Predicting the Change Impact of Resolving Defects by Leveraging the Topics of Issue Reports in Open Source Software Systems

Upon receiving a new issue report, practitioners start by investigating the defect type, the potential fixing effort needed to resolve the defect and the change impact. Moreover, issue reports contain valuable information, such as, the title, description and severity, and researchers leverage the topics of issue reports as a collective metric portraying similar characteristics of a defect. Nonetheless, none of the existing studies leverage the defect topic, i.e., a semantic cluster of defects of the same nature, such as Performance, GUI, and Database , to estimate the change impact that represents the amount of change needed in terms of code churn and the number of files changed. To this end, in this article, we conduct an empirical study on 298,548 issue reports belonging to three large-scale open-source systems, i.e., Mozilla, Apache, and Eclipse, to estimate the change impact in terms of code churn or the number of files changed while leveraging the topics of issue reports. First, we adopt the Embedded Topic Model (ETM), a state-of-the-art topic modelling algorithm, to identify the topics. Second, we investigate the feasibility of predicting the change impact using the identified topics and other information extracted from the issue reports by building eight prediction models that classify issue reports requiring small or large change impact along two dimensions, i.e., the code churn size and the number of files changed. Our results suggest that XGBoost is the best-performing algorithm for predicting the change impact, with an AUC of 0.84, 0.76, and 0.73 for the code churn and 0.82, 0.71, and 0.73 for the number of files changed metric for Mozilla, Apache, and Eclipse, respectively. Our results also demonstrate that the topics of issue reports improve the recall of the prediction model by up to 45%.

Improved DevOps Lifecycle by Integrating a Novel Tool V-Git Lab

Aims: we propose a tool that can automatically generate datasets for software defect prediction from GitHub repositories Background: DevOps is a software development approach that emphasizes collaboration, communication, and automation in order to improve the speed and quality of software delivery Objective: This study aims to demonstrate the effectiveness of the tool, and in order to do so, a series of experiments were conducted on several popular GitHub repositories and compared the performance of our generated datasets with existing datasets. Method: The tool works by analyzing the commit history of a given repository and extracting relevant features that can be used to predict defects. These features include code complexity metrics, code churn, and the number of developers involved in a particular code change. Result: Our results show that the datasets generated by our tool are comparable in quality to existing datasets and can be used to train effective software defect prediction models Conclusion: Overall, the proposed tool provides a convenient and effective way to generate high-quality datasets for software defect prediction, which can significantly improve the accuracy and reliability of prediction models.

Inherent Quality Metrics for Continuous Software Quality Enhancement

Traditional software quality metrics based on bug number and pass rate can only provide us afterthought post validation & product release. In this paper, we propose some new inherent software quality metrics for proactive quality control during development phase, including Lines of Code (LOC#), Cyclomatic Complexity and Code Churn. In this paper, citing one ultra-large-scale software - Intel Media Driver as one example, we introduce the reason to choose those metrics, our experience on leveraging those metrics to improve the software quality, and our turn-key solution for automatic metrics data collection and analysis. We expect the identified metrics can help more researchers to form the corresponding research agendas and the experiences sharing can help following practitioners to apply similar enhancements.

Comparing fine-grained source code changes and code churn for bug prediction - A retrospective

More than two decades ago, researchers started to mine the data stored in software repositories to help software developers in making informed decisions for developing and testing software systems. Bug prediction was one of the most promising and popular research directions that uses the data stored in software repositories to predict the bug-proneness or number of bugs in source files. On that topic and as part of Emanuel's PhD studies, we submitted a paper with the title Comparing fine-grained source code changes and code churn for bug prediction [8] to the 8th Working Conference on Mining Software Engineering, held 2011 in beautiful Honolulu, Hawaii. Ten years later, it got selected as one of the finalists to receive the MSR 2021 Most Influential Paper Award. In the following, we provide a retrospective on our work, describing the road to publishing this paper, its impact in the field of bug prediction, and the road ahead.

Studying the Relationship Between the Usage of APIs Discussed in the Crowd and Post-Release Defects

Software development nowadays is heavily based on libraries, frameworks and their proposed Application Programming Interfaces (APIs). However, due to challenges such as the complexity and the lack of documentation, these APIs may introduce various obstacles for developers and common defects in software systems. To resolve these issues, developers usually utilize Question and Answer (Q&A) websites such as Stack Overflow by asking their questions and finding proper solutions for their problems on APIs. Therefore, these websites have become inevitable sources of knowledge for developers, which is also known as the crowd knowledge.However, the relation of this knowledge to the software quality has never been adequately explored before. In this paper, we study whether using APIs which are challenging according to the discussions of the Stack Overflow is related to code quality defined in terms of post-release defects. To this purpose, we define the concept of challenge of an API, which denotes how much the API is discussed in high-quality posts on Stack Overflow. Then, using this concept, we propose a set of products and process metrics. We empirically study the statistical correlation between our metrics and post-release defects as well as added explanatory and predictive power to traditional models through a case study on five open source projects including Spring, Elastic Search, Jenkins, K-8 Mail Android Client, and OwnCloud Android client.Our findings reveal that our metrics have a positive correlation with post-release defects which is comparable to known high-performance traditional process metrics, such as code churn and number of pre-release defects. Furthermore, our proposed metrics can provide additional explanatory and predictive power for software quality when added to the models based on existing products and process metrics. Our results suggest that software developers should consider allocating more resources on reviewing and improving external API usages to prevent further defects.

How different are different diff algorithms in Git?

Automatic identification of the differences between two versions of a file is a common and basic task in several applications of mining code repositories. Git, a version control system, has a diff utility and users can select algorithms of diff from the default algorithm Myers to the advanced Histogram algorithm. From our systematic mapping, we identified three popular applications of diff in recent studies. On the impact on code churn metrics in 14 Java projects, we obtained different values in 1.7% to 8.2% commits based on the different diff algorithms. Regarding bug-introducing change identification, we found 6.0% and 13.3% in the identified bug-fix commits had different results of bug-introducing changes from 10 Java projects. For patch application, we found that the Histogram is more suitable than Myers for providing the changes of code, from our manual analysis. Thus, we strongly recommend using the Histogram algorithm when mining Git repositories to consider differences in source code.

The impact of context metrics on just-in-time defect prediction

Traditional just-in-time defect prediction approaches have been using changed lines of software to predict defective-changes in software development. However, they disregard information around the changed lines. Our main hypothesis is that such information has an impact on the likelihood that the change is defective. To take advantage of this information in defect prediction, we consider n-lines (n = 1,2,…) that precede and follow the changed lines (which we call context lines), and propose metrics that measure them, which we call “Context Metrics.” Specifically, these context metrics are defined as the number of words/keywords in the context lines. In a large-scale empirical study using six open source software projects, we compare the performance of using our context metrics, traditional code churn metrics (e.g., the number of modified subsystems), our extended context metrics which measure not only context lines but also changed lines, and combination metrics that use two extended context metrics at a prediction model for defect prediction. The results show that context metrics that consider the context lines of added-lines achieve the best median value in all cases in terms of a statistical test. Moreover, using few number of context lines is suitable for context metric that considers words, and using more number of context lines is suitable for context metric that considers keywords. Finally, the combination metrics of two extended context metrics significantly outperform all studied metrics in all studied projects w. r. t. the area under the receiver operation characteristic curve (AUC) and Matthews correlation coefficient (MCC).

Scripted GUI testing of Android open-source apps: evolution of test code and fragility causes

Evidence from empirical studies suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular, GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI testing fragility, i.e., GUI test classes failing or needing interventions because of modifications in the AUT or in its GUI arrangement and definition. The objective of our study is to examine the diffusion of test classes created with a set of popular GUI Automation Frameworks for Android apps, the amount of changes required to keep test classes up to date, and the amount of code churn in existing test suites, along with the underlying modifications in the AUT that caused such modifications. We defined 12 metrics to characterize the evolution of test classes and test methods, and a taxonomy of 28 possible causes for changes to test code. To perform our experiments, we selected six widely used open-source GUI Automation Frameworks for Android apps. We evaluated the diffusion of the tools by mining the GitHub repositories featuring them, and computed our set of metrics on the projects. Applying the Grounded Theory technique, we then manually analyzed diff files of test classes written with the selected tools, to build from the ground up a taxonomy of causes for modifications of test code. We found that none of the considered GUI automation frameworks achieved a major diffusion among open-source Android projects available on GitHub. For projects featuring tests created with the selected frameworks, we found that test suites had to be modified often – specifically, about 8% of developers’ modified LOCs belonged to test code and that a relevant portion (around 50% on average) of those modifications were induced by modifications in GUI definition and arrangement. Test code written with GUI automation fromeworks proved to need significant interventions during the lifespan of a typical Android open-source project. This can be seen as an obstacle for developers to adopt this kind of test automation. The evaluations and measurements of the maintainance needed by test code wrtitten with GUI automation frameworks, and the taxonomy of modification causes, can serve as a benchmark for developers, and the basis for the formulation of actionable guidelines and the development of automated tools to help mitigating the issue.

Bringing Parallel Patterns Out of the Corner

High-level parallel programming is an active research topic aimed at promoting parallel programming methodologies that provide the programmer with high-level abstractions to develop complex parallel software with reduced time to solution. Pattern-based parallel programming is based on a set of composable and customizable parallel patterns used as basic building blocks in parallel applications. In recent years, a considerable effort has been made in empowering this programming model with features able to overcome shortcomings of early approaches concerning flexibility and performance. In this article, we demonstrate that the approach is flexible and efficient enough by applying it on 12 out of 13 PARSEC applications. Our analysis, conducted on three different multicore architectures, demonstrates that pattern-based parallel programming has reached a good level of maturity, providing comparable results in terms of performance with respect to both other parallel programming methodologies based on pragma-based annotations (i.e., Open mp and O mp S s ) and native implementations (i.e., P threads ). Regarding the programming effort, we also demonstrate a considerable reduction in lines of code and code churn compared to P threads and comparable results with respect to other existing implementations.

Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical Study

Background . Slice-based cohesion metrics leverage program slices with respect to the output variables of a module to quantify the strength of functional relatedness of the elements within the module. Although slice-based cohesion metrics have been proposed for many years, few empirical studies have been conducted to examine their actual usefulness in predicting fault-proneness. Objective . We aim to provide an in-depth understanding of the ability of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction, i.e. their effectiveness in helping practitioners find post-release faults when taking into account the effort needed to test or inspect the code. Method . We use the most commonly used code and process metrics, including size, structural complexity, Halstead's software science, and code churn metrics, as the baseline metrics. First, we employ principal component analysis to analyze the relationships between slice-based cohesion metrics and the baseline metrics. Then, we use univariate prediction models to investigate the correlations between slice-based cohesion metrics and post-release fault-proneness. Finally, we build multivariate prediction models to examine the effectiveness of slice-based cohesion metrics in effort-aware post-release fault-proneness prediction when used alone or used together with the baseline code and process metrics. Results . Based on open-source software systems, our results show that: 1) slice-based cohesion metrics are not redundant with respect to the baseline code and process metrics; 2) most slice-based cohesion metrics are significantly negatively related to post-release fault-proneness; 3) slice-based cohesion metrics in general do not outperform the baseline metrics when predicting post-release fault-proneness; and 4) when used with the baseline metrics together, however, slice-based cohesion metrics can produce a statistically significant and practically important improvement of the effectiveness in effort-aware post-release fault-proneness prediction. Conclusion . Slice-based cohesion metrics are complementary to the most commonly used code and process metrics and are of practical value in the context of effort-aware post-release fault-proneness prediction.

Proactive Software Engineering Approach to Ensure Rapid Software Development and Scalable Production with Limited Resources

Nowadays, the need for building scalable systems in narrow time window is needed. While the efforts and accuracy usually required for building high scale systems is not simple, the agile nature of system requirements spawn a need for enhancing some software engineering practices. These practices should be integrated together in order to help software (SW) development teams to build, and test scalable systems rapidly with a high confidence level in their scalability. This research explains the proposed Proactive Approach, which presents a set of software engineering practices that could help in producing scalable system while minimizing the wasted time within the production cycle. This set of practices have been validated, verified and tested through building 46 releases of one of the most important, mission critical and scalable systems. Applying these practices succeeded to enhance average response time of web pages by %1921.5, test code churn by more than % 5000, time to release by % 300, and succeeded to produce a system that could stand against 95375 users with % 99.921 scalability ratio.

Design Churn as Predictor of Vulnerabilities?

This paper evaluates a metric suite to predict vulnerable Java classes based on how much the design of an application has changed over time. It refers to this concept as design churn in analogy with code churn. Based on a validation on 10 Android applications, it shows that several design churn metrics are in fact significantly associated with vulnerabilities. When used to build a prediction model, the metrics yield an average precision of 0.71 and an average recall of 0.27.

Defect-Density Assessment in Evolutionary Product Development: A Case Study in Medical Imaging

Defect density is the ratio between the number of defects and software size. Properly assessing defect density in evolutionary product development requires a strong tool and rigid process support that enables defects to be traced to the offending source code. In addition, it requires waiting for field defects after the product is deployed. To ease the calculation in practice, a proposed method approximates the lifetime number of defects against the software by the number of defects reported in a development period even if the defects are reported against previous product releases. The method uses aggregated code churn to measure the software size. It was applied to two development projects in medical imaging that involved three geographical locations (sites) with about 30 software engineers and 1.354 million lines of code in the released products. The results suggest the approach has some merits and validity, which the authors discuss in the distributed development context. The method is simple and operable and can be used by others with situations similar to ours.

Interactive churn metrics

A central part of software quality is finding bugs. One method of finding bugs is by measuring important aspects of the software product and the development process. In recent history, researchers have discovered evidence of a "code churn" effect whereby the degree to which a given source code file has changed over time is correlated with faults and vulnerabilities. Computing the code churn metric comes from counting source code differences in version control repositories. However, code churn does not take into account a critical factor of any software development team: the human factor, specifically who is making the changes. In this paper, we introduce a new class of human-centered metrics, "interactive churn metrics" as variants of code churn. Using the git blame tool, we identify the most recent developer who changed a given line of code in a file prior to a given revision. Then, for each line changed in a given revision, determined if the revision author was changing his or her own code ("self churn"), or the author was changing code last modified by somebody else ("interactive churn"). We derive and present several metrics from this concept. Finally, we conducted an empirical analysis of these metrics on the PHP programming language and its post-release vulnerabilities. We found that our interactive churn metrics are statistically correlated with post-release vulnerabilities and only weakly correlated with code churn metrics and source lines of code. The results indicate that interactive churn metrics are associated with software quality and are different from the code churn and source lines of code.

Can traditional fault prediction models be used for vulnerability prediction?

Finding security vulnerabilities requires a different mindset than finding general faults in software—thinking like an attacker. Therefore, security engineers looking to prioritize security inspection and testing efforts may be better served by a prediction model that indicates security vulnerabilities rather than faults. At the same time, faults and vulnerabilities have commonalities that may allow development teams to use traditional fault prediction models and metrics for vulnerability prediction. The goal of our study is to determine whether fault prediction models can be used for vulnerability prediction or if specialized vulnerability prediction models should be developed when both models are built with traditional metrics of complexity, code churn, and fault history. We have performed an empirical study on a widely-used, large open source project, the Mozilla Firefox web browser, where 21% of the source code files have faults and only 3% of the files have vulnerabilities. Both the fault prediction model and the vulnerability prediction model provide similar ability in vulnerability prediction across a wide range of classification thresholds. For example, the fault prediction model provided recall of 83% and precision of 11% at classification threshold 0.6 and the vulnerability prediction model provided recall of 83% and precision of 12% at classification threshold 0.5. Our results suggest that fault prediction models based upon traditional metrics can substitute for specialized vulnerability prediction models. However, both fault prediction and vulnerability prediction models require significant improvement to reduce false positives while providing high recall.

Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities

Security inspection and testing require experts in security who think like an attacker. Security experts need to know code locations on which to focus their testing and inspection efforts. Since vulnerabilities are rare occurrences, locating vulnerable code locations can be a challenging task. We investigated whether software metrics obtained from source code and development history are discriminative and predictive of vulnerable code locations. If so, security experts can use this prediction to prioritize security inspection and testing efforts. The metrics we investigated fall into three categories: complexity, code churn, and developer activity metrics. We performed two empirical case studies on large, widely used open-source projects: the Mozilla Firefox web browser and the Red Hat Enterprise Linux kernel. The results indicate that 24 of the 28 metrics collected are discriminative of vulnerabilities for both projects. The models using all three types of metrics together predicted over 80 percent of the known vulnerable files with less than 25 percent false positives for both projects. Compared to a random selection of files for inspection and testing, these models would have reduced the number of files and the number of lines of code to inspect or test by over 71 and 28 percent, respectively, for both projects.

Code churn estimation using organisational and code metrics: An experimental comparison

Context Source code revision control systems contain vast amounts of data that can be exploited for various purposes. For example, the data can be used as a base for estimating future code maintenance effort in order to plan software maintenance activities. Previous work has extensively studied the use of metrics extracted from object-oriented source code to estimate future coding effort. In comparison, the use of other types of metrics for this purpose has received significantly less attention. Objective This paper applies machine learning techniques to unveil predictors of yearly cumulative code churn of software projects on the basis of metrics extracted from revision control systems. Method The study is based on a collection of object-oriented code metrics, XML code metrics, and organisational metrics. Several models are constructed with different subsets of these metrics. The predictive power of these models is analysed based on a dataset extracted from eight open-source projects. Results The study shows that a code churn estimation model built purely with organisational metrics is superior to one built purely with code metrics. However, a combined model provides the highest predictive power. Conclusion The results suggest that code metrics in general, and XML metrics in particular, are complementary to organisational metrics for the purpose of estimating code churn.

Predicting the maintainability of XSL transformations

XSLT is a popular language for implementing both presentation templates in Web applications as well as document and message converters in enterprise applications. The widespread adoption and popularity of XSLT raises the challenge of efficiently managing the evolution of significant amounts of XSLT code. This challenge calls for guidelines and tool support for developing maintainable XSLT code. In this setting, this paper addresses the following question: Can the maintainability of XSL transformations, measured in terms of code churn in the next revision of a transformation, be predicted using a combination of simple metrics? This question is studied using a dataset extracted from open-source software project repositories. An outcome of this empirical study is a set of statistical models for predicting the maintainability of XSL transformations with relatively high accuracy. In addition, by analyzing the major influencers of code churn in these models, the paper identifies guidelines for designing XSL transformations with reduced future churn.

Does distributed development affect software quality?

Existing literature on distributed development in software engineering, and other fields discusses various challenges, including cultural barriers, expertise transfer difficulties, and communication and coordination overhead. Conventional wisdom, in fact, holds that distributed software development is riskier and more challenging than collocated development. We revisit this belief, empirically studying the overall development of Windows Vista and comparing the post-release failures of components that were developed in a distributed fashion with those that were developed by collocated teams. We found a negligible difference in failures. This difference becomes even less significant when controlling for the number of developers working on a binary. Furthermore, we also found that component characteristics (such as code churn, complexity, dependency information, and test code coverage) differ very little between distributed and collocated components. Finally, we examine the software process used during the Vista development cycle and examine how it may have mitigated some of the difficulties of distributed development introduced in prior work in this area.