Program Repair Research Articles

Multi-mechanism neural machine translation framework for automatic program repair

Automatic program repair (APR) is crucial to improve software quality. Recently, neural machine translation (NMT) based modeling for bug fixes has demonstrated great potential. However, these approaches still have two major challenges. One is that their search space is limited due to the out-of-vocabulary (OOV) problem. The other is that the NMT-based APR models tend to ignore past translation information, which often leads to over-translation and under-translation. To address the above challenges, we propose MNRepair, a new NMT-based APR approach that combines multiple mechanisms to fix bugs in source code. Specifically, we devise an encoder-decoder NMT framework with the attention mechanism. Our framework combines the copy mechanism to overcome the OOV problem that occurs with source code. To deal with the over-translation and under-translation, we utilize a coverage mechanism to record past translation information. MNRepair is able to capture a wide range of repair operators and fix 26 bugs in Defects4J. Our evaluation shows the effectiveness of multiple mechanisms in the repair process.

Exploring the Potential of Pre-Trained Language Models of Code for Automated Program Repair

In the realm of software development, automated program repair (APR) emerges as a pivotal technique, autonomously debugging faulty code to boost productivity. Despite the notable advancements of large pre-trained language models of code (PLMCs) in code generation, their efficacy in complex tasks like APR remains suboptimal. This limitation is attributed to the generic development of PLMCs, whose specialized potential for APR is yet be to fully explored. In this paper, we propose a novel approach designed to enhance PLMCs’ APR performance through source code augmentation and curriculum learning. Our approach employs code augmentation operators to generate a spectrum of syntactically varied yet semantically congruent bug-fixing programs, thus enriching the dataset’s diversity. Furthermore, we design a curriculum learning strategy, enabling PLMCs to develop a deep understanding of program semantics from these enriched code variants, thereby refining their APR fine-tuning prowess. We apply our approach across different PLMCs and systematically evaluate it on three benchmarks: BFP-small, BFP-medium, and Defects4J. The experimental results show that our approach outperforms both original models and existing baseline methods, demonstrating the promising future of adapting PLMCs for code debugging in practice.

SourcererJBF: A Java Build Framework For Large-Scale Compilation

Researchers and tool developers working on dynamic analysis, software testing, automated program repair, verification, and validation, need large compiled, compilable, and executable code corpora to test their ideas. The publicly available corpora are relatively small, and/or non-compilable, and/or non-executable. Developing a compiled code corpus is a laborious activity demanding significant manual effort and human intervention. To facilitate large-scale program analysis research, we develop SourcererJBF , a J ava B uild F ramework that can automatically build a large Java code corpus without project-specific instructions and human intervention. To generate a compiled code corpus, SourcererJBF creates an offline knowledge base by collecting external dependencies from the project directories and existing build scripts (if available). It constructs indices of those collected external dependencies that enable a fast search for resolving dependencies during the project compilation. As the output of the large-scale compilation, it produces JAigantic, a compilable Java corpus containing compiled projects, their bytecode, dependencies, normalized build script, and build command. We evaluated SourcererJBF’s effectiveness, correctness, performance, and scalability in a large collection of Java projects. Our experimental results demonstrate that SourcererJBF is significantly effective and scalable in building large Java code corpus. Besides, it substantiates reasonable performance and correctness similar to projects’ existing build systems.

Improving Automated Program Repair with Domain Adaptation

Automated Program Repair (APR) is defined as the process of fixing a bug/defect in the source code, by an automated tool. APR tools have recently experienced promising results by leveraging state-of-the-art Neural Language Processing (NLP) techniques. APR tools such as TFix and CodeXGLUE that combine text-to-text transformers with software-specific techniques are outperforming alternatives, these days. However, in most APR studies, the train and test sets are chosen from the same set of projects (i.e., when APR fixes a bug in the test set from project A, the model has already seen example fixed bugs from project A in the training set). In the real world, however, APR models are meant to be generalizable to new and different projects. Therefore, there is a potential threat that reported APR models with high effectiveness perform poorly when the characteristics of the new project or its bugs are different than the training set’s (“Domain Shift”). In this study, we first define the problem of domain shift in automated program repair. Next, we measure the potential damage of domain shift on two recent APR models (TFix and CodeXGLUE). Based on this observation, we then propose a domain adaptation framework that can adapt an APR model for a given target project. We conduct an empirical study with three domain adaptation methods FullFineTuning , TuningWithLightWeightAdapterLayers , and CurriculumLearning and two APR models on 2,672 bugs from 12 projects. The results show that our proposed framework on average can improve the effectiveness of TFix by 13.05% and CodeXGLUE by 48.78%, in terms of “Exact Match”. Through experiments, we also show that the framework provides high efficiency and reliability (in terms of “Exposure Bias”). Using synthetic data to domain adapt TFix and CodeXGLUE on the projects with no data (Zero-shot learning), also results in an average improvement of 5.76% and 17.62% for TFix and CodeXGLUE, respectively.

A method to identify overfitting program repair patches based on expression tree

The primary aim of Automatic Program Repair (APR) is to automatically repair defective programs, with the intention of reducing the amount of effort required by developers. However, APR techniques may produce overfitting patches that do not truly repair the program, allowing the program to pass all test cases. This paper provides a comprehensive review of the overfitting problem and adds to the existing research on overfitting in conditional statements. Our proposed method, ETPAT (Expression Tree-based Patch Assessment Technique), implements expression trees and targeted coverage criteria to identify differences between the original and the patched program. We utilize ETPAT to verify test case adequacy. In parallel, ETPAT also guides the generation of corresponding test cases via equivalence class information, which may be added to the original test suite, making it more robust while also preventing the repair technique from generating comparable overfitting patches. With reference to the patch set in the BuggyJavaJML benchmark, ETPAT recognized 77/82 (93.9%) overfitting patches out of 120 patches related to conditional constraints, displaying superior accuracy rates and fewer test cases required than the original repair tool.

Accelerating Patch Validation for Program Repair With Interception-Based Execution Scheduling

Accelerating Patch Validation for Program Repair With Interception-Based Execution Scheduling

Automatic program bug fixing by focusing on finding the shortest sequence of changes

Automatic bug repair as the last step in program repair has attracted a lot of research attention. Various ideas and techniques have been presented in this field. Recent bug fixing techniques use machine learning and graphs to generate fixes. Despite the promising results of recent approaches, maintaining high speed and accuracy as well as recording a wide range of errors may still be a problem. In this paper, a new approach is presented in the field of automatic bug fixing based on graphs and model checking. For this purpose, we have used the graph transformation and model checking system to create a sequence of edits and produce fixes. Then, using meta-heuristic algorithms, we have selected the best solution and fix from the generated solutions. We use the extracted graphs from the buggy JavaScript code and their corresponding bug-free ones. In evaluating the effectiveness of the proposed method, we implement it in GROOVE, which is a toolbox used to design and check graph transformation systems. Experimental results on identical dataset demonstrate that the proposed method outperforms other related methods in generating fixes. Also, this method covers a wider range of bugs compared to previous methods.

Automated Program Repair for Introductory Programming Assignments

Automated Program Repair for Introductory Programming Assignments

A Survey of Learning-based Automated Program Repair

Automated program repair (APR) aims to fix software bugs automatically and plays a crucial role in software development and maintenance. With the recent advances in deep learning (DL), an increasing number of APR techniques have been proposed to leverage neural networks to learn bug-fixing patterns from massive open-source code repositories. Such learning-based techniques usually treat APR as a neural machine translation (NMT) task, where buggy code snippets (i.e., source language) are translated into fixed code snippets (i.e., target language) automatically. Benefiting from the powerful capability of DL to learn hidden relationships from previous bug-fixing datasets, learning-based APR techniques have achieved remarkable performance. In this article, we provide a systematic survey to summarize the current state-of-the-art research in the learning-based APR community. We illustrate the general workflow of learning-based APR techniques and detail the crucial components, including fault localization, patch generation, patch ranking, patch validation, and patch correctness phases. We then discuss the widely adopted datasets and evaluation metrics and outline existing empirical studies. We discuss several critical aspects of learning-based APR techniques, such as repair domains, industrial deployment, and the open science issue. We highlight several practical guidelines on applying DL techniques for future APR studies, such as exploring explainable patch generation and utilizing code features. Overall, our article can help researchers gain a comprehensive understanding about the achievements of the existing learning-based APR techniques and promote the practical application of these techniques. Our artifacts are publicly available at the repository: https://github.com/iSEngLab/AwesomeLearningAPR .

Poracle: Testing Patches under Preservation Conditions to Combat the Overfitting Problem of Program Repair

To date, the users of test-driven program repair tools suffer from the overfitting problem; a generated patch may pass all available tests without being correct. In the existing work, users are treated as merely passive consumers of the tests. However, what if they are willing to modify the test to better assess the patches obtained from a repair tool? In this work, we propose a novel semi-automatic patch-classification methodology named Poracle . Our key contributions are three-fold. First, we design a novel lightweight specification method that reuses the existing test. Specifically, the users extend the existing failing test with a preservation condition —the condition under which the patched and pre-patched versions should produce the same output. Second, we develop a fuzzer that performs differential fuzzing with a test containing a preservation condition. Once we find an input that satisfies a specified preservation condition but produces different outputs between the patched and pre-patched versions, we classify the patch as incorrect with high confidence. We show that our approach is more effective than the four state-of-the-art patch classification approaches. Last, we show through a user study that the users find our semi-automatic patch assessment method more effective and preferable than the manual assessment.

Variable-based Fault Localization via Enhanced Decision Tree

Fault localization, aiming at localizing the root cause of the bug under repair, has been a longstanding research topic. Although many approaches have been proposed in past decades, most of the existing studies work at coarse-grained statement or method levels with very limited insights about how to repair the bug ( granularity problem ), but few studies target the finer-grained fault localization. In this article, we target the granularity problem and propose a novel finer-grained variable-level fault localization technique. Specifically, the basic idea of our approach is that fault-relevant variables may exhibit different values in failed and passed test runs, and variables that have higher discrimination ability have a larger possibility to be the root causes of the failure. Based on this, we propose a program-dependency-enhanced decision tree model to boost the identification of fault-relevant variables via discriminating failed and passed test cases based on the variable values. To evaluate the effectiveness of our approach, we have implemented it in a tool called VarDT and conducted an extensive study over the Defects4J benchmark. The results show that VarDT outperforms the state-of-the-art fault localization approaches with at least 268.4% improvement in terms of bugs located at Top-1, and the average improvement is 351.3%. Besides, to investigate whether our finer-grained fault localization result can further improve the effectiveness of downstream APR techniques, we have adapted VarDT to the application of patch filtering, where we use the variables located by VarDT to filter incorrect patches. The results denote that VarDT outperforms the state-of-the-art PATCH-SIM and BATS by filtering 14.8% and 181.8% more incorrect patches, respectively, demonstrating the effectiveness of our approach. It also provides a new way of thinking for improving automatic program repair techniques.

Extending the range of bugs that automated program repair can handle

Modern automated program repair (APR) is well-tuned to finding and repairing bugs that introduce observable erroneous behavior to a program. However, a significant class of bugs does not lead to observable behavior (e.g., termination bugs and non-functional bugs). Such bugs can generally not be handled with current APR approaches, so complementary techniques are needed. To stimulate the systematic study of alternative approaches and hybrid combinations, we devise a novel bug classification system that enables methodical analysis of their bug detection power and bug repair capabilities. To demonstrate the benefits, we study the repair of termination bugs in sequential and concurrent programs. Our analysis shows that integrating dynamic APR with formal analysis techniques, such as termination provers and software model checkers, reduces complexity and improves the overall reliability of these repairs. We empirically investigate how well the hybrid approach can repair termination and performance bugs by experimenting with hybrids that integrate different APR approaches with termination provers and execution time monitors. Our findings indicate that hybrid repair holds promise for handling termination and performance bugs. However, the capability of the chosen tools and the completeness of the available correctness specification affects the quality of the patches that can be produced.

APR4Vul: an empirical study of automatic program repair techniques on real-world Java vulnerabilities

Security vulnerability fixes could be a promising research avenue for Automated Program Repair (APR) techniques. In recent years, APR tools have been thoroughly developed for fixing generic bugs. However, the area is still relatively unexplored when it comes to fixing security bugs or vulnerabilities. In this paper, we evaluate nine state-of-the-art APR tools and one vulnerability-specific repair tool. In particular, we investigate their ability to generate patches for 79 real-world Java vulnerabilities in the Vul4J dataset, as well as the level of trustworthiness of these patches. We evaluate the tools with respect to their ability to generate security patches that are (i) testable, (ii) having the positive effect of closing the vulnerability, and (iii) not having side effects from a functional point of view. Our results show that the evaluated APR tools were able to generate testable patches for around 20% of the considered vulnerabilities. On average, nearly 73% of the testable patches indeed eliminate the vulnerabilities, but only 44% of them could actually fix security bugs while maintaining the functionalities. To understand the root cause of this phenomenon, we conduct a detailed comparative study of the general bug fix patterns in Defect4J and the vulnerability fix patterns in ExtraVul (which we extend from Vul4J). Our investigation shows that, although security patches are short in terms of lines of code, they contain unique characteristics in their fix patterns compared to general bugs. For example, many security fixes require adding method calls. These method calls contain specific input validation-related keywords, such as encode, normalize, and trim. In this regard, our study suggests that additional repair patterns should be implemented for existing APR tools to fix more types of security vulnerabilities.

Multi-View Feature Fusion Model for Software Bug Repair Pattern Prediction

Many search-based Automatic Program Repair (APR) techniques employ a set of repair patterns to generate candidate patches. Regarding repair pattern selection, existing search-based APR techniques either randomly select a repair pattern from the repair pattern set to apply or prioritize all repair patterns based on the bug's context information. In this paper, we introduce PatternNet, a multi-view feature fusion model capable of predicting the repair pattern for a reported software bug. To accomplish this task, PatternNet first extracts multi-view features from the pair of buggy code and bug report using different models. Specifically, a transformer-based model (i.e., UniXcoder) is utilized to obtain the bimodal feature representation of the buggy code and bug report. Additionally, an Abstract Syntax Tree (AST)-based neural model (i.e., ASTNN) is employed to learn the feature representation of the buggy code. Second, a co-attention mechanism is adopted to capture the dependencies between the statement trees in the AST of the buggy code and the textual tokens of the reported bug, resulting in co-attentive features between statement trees and reported bug's textual tokens. Finally, these multi-view features are combined into a unified representation using a feature fusion network. We quantitatively demonstrate the effectiveness of PatternNet and the feature fusion network for predicting software bug repair patterns.

Faire: Repairing Fairness of Neural Networks via Neuron Condition Synthesis

Deep Neural Networks (DNNs) have achieved tremendous success in many applications, while it has been demonstrated that DNNs can exhibit some undesirable behaviors on concerns such as robustness, privacy, and other trustworthiness issues. Among them, fairness (i.e., non-discrimination) is one important property, especially when they are applied to some sensitive applications (e.g., finance and employment). However, DNNs easily learn spurious correlations between protected attributes (e.g., age, gender, race) and the classification task and develop discriminatory behaviors if the training data is imbalanced. Such discriminatory decisions in sensitive applications would introduce severe social impacts. To expose potential discrimination problems in DNNs before putting them in use, some testing techniques have been proposed to identify the discriminatory instances (i.e., instances that show defined discrimination 1 ). However, how to repair DNNs after detecting such discrimination is still challenging. Existing techniques mainly rely on retraining on a large number of discriminatory instances generated by testing methods, which requires huge time overhead and makes the repairing inefficient. In this work, we propose the method Faire to effectively and efficiently repair the fairness issues of DNNs, without using additional data (e.g., discriminatory instances). Our basic idea is inspired by the traditional program repair method that synthesizes proper condition checking. To repair traditional programs, a typical method is to localize the program defects and repair the program logic by adding condition checking. Similarly, for DNNs, we try to understand the unfair logic and reformulate it with well-designed condition checking. In this article, we synthesize the condition that can reduce the effect of features relevant to the protected attributes in the DNN. Specifically, we first perform the neuron-based analysis and check the functionalities of neurons to identify neurons whose outputs could be regarded as features relevant to protected attributes and original tasks. Then a new condition layer is added after each hidden layer to penalize neurons that are accountable for the protected features (i.e., intermediate features relevant to protected attributes) and promote neurons that are accountable for the non-protected features (i.e., intermediate features relevant to original tasks). In sum, the repair rate 2 of Faire reaches up to more than 99%, which outperforms other methods, and the whole repairing process only takes no more than 340 s. The evaluation results demonstrate that our approach can effectively and efficiently repair the individual discriminatory instances of the target model.

Automatic program repair: A systematic literature review

Automatic Program Repair works by generating patches automatically to fix software bugs. This domain is very popular among the software engineering researchers as it helps the developers to mitigate tiresome debugging effort and increase productivity. This domain mostly focuses on how to develop better program repair techniques. However, a systematic literature review is needed to help the researchers for getting specific and scientific overview of this domain. This paper conducts a systematic literature review on 324 papers for analyzing the trendiness and associated factors of this domain. After that, this paper considers 16 papers to thoroughly review and states the evolution of automatic program repair. Hopefully, this study will be a valuable resource for the researchers of this domain.

Enhanced evolutionary automated program repair by finer‐granularity ingredients and better search algorithms

SummaryBug repair is time consuming and tedious, which hampers software maintenance. To alleviate the burden, automated program repair (APR) is proposed and has been fruitful in the last decade. Evolutionary repair is the seminal work of this field and proliferated a family of approaches. The performance of evolutionary repair approaches is affected by two main factors: (1) search space, which defines all possible patches, and (2) search algorithms, which navigate the space. Although recent approaches have achieved remarkable progress, the main challenges of the two factors still remain. On one hand, the different kinds of search space are very coarse for containing correct patches. On the other hand, the search process guided by genetic algorithms is inefficient in finding the correct patches in an appropriate time budget. In this paper, we propose MicroRepair, a new evolutionary repair approach to address the two challenges. Rather than finding statement‐level patches like existing genetic repair approaches, MicroRepair enlarges the search space by breaking the statements into finer‐granularity ingredients that consist of AST leaves. As the search space grows exponentially, the former search algorithms may become inefficient in navigating the larger space. We utilize the best multiobjective search algorithm selected from our empirical comparison of a set of search algorithms. At last, we find redundancies search in the existing genetic process, and we further design a history‐aware search strategy to accelerate the process.We evaluated MicroRepair on 224 bugs of real‐world from the benchmark Defects4J and compared it with several state‐of‐the‐art repair approaches. The evaluation results show that MicroRepair correctly repaired 26 bugs with a precision of 62%, which significantly outperforms the state‐of‐the‐art evolutionary APR approaches in terms of precision. Moreover, the history‐aware search boosts the repair execution speed by 4% on average.

Mining on Students’ Execution Logs and Repairing Compilation Errors Based on Deep Learning

Automatic program repair techniques based on deep neural networks have attracted widespread attention from researchers due to the high degree of automation and generality. However, there is a scarcity of high-quality labeled datasets available for training program repair models. This study proposes a method of mining reasonable program repair examples from student program execution logs. Additionally, we introduce the Rookie Simulator (RS), which simulates the error patterns commonly made by novice programmers and generates a large number of program repair sample pairs. To address the issue of low repair rates for infrequent and complex error patterns in compilation errors, the study proposes the attention-enhanced capsule network for program repair (ACNPR), a program repair model that integrates compiler feedback information and utilizes capsule networks to capture complex semantic features. Experimental evaluations were conducted using publicly available datasets, including the DeepFix, TEGCER, and a real course dataset named SUES-COJ mined in this study. The results indicate that our method consistently outperforms current state-of-the-art models in terms of full repair rates.

멀티 청크 버그 정정을 위한 버기 블록 대상 프로그램 정정 최적화 적용

멀티 청크 버그 정정을 위한 버기 블록 대상 프로그램 정정 최적화 적용

Doctor Code: A Machine Learning-Based Approach to Program Repair

Doctor Code: A Machine Learning-Based Approach to Program Repair