Modification Point Aware Test Prioritization and Sampling to Improve Patch Validation in Automatic Program Repair

Yazhini Venugopal,Lee Eunseok,Phung Quang-Ngoc

doi:10.3390/app10051593

Abstract

Recently, Automatic Program Repair (APR) has shown a high capability of repairing software bugs automatically. In general, most of the APR techniques require test suites to validate automatically generated patches. However, the test suites used for patch validation might contain thousands of test cases. Running these whole test suites to validate every program variant makes the validation process not only time-consuming but also expensive. To mitigate this issue and to enhance the patch validation in APR, we introduce (1) MPTPS (Modification Point-aware Test Prioritization and Sampling), which iteratively records test execution. Based on the failed test information, it performs test prioritization, then sampling to reduce the test execution time by moving forward the test cases that are most likely to fail in the test suite; and (2) a new fitness function that refines the existing one to improve repair efficiency. We implemented our MPPEngine approach in the Astor workspace by extending jGenProg. And the experiments on the Defects4j benchmark against jGenProg show that, on average, jGenProg consumes 79.27 s to validate one program variant, where MPPEngine takes only 33.70 s for results in 57.50% of validation time reduction. Also, MPPEngine outperforms jGenProg by finding patches for six more bugs than jGenProg.

Highlights

The increase in software complexity often results in high debugging and maintenance costs.A recent study on software depicted that the debugging process of the complex software is time-consuming and tedious
We investigated the effectiveness of MPPEngine against jGenProg in terms of reducing test execution time and cost
As per the experiments conducted among other tools in Astor [11,12,13], jGenProg generated patches faster using test suites

Summary

Introduction

The increase in software complexity often results in high debugging and maintenance costs.A recent study on software depicted that the debugging process of the complex software is time-consuming and tedious. The developer or tester finds the fault(s) in the program and fixes it manually. It requires time, effort, and manpower to find and fix issues in the software. Automatic Program Repair (APR) is a technique that automatically generates patches to fix the buggy program. APR tools have three main phases: Fault Localization, Program Modification, and Program Validation. The Fault localization tool takes Faulty Program and Repair Tests as inputs and automatically finds suspicious statements in the program. Candidate solutions are generated based on the given number of generations or time limit These generated candidate solutions are applied to the buggy program to produce patches.

Methods

Results

Conclusion