Should fixing these failures be delegated to automated program repair?

Abstract

Program repair constitutes one of the major components of software maintenance that usually incurs a significant cost in software production. Automated program repair is supposed to help in reducing the software maintenance cost by automatically fixing software defects. Despite the recent advances in automated software repair, it is still very costly to wait for repair tools to produce valid repairs of defects. This paper addresses the following question: Will an automated program repair technique find a repair for a defect within a reasonable time?. To answer this question, we build an oracle that can predict whether fixing a failure should be delegated to an automated repair technique. If the repair technique is predicted to take too long to produce a repair, the bug fixing process should rather be assigned to a developer or other appropriate techniques available. Our oracle is built for genetic-programming-based automated program repair approaches, which have recently received considerable attention due to their capability to automatically fix real-world bugs. These approaches search for a valid repair over a large number of variants that are syntactically mutated from the original program. At an early stage of running a repair tool, we extract a number of features that are potentially related to the effectiveness of the tool. Leveraging advances in machine learning, we process the values of these features to learn a discriminative model that is able to predict whether continuing a genetic programming search will lead to a repair within a desired time limit. We perform experiments to evaluate the ability of our approach to predict the effectiveness of GenProg, a well-known genetic-programming-based automated program repair approach, in fixing 105 real bugs. Our experiments show that our approach can identify effective cases from ineffective ones (i.e., bugs for which GenProg cannot produce correct fixes after a long period of time) with a precision, recall, F-measure, and AUC of 72%, 74%, 73%, and 76% respectively.