Empirical Comparison of Search Heuristics for Genetic Improvement of Software

Abstract

Genetic improvement (GI) uses automated search to improve existing software. It has been successfully used to optimize various program properties, such as runtime or energy consumption, as well as for the purpose of bug fixing. GI typically navigates a space of thousands of patches in search for the program mutation that best improves the desired software property. While genetic programming (GP) has been dominantly used as the search strategy, more recently other search strategies, such as local search, have been tried. It is, however, still unclear which strategy is the most effective and efficient. In this article, we conduct an in-depth empirical comparison of a total of 18 search processes using a set of eight improvement scenarios. Additionally, we also provide new GI benchmarks and we report on new software patches found. Our results show that, overall, local search approaches achieve better effectiveness and efficiency than GP approaches. Moreover, improvements were found in all scenarios (between 15% and 68%). A replication package can be found online: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/bloa/tevc_2020_artefact</uri> .