The evolutionary landscape of SBST: a 10 year perspective

Abstract

A key indicator of the health and quality of any evolutionary algorithm is the landscape of its search. By analyzing the landscape one can determine the peaks (local maxima) where significant solutions exist. In this paper we examine the landscape for the history of the International Workshop on Search-Based Software Testing (SBST) within the context of the broader field of search-based software testing. We study the evolution of the field, highlighting key advances during three phases of its ten year history. In 2008 the focus of SBST was inner looking, with advances in existing search techniques, improvements to individual generation techniques, and methods to transform the problem space for search effectiveness. However, diverse seeds of new ideas (such as automated program repair) were already being injected into the population. A few SBST tools existed, but the engineer still required skill and expertise to effectively apply search based approaches. During the middle years, open source tools were created and released, whole test suite generation appeared, and searches hybridized. Tool competitions began and industry started to play a stronger role. As we move to the most recent workshop years and look towards the future, more sophisticated techniques such as those that incorporate hyper-heuristics via learning, and/or balance multiple objectives at once are now common. SBST has become a mainstream topic in the testing community, tools are being commercialized and these tools often hide their inner workings, leading to a future that is optimized towards SBST for all.