A Journey from Manual Testing to Automated Test Generation in an Industry Project

Abstract

Test automation is essential in fast-paced agile de-velopment environments. The main goal is to speed up test execu-tion cycles and to reduce the effort involved in running tests manually. We took test automation one step further and applied test generation to a GUI-based application developed in a large industry project. The paper describes the transition from manual exploratory testing to automated GUI test generation. Key les-sons to be learned are: (1) the test automation pyramid proposed for agile development tends to underestimate the need for high-level GUI testing, (2) automated test generation does not reduce test effort but shifts it to writing test adapters and checks, and (3) the effort for analyzing results produced by generated tests limits the practical application of automated test generation. The report describes the successful application of test generation in a real-world industry project, but it also highlights several open issues to be addressed by future research on test automation.