Introducing Automated GUI Testing and Observing Its Benefits: An Industrial Case Study in the Context of Law-Practice Management Software

Abstract

Motivated by a real-world industrial need in the context of a large IT solutions company based in Turkey, the authors and their colleagues developed and introduced automated test suites for GUI testing of two large-scale law-practice management software (comprising of 414 and 105 KLOC). We report in this paper our experience in developing and introducing a set of large automated test suites (more than 50 KLOC in total), using best practices in state-of-the art and –practice, and to report its observed benefits by conducting cost-benefit analysis in the specific industrial context. The project was conducted based on the principles of case-study and action research in which the real industrial needs drove the research. Among the best practices that we used are the followings: (1) the page-object test pattern, (2) modularity in test code, (3) creating test-specific libraries, and (4) using systematic guidelines to decide when and what (test cases) to automate. To assess the cost-benefit and Return On Investment (ROI) of test automation, we followed a hybrid measurement approach to assess both the quantitative and qualitative (intangible) benefits of test automation. The empirical findings showed that the automated GUI testing approach has indeed benefitted the test and QA team in the company under study and automation has been highly welcome by the test engineers. By serving as a success story and experience report in development and introduction of automated test suites in an industrial setting, this paper adds to the body of evidence in this area and it aims at sharing both technical (e.g., using automated test patterns) and process aspects (e.g., test process improvement) of our project with other practitioners and researchers with the hope of encouraging more industry-academia collaborations in test automation.