A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

S Reine De Reanzi,P Ranjit Jeba Thangaiah

doi:10.1177/18479790211062044

S Reine De Reanzi, P Ranjit Jeba Thangaiah

Open Access

https://doi.org/10.1177/18479790211062044

Copy DOI

Abstract

Software industry has adopted automated testing widely. The most common method adopted is graphical user interface test automation for the functional scenarios to reduce manual testing and increase the repeatability. Reducing execution time and redundancy to achieve quality “go/no go decisions” provides rational for the executive management to allocate funds to adopt automation and invest in the setup including people, process, and tools to achieve faster time to market. There are a variety of practices engaged by testers, like frameworks, tools, methods, procedures, models, and technologies to achieve automation. Nonetheless, the actual effectiveness in terms of return on investment (ROI) is not known, though there are various formulas to calculate ROI of test automation. The factors that determine the ROI are maturity of test automation, purpose, or intent, picking right tests for automation, knowledge of the tester to derive test coverage, domain expertise, defining right metrics like defects found by automation runs, cost versus reduction of time, labor versus quality of scripts, repeatability, and results. These factors form the base of the questions designed for the survey. The paper presents a survey and analysis to understand the ROI of test automation from industry test professionals from both product and services organizations.

Highlights

Executives in the commercial IT industry strive to automate all test cases to save time, get repeatability, identify defects early, increase quality, and cut cost.[1]
Estimation helps to justify the time and material spent on the automation suite with respect to quality.[106,107,108,109,110] derived from the primary data to correlate unit test coverage to functional coverage, automation return on investment (ROI) from the perspective of industry test professionals who work for product organizations versus services organizations
In the real-world scenario, the services industry mostly deploys their professionals in the product organizations for aiding the test automation in the product organizations as that is the business model on which they operate

Summary

Introduction

Executives in the commercial IT industry strive to automate all test cases to save time, get repeatability, identify defects early, increase quality, and cut cost.[1]. From the data collected, the expectations of the services and the product industry on ROI of automation are illustrated using correlation. Estimation helps to justify the time and material spent on the automation suite with respect to quality.[106,107,108,109,110] derived from the primary data to correlate unit test coverage to functional coverage, automation ROI from the perspective of industry test professionals who work for product organizations versus services organizations.

Results

Conclusion