Regression Test Suite Execution Time Analysis using Statistical Techniques

Abstract

After applying techniques such as test design generation using combinatorial approach, test case selection, test suite minimization and prioritization, further reduction in execution time of the test suite is possible. This is necessary when there is scarcity of resources and time to execute the test suites. We investigate the statistical techniques which can be employed for analysis and reduction of test execution time further. The approach can help in extrapolating the execution time of test suite when the regression team wants to augment the test suite. The statistical techniques can aid in choosing the best test setup in terms of operating system, tool and Java virtual machine combination for given test setup. Further, these techniques are not the activities that need to be carried out before every execution of the test suite. Once the best combination is chosen the same can be used unless there is change in one of the layers of test setup. The activities mentioned in the paper can be part of test setup planning and maintenance phase.