Test case selection-prioritization approach based on memoization dynamic programming algorithm

Abstract

In the software industry, selection and prioritization techniques become a necessity in the regression and validation testing phases because a lot of test cases are available for reuse, yet time and project specific constraints must be respected. In this paper we propose a dynamic programming approach in solving test case selection-prioritization problems. We focus on low memory consumption in pseudo-polynomial time complexity applicable in both selection and selection-prioritization problems over sets of test cases or test suites. In dynamic programming optimization solutions, huge amounts of memory are required and unfortunately the memory is limited. Therefore, lower memory consumption leads to a higher number of test cases to be involved in the selection process. Our approach is suited for medium to large projects where the required memory space is not higher than the order of tens of GBytes. We employed both objective methods as the dynamic programming algorithm and subjective and empiric human decision as defining the prioritization criteria. Furthermore, we propose a method of employing multiple project specific criteria in evaluating the importance of a test case in the project context. To evaluate the proposed solution relative to the classical dynamic programming knapsack solution, we developed a suite of comparative case studies based on 1000 generated scenarios as close as possible to real project scenarios. The results of the comparative study reported the proposed algorithm requires up to 400 times less memory in the best-case scenarios and about 40 times less memory in average. The solution delivers optimal results in pseudo-polynomial time complexity, is effective for amounts of test cases up to the order of millions and compared with the classical dynamic programming methods leads to higher number of test cases to be involved in the selection process due to reduced memory consumption.