Applying Design of Experiments to Software Testing

Abstract

Recently, a class of experimental designs has been devised that guarantee input domain coverage up to all combinations of k test factors taken t at a time. With such designs, all pair-wise combinations (or triplets or quadruplets, etc.) are selected at least once. To evaluate their applicability to software testing, we analyzed the extent to which software coverage (i.e., code execution) achieved by these designs for r = 1 ,...,k is representative of that achieved by exhaustively testing all factor combinations. The block coverage obtained for tS2 was comparable with that achieved by exhaustively testing all factor combinations but higher-order values of t were required for path coverage. Implications of these results for software testing are discussed. field, and that there is a total of 151 characters that can be entered across all 20 screen fields, then there are 95’5’~4.3x102g8 different inputs or test cases corresponding to the different ways a user can populate the screen. Clearly, exhaustive input testing is not feasible and hence, a software tester needs to efficiently generate an effective set of test cases as a means of verifying the correct operation of the software.