How much testing is enough? Applying stopping rules to behavioral model testing

Abstract

Testing behavioral models before they are released to the synthesis and logic design phase is a tedious process, to say the least. A common practice is the test-it-to-death approach in which millions or even billions of vectors are applied and the results are checked for possible bugs. The vectors applied to behavioral models include functional vectors, but the significant amount of the vectors are random in nature, including random combinations of instructions. In this paper, we present and evaluate a stopping rule that can be used to determine when to stop the current testing phase using a given testing technique. We demonstrate the use of the stopping rule on two complex VHDL models that were tested for branch coverage with 4 different testing phases. We compare savings and quality of testing both with and without using the stopping rule.