Abstract
A general theory introducing asymptotic complexity to testing is presented. Our goal is measuring how fast the effort of testing must increase to reach higher levels of partial certainty on the correctness of the implementation under test (IUT). By recent works it is known that, for many practical testing scenarios, any partial level of correctness certainty less than 1 (where 1 means full certainty) can be reached by some finite test suite. In this paper we address the problem of finding out how fast must these test suites grow as long as the target level gets closer to 1. More precisely, we want to study how test suites grow with α, where α is the inverse of the distance to 1 (e.g., if α=4 then our target level is 0.75=1−14). A general theory to measure this testing complexity is developed. We use this theory to analyze the testing complexity of some general testing problems, as well as the complexity of some specific testing strategies for these problems, and discover that they are within e.g., O(logα), O(log2α), O(α), O(αlogα), or O(α). Similarly as the computational complexity theory conceptually distinguishes between the complexity of problems and algorithms, tightly identifying the complexity of a testing problem will require reasoning about any testing strategy for the problem. The capability to identify testing complexities will provide testers with a measure of the productivity of testing, that is, a measure of the utility of applying the (n+1)-th planned test case (after having passed the n previous ones) in terms of how closer would that additional test case get us to the (ideal) complete certainty on the IUT (in-)correctness.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Journal of Logical and Algebraic Methods in Programming
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.