Coverage-Based Software Testing

Abstract

Abstract Code coverage is one of the core quality metrics adopted by software testing practitioners nowadays. Researchers have devised several coverage criteria that testers use to assess the quality of test suites. A coverage criterion operates by: (1) defining a set of test requirements that need to be satisfied by the given test suite and (2) computing the percentage of the satisfied requirements, thus yielding a quality metric that quantifies the potential adequacy of the test suite at revealing program defects. What differentiates one coverage criterion from another is the set of test requirements involved. For example, function coverage is concerned with whether every function in the program has been called, and statement coverage is concerned with whether every statement in the program has executed. The use of code coverage in testing is not restricted to assessing the quality of test suites. For example, researchers have devised test suite minimization and test case generation techniques that also leverage coverage. Early coverage-based software testing techniques involved basic test requirements such as functions, statements, branches, and predicates, whereas recent techniques involved (1) test requirements that are complex code constructs such as paths, program dependences, and information flows or (2) test requirements that are not necessarily code constructs such as program properties and user-defined test requirements. The focus of this chapter is to compare these two generations of techniques in regard to their effectiveness at revealing defects. The chapter will first present preliminary background and definitions and then describe impactful early coverage techniques followed by selected recent work.