Formal methods for software testing (invited paper)

Abstract

This extended abstract takes advantage of a theory of software testing based on formal specifications to point out the benefits and limits of the use of formal methods to this end. A notion of exhaustive test set is defined according to the semantics of the formal notation, the considered conformance relation, and some testability hypotheses on the system under test. This gives a framework for the formalisation of test selection, test execution, and oracles, and, moreover, leads to the explicitation of those hypotheses underlying test selection strategies, such as uniformity hypotheses or regularity hypotheses. This explicitation provides some guides to complementary proofs, or tests, or instrumentations of the system under test. This approach has been applied to various formalisms: axiomatic specifications of data types, model-based specifications, process algebras, transition systems, etc. It provides some guiding principles for the development of testing methods given a formal specification notation and an associated conformance/refinement relation. It is at the origin of the development of some test environments based on SMT solvers and theorem provers.