Controlling the Generation of Multiple Counterexamples in LTL Model Checking

Abstract

The focus of traditional model checking has been on the verification problem where counterexamples play a secondary role. In many potential uses of model checkers, however, counterexamples play a primary role. For example, in safety analysis, achieving perfect safety in the system being analysed is often impossible or too expensive. In such a case, the analyst is interested in discovering all of the situations that can lead to unsafe conditions in order to assess their likelihood. These situations appear as counterexamples to a system safety property expressed as a temporal logic formula in model checking. This thesis proposes an approach to model checking when counterexample generation is the primary goal. Model checking is viewed as a search for counterexamples rather than simply ensuring that a specification is satisfied by a model. The temporal logic used is Linear Temporal Logic (LTL). Most existing model checkers stop after the first counterexample is found. The few that can generate multiple counterexample paths typically generate too many counterexample paths that are slight variations of each other. For LTL, a counterexample path is an infinite sequence of states, and the number of counterexample paths for a model checking problem can be infinite. Typically, the analyst is interested in a finite number of classes of counterexample, with each class represented by a single counterexample path. However, the classes of interest are often specific to the problem domain. An approach explored in this thesis is to control the generation of counterexample paths by allowing the analyst to direct the search for a counterexample path to rule in or rule out certain classes of counterexamples. The counterexample paths generated are of the so-called lasso form, each consisting of a prefix part (a possibly empty finite sequence of states) and a cycle part (a non-empty finite sequence of states that is repeated forever). The main technique proposed for controlled generation of counterexamples within a symbolic framework is called directed counterexample generation. The search for a counterexample path is directed using two kinds of constraints: a global constraint which is a state property that must be satisfied by all states in the counterexample path, and a cycle constraint which is a state property that must be satisfied by at least one state in the cycle part of a counterexample path. While global constraints can be easily integrated with existing techniques for counterexample path generation, cycle constraints entail a search technique different from the existing techniques. As well as controlling the generation of multiple counterexample paths, the use of constraints can greatly reduce the search space in generating individual counterexample paths. The framework together with directed counterexample generation provide an infrastructure for exploring the counterexample space in a model checking problem. Model checking, and thus counterexample generation, suffers from the state explosion problem. Although many techniques have been developed to mitigate the state explosion problem in model checking, including symbolic model checking, no best single combination of techniques for model checking and counterexample generation has been found and it is unlikely that one will be found, since the state explosion problem cannot in general be eliminated. The approach in this thesis is to develop a framework for LTL model checking and counterexample generation where different techniques and strategies can be mixed and matched, including fixpoint and on-the- y techniques. The proposed framework is intended to support the analysis of finite-state asynchronous systems with interleaving semantics. The framework is independent of the modelling notation, but the Behavior Tree (BT) notation is used as a concrete example notation for modelling finite-state asynchronous systems. A method for translating models in a substantial subset of the BT notation into objects in the framework is provided. As a proof of concept, a prototype that incorporates the proposed techniques within the framework has been developed. The prototype includes a translator from the BT notation. Experiments with the prototype were performed to demonstrate the advantages of the proposed approach and assess the effects of model checking strategies on directed counterexample generation.