Counterexample Generation Research Articles

Specification and counterexample generation for cyber-physical systems

Specification and counterexample generation for cyber-physical systems

Optimization Techniques for Model Checking Leads-to Properties in a Stratified Way

We devised the L +1-layer divide & conquer approach to leads-to model checking ( L +1-DCA2L2MC) and its parallel version, and developed sequential and parallel tools for L +1-DCA2L2MC. In a temporal logic called UNITY , designed by Chandy and Misra, the leads-to temporal connective plays an important role and many case studies have been conducted in UNITY, demonstrating that many systems requirements can be expressed as leads-to properties. Hence, it is worth dedicating to these properties. Counterexample generation is one of the main tasks in the L +1-DCA2L2MC technique that can be optimized to improve its running performance. This article proposes a technique to find all counterexamples at once in model checking with a new model checker. Furthermore, layer configuration selection is essential to make the best use of the L +1-DCA2L2MC technique. This work also proposes an approach to finding a good layer configuration for the technique with an analysis tool. Some experiments are conducted to demonstrate the power and usefulness of the two optimization techniques, respectively. Moreover, our sequential and parallel tools are compared with SPIN and LTSmin model checkers, showing a promising way to mitigate the state space explosion and improve the running performance of model checking when dealing with large state spaces.

Formal Synthesis of Neural Barrier Certificates for Continuous Systems via Counterexample Guided Learning

This paper presents a novel approach to safety verification based on neural barrier certificates synthesis for continuous dynamical systems. We construct the synthesis framework as an inductive loop between a Learner and a Verifier based on barrier certificate learning and counterexample guidance. Compared with the counterexample-guided verification method based on the SMT solver, we design and learn neural barrier functions with special structure, and use the special form to convert the counterexample generation into a polynomial optimization problem for obtaining the optimal counterexample. In the verification phase, the task of identifying the real barrier certificate can be tackled by solving the Linear Matrix Inequalities (LMI) feasibility problem, which is efficient and makes the proposed method formally sound. The experimental results demonstrate that our approach is more effective and practical than the traditional SOS-based barrier certificates synthesis and the state-of-the-art neural barrier certificates learning approach.

A Debugging Game for Probabilistic Models

One of the major advantages of model checking over other formal methods is its ability to generate a counterexample when a model does not satisfy is its specification. A counterexample is an error trace that helps to locate the source of the error. Therefore, the counterexample represents a valuable tool for debugging. In Probabilistic Model Checking (PMC), the task of counterexample generation has a quantitative aspect. Unlike the previous methods proposed for conventional model checking that generate the counterexample as a single path ending with a bad state representing the failure, the task in PMC is completely different. A counterexample in PMC is a set of evidences or diagnostic paths that satisfy a path formula, whose probability mass violates the probability threshold. Counterexample generation is not sufficient for finding the exact source of the error. Therefore, in conventional model checking, many debugging techniques have been proposed to act on the counterexamples generated to locate the source of the error. In PMC, debugging counterexamples is more challenging, since the probabilistic counterexample consists of multiple paths and it is probabilistic. In this article, we propose a debugging technique based on stochastic games to analyze probabilistic counterexamples generated for probabilistic models described as Markov chains in PRISM language. The technique is based mainly on the idea of considering the modules composing the system as players of a reachability game, whose actions contribute to the evolution of the game. Through many case studies, we will show that our technique is very effective for systems employing multiple components. The results are also validated by introducing a debugging tool called GEPCX (Game Explainer of Probabilistic Counterexamples).

Using ACL2 To Teach Students About Software Testing

We report on our experience using ACL2 in the classroom to teach students about software testing. The course COSC2300 at the University of Wyoming is a mostly traditional Discrete Mathematics course, but with a clear focus on computer science applications. For instance, the section on logic and proofs is motivated by the desire to write proofs about computer software. We emphasize that the importance of software correctness falls along a spectrum with casual programs on one end and mission-critical ones on the other. Corresponding to this spectrum is a variety of tools, ranging from unit tests, randomized testing of properties, and even formal proofs. In this paper, we describe one of the major activities, in which students use the ACL2 Sedan's counter-example generation facility to investigate properties of various existing checksum algorithms used in error detection. Students are challenged to state the relevant properties correctly, so that the counter-example generation tool is used effectively in all cases, and ACL2 can find formal proofs automatically in some of those.

Counterexample generation in CPS model checking based on ARSG algorithm

With the rapid development of software and physical devices, cyber-physical systems (CPS) are widely adopted in many application areas. Due to the increasing complexities of these systems, it is difficult to detect defects in CPS models. Counterexample generation in CPS model checking is a good choice as it is able to find defects in CPS models efficiently and can provide meaningful diagnostic feedback to facilitate debugging. In many studies, robustness-guided counterexample generation of CPS is investigated by various optimisation methods, which falsify the given properties of a CPS. In this paper, we combine genetic algorithm (GA) with acceptance-rejection technique based on the neighbourhood of the input sequence space, and propose a novel algorithm which is called ARSG algorithm. The idea of this algorithm is similar to 'exploration-exploitation' in reinforcement learning. Finally, the new algorithm is compared with the cross-entropy algorithm and the genetic algorithm under different parameters, and the performance of the new algorithm is better than the other two algorithms.

Counterexample generation in CPS model checking based on ARSG algorithm

With the rapid development of software and physical devices, cyber-physical systems (CPS) are widely adopted in many application areas. Due to the increasing complexities of these systems, it is difficult to detect defects in CPS models. Counterexample generation in CPS model checking is a good choice as it is able to find defects in CPS models efficiently and can provide meaningful diagnostic feedback to facilitate debugging. In many studies, robustness-guided counterexample generation of CPS is investigated by various optimisation methods, which falsify the given properties of a CPS. In this paper, we combine genetic algorithm (GA) with acceptance-rejection technique based on the neighbourhood of the input sequence space, and propose a novel algorithm which is called ARSG algorithm. The idea of this algorithm is similar to 'exploration-exploitation' in reinforcement learning. Finally, the new algorithm is compared with the cross-entropy algorithm and the genetic algorithm under different parameters, and the performance of the new algorithm is better than the other two algorithms.

Gordian

Accurate localization from Cyber-Physical Systems (CPS) is a critical enabling technology for context-aware applications and control. As localization plays an increasingly safety-critical role, location systems must be able to identify and eliminate faulty measurements to prevent dangerously inaccurate localization. In this article, we consider the range-based localization problem and propose a method to detect coordinated adversarial corruption on anchor positions and distance measurements. Our algorithm, G ordian , rapidly finds attacks by identifying geometric inconsistencies at the graph level without requiring assumptions about hardware, ranging mechanisms, or cryptographic protocols. We give necessary conditions for which attack detection is guaranteed to be successful in the noiseless case, and we use that intuition to extend G ordian to the noisy case where fewer guarantees are possible. In simulations generated from real-world sensor noise, we empirically show that G ordian ’s trilateration counterexample generation procedure enables rapid attack detection even for combinatorially difficult problems.

Formalizing Constructive Quantifier Elimination in Agda

In this paper a constructive formalization of quantifier elimination is presented, based on a classical formalization by Tobias Nipkow. The formalization is implemented and verified in the programming language/proof assistant Agda. It is shown that, as in the classical case, the ability to eliminate a single existential quantifier may be generalized to full quantifier elimination and consequently a decision procedure. The latter is shown to have strong properties under a constructive metatheory, such as the generation of witnesses and counterexamples. Finally, this is demonstrated on a minimal theory on the natural numbers.

Instrumenting a weakest precondition calculus for counterexample generation

A major issue in the activity of deductive program verification is to understand why automated provers fail to discharge a proof obligation. To help the user understand the problem and decide what needs to be fixed in the code or the specification, it is essential to provide means to investigate such a failure. We present our approach for the design and the implementation of counterexample generation, exhibiting values for the variables of the program where a given part of the specification fails to be validated. To produce a counterexample, we exploit the ability of SMT solvers to propose, when a proof of a formula is not found, a counter-model. Turning such a counter-model into a counterexample for the initial program is not trivial because of the many transformations leading from a particular piece of code and its specification to a set of proof goals given to external provers.

Branching Path Planning with Modal Logics

Path planning concerns about finding a route for an agent, such that this agent move along the route from an initial to a final goal. Some additional constraints may also have to be satisfied for the agent, such as avoiding obstacles or collisions. Path planning has been recently studied in the context of linear temporal logic with great success. Expressive constraint specifications involving temporal ordering can be succinctly expressed by logic formulas, whereas environments are abstracted as transition systems. The plan is obtained by counterexample generation in a model checking tool: finding a path, if any, such that a given formula (constraints) satisfies a given model (agent environment). Due to the expressive power of linear temporal logic, only linear planning has mostly been considered so far, that is, plans corresponding to tasks to be performed in a linear successive order. In this work, we study branching shaped (tree) plans in the context of the μ -calculus, an expressive modal logic which subsumes many program logics such as LTL, PDL and CTL. Branching plans can be succinctly expressed by logic formulas so that a team of mobile devices can concurrently satisfy the plan. In the current work, we provide a plan generator based on a model checking algorithm for the μ -calculus. We show the algorithm is sound and complete, that is, for any environment, there a satisfying plan for a given set of constraints, if and only if, the plan generator succeeds.

Counterexample Generation in Stochastic Model Checking Based on PSO Algorithm with Heuristic

Providing counterexample for the refutation of a property is an essential feature of model checking, if it is not the most important. However, generating counterexample in stochastic model checking needs a dedicated algorithm. It usually costs too much time and memory, and sometimes it cannot find the counterexample. What is worse, generating smallest counterexample in stochastic model checking has been proved to be NP-complete, and it is unlikely to be efficiently approximable. Although there are a few heuristic methods that are applied to the construction of the counterexample, it is usually difficult to determine the heuristic function which is critical for counterexample generating. In this paper, we present a particle swarm optimization (PSO)-based approach to generating counterexample for stochastic model checking. We define the diagnostic sub-graph as counterexample, and extend PSO algorithm with heuristic (HPSO) to generate counterexample. It adopts indirect path coding scheme to expand the scope of the search space, and employs heuristic operator to generate more effective path. The validity of our approach is illustrated by some case studies in a prototype tool. The experiments show that HPSO algorithm can significantly outperform the present algorithm for counterexample generation in stochastic model checking.

Debugging of Markov Decision Processes (MDPs) Models

In model checking, a counterexample is considered as a valuable tool for debugging. In Probabilistic Model Checking (PMC), counterexample generation has a quantitative aspect. The counterexample in PMC is a set of paths in which a path formula holds, and their accumulative probability mass violates the probability threshold. However, understanding the counterexample is not an easy task. In this paper we address the task of counterexample analysis for Markov Decision Processes (MDPs). We propose an aided-diagnostic method for probabilistic counterexamples based on the notions of causality, responsibility and blame. Given a counterexample for a Probabilistic CTL (PCTL) formula that does not hold over an MDP model, this method guides the user to the most relevant parts of the model that led to the violation.

Generating Counterexamples for Model Checking by Transformation

Counterexamples explain why a desired temporal logic property fails to hold. The generation of counterexamples is considered to be one of the primary advantages of model checking as a verification technique. Furthermore, when model checking does succeed in verifying a property, there is typically no independently checkable witness that can be used as evidence for the verified property. Previously, we have shown how program transformation techniques can be used for the verification of both safety and liveness properties of reactive systems. However, no counterexamples or witnesses were generated using the described techniques. In this paper, we address this issue. In particular, we show how the program transformation technique distillation can be used to facilitate the construction of counterexamples and witnesses for temporal properties of reactive systems. Example systems which are intended to model mutual exclusion are analysed using these techniques with respect to both safety (mutual exclusion) and liveness (non-starvation), with counterexamples being generated for those properties which do not hold.

Extreme cognitions are associated with diminished ability to use disconfirming evidence.

An Integrative Cognitive Model of mood swings and bipolar disorder proposes that cognitive styles characterized by extreme self-referent appraisals of internal states (e.g., 'If I have a bad night's sleep it means that I am about to have a breakdown') interfere with mood regulation. The aim of this study is to determine whether strong endorsement of such appraisals is predicted by a diminished ability to access disconfirming counterexamples. We examined whether the ability to access two different categories of counterexample (known as Disabling Conditions and Alternative Causes) would predict endorsement of extreme appraisals (measured by the Hypomanic Attitudes and Positive Predictions Inventory; HAPPI) and mania risk (measured by the Hypomanic Personality Scale; HPS). A non-clinical sample of 150 students completed the HAPPI, the HPS and a conditional reasoning task that indexed the ability to access Disabling Conditions and Alternative Causes. Current mood was controlled for using the Internal States Scale. The ability to make use of disabling counterexamples during the reasoning task was inversely related with scores on the HAPPI (r=-.19, p<.05); participants that were less able to make use of disabling counterexamples endorsed extreme self-referent appraisals to a greater extent. There was no association between the use of alternative cause counterexamples and the HAPPI, and no association between either measure of counterexample generation and the HPS. A diminished ability to use disconfirming evidence when reasoning about the world may reinforce problematic cognitive styles such as extreme, personalized appraisals of experience, which can interfere with mood regulation. Problematic cognitive styles such as extreme, personalized appraisals of experience may be reinforced by the inability to produce or access evidence that disconfirms these maladaptive beliefs. This reasoning bias may be associated with cognitive styles underlying psychopathology. There may be clinical utility in exploring the use of disabler generation in psychological interventions, to help disconfirm maladaptive beliefs.

Debugging of probabilistic systems using structural equation modelling

The counterexample in probabilistic model checking PMC is a set of paths in which a path formula holds, and their accumulated probability violates the probability bound. However, understanding the counterexample is not an easy task. In this paper we address the complementary task of counterexample generation, which is the counterexample analysis. We propose an aided-diagnostic method for probabilistic counterexamples based on the notions of causality and regression analysis. Given a counterexample for a probabilistic CTL PCTL/continuous stochastic logic CSL formula that does not hold over probabilistic models [discrete time Markov chain DTMC, Markov decision process MDP and continuous time Markov Chain CTMC], this method generates the causes of the violation, and describes their contribution to the error in the form of a regression model using structural equation modelling SEM. The interpretation of the regression model generated helps the designer to get better insight on the behaviour of the model, and thus helps him to understand how the error has emerged.

Automatic Generation of Minimal Cut Sets

A cut set is a collection of component failure modes that could lead to a system failure. Cut Set Analysis (CSA) is applied to critical systems to identify and rank system vulnerabilities at design time. Model checking tools have been used to automate the generation of minimal cut sets but are generally based on checking reachability of system failure states. This paper describes a new approach to CSA using a Linear Temporal Logic (LTL) model checker called BT Analyser that supports the generation of multiple counterexamples. The approach enables a broader class of system failures to be analysed, by generalising from failure state formulae to failure behaviours expressed in LTL. The traditional approach to CSA using model checking requires the model or system failure to be modified, usually by hand, to eliminate already-discovered cut sets, and the model checker to be rerun, at each step. By contrast, the new approach works incrementally and fully automatically, thereby removing the tedious and error-prone manual process and resulting in significantly reduced computation time. This in turn enables larger models to be checked. Two different strategies for using BT Analyser for CSA are presented. There is generally no single best strategy for model checking: their relative efficiency depends on the model and property being analysed. Comparative results are given for the A320 hydraulics case study in the Behavior Tree modelling language.

Relatively complete counterexamples for higher-order programs

In this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for constructing counterexamples for PCF programs. The method is relatively complete with respect to a first-order solver over the base types of PCF. In practice, this means an SMT solver can be used for the effective, automated generation of higher-order counterexamples for a large class of programs. We achieve this result by employing a novel form of symbolic execution for higher-order programs. The remarkable aspect of this symbolic execution is that even though symbolic higher-order inputs and values are considered, the path condition remains a first-order formula. Our handling of symbolic function application enables the reconstruction of higher-order counterexamples from this first-order formula. After establishing our main theoretical results, we sketch how to apply the approach to untyped, higher-order, stateful languages with first-class contracts and show how counterexample generation can be used to detect contract violations in this setting. To validate our approach, we implement a tool generating counterexamples for erroneous modules written in Racket.

MCMAS: an open-source model checker for the verification of multi-agent systems

We present MCMAS, a model checker for the verification of multi-agent systems. MCMAS supports efficient symbolic techniques for the verification of multi-agent systems against specifications representing temporal, epistemic and strategic properties. We present the underlying semantics of the specification language supported and the algorithms implemented in MCMAS, including its fairness and counterexample generation features. We provide a detailed description of the implementation. We illustrate its use by discussing a number of examples and evaluate its performance by comparing it against other model checkers for multi-agent systems on a common case study.

On the Fine-Structure of Regular Algebra

Regular algebra is the algebra of regular expressions as induced by regular language identity. We use Isabelle/HOL for a detailed systematic study of the regular algebra axioms given by Boffa, Conway, Kozen and Salomaa. We investigate the relationships between these systems, formalise a soundness proof for the smallest class (Salomaa’s) and obtain completeness for the largest one (Boffa’s) relative to a deep result by Krob. As a case study in formalised mathematics, our investigations also shed some light on the power of theorem proving technology for reasoning with algebras and their models, including proof automation and counterexample generation.