Reachability Algorithm Research Articles

Reachability Analysis of Sigmoidal Neural Networks

This paper extends the star set reachability approach to verify the robustness of feed-forward neural networks (FNNs) with sigmoidal activation functions such as Sigmoid and TanH. The main drawbacks of the star set approach in Sigmoid/TanH FNN verification are scalability, feasibility, and optimality issues in some cases due to the linear programming solver usage. We overcome this challenge by proposing a relaxed star (RStar) with symbolic intervals, which allows the usage of the back-substitution technique in DeepPoly to find bounds when overapproximating activation functions while maintaining the valuable features of a star set. RStar can overapproximate a sigmoidal activation function using four linear constraints (RStar4) or two linear constraints (RStar2), or only the output bounds (RStar0). We implement our RStar reachability algorithms in NNV and compare them to DeepPoly via robustness verification of image classification DNNs benchmarks. The experimental results show that the original star approach (i.e., no relaxation) is the least conservative of all methods yet the slowest. RStar4 is computationally much faster than the original star method and is the second least conservative approach. It certifies up to 40% more images against adversarial attacks than DeepPoly and on average 51 times faster than the star set. Last but not least, RStar0 is the most conservative method, which could only verify two cases for the CIFAR10 small Sigmoid network, δ = 0.014. However, it is the fastest method that can verify neural networks up to 3528 times faster than the star set and up to 46 times faster than DeepPoly in our evaluation.

Data-Driven Reachability Analysis From Noisy Data

We consider the problem of computing reachable sets directly from noisy data without a given system model. Several reachability algorithms are presented for different types of systems generating the data. First, an algorithm for computing over-approximated reachable sets based on matrix zonotopes is proposed for linear systems. Constrained matrix zonotopes are introduced to provide less conservative reachable sets at the cost of increased computational expenses and utilized to incorporate prior knowledge about the unknown system model. Then we extend the approach to polynomial systems and, under the assumption of Lipschitz continuity, to nonlinear systems. Theoretical guarantees are given for these algorithms in that they give a proper over-approximate reachable set containing the true reachable set. Multiple numerical examples and real experiments show the applicability of the introduced algorithms, and comparisons are made between algorithms.

Adaptive reachability algorithms for nonlinear systems using abstraction error analysis

In many reachability algorithms for nonlinear ordinary differential equations (ODEs), the tightness of the computed reachable sets mainly depends on abstraction errors and the choice of the set representation. One has to mitigate the resulting wrapping effects by suitable tuning of internally-used algorithm parameters since there exists no wrapping-free algorithm to date. In this work, we investigate the fundamentals governing the abstraction error in reachability algorithms – which we also refer to as set-based solvers – and its dependence on the time step size, leading to the introduction of a gain order. This order is related to measures for local and global abstraction errors and thus relates the well-known concept of convergence order from classical ODE solvers to set-based solvers. Furthermore, the simplification of the set representation is tackled by limiting the Hausdorff distance between the original and reduced sets; we demonstrate this for zonotopes. Both these theoretical advancements are incorporated in a modular adaptive parameter tuning algorithm suited for multiple classes of nonlinear ODEs whose efficiency is demonstrated on a wide range of benchmarks.

Verification of Neural-Network Control Systems by Integrating Taylor Models and Zonotopes

We study the verification problem for closed-loop dynamical systems with neural-network controllers (NNCS). This problem is commonly reduced to computing the set of reachable states. When considering dynamical systems and neural networks in isolation, there exist precise approaches for that task based on set representations respectively called Taylor models and zonotopes. However, the combination of these approaches to NNCS is non-trivial because, when converting between the set representations, dependency information gets lost in each control cycle and the accumulated approximation error quickly renders the result useless. We present an algorithm to chain approaches based on Taylor models and zonotopes, yielding a precise reachability algorithm for NNCS. Because the algorithm only acts at the interface of the isolated approaches, it is applicable to general dynamical systems and neural networks and can benefit from future advances in these areas. Our implementation delivers state-of-the-art performance and is the first to successfully analyze all benchmark problems of an annual reachability competition for NNCS.

Efficient algorithms for dynamic bidirected Dyck-reachability

Dyck-reachability is a fundamental formulation for program analysis, which has been widely used to capture properly-matched-parenthesis program properties such as function calls/returns and field writes/reads. Bidirected Dyck-reachability is a relaxation of Dyck-reachability on bidirected graphs where each edge u → ( i v labeled by an open parenthesis “( i ” is accompanied with an inverse edge v → ) i u labeled by the corresponding close parenthesis “) i ”, and vice versa. In practice, many client analyses such as alias analysis adopt the bidirected Dyck-reachability formulation. Bidirected Dyck-reachability admits an optimal reachability algorithm. Specifically, given a graph with n nodes and m edges, the optimal bidirected Dyck-reachability algorithm computes all-pairs reachability information in O ( m ) time. This paper focuses on the dynamic version of bidirected Dyck-reachability. In particular, we consider the problem of maintaining all-pairs Dyck-reachability information in bidirected graphs under a sequence of edge insertions and deletions. Dynamic bidirected Dyck-reachability can formulate many program analysis problems in the presence of code changes. Unfortunately, solving dynamic graph reachability problems is challenging. For example, even for maintaining transitive closure, the fastest deterministic dynamic algorithm requires O ( n 2 ) update time to achieve O (1) query time. All-pairs Dyck-reachability is a generalization of transitive closure. Despite extensive research on incremental computation, there is no algorithmic development on dynamic graph algorithms for program analysis with worst-case guarantees. Our work fills the gap and proposes the first dynamic algorithm for Dyck reachability on bidirected graphs. Our dynamic algorithms can handle each graph update ( i.e. , edge insertion and deletion) in O ( n ·α( n )) time and support any all-pairs reachability query in O (1) time, where α( n ) is the inverse Ackermann function. We have implemented and evaluated our dynamic algorithm on an alias analysis and a context-sensitive data-dependence analysis for Java. We compare our dynamic algorithms against a straightforward approach based on the O ( m )-time optimal bidirected Dyck-reachability algorithm and a recent incremental Datalog solver. Experimental results show that our algorithm achieves orders of magnitude speedup over both approaches.

Falsification of hybrid systems with symbolic reachability analysis and trajectory splicing

The falsification of a hybrid system aims at finding trajectories that violate a given safety property. This is a challenging problem, and the practical applicability of current falsification algorithms still suffers from their high time complexity. In contrast to falsification, verification algorithms aim at providing guarantees that no such trajectories exist. Recent symbolic reachability techniques are capable of efficiently computing linear constraints that enclose all trajectories of the system with reasonable precision. In this paper, we leverage the power of symbolic reachability algorithms to improve the scalability of falsification techniques. Recent approaches to falsification reduce the problem to a nonlinear optimization problem. We propose to reduce the search space of the optimization problem by adding linear state constraints obtained with a reachability algorithm. An empirical study of how varying abstractions during symbolic reachability analysis affect the performance of solving a falsification problem is presented. In addition, for solving a falsification problem, we propose an alternating minimization algorithm that solves a linear programming problem and a non-linear programming problem in alternation finitely many times. We showcase the efficacy of our algorithms on a number of standard hybrid systems benchmarks demonstrating the performance increase and number of falsifyable instances.

Verification of piecewise deep neural networks: a star set approach with zonotope pre-filter

Abstract Verification has emerged as a means to provide formal guarantees on learning-based systems incorporating neural network before using them in safety-critical applications. This paper proposes a new verification approach for deep neural networks (DNNs) with piecewise linear activation functions using reachability analysis. The core of our approach is a collection of reachability algorithms using star sets (or shortly, stars), an effective symbolic representation of high-dimensional polytopes. The star-based reachability algorithms compute the output reachable sets of a network with a given input set before using them for verification. For a neural network with piecewise linear activation functions, our approach can construct both exact and over-approximate reachable sets of the neural network. To enhance the scalability of our approach, a star set is equipped with an outer-zonotope (a zonotope over-approximation of the star set) to quickly estimate the lower and upper bounds of an input set at a specific neuron to determine if splitting occurs at that neuron. This zonotope pre-filtering step reduces significantly the number of linear programming optimization problems that must be solved in the analysis, and leads to a reduction in computation time, which enhances the scalability of the star set approach. Our reachability algorithms are implemented in a software prototype called the neural network verification tool, and can be applied to problems analyzing the robustness of machine learning methods, such as safety and robustness verification of DNNs. Our experiments show that our approach can achieve runtimes twenty to 1400 times faster than Reluplex, a satisfiability modulo theory-based approach. Our star set approach is also less conservative than other recent zonotope and abstract domain approaches.

Quantitative Verification of Stochastic Regular Expressions

In this article, we introduce a probabilistic verification algorithm for stochastic regular expressions over a probabilistic extension of the Action based Computation Tree Logic (ACTL*). The main results include a novel model checking algorithm and a semantics on the probabilistic action logic for stochastic regular expressions (SREs). Specific to our model checking algorithm is that SREs are defined via local probabilistic functions. Such functions are beneficial since they enable to verify properties locally for sub-components. This ability provides a flexibility to reuse the local results for the global verification of the system; hence, the framework can be used for iterative verification. We demonstrate how to model a system with an SRE and how to verify it with the probabilistic action based logic and present a preliminary performance evaluation with respect to the execution time of the reachability algorithm.

Reachability of Black-Box Nonlinear Systems after Koopman Operator Linearization

Abstract Reachability analysis of nonlinear dynamical systems is a challenging and computationally expensive task. Computing the reachable states for linear systems, in contrast, can often be done efficiently in high dimensions. In this paper, we explore verification methods that leverage a connection between these two classes of systems based on the concept of the Koopman operator. The Koopman operator links the behaviors of a nonlinear system to a linear system embedded in a higher dimensional space, with an additional set of so-called observable variables. Although the new dynamical system has linear differential equations, the set of initial states is defined with nonlinear constraints. For this reason, existing approaches for linear systems reachability cannot be used directly. We propose the first reachability algorithm that deals with this unexplored type of reachability problem. Our evaluation examines several optimizations, and shows the proposed workflow is a promising avenue for verifying behaviors of nonlinear systems.

Reachability-Based Approach to Design Interval State Observers

This work aims at showing the advantage to consider the design problem of interval observers for uncertain discrete-time linear systems as a reachability problem. Based on the classical equations of the Lunenberger observer, a reachability algorithm is proposed to compute tight outer approximations of the reachable set of the system. Moreover, based on the symmetric property of interval vectors a more convenient version of this reachability algorithm is introduced. Through numerical examples, the simulation results of the proposed reachability approach are compared to those obtained by interval observers.

Parameterized Model Checking on the TSO Weak Memory Model

We present an extended version of the model checking modulo theories framework for verifying parameterized systems under the TSO weak memory model. Our extension relies on three main ingredients: (1) an axiomatic theory of the TSO memory model based on relations over (read, write) events, (2) a TSO-specific backward reachability algorithm and (3) an SMT solver for reasoning about TSO formulas. One of the main originality of our work is a partial order reduction technique that exploits specificities of the TSO memory model. We have implemented this framework in a new version of the Cubicle model checker called Cubicle- $$\mathscr {W}$$ . Our experiments show that Cubicle- $$\mathscr {W}$$ is expressive and efficient enough to automatically prove safety of concurrent algorithms, for an arbitrary number of processes, ranging from mutual exclusion to synchronization barriers translated from actual x86-TSO implementations.

Verification and refutation of C programs based on k-induction and invariant inference

DepthK is a source-to-source transformation tool that employs bounded model checking (BMC) to verify and falsify safety properties in single- and multi-threaded C programs, without manual annotation of loop invariants. Here, we describe and evaluate a proof-by-induction algorithm that combines k-induction with invariant inference to prove and refute safety properties. We apply two invariant generators to produce program invariants and feed these into a k-induction-based verification algorithm implemented in DepthK, which uses the efficient SMT-based context-bounded model checker (ESBMC) as sequential verification back-end. A set of C benchmarks from the International Competition on Software Verification (SV-COMP) and embedded-system applications extracted from the available literature are used to evaluate the effectiveness of the proposed approach. Experimental results show that k-induction with invariants can handle a wide variety of safety properties, in typical programs with loops and embedded software applications from the telecommunications, control systems, and medical domains. The results of our comparative evaluation extend the knowledge about approaches that rely on both BMC and k-induction for software verification, in the following ways. (1) The proposed method outperforms the existing implementations that use k-induction with an interval-invariant generator (e.g., 2LS and ESBMC), in the category ConcurrencySafety, and overcame, in others categories, such as SoftwareSystems, other software verifiers that use plain BMC (e.g., CBMC). Also, (2) it is more precise than other verifiers based on the property-directed reachability (PDR) algorithm (i.e., SeaHorn, Vvt and CPAchecker-CTIGAR). This way, our methodology demonstrated improvement over existing BMC and k-induction-based approaches.

Sum-of-Squares Flight Control Synthesis for Deep-Stall Recovery

Certification of inflight loss-of-control recovery is complicated by the highly nonlinear flight dynamics beyond stall. In lieu of extensive Monte Carlo simulations for flight control certification, sum-of-squares programming techniques provide an algebraic approach to the problem of nonlinear control synthesis and analysis. However, reliance on polynomial models has hitherto limited applicability to aeronautical control problems. Taking advantage of recently proposed piecewise polynomial models, this paper revisits sum-of-squares techniques for recovery of an aircraft from deep-stall conditions using a realistic yet tractable aerodynamic model. The local stability analysis of classical controllers is presented as well as synthesis of polynomial feedback laws with the objective of enlarging their nonlinear region of attraction. A newly developed synthesis algorithm for infinite-horizon backward reachability facilitates the design of recovery control laws, ensuring stable recovery by design. The paper’s results motivate future research in aeronautical sum-of-squares applications.

Special Section on the Fiftieth Annual ACM Symposium on Theory of Computing (STOC 2018)

This issue of SICOMP contains 10 specially selected papers from the Fiftieth Annual ACM Symposium on Theory of Computing, otherwise known as STOC 2018, held June 25 to 29 in Los Angeles, California...

One Edge at a Time: A Novel Approach Towards Efficient Transitive Reduction Computation on DAGs

Given a directed acyclic graph ( ${DAG}$ ) $G$ , $G$ ’s transitive reduction ( TR ) $G^{tr}$ is the unique ${DAG}$ satisfying that $G^{tr}$ has the minimum number of edges and has the same transitive closure ( ${TC}$ ) as $G$ . ${TR}$ computation has been extensively studied during the past decades and was used in many applications, where the main problem is how to compute ${TR}$ efficiently for large graphs. However, existing approaches have either large space complexity or higher time complexity, which makes them cannot compute ${TR}$ efficiently on large dense graphs. We propose a novel approach for ${TR}$ computation, which takes every single edge as the basic processing unit, and utilizes existing reachability algorithms to test whether it is redundant or not. In this way, we avoid the costly graph traversal operation of existing approaches. We identify the performance bottleneck and propose a set of heuristics to sort edges, such that to reduce the average processing cost of each edge. We show by experimental results that our approach works much better than all the existing approaches, and can be faster than the state-of-the-art approach by more than two orders of magnitude on large dense graphs.

Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control

This paper proposes a new forward reachability analysis approach to verify safety of cyber-physical systems (CPS) with reinforcement learning controllers. The foundation of our approach lies on two efficient, exact and over-approximate reachability algorithms for neural network control systems using star sets, which is an efficient representation of polyhedra. Using these algorithms, we determine the initial conditions for which a safety-critical system with a neural network controller is safe by incrementally searching a critical initial condition where the safety of the system cannot be established. Our approach produces tight over-approximation error and it is computationally efficient, which allows the application to practical CPS with learning enable components (LECs). We implement our approach in NNV, a recent verification tool for neural networks and neural network control systems, and evaluate its advantages and applicability by verifying safety of a practical Advanced Emergency Braking System (AEBS) with a reinforcement learning (RL) controller trained using the deep deterministic policy gradient (DDPG) method. The experimental results show that our new reachability algorithms are much less conservative than existing polyhedra-based approaches. We successfully determine the entire region of the initial conditions of the AEBS with the RL controller such that the safety of the system is guaranteed, while a polyhedra-based approach cannot prove the safety properties of the system.

Modular Decomposition-Based Graph Compression for Fast Reachability Detection

Fast reachability detection is one of the key problems in graph applications. Most of the existing works focus on creating an index and answering reachability based on that index. For these approaches, the index construction time and index size can become a concern for large graphs. More recently query-preserving graph compression has been proposed, and searching reachability over the compressed graph has been shown to be able to significantly improve query performance as well as reducing the index size. In this paper, we introduce a multilevel compression scheme for DAGs, which builds on existing compression schemes, but can further reduce the graph size for many real-world graphs. We propose an algorithm to answer reachability queries using the compressed graph. Extensive experiments with four existing state-of-the-art reachability algorithms and 12 real-world datasets demonstrate that our approach outperforms the existing methods. Experiments with synthetic datasets ensure the scalability of this approach. We also provide a discussion on possible compression for k-reachability.

Set reachability and observability of probabilistic Boolean networks

In this paper, the set reachability and observability of probabilistic Boolean networks (PBNs) are investigated. Using a parallel extension technique, we proved that the observability problem of a PBN can be recast as a set reachability problem of an interconnected PBN. For set reachability analysis, we designed a random logic dynamical system (RLDS) from the PBN under consideration by reconstructing the state transfer graph (STG). We proved that, for a PBN, a target subset is reachable from an initial subset if and only if all solutions to the corresponding RLDS starting from the initial subset converge to the zero state. Based on the STG reconstruction technique and using the largest invariant subset algorithm, the necessary and sufficient conditions for finite-time set reachability with probability one and asymptotical set reachability in distribution were obtained. All the results are expressed in terms of the transition probability matrix between non-zero states of the RLDS. Further, the results related to set reachability were applied to the observability problem of PBNs. The necessary and sufficient conditions for finite-time observability with probability one and asymptotical observability in distribution were obtained. Finally, examples were presented to illustrate the effectiveness of the proposed method.

Min/Max-Poly Weighting Schemes and the NL versus UL Problem

For a graph G ( V , E ) (| V | = n ) and a vertex s ∈ V , a weighting scheme ( W : E ↦ Z + ) is called a min-unique (resp. max-unique ) weighting scheme if, for any vertex v of the graph G , there is a unique path of minimum (resp. maximum) weight from s to v , where weight of a path is the sum of the weights assigned to the edges. Instead, if the number of paths of minimum (resp. maximum) weight is bounded by n c for some constant c , then the weighting scheme is called a min-poly (resp. max-poly ) weighting scheme. In this article, we propose an unambiguous nondeterministic log-space (UL) algorithm for the problem of testing reachability graphs augmented with a min-poly weighting scheme. This improves the result in Reinhardt and Allender [2000], in which a UL algorithm was given for the case when the weighting scheme is min-unique . Our main technique involves triple inductive counting and generalizes the techniques of Immerman [1988], Szelepcsényi [1988], and Reinhardt and Allender [2000], combined with a hashing technique due to Fredman et al. [1984] (also used in Garvin et al. [2014]). We combine this with a complementary unambiguous verification method to give the desired UL algorithm. At the other end of the spectrum, we propose a UL algorithm for testing reachability in layered DAGs augmented with max-poly weighting schemes. To achieve this, we first reduce reachability in layered DAGs to the longest path problem for DAGs with a unique source, such that the reduction also preserves the max-unique and max-poly properties of the graph. Using our techniques, we generalize the double inductive counting method in Limaye et al. [2009], in which the UL algorithm was given for the longest path problem on DAGs with a unique sink and augmented with a max-unique weighting scheme. An important consequence of our results is that, to show NL = UL, it suffices to design log-space computable min-poly (or max-poly ) weighting schemes for layered DAGs.

A Light-Weight Approach for Verifying Multi-Threaded Programs with CPAchecker

Verifying multi-threaded programs is becoming more and more important, because of the strong trend to increase the number of processing units per CPU socket. We introduce a new configurable program analysis for verifying multi-threaded programs with a bounded number of threads. We present a simple and yet efficient implementation as component of the existing program-verification framework CPAchecker. While CPAchecker is already competitive on a large benchmark set of sequential verification tasks, our extension enhances the overall applicability of the framework. Our implementation of handling multiple threads is orthogonal to the abstract domain of the data-flow analysis, and thus, can be combined with several existing analyses in CPAchecker, like value analysis, interval analysis, and BDD analysis. The new analysis is modular and can be used, for example, to verify reachability properties as well as to detect deadlocks in the program. This paper includes an evaluation of the benefit of some optimization steps (e.g., changing the iteration order of the reachability algorithm or applying partial-order reduction) as well as the comparison with other state-of-the-art tools for verifying multi-threaded programs.