Falsification of Cyber-Physical Systems with Robustness Uncertainty Quantification Through Stochastic optimization with Adaptive Restart

Abstract

This work is in the field of requirements driven search-based test case generation methods for Cyber-Physical Systems (CPS). The basic characteristic of search-based testing methods is that the search process is guided by high level requirements captured in formal logic and, in particular, Signal Temporal Logic (STL). Given a system trajectory, STL specifications can be equipped with quantitative semantics which evaluate the closeness of the given trajectory from violating the requirement. Hence, by searching for trajectories of decreasing value with respect to the specification, a test generation method can be formulated which searches for system behaviors with a closeness to violation value of less than 0. These system behaviors, i.e., trajectories that violate the requirements and yield STL closeness value less than 0, are referred to as falsiping behaviors. In addition, signed distance can be utilized when searching for trajectories that maximally violate the specification (negative specification valuations). In this work, we propose the use of a stochastic search method that mixes global and local search for system test case generation. The implemented search method models input-output relationships between test cases and the observed STL closeness values of the yielded system trajectories, adaptively linking input-out of both global and local regional modeling. The method shows improved finite time performance, i.e., quick identification of falsification behaviors, over current search-based test case generation methods. Further, given no falsifying behaviors are found in finite time our method is capable of quantifying the certainty that no falsifying behaviors exist.