State Space Exploration Research Articles

Failure effects analysis on safety properties for hard logic in reactor protection system using model checking

Ensuring the reliability and safety of safety-critical systems within nuclear power plant hinges upon efficient failure effects analysis. Conventional approaches to failure effects analysis in reactor protection systems encounter notable challenges, including labor-intensive manual analysis and limitations in ensuring comprehensive analysis coverage. To tackle these issues head-on, we introduce a novel methodology termed Failure Effects Analysis on Safety Properties (FEA-SP). Grounded in model checking technology, this method facilitates automated failure analysis processes. By harnessing the exhaustive state space exploration capabilities inherent in model checking, the FEA-SP methodology adopts the safety properties of the system as its granularity and verification focal point. A detailed component-level case study involving a hard logic within the HPR1000 nuclear reactor protection system underscores the efficacy and practicality of the proposed approach, especially in the thorough examination of system spurious actions. The failure effects analysis method delineated in this paper holds broad applicability and serves as a valuable reference for the analysis of failure effects in safety systems.

Mutation‐Guided Metamorphic Testing of Optimality in AI Planning

ABSTRACTAutonomous systems such as space‐ or underwater‐exploration robots or elderly people assistance robots often include an artificial intelligence (AI) planner as a component. Starting from the initial state of a system, an AI planner automatically generates sequential plans to reach final states that satisfy user‐specified goals. Generating plans having a minimum number of intermediate steps or taking the least time to execute is usually strongly desired, as these plans exhibit minimal costs. Unfortunately, testing if an AI planner generates optimal plans is almost impossible because the expected cost of these plans is usually unknown. Based on mutation adequacy test suite selection, this article proposes a novel metamorphic testing framework for detecting the lack of optimality in AI planners. The general idea is to perform a systematic but non‐exhaustive state space exploration from the initial state and to select mutant‐adequate states to instantiate new planning tasks as follow‐up test cases. We then check a metamorphic relation between the automatically generated solutions of the AI planner for these new test cases and the cost of the initial plan. We implemented this metamorphic testing framework in a tool called MorphinPlan. Our experimental evaluation shows that MorphinPlan can detect non‐optimal behaviour in both mutated AI planners and off‐the‐shelf, configurable planners. It also shows that our proposed mutation adequacy test selection strategy outperforms three alternative test generation and selection strategies, including both random state selection and random walks through the state space in terms of mutation scores.

The fast and the capacious: memory-efficient multi-GPU accelerated explicit state space exploration with GPUexplore 3.0

The GPU acceleration of explicit state space exploration, for explicit-state model checking, has been the subject of previous research, but to date, the tools have been limited in their applicability and in their practical use. Considering this research, to our knowledge, we are the first to use a novel tree database for GPUs. This novel tree database allows high-performant, memory-efficient storage of states in the form of binary trees. Besides the tree compression this enables, we also propose two new hashing schemes, compact-cuckoo and compact multiple-functions. These schemes enable the use of Cleary compression to compactly store tree roots. Besides an in-depth discussion of the tree database algorithms, the input language and workflow of our tool, called GPUexplore 3.0, are presented. Finally, we explain how the algorithms can be extended to exploit multiple GPUs that reside on the same machine. Experiments show single-GPU processing speeds of up to 144 million states per second compared to 20 million states achieved by 32-core LTSmin. In the multi-GPU setting, workload and storage distributions are optimal, and, frequently, performance is even positively impacted when the number of GPUs is increased. Overall, a logarithmic acceleration up to 1.9× was achieved with four GPUs, compared to what was achieved with one and two GPUs. We believe that a linear speedup can be easily accomplished with faster P2P communications between the GPUs.

A survey on compositional algorithms for verification and synthesis in supervisory control

AbstractThis survey gives an overview of the current research on compositional algorithms for verification and synthesis of modular systems modelled as interacting finite-state machines. Compositional algorithms operate by repeatedly simplifying individual components of a large system, replacing them by smaller so-called abstractions, while preserving critical properties. In this way, the exponential growth of the state space can be limited, making it possible to analyse much bigger state spaces than possible by standard state space exploration. This paper gives an introduction to the principles underlying compositional methods, followed by a survey of algorithmic solutions from the recent literature that use compositional methods to analyse systems automatically. The focus is on applications in supervisory control of discrete event systems, particularly on methods that verify critical properties or synthesise controllable and nonblocking supervisors.

A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots

When a mobile robot inspects tasks with complex requirements indoors, the traditional backstepping method cannot guarantee the accuracy of the trajectory, leading to problems such as the instrument not being inside the image and focus failure when the robot grabs the image with high zoom. In order to solve this problem, this paper proposes an adaptive backstepping method based on double Q-learning for tracking and controlling the trajectory of mobile robots. We design the incremental model-free algorithm of Double-Q learning, which can quickly learn to rectify the trajectory tracking controller gain online. For the controller gain rectification problem in non-uniform state space exploration, we propose an incremental active learning exploration algorithm that incorporates memory playback as well as experience playback mechanisms to achieve online fast learning and controller gain rectification for agents. To verify the feasibility of the algorithm, we perform algorithm verification on different types of trajectories in Gazebo and physical platforms. The results show that the adaptive trajectory tracking control algorithm can be used to rectify the mobile robot trajectory tracking controller’s gain. Compared with the Backstepping-Fractional-Older PID controller and Fuzzy-Backstepping controller, Double Q-backstepping has better robustness, generalization, real-time, and stronger anti-disturbance capability.

Auto adaptation of closed-loop insulin delivery system using continuous reward functions and incremental discretization

Several closed or hybrid loop controllers for Blood Glucose (BG) regulation, which are also known as Artificial Pancreas (AP) Systems or Automated Insulin Delivery systems (AIDs), are in development worldwide. Most AIDs are designed and evaluated for short-term performance, with a particular emphasis on the post-meal period. However, if controllers are not adapted properly to account for variations in physiology that affect Insulin Sensitivity (IS), the AIDs may perform inadequately. In this work, the performance of two Reinforcement Learning (RL) agents trained under both piecewise and continuous reward functions is evaluated in-silico for long-term adaptation of a Fully Automated Insulin Delivery (fAID) system. An automatic adaptive discretization scheme that expands the state space as needed is also implemented to avoid disproportionate state space exploration. The proposed agents are evaluated for long-term adaptation of the Automatic Regulation of Glucose (ARG) algorithm, considering variations in IS. Results show that both RL agents have improved performance compared to a rule-based decision-making approach and the baseline controller for the majority of the adult population. Moreover, the use of a continuous shaped reward function proves to enhance the performance of the agents further than a piecewise one.

Deceptive Reinforcement Learning in Model-Free Domains

This paper investigates deceptive reinforcement learning for privacy preservation in model-free and continuous action space domains. In reinforcement learning, the reward function defines the agent's objective. In adversarial scenarios, an agent may need to both maximise rewards and keep its reward function private from observers. Recent research presented the ambiguity model (AM), which selects actions that are ambiguous over a set of possible reward functions, via pre-trained Q-functions. Despite promising results in model-based domains, our investigation shows that AM is ineffective in model-free domains due to misdirected state space exploration. It is also inefficient to train and inapplicable in continuous action spaces. We propose the deceptive exploration ambiguity model (DEAM), which learns using the deceptive policy during training, leading to targeted exploration of the state space. DEAM is also applicable in continuous action spaces. We evaluate DEAM in discrete and continuous action space path planning environments. DEAM achieves similar performance to an optimal model-based version of AM and outperforms a model-free version of AM in terms of path cost, deceptiveness and training efficiency. These results extend to the continuous domain.

Coverage visualization and analysis of net inscriptions in coloured Petri net models

High-level Petri nets such as coloured Petri nets (CPNs) are characterized by the combination of Petri nets and a high-level programming language. In CPNs and CPN Tools, the inscriptions (e.g. arc expressions and guards) are specified using Standard ML. The application of simulation and state space exploration for validating CPN models traditionally focuses on behavioural properties related to net structure, i.e. places and transitions. This means that the net inscriptions are only implicitly validated, and the extent to which their sub-expressions have been covered is not made explicit. This paper extends our previous work on coverage analysis of net inscriptions of CPN models. In particular, we improve the CPN Tools library responsible for annotating, instrumenting and collecting the evaluation of Boolean conditions for determining the coverage criteria based on model executions. The library now automates most of the instrumentation parts that were done manually before and integrates the reports of the coverage analysis into the CPN Tools GUI. We evaluate our approach on new publicly available CPN models.

Grid-based state space exploration for molecular binding

Binding processes are difficult to sample with molecular dynamics (MD) simulations. In particular, the state space exploration is often incomplete. Evaluating the molecular interaction energy on a grid circumvents this problem but is heavily limited by state space dimensionality. Here, we make the first steps towards a low-dimensional grid-based model of molecular binding. We discretize the state space of relative positions and orientations of the two molecules under the rigid body assumption. The corresponding program is published as the Python package molgri. For the rotational component of the grids, we test algorithms based on Euler angles, polyhedra and quaternions, of which the polyhedra-based are the most uniform. The program outputs a sequence of molecular structures that can be easily processed by standard MD programs to calculate grid point energies. We demonstrate the grid-based approach on two molecular systems: a water dimer and a coiled-coil protein interacting with a chloride anion. For the second system we relax the rigid-body assumption and improve the accuracy of the grid point energies by an energy minimization. In both cases, oriented bonding patterns and energies confirm expectations from chemical intuition and MD simulations. We also demonstrate how analysis of energy contributions on a grid can be performed and demonstrate that electrostatically driven association is sufficiently resolved by point-energy calculations. Overall, grid-based models of molecular binding are potentially a powerful complement to molecular sampling approaches, and we see the potential to expand the method to quantum chemistry and flexible docking applications.

Neural assemblies uncovered by generative modeling explain whole-brain activity statistics and reflect structural connectivity.

Patterns of endogenous activity in the brain reflect a stochastic exploration of the neuronal state space that is constrained by the underlying assembly organization of neurons. Yet, it remains to be shown that this interplay between neurons and their assembly dynamics indeed suffices to generate whole-brain data statistics. Here, we recorded the activity from ∼40,000 neurons simultaneously in zebrafish larvae, and show that a data-driven generative model of neuron-assembly interactions can accurately reproduce the mean activity and pairwise correlation statistics of their spontaneous activity. This model, the compositional Restricted Boltzmann Machine (cRBM), unveils ∼200 neural assemblies, which compose neurophysiological circuits and whose various combinations form successive brain states. We then performed in silico perturbation experiments to determine the interregional functional connectivity, which is conserved across individual animals and correlates well with structural connectivity. Our results showcase how cRBMs can capture the coarse-grained organization of the zebrafish brain. Notably, this generative model can readily be deployed to parse neural data obtained by other large-scale recording techniques.

A Symbolic Approach to Detecting Hardware Trojans Triggered by Don’t Care Transitions

Due to the globalization of Integrated Circuit supply chain, hardware Trojans and the attacks that can trigger them have become an important security issue. One type of hardware Trojans leverages the “don’t care transitions” in Finite-state Machines (FSMs) of hardware designs. In this article, we present a symbolic approach to detecting don’t care transitions and the hidden Trojans. Our detection approach works at both register-transfer level (RTL) and gate level, does not require a golden design, and works in three stages. In the first stage, it explores the reachable states. In the second stage, it performs an approximate analysis to find the don’t care transitions and any discrepancies in the register values or output lines due to don’t care transitions. The second stage can be used for both predicting don’t care triggered Trojans and for guiding don’t care aware reachability analysis. In the third stage, it performs a state-space exploration from reachable states that have incoming don’t care transitions to explore the Trojan payload and to find behavioral discrepancies with respect to what has been observed in the first stage. We also present a pruning technique based on the reachability of FSM states. We present a methodology that leverages both RTL and gate-level for soundness and efficiency. Specifically, we show that don’t care transitions and Trojans that leverage them must be detected at the gate-level, i.e., after synthesis has been performed, for soundness. However, under specific conditions, Trojan payload exploration can be performed more efficiently at RTL. Additionally, the modular design of our approach also provides a fast Trojan prediction method even at the gate level when the reachable states of the FSM is known a priori . Evaluation of our approach on a set of benchmarks from OpenCores and TrustHub and using gate-level representation generated by two synthesis tools, YOSYS and Synopsis Design Compiler (SDC), shows that our approach is both efficient (up to 10× speedup w.r.t. no pruning) and precise (0% false positives both at RTL and gate-level netlist) in detecting don’t care transitions and the Trojans that leverage them. Additionally, the total analysis time can achieve up to 1.62× (using YOSYS) and 1.92× (using SDC) speedup when synthesis preserves the FSM structure, the foundry is trusted, and the Trojan detection is performed at RTL.

NExG: Provable and Guided State-Space Exploration of Neural Network Control Systems Using Sensitivity Approximation

We propose a new technique for performing state-space exploration of closed-loop control systems with neural network feedback controllers. Our approach involves approximating the sensitivity of the trajectories of the closed-loop dynamics. Using such an approximator and the system simulator, we present a guided state-space exploration method that can generate trajectories visiting the neighborhood of a target state at a specified time. We present a theoretical framework which establishes that our method will produce a sequence of trajectories that will reach a suitable neighborhood of the target state. We provide a thorough evaluation of our approach on various systems with neural network feedback controllers of different configurations. We outperform earlier state-space exploration techniques and achieve significant improvement in both the quality (explainability) and performance (convergence rate). Finally, we adopt our algorithm for the falsification of a class of temporal logic specification, assess its performance, and show its potential in supplementing existing falsification algorithms.

Learning-based symbolic abstractions for nonlinear control systems

Symbolic models or abstractions are known to be powerful tools for the control design of cyber–physical systems (CPSs) with logic specifications. In this paper, we investigate a novel learning-based approach to the construction of symbolic models for nonlinear control systems. In particular, the symbolic model is constructed based on learning the un-modeled part of the dynamics from training data based on state-space exploration, and the concept of an alternating simulation relation that represents behavioral relationships with respect to the original control system. Moreover, we aim at achieving safe exploration, meaning that the trajectory of the system is guaranteed to be in a safe region for all times while collecting the training data. In addition, we provide some techniques to reduce the computational load, in terms of memory and computation time,of constructing the symbolic models and the safety controller synthesis, so as to make our approach practical. Finally, a numerical simulation illustrates the effectiveness of the proposed approach.

EtWExplorer: Multi-Priority Scheduling Path Exploration Technology Based on Abstract Syntax Tree Analysis

Symbolic execution is well known as a dynamic vulnerability discovery technique. Its greatest advantage is the capability to analyze the execution information of the program and to explore the path in the program deterministically. This is a more accurate way to determine if there are vulnerabilities in a program than randomized testing by fuzzing. In addition, symbolic execution does not suffer from the problem of decreasing the capability to discover new paths as more paths are discovered, similar to that caused by random-based fuzzing. However, the reason why symbolic execution is not widely used in vulnerability discovery is mainly due to the state space explosion in the program. The state space explosion severely affects the applicability of symbolic execution. To further improve the applicability of symbolic execution, this paper proposes a path exploration technology based on abstract syntax tree analysis. With the distance between the expression generated by the symbolic execution of the repeat location and the “unsatisfiable” condition of the “unsat” state, we can perform multi-priority scheduling for the repeat location state, thus mitigating the impact of the state space explosion on path exploration. We proposed and implemented EtWExplorer, a multi-priority scheduling technique based on abstract syntax tree analysis. With this technique, we can significantly improve the capability of symbolic execution to discover unknown paths even in state space exploration. Experiments show that EtWExplorer introduces a performance overhead of 72% in the worst case and can improve performance by 294% in the best case. EtWExplorer has a 95% improvement in state space explosion mitigation capability and a 199% to 983% improvement in the path exploration capability of block coverage and a 181% to 1047% improvement in the path exploration capability of edge coverage when facing programs that cause a state space explosion.

Feature-Based MPPI Control with Applications to Maritime Systems

In this paper, a novel feature-based sampling strategy for nonlinear Model Predictive Path Integral (MPPI) control is presented. Using the MPPI approach, the optimal feedback control is calculated by solving a stochastic optimal control (OCP) problem online by evaluating the weighted inference of sampled stochastic trajectories. While the MPPI algorithm can be excellently parallelized, the closed-loop performance strongly depends on the information quality of the sampled trajectories. To draw samples, a proposal density is used. The solver’s and thus, the controller’s performance is of high quality if the sampled trajectories drawn from this proposal density are located in low-cost regions of state-space. In classical MPPI control, the explored state-space is strongly constrained by assumptions that refer to the control value’s covariance matrix, which are necessary for transforming the stochastic Hamilton–Jacobi–Bellman (HJB) equation into a linear second-order partial differential equation. To achieve excellent performance even with discontinuous cost functions, in this novel approach, knowledge-based features are introduced to constitute the proposal density and thus the low-cost region of state-space for exploration. This paper addresses the question of how the performance of the MPPI algorithm can be improved using a feature-based mixture of base densities. Furthermore, the developed algorithm is applied to an autonomous vessel that follows a track and concurrently avoids collisions using an emergency braking feature. Therefore, the presented feature-based MPPI algorithm is applied and analyzed in both simulation and full-scale experiments.

SATFuzz: A Stateful Network Protocol Fuzzing Framework from a Novel Perspective

Stateful network protocol fuzzing is one of the essential means for ensuring network communication security. However, the existing methods have problems, including frequent auxiliary message interaction, no in-depth state-space exploration, and high shares of invalid interaction time. To this end, we propose SATFuzz, a stateful network protocol fuzzing framework. SATFuzz first prioritizes the states identified by the status codes in response messages, then randomly selects a state to test among the high-priority states, and determines its corresponding optimal test sequence, which is composed of the minimum pre-lead sequence, the test case, and the fittest post-end sequence. Finally, SATFuzz uses a quasi-recurrent neural network (QRNN) to filter the test cases before performing interaction, and only the optimal test sequence, including the valid test case, can be fed to the protocol entity. To verify the proposed framework, we conduct extensive experiments with the state-of-the-art fuzzer on two popular protocols. The results show that the vulnerability discovery efficiency of the proposed approach increases by at least 1.48 times (at most by 3.06 times), making it superior to the rival methods. This not only confirms the effectiveness of SATFuzz in terms of improving the vulnerability discovery efficiency but also shows that SATFuzz has significant advantages.

Plateau proposal distributions for adaptive component-wise multiple-try metropolis

Markov chain Monte Carlo (MCMC) methods are sampling methods that have become a commonly used tool in statistics, for example to perform Monte Carlo integration. As a consequence of the increase in computational power, many variations of MCMC methods exist for generating samples from arbitrary, possibly complex, target distributions. The performance of an MCMC method, in particular that of a Metropolis–Hastings MCMC method, is predominately governed by the choice of the so-called proposal distribution used. In this paper, we introduce a new type of proposal distribution for the use in Metropolis–Hastings MCMC methods that operates component-wise and with multiple trials per iteration. Specifically, the novel class of proposal distributions, called Plateau distributions, does not overlap, thus ensuring that the multiple trials are drawn from different regions of the state space. Furthermore, the Plateau proposal distributions allow for a bespoke adaptation procedure that lends itself to a Markov chain with efficient problem dependent state space exploration and favourable burn-in properties. Simulation studies show that our novel MCMC algorithm outperforms competitors when sampling from distributions with a complex shape, highly correlated components or multiple modes.

Scales of Cancer Evolution: Selfish Genome or Cooperating Cells?

Simple SummaryCancer continuously evolves its ability to survive in time-varying microenvironment, which results, regarding the therapeutic context, in its therapeutic resistance. As it is accepted that the development of resistance is the direct consequence of intratumour heterogeneity, its evolutionary etiology is intensively studied. Models of carinogenesis are often assessed accordingly to how well they fit into the evolutionary scenario. In the paper, the relevant observations and concepts in cancer research, such as intratumour heterogeneity, cell plasticity, and Markov cell state dynamics, are reviewed and integrated into an evolutionary model. The possibility that the interaction between cancer cells can be interpreted as cooperation is proposed.The exploitation of the evolutionary modus operandi of cancer to steer its progression towards drug sensitive cancer cells is a challenging research topic. Integrating evolutionary principles into cancer therapy requires properly identified selection level, the relevant timescale, and the respective fitness of the principal selection unit on that timescale. Interpretation of some features of cancer progression, such as increased heterogeneity of isogenic cancer cells, is difficult from the most straightforward evolutionary view with the cancer cell as the principal selection unit. In the paper, the relation between the two levels of intratumour heterogeneity, genetic, due to genetic instability, and non-genetic, due to phenotypic plasticity, is reviewed and the evolutionary role of the latter is outlined. In analogy to the evolutionary optimization in a changing environment, the cell state dynamics in cancer clones are interpreted as the risk diversifying strategy bet hedging, optimizing the balance between the exploitation and exploration of the cell state space.

A systematic exploration of local network state space in neocortical mouse brain slices

The ex vivo cortical slice is an extremely versatile preparation, but its utility ultimately depends on understanding its limitations and functional constraints. A question for experimentalists new to the field of cortical slice electrophysiology might be — what are the different network dynamical states available to a cortical slice as a function of excitatory drive? The purpose of this study is to provide a coherent answer to this question, within the context of extracellularly recorded population field potentials.Cortical slices (400 µm) were prepared from adult male or female C57 mice. Evoked responses were recorded within cortical layer III/IV using extracellularly positioned metal electrodes. In the first part of the study, slice excitatory drive was increased by reducing the concentration of magnesium ions in the artificial cerebrospinal fluid — and the evoked responses categorized during the transition. In the second part, each of the identified functional states were explored in greater detail with tissue perfusion conditions and excitatory drive optimised for the requisite response state.As expected, rodent cortical slices did not generate spontaneous, persistent EEG-like field potential activity. However, distinct response states (spontaneous and evoked) characterized by intermittent population bursts could be differentiated as a function of excitatory drive. Each state reflected different modes of neocortical activation: “monosynaptic” responses were brief, non-propagating activations, reflecting an inhibited cortex with sensory processing blocked; polysynaptic and epileptiform activity propagated intra-cortically, the latter reflecting a hyperactivated, hypersynchronous “seizing” cortex. Polysynaptic activity most closely resembled sensory “up states” associated with intracortical sensory processing.Understanding the functional distinction between the different cortical slice response states is the starting point for designing experiments that maximise the utility of this ex vivo model. The results and descriptions in this study should help slice experimentalists less experienced in the nuances of cortical slice neurophysiology to make informed choices about how to tailor the parameters of the model to suit the specific aims of their research.

Expressiveness and analysis of Delayable Timed Petri Net*

We consider an extension of Timed Petri Nets “à la Ramchandani” where the transitions are partitioned into delayable and non-delayable transitions which has proven to be suitable for the design of synchronous circuits. For this model called Delayable Timed Petri Net (DTPN), the firing delay of a non-delayable transition is strict whereas a delayable transition can miss its firing delay. Since the delays are natural numbers, this model can be studied as a discrete time model.We deal with the expressiveness of DTPN by a comparison with the well known Merlin's Time Petri Net model for which transitions can fire in a time interval. We show that DTPN are strictly more expressive w.r.t. weak timed bisimilarity than Merlin's model under the discrete-time semantics. We then deal with the symbolic reachability analysis of DTPN, for which we show the complexity of the successor symbolic state computation to be O(n). In addition, we propose a reduction of the number of edges to explore that preserves the markings and the firing sequences. The symbolic state space exploration is implemented in a prototype tool, which is evaluated on a classical TPN problem and a circuit design application.