Statistical Model Checking Research Articles

Behavior Safety Decision-Making Based on Deep Deterministic Policy Gradient and Its Verification Method

As an emerging mode of transportation, autonomous vehicles are increasingly attracting widespread attention. To address the issues of the traditional reinforcement learning algorithm, which only considers discrete actions within the system and cannot ensure the safety of decision-making, this paper proposes a behavior decision-making method based on the deep deterministic policy gradient. Firstly, to enable autonomous vehicles to drive as close to the center of the road as possible while sensitively avoiding surrounding obstacles, the reward function for reinforcement learning is constructed by comprehensively considering road boundaries and nearby vehicles. We account for the symmetry of the road by calculating the distances between the vehicle and both the left and right road boundaries, ensuring that the vehicle remains centered within the road. Secondly, to ensure the safety of decision-making, the safety constraints in autonomous driving scenarios are described using probabilistic computation tree logic, and the scenario is modeled as a stochastic hybrid automaton. Finally, the model is verified by the statistical model checker UPPAAL. The above method enables autonomous vehicles not only to independently acquire driving skills across diverse driving environments but also significantly enhances their obstacle avoidance capabilities, thereby ensuring driving safety.

Symbolic state-space exploration meets statistical model checking

Efficient reachability analysis, as well as statistical model checking have been proposed for the evaluation of Hybrid Petri nets with general transitions (HPnGs). Both have different (dis-)advantages. The performance of statistical simulation suffers in large models and the number of required simulation runs to achieve a relatively small confidence interval increases considerably. The approach introduced for analytical reachability analysis of HPnGs, however, becomes infeasible for a large number of random variables. To overcome these limitations, this paper applies statistical model checking (SMC) for a stochastic variant of the Signal Temporal Logic (STL) to a pre-computed symbolic state-space representation of HPnGs, i.e., the Parametric Location Tree (PLT), which has previously been used for model checking HPnGs. Furthermore, we define how to reduce the PLT for a given state-based and path-based STL property, by introducing a three-valued interpretation of a given STL property for every location of the PLT. This paper applies learning in the presence of nondeterminism and considers four different scheduler classes. The proposed improvement is especially useful if a large number of training runs is necessary to optimize the probability that a given STL property holds. A case study on a water tank model shows the feasibility of the approach, as well as improved computation times, when applying the above-mentioned reduction for varying time bounds. We validate our results with existing analytical and simulation tools, as applicable for different types of schedulers.

Scaling up statistical model checking of cyber-physical systems via algorithm ensemble and parallel simulations over HPC infrastructures

Model-based formal verification of industry-relevant Cyber-Physical Systems (CPSs) is often a computationally prohibitive task. In most cases, the complexity of the models precludes any prospect of symbolic analysis, leaving numerical simulation as the only viable option. Unfortunately, exhaustive simulation of a CPS model over the entire set of plausible operational scenarios is rarely possible in practice, and alternative strategies such as Statistical Model Checking (SMC) must be used instead.In this article, we show that the number of model simulations (samples) required by SMC techniques to converge can be significantly reduced by considering multiple (an ensemble of) Adaptive Stopping Algorithms (SAs) at once, and that the simulations themselves (by far the most expensive step of the entire workload) can be efficiently sped up by exploiting massively parallel platforms.With three industry-scale CPS models, we experimentally show that the use of an ensemble of two state-of-the-art SAs (AA and EBGStop) may require dozens of millions fewer samples when compared to running a single algorithm, with reductions in sample size of up to 78%. Furthermore, we show that our implementation, by massively parallelizing system model simulations on a HPC infrastructure, yields speed-ups for the completion time of the verification tasks which are practically linear with respect to the number of computational nodes, thus achieving an efficiency of virtually 100%, even on very large platforms. This makes it possible to complete tasks of model-based SMC verification for complex CPSs in a matter of hours or days, whereas a naïve sequential execution would require from months to many years.

Quantum Probabilistic Model Checking for Time-Bounded Properties

Probabilistic model checking (PMC) is a verification technique for analyzing the properties of probabilistic systems. However, existing techniques face challenges in verifying large systems with high accuracy. PMC struggles with state explosion, where the number of states grows exponentially with the size of the system, making large system verification infeasible. While statistical model checking (SMC) avoids PMC’s state explosion problem by using a simulation approach, it suffers from runtime explosion, requiring numerous samples for high accuracy. To address these limitations in verifying large systems with high accuracy, we present quantum probabilistic model checking (QPMC), the first method leveraging quantum computing for PMC with respect to time-bounded properties. QPMC addresses state explosion by encoding PMC problems into quantum circuits that superpose states within qubits. Additionally, QPMC resolves runtime explosion through Quantum Amplitude Estimation, efficiently estimating the probabilities of specified properties. We prove that QPMC correctly solves PMC problems and achieves a quadratic speedup in time complexity compared to SMC.

PAC statistical model checking of mean payoff in discrete- and continuous-time MDP

PAC statistical model checking of mean payoff in discrete- and continuous-time MDP

Serial and parallel algorithms for short time horizon multi-attribute queries on stochastic multi-agent systems

Statistical model checking (SMC) for the analysis of multi-agent systems has been studied in the recent past. A feature peculiar to multi-agent systems in the context of statistical model checking is that of aggregate queries–temporal logic formula that involves a large number of agents. To answer such queries through Monte Carlo sampling, the statistical approach to model checking simulates the entire agent population and evaluates the query. This makes the simulation overhead significantly higher than the query evaluation overhead. This problem becomes particularly challenging when the model checking queries involve multiple attributes of the agents. To alleviate this problem, we propose a population sampling algorithm that simulates only a subset of all the agents and scales to multiple attributes, thus making the solution generic. The population sampling approach results in increased efficiency (a gain in running time of 50%–100%) for a marginal loss in accuracy (between 1% and 5%) when compared with the exhaustive approach (which simulates the entire agent population to evaluate the query), especially for queries that involve limited time horizons. Finally, we report parallel versions of the above algorithms. We explore different strategies of core allocation, both for exhaustive simulations of all agents and the sampling approach. This yields further gains in running time, as expected. The parallel approach, when combined with the sampling idea, results in improving the efficiency (a gain in running time of 100%–150%) with a minor loss when compared with the exhaustive approach in accuracy (between 1% and 5%).

Computational assessment of Amazon forest plots regrowth capacity under strong spatial variability for simulating logging scenarios

In this paper, we assess the regrowth capacity of tropical forest plots by developing an original computational procedure based on statistical model checking methods. We calibrate a new mathematical model of forest dynamics with respect to post-logging data, produced in the Amazon basin. Our mathematical model is determined by a mechanistic system of ordinary differential equations and integrates a new nonlinear aging term, which is necessary to reproduce the complex post-logging dynamics of the tropical forest plots. Our method is based on an efficient algorithmic procedure that explores the parameter space of the model and computes a score for each parameter value, depending on the distance between the trajectory of the model and the data. We distinguish a group of four reference plots, for which the logging was the most intense, from another group of five non-reference plots, which are fitted a posteriori. Our results provide a set trajectories of the new model, which successfully fit the non-monotone post-logging data on each forest plot, in spite of the high level of biological variability identified in the study site. Rather than a unique set of parameters, we return a small cell of parameters, extracted from the parameter space, which contains in its close vicinity several relevant sets of parameters that are equally able to reproduce the regrowth dynamics of distinct tropical forest plots. The size of the cells that should be extracted from the parameter space increases with the level of biological variability. Finally, we show how to use the calibrated mathematical model for simulating logging scenarios, so as to better understand the temporal dynamics of forest regrowth.

End-to-End Statistical Model Checking for Parameterization and Stability Analysis of ODE Models

We propose a simulation-based technique for the parameterization and the stability analysis of parametric Ordinary Differential Equations. This technique is an adaptation of Statistical Model Checking, often used to verify the validity of biological models, to the setting of Ordinary Differential Equations systems. The aim of our technique is to estimate the probability of satisfying a given property under the variability of the parameter or initial condition of the ODE, with any metrics of choice. To do so, we discretize the values space and use statistical model checking to evaluate each individual value w.r.t. provided data. Contrary to other existing methods, we provide statistical guarantees regarding our results that take into account the unavoidable approximation errors introduced through the numerical integration of the ODE system performed while simulating. In order to show the potential of our technique, we present its application to two case studies taken from the literature, one relative to the growth of a jellyfish population, and the other concerning a well-known oscillator model.

Correctness Verification of Mutual Exclusion Algorithms by Model Checking

Mutual exclusion algorithms are at the heart of concurrent/parallel and distributed systems. It is well known that such algorithms are very difficult to analyze, and in the literature, different conjectures about starvation freedom and the number of by-passes (also called the overtaking factor) exist. The overtaking factor affects the (hopefully) bounded waiting time that a process competing for entering the critical section has to suffer before accessing the shared resource. This paper proposes a novel modeling approach based on Timed Automata and the Uppaal toolset, which proves effective for studying all the properties of a mutual exclusion algorithm for N≥2 processes, by exhaustive model checking. Although the approach, as already confirmed by similar experiments reported in the literature, is not scalable due to state explosion problems and can be practically applied until N≤5, it is of great value for revealing the true properties of analyzed algorithms. For dimensions N>5, the Statistical Model Checker of Uppaal can be used, which, although based on simulations, can confirm properties by estimations and probabilities. This paper describes the proposed modeling and verification method and applies it to several mutual exclusion algorithms, thus retrieving known properties but also showing new results about properties often studied by informal reasoning.

Statistical Verification using Surrogate Models and Conformal Inference and a Comparison with Risk-Aware Verification

Uncertainty in safety-critical cyber-physical systems can be modeled using a finite number of parameters or parameterized input signals. Given a system specification in Signal Temporal Logic (STL), we would like to verify that for all (infinite) values of the model parameters/input signals, the system satisfies its specification. Unfortunately, this problem is undecidable in general. Statistical model checking (SMC) offers a solution by providing guarantees on the correctness of CPS models by statistically reasoning on model simulations. We propose a new approach for statistical verification of CPS models for user-provided distribution on the model parameters. Our technique uses model simulations to learn surrogate models , and uses conformal inference to provide probabilistic guarantees on the satisfaction of a given STL property. Additionally, we can provide prediction intervals containing the quantitative satisfaction values of the given STL property for any user-specified confidence level. We compare this prediction interval with the interval we get using risk estimation procedures. We also propose a refinement procedure based on Gaussian Process (GP)-based surrogate models for obtaining fine-grained probabilistic guarantees over sub-regions in the parameter space. This in turn enables the CPS designer to choose assured validity domains in the parameter space for safety-critical applications. Finally, we demonstrate the efficacy of our technique on several CPS models.

Formal Modeling and Verification of Embedded Real-Time Systems: An Approach and Practical Tool Based on Constraint Time Petri Nets

Modeling and verification of the correct behavior of embedded real-time systems with strict timing constraints is a well-known and important problem. Failing to fulfill a deadline in system operation can have severe consequences in the practical case. This paper proposes an approach to formal modeling and schedulability analysis. A novel extension of Petri Nets named Constraint Time Petri Nets (C-TPN) is developed, which enables the modeling of a collection of interdependent real-time tasks whose execution is constrained by the use of priority and shared resources like processors and memory data. A C-TPN model is reduced to a network of Timed Automata in the context of the popular Uppaal toolbox. Both functional and, most importantly, temporal properties can be assessed by exhaustive model checking and/or statistical model checking based on simulations. This paper first describes and motivates the proposed C-TPN modeling language and its formal semantics. Then, a Uppaal translation is shown. Finally, three models of embedded real-time systems are considered, and their properties are thoroughly verified.

White-box validation of quantitative product lines by statistical model checking and process mining

We propose a novel methodology to validate software product line (PL) models by integrating Statistical Model Checking (SMC) with Process Mining (PM). We consider the feature-oriented language QFLan from the PL engineering domain. QFLan allows to model PL equipped with rich cross-tree and quantitative constraints, as well as aspects of dynamic PLs such as the staged configurations. This richness allows us to easily obtain models with infinite state-space, calling for simulation-based analysis techniques, like SMC. For example, we use a running example with infinite state space. SMC is a family of analysis techniques based on the generation of samples of the dynamics of a system. SMC aims at estimating properties of a system like the probability of a given event (e.g., installing a feature), or the expected value of quantities in it (e.g., the average price of products from the studied family). Instead, PM is a family of data-driven techniques that uses logs collected on the execution of an information system to identify and reason about its underlying execution process. This often regards identifying and reasoning about process patterns, bottlenecks, and possibilities for improvement. In this paper, to the best of our knowledge, we propose, for the first time, the application of Process Mining (PM) techniques to the byproducts of Statistical Model Checking (SMC) simulations. This aims to enhance the utility of SMC analyses. Typically, if SMC gives unexpected results, the modeler has to discover whether these come from actual characteristics of the system, or from bugs in the model. This is done in a black-box manner, only based on the obtained numerical values. We improve on this by using PM to get a white-box perspective on the dynamics of the system observed by SMC. Roughly speaking, we feed the samples generated by SMC to PM tools, obtaining a compact graphical representation of the observed dynamics. This mined PM model is then transformed into a mined QFLan model, making it accessible to PL engineers. Using two well-known PL models, we show that our methodology is effective (helps in pinpointing issues in models, and in suggesting fixes), and that it scales to complex models. We also show that it is general, by applying it to the security domain.

Statistical Model Checking in Process Mining: A Comprehensive Approach to Analyse Stochastic Processes

The study of business process analysis and optimisation has attracted significant scholarly interest in the recent past, due to its integral role in boosting organisational performance. A specific area of focus within this broader research field is process mining (PM). Its purpose is to extract knowledge and insights from event logs maintained by information systems, thereby discovering process models and identifying process-related issues. On the other hand, statistical model checking (SMC) is a verification technique used to analyse and validate properties of stochastic systems that employs statistical methods and random sampling to estimate the likelihood of a property being satisfied. In a seamless business setting, it is essential to validate and verify process models. The objective of this paper is to apply the SMC technique in process mining for the verification and validation of process models with stochastic behaviour and large state space, where probabilistic model checking is not feasible. We propose a novel methodology in this research direction that integrates SMC and PM by formally modelling discovered and replayed process models and apply statistical methods to estimate the results. The methodology facilitates an automated and proficient evaluation of the extent to which a process model aligns with user requirements and assists in selecting the optimal model. We demonstrate the effectiveness of our methodology with a case study of a loan application process performed in a financial institution that deals with loan applications submitted by customers. The case study highlights our methodology’s capability to identify the performance constraints of various process models and aid enhancement efforts.

DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version

Neural networks (NN) are gaining importance in sequential decision-making. Deep reinforcement learning (DRL), in particular, is extremely successful in learning action policies in complex and dynamic environments. Despite this success, however, DRL technology is not without its failures, especially in safety-critical applications: (i) the training objective maximizes average rewards, which may disregard rare but critical situations and hence lack local robustness; (ii) optimization objectives targeting safety typically yield degenerated reward structures, which, for DRL to work, must be replaced with proxy objectives. Here, we introduce a methodology that can help to address both deficiencies. We incorporate evaluation stages (ES) into DRL, leveraging recent work on deep statistical model checking (DSMC), which verifies NN policies in Markov decision processes. Our ES apply DSMC at regular intervals to determine state space regions with weak performance. We adapt the subsequent DRL training priorities based on the outcome, (i) focusing DRL on critical situations and (ii) allowing to foster arbitrary objectives. We run case studies on two benchmarks. One of them is the Racetrack, an abstraction of autonomous driving that requires navigating a map without crashing into a wall. The other is MiniGrid, a widely used benchmark in the AI community. Our results show that DSMC-based ES can significantly improve both (i) and (ii).

OSM: Leveraging model checking for observing dynamic behaviors in aspect-oriented applications

In the intricate domain of software systems verification, dynamically model checking multifaceted system characteristics remains paramount, yet challenging. This research proposes the advanced observe-based statistical model-checking (OSM) framework, devised to craft executable formal models directly from foundational system code. Leveraging model checking predicates, the framework melds seamlessly with aspect-oriented programming paradigms, yielding a potent method for the analytical verification of varied behavioral attributes. Exploiting the transformative capacity of OSM framework, primary system code undergoes a systematic metamorphosis into multifaceted analysis constructs. This not only simplifies the model verification process but also orchestrates feature interactions using an innovative observing join point abstraction mechanism. Within this framework, components encompassing parsing, formal verification, computational analytics, and rigorous validation are intrinsically interwoven. Marrying the principles of model checking with aspect-oriented (AO) modularization, OSM framework stands as a paragon, proficiently scrutinizing and affirming system specifications. This ensures the unyielding performance of electronic health record systems amidst shifting preconditions. OSM framework offers runtime verification of both object-oriented and AO deployments, positioning itself as an indispensable open-source resource, poised to automate the enhancement of system performance and scalability.

Stochastic formal model of PI3K/mTOR pathway in Alzheimer's disease for drug repurposing: An evaluation of rapamycin, LY294002, and NVP-BEZ235

Alzheimer's disease is the most common form of dementia characterized by the gradual loss of memory and cognition of patients. There is no drug capable of curing or preventing the disease, being the discovery of an efficient treatment for the disease is of vital importance. An approach that can contribute to this purpose of reducing time and costs of new discoveries is drug repositioning and in silico techniques. Statistical Model Checking is a formal verification technique that can aid in analyzing protein and drugs interactions and test pharmacological strategies, contributing to drug discovery and repurposing. In this work, we present a stochastic formal model that allows us to test different drugs, or combination of drugs, that target the PI3K/mTOR pathway and to evaluate the effect on tau protein and A β peptide, which are two important components that contribute to the progression of Alzheimer's disease. We have analyzed the effect of rapamycin, LY294002, and NVP-BEZ235 on those proteins. Our results have shown that rapamycin has the potential to slow down one of the biological processes that cause neuronal death. Moreover, we have identified the ideal dose of rapamycin to obtain such results. However, our findings have unveiled that LY294002 and NVP-BEZ235 can increase tau phosphorylation in all scenarios tested. This is an alert to the scientific community, as this deleterious effect can be caused by other PI3K inhibitor drugs used to treat different diseases. These results suggest the inhibition of some proteins of the pathway can trigger an unexpected pathology. Besides that, rapamycin has not reduced this side effect when administered together with LY294002. The proposed methodology has been able to model both the disease and drug interactions. We have analyzed three drugs and our model is flexible to test other drugs and pharmacological strategies and efficient to analyze the properties of the model generating new insights.

Modular Modeling and Statistical Validation for Grid-Connected FS-MPC-Controlled Matrix Converters

In recent publications statistical model checking (SMC) has been proposed as a method for verifying the performance of finite-set model predictive control (FS-MPC) algorithms applied to power electronics converters. One of the reasons the full potential of the method in the power electronics systems (PES) has not yet been explored is the time consuming modelling process. In this paper we propose a modular method of modelling the power electronics system components by providing simple building blocks, which can be connected to build different PES. The modelling method is here demonstrated on a direct matrix converter, which operates in a stochastic grid with different harmonic distortion levels and voltage sags. By applying the SMC, the performance of the control algorithm in terms of the output current distortion, effects of the weighting factor selection and grid distortions on the device utilization can be evaluated. The obtained results confirm, that high grid distortions and voltage sags will increase the stress of several devices. This information can be of great importance to identify the most stressed components and how the control algorithm can be adapted to extend the lifetime of the components and thereby the system during different grid conditions. The verified FS-MPC algorithm has also been implemented in an experimental set-up.

Modelling and Analysis of a Sigfox-Based IoT Network Using uppaalsmc

Wireless sensor nodes are usually powered by batteries that have limited energy capacity. In many applications, the nodes are installed in inaccessible locations where they are problematic to replace or recharge. Therefore, energy optimisation is crucial for increasing the node’s lifetime. This study presents a method for the analysis and prediction of the energy consumption of Sigfox-based Wireless Sensor Nodes. The method is illustrated in a use case where the nodes monitor the water level in drainage lines in cities to improve surface and wastewater management. We propose a formal model-based technique using the UPPAAL Statistical model checker tool to model and analyse the node’s lifetime. Statistical model checking provides a highly scalable technique for performance analysis of complex cyber-physical systems. The model captures the energy-related behaviour of the node, the Sigfox radio specification, and the sensor, each parameterised with values from the device’s datasheets. Furthermore, we calibrate the model using measurements obtained from real-world hardware. Finally, we evaluate a collection of strategies to optimise the battery lifetime of the node. We simulate the model with a 10,000 mAh battery, and the results indicate that we can extend the node’s lifetime from 202 days to 2.71 years using our most optimised transmission strategy.

On the degradation of forest ecosystems by extreme events: Statistical Model Checking of a hybrid model

In this paper, we study the vulnerability of forest ecosystems perturbed by extreme events, such as those arising from climate change. To investigate the complex interactions between the biological dynamics of the forest and the climatic activity, we construct an original hybrid model, obtained by coupling a continuous reaction–diffusion system, which describes the spatio-temporal dynamics of the forest ecosystem, with a discrete probabilistic process, which models the possible occurrences of extreme events. Properties of ecological interest are considered: invariance of the persistence equilibrium, attraction to the extinction equilibrium and emergence of degraded states. Those properties of the hybrid model are verified through an extension of the Statistical Model Checking framework. We establish the existence of a threshold above which the persistence equilibrium of the forest ecosystem is compromised and give a numerical assessment of this threshold in terms of the probability and intensity of extreme events. We also present non-trivial parameter conditions for which the forest ecosystem converges to a degraded savanna-like state.

A framework for modeling and analyzing cyber-physical systems using statistical model checking

The trustworthiness of a cyber–physical system is essential for it to be qualified for utilization in most real-life deployments. This is especially critical for systems that deal with precious human lives. Although these safety-critical systems can be investigated using both experimental testing and model-based verification, accurate models have the potential to permit risk-free mimicking of the system behavior even in the most extreme scenarios. To overcome the CPS modeling and design challenges, the INCOSE/OMG standard System Modeling Language (SysML) is utilized in this work to accurately specify cyber–physical systems. For that, a bounded set of SysML constructs are defined to precisely capture the semantics of continuous-time and discrete-time system behaviors. Then, the SysML constructs are substituted by developing a new algebra, called Enhanced Activity Calculus (EAC). So, EAC helps construct equivalent priced timed automata models by developing a new systematic procedure to correctly translate the SysML models into the statistical model checking tool UPPAAL-SMC inputs. The latter checks whether the system is correct and safe or not. Moreover, the soundness of the developed translation mechanism has been proved and its effectiveness has been shown on a real use case, namely the artificial pancreas.