Discrete-time Stochastic Systems Research Articles

Stability Analysis of Discrete-Time Stochastic Delay Systems with Impulses

This paper is concerned with stability analysis of discrete-time stochastic delay systems with impulses. By using the sums average value of the time-varying coefficients and the average impulsive interval, two sufficient criteria for exponential stability of discrete-time impulsive stochastic delay systems are derived, which are more convenient to be applied than those Razumikhin-type conditions in previous literature. Both pth moment asymptotic stability and pth moment exponential stability are considered. Finally, two numerical examples to illustrate the effectiveness.

Fault reconstruction and resilient control for discrete-time stochastic systems

In this paper, a novel resilient control technique is proposed for discrete-time stochastic Brownian systems with simultaneous unknown inputs and unexpected faults. Prior to previous work, the stochastic Brownian system under consideration is quite general, where stochastic perturbations exist in states, control inputs, uncertainties, and faults. Moreover, the unknown input uncertainties concerned cannot be fully decoupled. Innovative observer by employing augmented system approach, decomposition observer, and optimization algorithms is proposed to achieve simultaneous estimates of both states and faults. Furthermore, fault reconstruction-based signal compensation is formulated to alleviate the effects from actuator faults and sensor faults. An observer-based controller is eventually constructed to enhance the stability and robustness of the closed-loop dynamic system. The integrated resilient control technique can ensure the system has reliable output even under faults. Both linear systems and Lipschitz nonlinear systems are investigated and the design procedures are addressed, respectively. Finally, the proposed resilient control techniques are validated via an electromechanical servo-system, and an aircraft system.

Quantitative verification of Kalman filters

AbstractKalman filters are widely used for estimating the state of a system based on noisy or inaccurate sensor readings, for example in the control and navigation of vehicles or robots. However, numerical instability or modelling errors may lead to divergence of the filter, leading to erroneous estimations. Establishing robustness against such issues can be challenging. We propose novel formal verification techniques and software to perform a rigorous quantitative analysis of the effectiveness of Kalman filters. We present a general framework for modelling Kalman filter implementations operating on linear discrete-time stochastic systems, and techniques to systematically construct a Markov model of the filter's operation using truncation and discretisation of the stochastic noise model. Numerical stability and divergence properties are then verified using probabilistic model checking. We evaluate the scalability and accuracy of our approach on two distinct probabilistic kinematic models and four Kalman filter implementations.

Control of Discrete-Time Stochastic Systems With Packet Loss by Event-Triggered Approach

Event-triggered schemes are characterized in less communication traffic while maintaining the resulting controlled plant's desired stability and performance criteria. In the presence of packet dropouts, this paper is concerned with the modeling and control problems for a class of discrete-time stochastic systems with event-triggered schemes. Two different mathematical analysis methods are proposed to model the packet loss when an event-triggered scheme is subject to packet loss. First, a stochastic distributed sequence satisfying the Bernoulli process is utilized to model the triggered packets transmitted in the communication networks. Considering the difference between time-triggered and event-triggered schemes, an equivalent model with a random sequence not satisfying a Bernoulli distribution process is also analyzed, which is not the same as some existing results in the literature. Then, the mean-square exponential stability of resulting augmented system is guaranteed and the prescribed H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> performance level is achieved by solving resulting discrete P-problem. Two examples with simulations are provided to validate the analytical results and demonstrate the effectiveness of the proposed co-design techniques.

Observers Design for a Class of Nonlinear Stochastic Discrete-Time Systems

This paper mainly studied the observer design of Lipschitz stochastic discrete system. For the first time, generalized Lipschitz conditions are introduced into the observer design of a class of nonlinear stochastic discrete systems. From this paper, it can be seen that the generalized Lipschitz condition can better reflect the structural information of the nonlinear part, and thus has more advantages than the classical Lipschitz condition in the observer design. The stability criterion and observer design method of nonlinear stochastic discrete systems were proposed. Numerical examples illustrated the advantages and feasibility of the theoretical results obtained.

A New Approach to Characterize Successive Packet Losses in Stochastic Networked Systems

In this paper, we discuss the modeling and stabilization problems for a class of discrete-time stochastic networked systems (DSNSs) subject to successive packet losses. The system under consideration is transformed into a stochastic one with bounded stochastic delay. Considering that the stochastic delay is characterized by nonuniform distribution, a new equivalent model is then constructed that enables the DSNS’s controller design to benefit from knowing the probability characteristic of packet losses. Specially, to verify this feature, the probabilities of the delay can be explicitly obtained by utilizing the formula of total probability, which is critical to model transformation and analyze practical problems that exist in DSNSs. Based on the proposed model, sufficient conditions are established under which the globally mean-square asymptotic stability of resulting stochastic delay closed-loop system is guaranteed, and a delay-distribution-dependent design procedure is then proposed. A numerical example with simulation is provided to validate the analytical results and demonstrate the effectiveness of the design procedure.

Power Systems Dynamic State Estimation With the Two-Step Fault Tolerant Extended Kalman Filtering

Bad data may lead to performance degradation or even instability of a power system, which can be caused by various factors: unintentional PMU abnormalities, topology error, malicious cyber-attacks, electromagnetic interference, temporary loss of communication links, external disturbances, extraneous noise biases, etc. In order to develop a more resilient and reliable state estimation technique, this manuscript presents a novel two-step fault tolerant extended Kalman filter framework for discrete-time stochastic power systems, under bad data, PMU failures, external disturbances, extraneous noise, and bounded observer-gain perturbation conditions. The failure mechanisms of multiple phasor measurement units are assumed to be independent of each other with various bad data or malfunction rates. The benchmark IEEE standard test systems are utilized as a demonstrative example to carry out computer simulation studies and to examine different estimation algorithms. Experimental results demonstrates that the proposed second-order fault tolerant extended Kalman filter provides more accurate estimation results, in comparison with traditional first- and second-order extended Kalman filter, and the unscented Kalman filter. The proposed two-step fault-tolerant extended Kalman filter can serve as a powerful alternative to the existing dynamic power system state estimation techniques.

Robust Fault Detection Based on l1 Regularization

Herein, regularization‐based fault detection technique for stochastic discrete time systems in state space form is discussed. Compared with the deterministic nature of a fault which usually causes an abrupt change, the state of a system smoothly evolves in most cases and the disturbance in the process and the sensor measurement has a stochastic characteristic. The modeling uncertainty in the state space of a system can induce a bias in deterministic way and it can be combined to a fault. In the fault detection community, the modeling uncertainty is called a multiplicative fault and the abruptly‐changing fault is called an additive fault. Inspired by the fact that norm is useful for estimating smoothly evolving states and reducing a bias and norm for detecting abrupt change with spare structure, this research develops the technique for detecting fault which changes abruptly under stochastic disturbance and modeling uncertainty. The norm for bias compensation (due to modeling uncertainty and/or additive fault) is combined with norm for fault detection (especially abruptly changing fault) and the two norms are set up as a regularization problem. The regularization problem is convex thus, global solution can be found in efficient way.

Optimal Control of a Discrete-Time Stochastic System with a Probabilistic Criterion and a Non-fixed Terminal Time

This paper considers an optimal control problem for a discrete-time stochastic system with the probability of first reaching the boundaries of a given domain as the optimality criterion. Dynamic programming-based sufficient conditions of optimality are formulated and proved. The isobells of levels 1 and 0 of the Bellman function are used for obtaining two-sided estimates of the right-hand side of the dynamic programming equation, two-sided estimates of the Bellman function, and two-sided estimates of the optimal-value function of the probabilistic criterion. A suboptimal control design method is proposed. The conditions of equivalence to an optimal control problem with a probabilistic terminal criterion are established. An illustrative example is given.

Compositional (In)Finite Abstractions for Large-Scale Interconnected Stochastic Systems

This article is concerned with a compositional approach for constructing both infinite (reduced-order models) and finite abstractions [a.k.a. finite Markov decision processes (MDPs)] of large-scale interconnected discrete-time stochastic systems. The proposed framework is based on the notion of stochastic simulation functions enabling us to employ an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds. In the first part of this article, we derive sufficient small-gain-type conditions for the compositional quantification of the probabilistic distance between the interconnection of stochastic control subsystems and that of their infinite abstractions. We then construct infinite abstractions together with their corresponding stochastic simulation functions for a particular class of discrete-time nonlinear stochastic control systems. In the second part of this article, we leverage small-gain-type conditions for the compositional construction of finite abstractions. We propose an approach to construct finite MDPs as finite abstractions of concrete models or their reduced-order versions satisfying an incremental input-to-state stability property. We also show that for the particular class of nonlinear stochastic control systems, the aforementioned property can be readily checked by matrix inequalities. We demonstrate the effectiveness of the proposed results by applying our approaches to a fully interconnected network of 20 nonlinear subsystems (totally 100 dimensions). We construct finite MDPs from their reduced-order versions (together 20 dimensions) with guaranteed error bounds on their output trajectories. We also apply the proposed results to a temperature regulation in a circular building and construct compositionally a finite abstraction of a network containing 1000 rooms. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies regulating the temperature in each room for a bounded time horizon.

A Two-Phase Distributed Filtering Algorithm for Networked Uncertain Systems with Fading Measurements under Deception Attacks.

In this paper, the distributed filtering problem is addressed for a class of discrete-time stochastic systems over a sensor network with a given topology, susceptible to suffering deception attacks, launched by potential adversaries, which can randomly succeed or not with a known success probability, which is not necessarily the same for the different sensors. The system model integrates some random imperfections and features that are frequently found in real networked environments, namely: (1) fading measurements; (2) multiplicative noises in both the state and measurement equations; and (3) sensor additive noises cross-correlated with each other and with the process noise. According to the network communication scheme, besides its own local measurements, each sensor receives the measured outputs from its adjacent nodes. Based on such measurements, a recursive algorithm is designed to obtain the least-squares linear filter of the state. Thereafter, each sensor receives the filtering estimators previously obtained by its adjacent nodes, and these estimators are all fused to obtain the desired distributed filter as the minimum mean squared error matrix-weighted linear combination of them. The theoretical results are illustrated by a simulation example, where the efficiency of the developed distributed estimation strategy is discussed in terms of the error variances.

An Optimal Recurrent Logical-Dynamical Filter of a High Order and Its Covariance Approximations

The problem of logical recognition of the operating mode and dynamic estimation of the intra-mode state vector of a discrete-time stochastic Markov system with a random structure is considered. To develop an estimation algorithm implemented at the pace of system time on a computer of limited power, a method for designing a new finite-dimensional optimal structure filter is proposed. Its state vector consists of several latest estimates, and the current estimate is found in the form of an optimal-accuracy dependence on the last measurement and the vector of the previous filter state. The structural functions of the filter are designed in advance, which can be performed by the Monte Carlo method by obtaining their multivariate histograms. Due to the computational complexity of this procedure, algorithms are also proposed for constructing two numerical-analytical approximations of the filter, which take into account only the first two moments of random variables.

Robust Distributed H∞ State Estimation for Stochastic Periodic Systems Over Constraint Sensor Networks

In this paper, the distributed H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> state estimators are designed for discrete-time stochastic periodic systems with randomly occurred uncertainties and topology-dependent quantizers over sensor networks. A norm-bounded uncertain model is introduced to describe the randomly occurred norm-bounded parameter uncertainties, where each component of the system occurs uncertainty independently. According to the changeable topologies of the sensor networks, a topology-dependent quantizer is proposed to enhance the channel utilization. Sufficient conditions are established to ensure the asymptotic stability and the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> performance simultaneously for the augmented estimation error system. Then the gains of estimators are derived in every period. Finally, the average of one thousand simulations are depicted in the figures. Moreover, simulation comparisons are given to explain the quantizer design method.

Event-Based Security Control for Stochastic Networked Systems Subject to Attacks

This paper is concerned with the event-based security control problems for a set of discrete-time stochastic systems suffered from randomly occurred attacks, especially the denial-of-service attacks and the deception attacks. An attack model with compensation is established to describe combinations of attacks to the networked control systems. An event-triggered mechanism is employed to debase the communication load by transmitting measurement signals when a definite triggering condition is satisfied. A definition of security probability is proposed to describe the transient state of controller systems. The aim of this paper is to design a dynamic output feedback controller, thereupon the closed-loop system achieves the described security in probability. Some novel sufficient conditions are proposed to guarantee the input-to-state stability of the system in probability, and the controller gains are designed by solving a set of matrix inequalities. The upper bound of the quadratic cost function is also derived. Finally, simulation examples are applied to illustrate the effectiveness of the proposed design scheme.

Probabilistic Criterion-Based Optimal Retention of Trajectories of a Discrete-Time Stochastic System in a Given Tube: Bilateral Estimation of the Bellman Function

This paper examines an optimal control problem with a probabilistic criterion for retaining the trajectories of a discrete-time stochastic system in given sets. The dynamic programming method is employed for obtaining the isobells of levels 1 and 0 of the Bellman function, two-sided estimates for the right-hand side of the dynamic programming equation, two-sided estimates for the Bellman function, and the optimal-value function of the probabilistic criterion. These results are then used for deriving an approximate formula for the optimal control. As an illustrative example the problem of keeping an inverted pendulum in the neighborhood of an unstable equilibrium is considered.

Model predictive control design for constrained Markov jump bilinear stochastic systems with an application in finance

In this study, we propose a solution to the model predictive control problem for a class of constrained discrete-time bilinear stochastic systems consisting of two coupled subsystems with Markov jumps. The first one includes a bilinear term in the state variables of the second subsystem and the input, whereas the second subsystem is described by a Markov switching vector autoregressive model. Furthermore, hard constraints imposed on the input manipulated variables. The results obtained are applied to the dynamic investment portfolio selection problem for a financial market with serially dependent returns and switching modes, subject to hard constraints on trading amounts. Our approach is tested on a real dataset from the New York Stock Exchange and the Russian Stock Exchange MOEX.

Formal Synthesis of Stochastic Systems via Control Barrier Certificates

This paper focuses on synthesizing control policies for discrete-time stochastic control systems together with a lower bound on the probability that the systems satisfy the complex temporal properties. The desired properties of the system are expressed as linear temporal logic (LTL) specifications over finite traces. In particular, our approach decomposes the given specification into simpler reachability tasks based on its automata representation. We then propose the use of so-called \emph{control barrier certificate} to solve those simpler reachability tasks along with computing the corresponding controllers and probability bounds. Finally, we combine those controllers to obtain a hybrid control policy solving the considered problem. Under some assumptions, we also provide two systematic approaches for uncountable and finite input sets to search for control barrier certificates. We demonstrate the effectiveness of the proposed approach on a room temperature control and lane-keeping of a vehicle modeled as a four-dimensional single-track kinematic model. We compare our results with the discretization-based methods in the literature.

Pth moment exponential stability of general nonlinear discrete-time stochastic systems

pth moment exponential stability of general nonlinear discrete-time stochastic systems

Fusion of PDF compensation and gain-scheduled control for discrete stochastic systems with randomly occurring nonlinearities

A fusion control strategy, including probability density function (PDF) compensation and gain-scheduled control, is proposed for discrete-time stochastic systems with randomly occurring nonlinearities. Usually, it is difficult to drive the tracking error strictly to zero in a random environment. So the proposed method aims to stabilize the closed-loop system in the exponentially mean square sense and guarantee the distribution of output error as close to the expected PDF as possible. The parameters of the gain-scheduled controller are obtained by solving appropriate linear matrix inequalities, and the parameters of the PDF compensator are updated in real time according to the Kullback–Leibler divergence between the PDF of output error and the desired one. The proposed method has the characteristics of low conservativeness and easy to implement. The effectiveness of the proposed method is shown by two examples.

Specification-Guided Verification and Abstraction Refinement of Mixed Monotone Stochastic Systems

This article addresses the problem of verifying discrete-time stochastic systems against omega-regular specifications using finite-state abstractions. Omega-regular properties allow specifying complex behavior and encompass, for example, linear temporal logic. We focus on a class of systems with mixed monotone dynamics. This class is shown to be amenable to efficient reachable set computation and models a wide range of physically relevant systems. In general, finite-state abstractions of continuous state stochastic systems give rise to augmented Markov chains wherein the probabilities of transition between states are restricted to an interval. We present a procedure to compute a finite-state interval-valued Markov chain (IMC) abstraction of discrete-time, mixed monotone stochastic systems subject to affine disturbances given a rectangular partition of the state space. Then, we suggest an algorithm for performing verification against omega-regular properties in IMCs. Specifically, we aim to compute bounds on the probability of satisfying a specification from any initial state in the IMC. This is achieved by solving a reachability problem on the sets of so-called winning and losing components in the Cartesian product between the IMC and a Rabin automaton representing the specification. Next, the verification of IMCs may yield a set of states whose acceptance status is undecided with respect to the specification, requiring a refinement of the abstraction. We describe a specification-guided approach that compares the best and worst case behaviors of accepting paths in the IMC and targets the appropriate states accordingly. Finally, we show a case study.