Discrete-time Nonlinear Stochastic Systems Research Articles

Partial Stability in Probability of Nonlinear Stochastic Discrete-Time Systems with Delay

A system of nonlinear stochastic functional-difference equations with finite delay is considered. By assumption, this system has a “partial” trivial equilibrium (with respect to part of the state variables). The problem under study is to analyze partial stability in probability of this equilibrium: stability is considered with respect to part of the variables determining it. The problem is solved using a discrete-stochastic modification of the method of Lyapunov–Krasovskii functionals. Conditions for partial stability in probability are established. An example is provided to illustrate the features of the proposed approach and the rationale for introducing a one-parameter family of functionals.

Partial Stability in Probability of Nonlinear Stochastic Discrete-Time Systems with Delay

Partial Stability in Probability of Nonlinear Stochastic Discrete-Time Systems with Delay

Finite-Time Robust Distributed Estimate for Nonlinear Systems With Heterogeneous Sensors.

This article proposes a finite-time distributed state estimation (DSE) algorithm for discrete-time stochastic nonlinear systems with heterogeneous sensors. Considering the network with heterogeneous sensors, the distributed estimate framework is designed by three phases, namely, priori prediction, measurement update, and consensus fusion. To obtain the accurate priori prediction results, the interactive multiple model (IMM) method is adopted to calculate the priori state value in the priori prediction phase. By introducing the measurement probability matrix, a novel heterogeneous measurement information fusion algorithm is designed. Then the measurement information of each sensor is used to update the priori prediction estimates to calculate the estimate results in the measurement update phase. Based on the consensus method, the estimate results of each sensor are fused with consensus weight to calculate the distributed state estimates of nonlinear systems in the consensus fusion phase. Besides, with finite consensus fusion steps, the bounds of the proposed distributed estimate algorithm are proved to be existed. Finally, distributed state estimate simulation example for nonlinear system is set to validate the performance.

Learning Control Policies for Stochastic Systems with Reach-Avoid Guarantees

We study the problem of learning controllers for discrete-time non-linear stochastic dynamical systems with formal reach-avoid guarantees. This work presents the first method for providing formal reach-avoid guarantees, which combine and generalize stability and safety guarantees, with a tolerable probability threshold p in [0,1] over the infinite time horizon. Our method leverages advances in machine learning literature and it represents formal certificates as neural networks. In particular, we learn a certificate in the form of a reach-avoid supermartingale (RASM), a novel notion that we introduce in this work. Our RASMs provide reachability and avoidance guarantees by imposing constraints on what can be viewed as a stochastic extension of level sets of Lyapunov functions for deterministic systems. Our approach solves several important problems -- it can be used to learn a control policy from scratch, to verify a reach-avoid specification for a fixed control policy, or to fine-tune a pre-trained policy if it does not satisfy the reach-avoid specification. We validate our approach on 3 stochastic non-linear reinforcement learning tasks.

Event-Triggered Finite-Time <i>H</i><sub>∞</sub> Filtering for Discrete-Time Nonlinear Stochastic Systems

This paper addresses the problem of event-triggered finite-time H∞ filter design for a class of discrete-time nonlinear stochastic systems with exogenous disturbances. The stochastic Lyapunov-Krasoviskii functional method is adopted to design a filter such that the filtering error system is stochastic finite-time stable (SFTS) and preserves a prescribed performance level according to the pre-defined event-triggered criteria. Based on stochastic differential equations theory, some sufficient conditions for the existence of H∞ filter are obtained for the suggested system by employing linear matrix inequality technique. Finally, the desired H∞ filter gain matrices can be expressed in an explicit form.

An Approximate Method of System Entropy in Discrete-Time Nonlinear Biological Networks

This study discusses the calculation of entropy of discrete-time stochastic biological systems. First, measurement methods of the system entropy of discrete-time linear stochastic networks are introduced. The system entropy is found to be characterized by system matrices of the discrete-time biological systems. Secondly, the system entropy of nonlinear discrete-time stochastic biological systems is discussed and is calculated based on a global linearization method. The approximation of the values of system entropy of nonlinear stochastic systems needs to solve an optimization problem that is constrained by a kind of linear matrix inequality (LMI). Finally, a practical biochemical system is provided to verify the effectiveness of the proposed calculation method.

Stability Verification in Stochastic Control Systems via Neural Network Supermartingales

We consider the problem of formally verifying almost-sure (a.s.) asymptotic stability in discrete-time nonlinear stochastic control systems. While verifying stability in deterministic control systems is extensively studied in the literature, verifying stability in stochastic control systems is an open problem. The few existing works on this topic either consider only specialized forms of stochasticity or make restrictive assumptions on the system, rendering them inapplicable to learning algorithms with neural network policies. In this work, we present an approach for general nonlinear stochastic control problems with two novel aspects: (a) instead of classical stochastic extensions of Lyapunov functions, we use ranking supermartingales (RSMs) to certify a.s. asymptotic stability, and (b) we present a method for learning neural network RSMs. We prove that our approach guarantees a.s. asymptotic stability of the system and provides the first method to obtain bounds on the stabilization time, which stochastic Lyapunov functions do not. Finally, we validate our approach experimentally on a set of nonlinear stochastic reinforcement learning environments with neural network policies.

Robust Finite-Time Stability for Uncertain Discrete-Time Stochastic Nonlinear Systems with Time-Varying Delay.

The main concern of this paper is finite-time stability (FTS) for uncertain discrete-time stochastic nonlinear systems (DSNSs) with time-varying delay (TVD) and multiplicative noise. First, a Lyapunov–Krasovskii function (LKF) is constructed, using the forward difference, and less conservative stability criteria are obtained. By solving a series of linear matrix inequalities (LMIs), some sufficient conditions for FTS of the stochastic system are found. Moreover, FTS is presented for a stochastic nominal system. Lastly, the validity and improvement of the proposed methods are shown with two simulation examples.

Lyapunov theorems for stability and semistability of discrete-time stochastic systems

This paper develops Lyapunov and converse Lyapunov theorems for discrete-time stochastic semistable nonlinear dynamical systems expressed by Itô-type difference equations possessing a continuum of equilibria. Specifically, we provide necessary and sufficient Lyapunov conditions for stochastic semistability and show that stochastic semistability implies the existence of a continuous Lyapunov function whose difference operator involves a discrete-time analog of the infinitesimal generator for continuous-time Itô dynamical systems and decreases along the dynamical system sample solution sequences satisfying an inequality involving the average distance to the set of the system equilibria.

Finite Time Stability and Optimal Finite Time Stabilization for Discrete-Time Stochastic Dynamical Systems

In this paper, we address finite time stability in probability of discrete-time stochastic dynamical systems. Specifically, a stochastic comparison lemma is constructed along with a scalar system involving a generalized deadzone function to establish almost sure convergence and finite time stability in probability. This result is used to provide Lyapunov theorems for finite time stability in probability for Itô-type stationary nonlinear stochastic difference equations involving Lyapunov difference conditions on the minimum of the Lyapunov function itself along with a fractional power of the Lyapunov function. In addition, we establish sufficient conditions for almost sure lower semicontinuity of the stochastic settling-time capturing the average settling time behavior of the discrete-time nonlinear stochastic dynamical system. Furthermore, a stochastic finite-time optimal control framework is developed by exploiting connections between Lyapunov theory for finite time stability in probability and stochastic Bellman theory. In particular, we show that finite time stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic finite time stability and optimality.

Comment on “LaSalle-Type Theorem and Its Applications to Infinite Horizon Optimal Control of Discrete-Time Nonlinear Stochastic Systems”

This article corrects some errors existing in Definitions 3.2 and 4.1 and Theorem 4.1. of the reference [1].

Dissipativity, inverse optimal control, and stability margins for nonlinear discrete-time stochastic feedback regulators

ABSTRACT In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate.

Observers Design for a Class of Nonlinear Stochastic Discrete-Time Systems

This paper mainly studied the observer design of Lipschitz stochastic discrete system. For the first time, generalized Lipschitz conditions are introduced into the observer design of a class of nonlinear stochastic discrete systems. From this paper, it can be seen that the generalized Lipschitz condition can better reflect the structural information of the nonlinear part, and thus has more advantages than the classical Lipschitz condition in the observer design. The stability criterion and observer design method of nonlinear stochastic discrete systems were proposed. Numerical examples illustrated the advantages and feasibility of the theoretical results obtained.

Compositional (In)Finite Abstractions for Large-Scale Interconnected Stochastic Systems

This article is concerned with a compositional approach for constructing both infinite (reduced-order models) and finite abstractions [a.k.a. finite Markov decision processes (MDPs)] of large-scale interconnected discrete-time stochastic systems. The proposed framework is based on the notion of stochastic simulation functions enabling us to employ an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds. In the first part of this article, we derive sufficient small-gain-type conditions for the compositional quantification of the probabilistic distance between the interconnection of stochastic control subsystems and that of their infinite abstractions. We then construct infinite abstractions together with their corresponding stochastic simulation functions for a particular class of discrete-time nonlinear stochastic control systems. In the second part of this article, we leverage small-gain-type conditions for the compositional construction of finite abstractions. We propose an approach to construct finite MDPs as finite abstractions of concrete models or their reduced-order versions satisfying an incremental input-to-state stability property. We also show that for the particular class of nonlinear stochastic control systems, the aforementioned property can be readily checked by matrix inequalities. We demonstrate the effectiveness of the proposed results by applying our approaches to a fully interconnected network of 20 nonlinear subsystems (totally 100 dimensions). We construct finite MDPs from their reduced-order versions (together 20 dimensions) with guaranteed error bounds on their output trajectories. We also apply the proposed results to a temperature regulation in a circular building and construct compositionally a finite abstraction of a network containing 1000 rooms. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies regulating the temperature in each room for a bounded time horizon.

Recursive Filtering With Measurement Fading: A Multiple Description Coding Scheme

In this article, the recursive filtering problem is studied for a class of discrete-time nonlinear stochastic systems subject to fading measurements. In order to facilitate the data transmission in a resource-constrained communication network, the multiple description coding scheme is adopted to encode the fading measurements into two descriptions with the identical importance. Two independent Bernoulli distributed random variables are introduced to govern the occurrences of the packet dropouts in two channels from the encoders to the decoders. The channel fading phenomenon is characterized by the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> th-order Rice fading model whose coefficients are mutually independent random variables obeying certain probability distributions. The purpose of the problem addressed is to design a recursive filter, such that in the simultaneous presence of the stochastic noises, the channel fading and the data coding–decoding mechanism, an upper bound of the filtering error variance is obtained and then minimized at each time step. In virtue of the Riccati difference equation technique and the stochastic analysis approach, the explicit form of the desired filter parameters is derived by solving a sequence of coupled algebraic Riccati-like difference equations. Finally, a simulation experiment is provided to show the applicability of the developed filtering scheme.

Pth moment exponential stability of general nonlinear discrete-time stochastic systems

pth moment exponential stability of general nonlinear discrete-time stochastic systems

Outlier-resistant H∞ filtering for a class of networked systems under Round-Robin protocol

In this paper, the H∞ filtering problem is addressed for a class of nonlinear stochastic discrete-time systems subject to Round-Robin protocol, measurement outliers, randomly occurring parameter uncertainties and randomly occurring nonlinearities. First of all, the considered system is influenced by the parameter uncertainties, nonlinear dynamics and external disturbances. Next, mutually independent Bernoulli-distributed white sequences are employed to describe the phenomena of randomly occurring parameter uncertainties and nonlinearities, respectively. Besides, the Round-Robin protocol is introduced to prevent the occurrence of data collisions during the signal transmission. Moreover, a filter with adaptive saturation is constructed to weaken the impact of the measurement outliers. Through constructing a suitable Lyapunov function, sufficient conditions are established to ensure that the error system dynamics is exponentially mean-square stable and satisfies the H∞ performance. Subsequently, the filter gains are designed by solving a set of linear matrix inequalities. Finally, a simulation example is given to show the effectiveness of the filtering scheme proposed in this paper.

Resilient [formula omitted]-[formula omitted] filtering with dwell-time-based communication scheduling

This paper is concerned with the resilient filtering problem for a class of discrete-time stochastic nonlinear systems with both time-varying delays and probabilistic distributed delays. In order to save the limited communication resource, stochastic communication protocols (SCPs) with multiple rules governed by a switching signal are employed to schedule the data transmission between the sensors and the filter for complying with different network scenarios and preserving the desired performance. An approach based on average dwell time (ADT) is utilized to establish the rule of SCPs for different scenarios. The purpose of the filtering problem is to design a resilient filter such that, under the regulation of SCPs, the dynamics of the filtering error is exponentially mean-square stable, and satisfies a prescribed disturbance attenuation level in a ℓ2-ℓ∞ sense. A sufficient condition for the exponential mean-square stability is first derived and the ℓ2-ℓ∞ performance is then guaranteed. In terms of certain linear matrix inequalities, the solvability of the addressed problem is discussed and the explicit expression of the desired resilient filter is also parameterized. Finally, a numerical example is provided to demonstrate the validity of the proposed design approach.

Event-Based Distributed Adaptive Kalman Filtering With Unknown Covariance of Process Noises

In this article, the distributed adaptive Kalman filtering is investigated for discrete-time stochastic nonlinear systems with gain perturbation as well as unknown covariance of process noises. For the adopted event-triggered communication scheduling, a distributed Kalman filter with an event timestamp is first constructed to effectively fuse the information from neighbors and itself while guaranteeing the unbiasedness. In light of stochastic analysis, the desired filter gain, achieving the suboptimality of filtering performance, is obtained recursively by solving two optimization issues with the form of Riccati-like difference equations. With the help of the fashionable weighted fusion conception combined with the well-known law of large numbers, a recursive estimation of process noise covariance is derived step by step and consequently suits for online computation. Finally, the effectiveness of the proposed filtering scheme is verified via a “lineland” system model.

Stability of discrete-time stochastic nonlinear systems with event-triggered state-feedback control

This paper investigates a class of discrete-time stochastic nonlinear systems with external disturbances. To obtain the stability of the suggested system, we develop an event-triggered mechanism which is more general than those given in previous literatures. On this basis, many techniques such as stochastic analysis theory, classified discussion, comparison principle and reduction to absurdity are successfully applied in this paper. Under some suitable conditions, the asymptotic stability in mean square in the case of the state-dependent external disturbance and the input-to-state stability in mean square in the case of the state-independent external disturbance are obtained. Finally, some discussions and remarks are provided to show the significance of our results.