Discrete-time Stochastic Systems Research Articles

Finite Time Stability and Optimal Finite Time Stabilization for Discrete-Time Stochastic Dynamical Systems

In this paper, we address finite time stability in probability of discrete-time stochastic dynamical systems. Specifically, a stochastic comparison lemma is constructed along with a scalar system involving a generalized deadzone function to establish almost sure convergence and finite time stability in probability. This result is used to provide Lyapunov theorems for finite time stability in probability for Itô-type stationary nonlinear stochastic difference equations involving Lyapunov difference conditions on the minimum of the Lyapunov function itself along with a fractional power of the Lyapunov function. In addition, we establish sufficient conditions for almost sure lower semicontinuity of the stochastic settling-time capturing the average settling time behavior of the discrete-time nonlinear stochastic dynamical system. Furthermore, a stochastic finite-time optimal control framework is developed by exploiting connections between Lyapunov theory for finite time stability in probability and stochastic Bellman theory. In particular, we show that finite time stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic finite time stability and optimality.

Full Information H2 Control of Borel-Measurable Markov Jump Systems with Multiplicative Noises

This paper addresses an H2 optimal control problem for a class of discrete-time stochastic systems with Markov jump parameter and multiplicative noises. The involved Markov jump parameter is a uniform ergodic Markov chain taking values in a Borel-measurable set. In the presence of exogenous white noise disturbance, Gramian characterization is derived for the H2 norm, which quantifies the stationary variance of output response for the considered systems. Moreover, under the condition that full information of the system state is accessible to measurement, an H2 dynamic optimal control problem is shown to be solved by a zero-order stabilizing feedback controller, which can be represented in terms of the stabilizing solution to a set of coupled stochastic algebraic Riccati equations. Finally, an iterative algorithm is provided to get the approximate solution of the obtained Riccati equations, and a numerical example illustrates the effectiveness of the proposed algorithm.

Optimal linear filtering for networked control systems under time correlated fading channel and noise

In this paper, the optimal filtering problem for discrete-time networked control systems affected by fading channels is investigated where both the fading channel and channel noise are time-correlated. Each fading channel is modeled as a scalar uncertain stochastic discrete-time system, and the channel noise is described by a discrete-time linear system perturbed by white noise. By using multiple recursive terms and the correlations of the signals in the systems, an optimal linear filter is proposed such that it is implemented in a recursive manner. The optimal filter designed has time independent computation structure, hence is easier to be implemented in practice. A vehicle tracking example is provided to illustrate the effectiveness and performance of the theoretic results obtained.

Adaptive-triggered asynchronous control for stochastic jumping systems under sliding mode

This paper investigates the problem of event-triggered asynchronous control for discrete-time stochastic jumping systems. A novel adaptive event-triggered (AET) strategy is proposed to save network resources, and two asynchronous control approaches are presented. First, an asynchronous feedback control method on adaptive triggered mechanism is proposed, and the mean-square exponentially stable of the closed-loop system is analysed. Second, the reduced-order model for discrete-time stochastic system is established to deal with the actuator faults. Furthermore, combining with the AET scheme, an asynchronous sliding-mode surface is designed such that the discrete-time reduced-order sliding-mode dynamics is yielded. Third, the stability analysis for the sliding-mode dynamics is discussed and the sliding-mode parameters are solved. Moreover, in order to compensate the actuator faults, an asynchronous sliding-mode controller is synthesised, and the reachability of the sliding-mode surface is analysed. Finally, two examples are given to demonstrate the potential of the developed scheme.

Expected value based optimal control for discrete-time stochastic noncausal systems

Expected value based optimal control for discrete-time stochastic noncausal systems

Finite-time Stochastic Stability and Stabilization for Uncertain Discrete-time Stochastic Systems with Time-varying Delay

This paper deals with the problems of finite-time stochastic stability and stabilization for discrete-time stochastic systems with parametric uncertainties and time-varying delay. Using the Lyapunov-Krasovskii functional method, some sufficient conditions of finite-time stochastic stability for a class of discrete-time stochastic uncertain systems are established in term of matrix inequalities. Then, a new criterion is proposed to ensure the closed-loop system is finite-time stochastically stable. The controller gain is designed. Finally, two numerical examples are given to illustrate the effectiveness of the proposed results.

Security control of networked systems with deception attacks and packet dropouts: A discrete-time approach

This paper is concerned with the security control problem for a class of networked systems subject to deception attacks and packet dropouts. First, by taking into account the deception attacks and packet dropouts in an unified framework, a discrete-time stochastic system is presented. In virtue of matrix exponential computation, an equivalent discrete-time stochastic model is established. Based on this, the security analysis is given by the law of total expectation and some sufficient conditions are provided. Subsequently, a controller is designed. Finally, the effectiveness of the proposed method is illustrated by two examples including a practical power grid system.

Formula omitted]th Moment Exponential Stability of Switched Discrete-Time Stochastic Systems: A Multiple Lyapunov Functions Method

This paper studies the pth moment exponential stability of switched discrete-time stochastic systems with stable and unstable subsystems. By using multiple Lyapunov functions and mode-dependent average dwell time, we provide a sufficient condition for the pth moment exponential stability with fast convergence rate for deterministic switched discrete-time stochastic systems with stable and unstable subsystems. Obviously, our result has a weaker conservatism than those previous results where the effect of random factor was ignored. In addition, the criteria for stability and instability of discrete-time stochastic systems without switching are provided. Finally, the correctness of the results is verified by two numerical examples. To the best of our knowledge, it is the first time that the stability of discrete time switching system with random disturbance has been discussed.

Intermittent fault detection for linear discrete-time stochastic multi-agent systems

In this paper, the intermittent fault (IF) detection problem is investigated for a class of linear discrete-time stochastic multi-agent systems (MASs). By using the reduced-order observer method and introducing a sliding-time window, the truncated residuals are designed to detect the appearing time and disappearing time of the IF. Moreover, two sets of hypothesis tests are proposed to determine the detection and location thresholds of the IF, and the detectability of the IF is discussed in the framework of statistical analysis. Finally, a simulation example is provided to validate the feasibility and effectiveness of the proposed method.

Intermittent fault detection for delayed stochastic systems over sensor networks

This paper is concerned with the intermittent fault (IF) detection problem for a class of linear discrete-time stochastic systems over sensor networks with constant time delay. By utilizing the lifting method, the distributed decoupled observers are proposed based on the output information of neighbor nodes and the node itself. In order to detect the appearing time and disappearing time of the IF, the truncated residuals are designed by introducing a sliding-time window. Furthermore, the IF detection and location thresholds are determined based on the hypothesis testing technique and the detectability of the IF is analyzed in the framework of stochastic analysis. Finally, a simulation example is presented to illustrate the effectiveness of the derived results.

Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems

Discrete-time stochastic systems are an essential modeling tool for many engineering systems. We consider stochastic control systems that are evolving over continuous spaces. For this class of models, methods for the formal verification and synthesis of control strategies are computationally hard and generally rely on the use of approximate abstractions. Building on approximate abstractions, we compute control strategies with lower- and upper-bounds for satisfying unbounded temporal logic specifications. First, robust dynamic programming mappings over the abstract system are introduced to solve the control synthesis and verification problem. These mappings yield a control strategy and a unique lower bound on the satisfaction probability for temporal logic specifications that is robust to the incurred approximation errors. Second, upper-bounds on the satisfaction probability are quantified, and properties of the mappings are analyzed and discussed. Finally, we show the implications of these results to continuous state space of linear stochastic dynamic systems. This abstraction-based synthesis framework is shown to be able to handle infinite-horizon properties. Approximation errors expressed as deviations in the outputs of the models and as deviations in the probabilistic transitions are allowed and are quantified using approximate stochastic simulation relations.

Dissipativity, inverse optimal control, and stability margins for nonlinear discrete-time stochastic feedback regulators

ABSTRACT In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate.

First-order and second-order necessary optimality conditions concerning components for discrete-time stochastic systems

This paper deals with the first-order and second-order necessary optimality conditions concerning the components for discrete-time stochastic systems under weakened convexity assumptions. By means of a new discrete-time backward stochastic equation, we establish a first-order necessary optimality condition concerning every component in the form of a global stochastic maximum principle. Besides, we also obtain an optimality condition concerning one component of vector control in the form of a global stochastic maximum principle and concerning the other in the form of linearised stochastic maximum principle. Moreover, by introducing a new discrete-time backward stochastic matrix equation, we derive diverse second-order necessary optimality conditions of singular and quasi-singular controls concerning the components.

Optimal control for unknown mean-field discrete-time system based on Q-Learning

Solving the optimal mean-field control problem usually requires complete system information. In this paper, a Q-learning algorithm is discussed to solve the optimal control problem of the unknown mean-field discrete-time stochastic system. First, through the corresponding transformation, we turn the stochastic mean-field control problem into a deterministic problem. Second, the H matrix is obtained through Q-function, and the control strategy relies only on the H matrix. Therefore, solving H matrix is equivalent to solving the mean-field optimal control. The proposed Q-learning method iteratively solves H matrix and gain matrix according to input system state information, without the need for system parameter knowledge. Next, it is proved that the control matrix sequence obtained by Q-learning converge to the optimal control, which shows theoretical feasibility of the Q-learning. Finally, two simulation cases verify the effectiveness of Q-learning algorithm.

Robust observer-based sliding mode [formula omitted] control for stochastic Markovian jump systems subject to packet losses

This paper studies the robust observer-based sliding mode H∞ control issue for discrete-time stochastic Markovian jumping systems (MJSs) subject to packet losses (PLs). Here, a state variable observer of an initial system is designed to estimate the state and a series of Bernoulli random variables is utilized to model the PLs, whose occurrence probability is allowed to be imprecise. Then, a linear switching function is constructed based on the estimated state space. The aim of our paper is to design an observer-based sliding mode H∞ control strategy such that, the stochastic stability conditions with prescribed H∞ performance are established for augmented MJSs under the PLs with imprecise occurrence probability, which is composed of the sliding mode dynamics and the error system. The linear sliding surface can be determined by solving some linear matrix inequalities (LMIs). In addition, by proposing an improved reaching condition, the desired robust observer-based H∞ controller is designed in discrete-time setting to drive the system state from any initial one onto the designated sliding surface. Finally, the usefulness of observer-based H∞ control method is demonstrated via some numerical simulations.

H∞ Filtering for Markovian Jump Linear Systems with Uncertain Transition Probabilities

This paper studies the H∞ filtering problem of stochastic linear systems subject to Markovian jump and multiplicative noise. The transition probabilities are considered to be uncertain. A unified form of filters is constructed for both continuous-time and discrete-time stochastic systems. With the new decoupling technique for the coupling terms between Lyapunov matrices and filtering parameters, sufficient conditions of stochastic stability and H∞ performance of filtering error system are derived. Based on these conditions, the filter is designed with less coupling matrices and the filter gain matrices are obtained by calculating a set of linear matrix inequalities. Finally, three examples are presented to test the effectiveness of the obtained method.

A mean field absorbing control model for interacting objects systems

We study a class of discrete-time stochastic systems composed of a large number of N interacting objects, which are classified in a finite number of classes. The behavior of the objects is controlled by a central decision-maker as follows. At each stage, once the configuration of the system is observed, the controller takes a decision; then a cost is incurred and there is a positive probability the process stops, otherwise the objects move randomly among the classes according to a transition probability. That is, with positive probability, the system is absorbed by a configuration that represents the death of the system, and there it will remain without incurring cost. Due to the large number of objects, the control problem is studied according to the mean field theory. Thus, instead of analyzing a single object, we focus on the proportions of objects occupying each class, and then we study the limit as N goes to infinity.

Feedback Stabilization of Uncertain Networked Control Systems Over Delayed and Fading Channels

In this article, we characterize the joint effects of plant uncertainty, transmission delay, and fading channel on the stabilization problem of networked control systems (NCSs). We assume that the controller remotely controls the plant and the control packet is transmitted over a delayed and fading channel. We formulate the integrated system as an uncertain discrete-time stochastic system with both input delay and multiplicative noise. The innovative contributions of this article are described as follows. We propose a set of necessary and sufficient conditions for stabilizing an uncertain-free system. Moreover, for the general uncertain model, we derive the necessary and sufficient stabilization condition in terms of symmetric matrix homogeneous polynomials, which can be verified by linear matrix inequality (LMI) feasibility test. As an application, under the parallel transmission strategy, we derive an explicit formula for computing the maximum allowable delay bound, which is uniquely determined by system parameters and signal-to-noise ratio.

Pth moment $${\cal D}$$-stability/stabilization of linear discrete-time stochastic systems

pth moment $${\cal D}$$-stability/stabilization of linear discrete-time stochastic systems

Stability Analysis of Discrete-Time Stochastic Delay Systems with Impulses

This paper is concerned with stability analysis of discrete-time stochastic delay systems with impulses. By using the sums average value of the time-varying coefficients and the average impulsive interval, two sufficient criteria for exponential stability of discrete-time impulsive stochastic delay systems are derived, which are more convenient to be applied than those Razumikhin-type conditions in previous literature. Both pth moment asymptotic stability and pth moment exponential stability are considered. Finally, two numerical examples to illustrate the effectiveness.