Discrete-time Markov Jump Linear Systems Research Articles

Almost sure stability of discrete‐time Markov jump linear systems

This paper deals with transient analysis and almost sure stability for discrete-time Markov Jump Linear System (MJLS). The expectation of sojourn time and activation number of any mode, and switching number between any two modes of discrete-time MJLS are presented firstly. Then a result on transient behavior analysis of discrete-time MJLS is given. Finally a new deterministically testable condition for the exponential almost sure stability of discrete-time MJLS is proposed.

Formula omitted]-gain performance analysis and positive filter design for positive discrete-time Markov jump linear systems: A linear programming approach

This paper is concerned with the l1-gain performance analysis and positive filter design for a positive discrete-time Markov jump linear system (MJLS) by using a linear programming (LP) approach. First, by constructing a linear stochastic Lyapunov function and introducing an “equivalent” deterministic positive discrete-time linear system, necessary and sufficient conditions in the form of LP are derived for stochastic stability and l1-gain performance of the positive discrete-time MJLS. Then a sufficient condition on the existence of the desired positive l1-gain filter is provided. The desired positive l1-gain filter can be designed by solving a standard LP problem. The pest’s structured population dynamic model is employed to illustrate the effectiveness of the proposed method.

***ℋ∞ Filter Design through Multi-simplex Modeling for Discrete-time Markov Jump Linear Systems with Partly Unknown Transition Probability Matrix

This paper addresses the problem of ***ℋ∞ filter design for discrete-time Markov jump linear systems (MJLS) with transition probability matrix affected by uncertainties. The proposed methodology allows to take into account the different types of uncertainties usually adopted in MJLS in a systematic way. New conditions are given for ***ℋ∞ filter design with partial, complete or null Markov mode availability. Due to the presence of slack variables in the synthesis conditions and to the use of homogeneous polynomial solutions of arbitrary degrees, less conservative linear matrix inequality relaxations can be obtained. Numerical experiments illustrate the better performance and efficiency of the proposed approach when compared to other strategies available in the literature.

Stability and Linear Quadratic Differential Games of Discrete-Time Markovian Jump Linear Systems with State-Dependent Noise

We mainly consider the stability of discrete-time Markovian jump linear systems with state-dependent noise as well as its linear quadratic (LQ) differential games. A necessary and sufficient condition involved with the connection between stochasticTn-stability of Markovian jump linear systems with state-dependent noise and Lyapunov equation is proposed. And using the theory of stochasticTn-stability, we give the optimal strategies and the optimal cost values for infinite horizon LQ stochastic differential games. It is demonstrated that the solutions of infinite horizon LQ stochastic differential games are concerned with four coupled generalized algebraic Riccati equations (GAREs). Finally, an iterative algorithm is presented to solve the four coupled GAREs and a simulation example is given to illustrate the effectiveness of it.

Receding Horizon Control for Uncertain Markov Jump Linear Systems with Time Delay and Actuator Saturation

A robust receding horizon control (RHC) scheme is developed for uncertain discrete-time Markov Jump Linear Systems (MJLS) with time delay and actuator saturation where the system uncertainties and jumping transition probabilities are assumed to belong to some convex sets. Firstly, when time delay is considered, a sufficient condition of minimizing upper bound of the cost function and mean square stability of the closed-loop system are established based on the Lyapunov Krasovskii function which depend on the current time jump mode. At each sampling time, an optimal control gain can be obtained by solving the semi-definite programming (SDP) problem. Then, the proposed strategy is extended to design robust RHC scheme for uncertain MJLS with both time delay and actuator saturation. Moreover, the domain of attraction can be estimated through a modified invariant ellipsoid. Finally, a numerical example is given to illustrate the feasibility and effectiveness of the proposed method.

Formula omitted] robust and networked control of discrete-time MJLS through LMIs

This paper addresses the H∞ state-feedback control of Markov Jump Linear Systems (MJLS) through Linear Matrix Inequalities (LMIs). First, under the assumption that the Markov parameter is measured, the main contribution is on the LMI characterization of all linear feedback controllers such that the closed loop output remains bounded by a given norm level. This result allows the robust controller design to deal with convex bounded parameter uncertainty, probability uncertainty and cluster availability of the Markov mode. For partly unknown transition probabilities, the proposed design problem is proved to be less conservative than the one already available. For the case where each transition probability is considered unknown and belonging to a given interval, we also prove that our conditions are less conservative than a previous result from the literature. Under cluster availability of the Markov modes, we incorporate several models for measurement transmission network channels, such as the generalized Gilbert–Elliot, Fritchman or McCullough ones, into the MJLS framework. An example is solved for illustration and comparisons.

Optimal State Estimation for Discrete-Time Markovian Jump Linear Systems, in the Presence of Delayed Output Observations

We investigate the design of optimal state estimators for Markovian Jump Linear Systems. We consider that the output observations and the mode observations are affected by delays not necessarily identical. Our objective is to design optimal estimators for the current state, given current and past observations. We provide a solution to this paradigm by giving an optimal recursive estimator for the state, in the minimum mean square sense, and a finitely parameterized recursive scheme for computing the probability mass function of the current mode conditioned on the observed output. We also show that if the output delay is less then the one in observing the mode, then the optimal state estimation becomes nonlinear in the output observations.

H∞ State Feedback Control of Discrete-time Markov Jump Linear Systems through Linear Matrix Inequalities

AbstractThis paper addresses the H∞ state-feedback control design problem of discrete-time Markov jump linear systems. First, under the assumption that the Markov parameter is measured, the main contribution is on the LMI characterization of all linear feedback controllers such that the closed loop output remains bounded by a given norm level. This results allows the robust controller design to deal with convex bounded parameter uncertainty, probability uncertainty and cluster availability of the Markov mode. For partly unknown transition probabilities, the proposed design problem is proved to be less conservative than one available in the current literature. An example is solved for illustration and comparisons.

A BRL for A Class of Discrete-time Markov Jump Linear System with Piecewise-Constant TPs

AbstractThis paper aims to obtain a Bounded Real Lemma (BRL) for a class of Markov jump linear systems (MJLSs) with time-varying transition probabilities (TPs) in discrete-time domain. The time-varying character of TPs is considered as piecewise-constant and the variation of TP matrices is subject to average dwell time (ADT) switching, i.e., the number of switches in a nite interval is bounded and the er ge time between two consecutive switchings of TP matrices is not less than a constant. Combining the Lyapunov function approach and the linear matrix inequality technique, a BRL for the underlying system is derived in order to check whether the corresponding system is stochastically stable and has a guaranteed H∞ noise-attenuation performance index scheduled based on the variation of TP matrices. A numerical example is provided to demonstrate that the method obtained in this paper is a signi cant improvement over previous one.

Fault Detection and Isolation of discrete-time Markovian jump linear systems with application to a network of multi-agent systems having imperfect communication channels

This paper deals with the problem of Fault Detection and Isolation (FDI) for discrete-time Markovian Jump Linear Systems (MJLSs). A geometric property related to the unobservable subspace of an MJLS is first presented and the concept of unobservability subspace is introduced. Sufficient conditions for designing an H ∞ -based FDI algorithm for MJLSs subject to input disturbances and measurement noise are presented and developed. Our proposed approach is then applied to the problem of fault detection and isolation in a network of multi-agent systems when imperfect communication channels exist among the agents. A discrete-time communication link with a stochastic packet dropping effect is considered based on the Gilbert–Elliott model and the entire network is modeled as a discrete-time MJLS. Simulation results are presented for formation flight of satellites to demonstrate and verify the effectiveness and performance capabilities of our proposed FDI algorithm.

Dynamic Output Feedback Control of Discrete-Time Markov Jump Linear Systems through Linear Matrix Inequalities

This paper addresses the ${\cal H}_2$ and ${\cal H}_\infty$ dynamic output feedback control design problems of discrete-time Markov jump linear systems. Under the mode-dependent assumption, which means that the Markov parameters are available for feedback, the main contribution is the complete characterization of all full order proper Markov jump linear controllers such that the ${\cal H}_2$ or ${\cal H}_\infty$ norm of the closed loop system remains bounded by a given prespecified level, yielding the global solution to the corresponding mode-dependent optimal control design problem, expressed in terms of pure linear matrix inequalities. Some academic examples are solved for illustration and comparison. As a more consequent practical application, the networked control of a vehicle platoon using measurement signals transmitted in a Markov channel, as initially proposed in [P. Seiler and R. Sengupta, IEEE Trans. Automat. Control, 50 (2005), pp. 356–364], is considered.

Output Feedback H2/H∞ Control of a Class of Discrete-time Reconfigurable Control Systems

In this paper, static output feedback control of a class of discrete-time reconfigurable systems with Markovian parameters is addressed. The main contribution is to formulate conditions for multi-performance design related to this class of systems. The specifications and objectives under consideration include stochastic stability, H2 and H∞ performances. Another problematic related to a more general class of stochastic hybrid systems, known as Markovian Jump Linear Systems (MJLS), is also considered. This problematic concerns the mode-independent output feedback control of discrete-time MJLS. Results are formulated as nonlinear matrix inequalities. A numerical algorithm is provided and its running is illustrated on a classical example from literature.

H2 and H∞ Filtering of Discrete-time Markov Jump Linear Systems through Linear Matrix Inequalities

This paper addresses the H2 and H∞ filtering design problems of discrete-time Markov jump linear systems. First, under the assumption that the Markov parameter is measured, the main contribution is on the LMI characterization of all filters such that the estimation error remains bounded by a given norm level, yielding the complete solution of the mode-dependent filtering design problem. Based on this result, a robust filter design to deal with convex bounded parameter uncertainty is considered. Second, from the same LMI characterization, a procedure for mode-independent filtering design is proposed. An example is solved for illustration and comparisons with the methods available in the literature.

Array Algorithm for Filtering of Discrete-Time Markovian Jump Linear Systems

This note develops an array algorithm for optimal filtering of discrete-time Markovian jump linear systems (DTMJLSs). The known advantages of this kind of algorithm, which was originally developed for normal state-space systems, remain valid when it is applied to linear systems subject to Markovian jumps. It is numerically more stable in the sense that it presents better conditioning and reduced dynamical range. A numerical example, based on fixed-point implementations, is presented in order to demonstrate the advantage of this algorithm.