Discrete-time Markov Jump Linear Systems Research Articles

Stabilization and Optimal Control for Discrete-Time Markov Jump Linear System With Multiplicative Noises and Input Delays: A Complete Solution

Stabilization and Optimal Control for Discrete-Time Markov Jump Linear System With Multiplicative Noises and Input Delays: A Complete Solution

Stabilization and Optimization of Discrete-Time Markovian Jump Linear Systems via Mode Feedback Control

This article is devoted to a specific kind of discrete-time switched linear systems, where the switching signal is governed by a Markov chain, i.e., Markovian jump linear systems. For such systems, a mode feedback control mechanism is adopted to adjust the mode transition probability matrix, which is referred to as the switching law design, and the optimal mode feedback controller is sought to minimize a quadratic performance index containing both the system state and the mode feedback control input. First, the admissible set of the mode feedback control is investigated, and a sufficient condition is derived to ensure the desired stochastic stability. Second, under the assumption that the concerned system is mode-cost sequential, the original performance index is approximated into a new tractable one and a suboptimal mode feedback controller is then derived within the admissible set. Third, we consider a general situation where no specific requirements are imposed on the Markovian jump linear systems’ dynamics, derive the optimal mode feedback controller via a value-iteration-based algorithm, and prove the convergence of the obtained optimal controller. Finally, simulation results are provided to illustrate the validity of the proposed mode feedback control mechanism.

Off-policy reinforcement learning for tracking control of discrete-time Markov jump linear systems with completely unknown dynamics

In this paper, a model-free off-policy reinforcement learning (RL) algorithm is proposed to address the optimal tracking control (OTC) problem for discrete-time Markov jump linear systems (MJLSs). The tracking reference signal is firstly augmented into discrete-time MJLSs, whereby the original tracking control problem is converted to the optimal control problem of the augmented system. The corresponding augmented coupled game algebraic Riccati equation (ACGARE) is then derived. On this basis, an online RL algorithm is developed to solve the OTC problem by using the policy iteration (PI) technique. Then, a novel model-free method is proposed, which eliminates the requirement of the system dynamics and transition probability. Finally, a simulation example is provided to prove the convergence and validate the effectiveness of the proposed algorithm.

Finite-time higher-order moment state estimation for Markov jump linear system with time-correlated measurement noise

In the present study, a higher-order moment state estimation problem in a finite time interval for the discrete-time Markov jump linear systems (MJLS) is investigated. Moreover, time-correlated noise in measurements is considered. Initially, the measruement differencing approach is applied to convert the time-correlated measurement noise to an uncorrelated noise. Then the cumulant generating function is utilized to solve the stochastic jumping problem of MJLS, by which the discrete-time MJLS is transformed into a deterministic system. In this way, the transformed deterministic system has the same norm with a higher-order moment of the original state. Finally, a finite-time state estimation algorithm is proposed to guarantee that the higher-order moment of error trajectory remains within a pre-specified bound over a given time interval. In order to evaluate the performance of the proposed method, some test cases are applied. Obtained results prove the accuracy and efficiency of the proposed method.

Optimal Control and Stabilization for Discrete-Time Markov Jump Linear Systems with Input Delay

This paper examines the finite horizon optimal control problem and infinite horizon stabilization problem for Markov jump linear system (MJLS) with input delay. The essential obstacle encountered i...

Complementary Stability of Markovian Systems: Series Elastic Actuators and Human-Robot Interaction

Abstract This paper presents an extension of the complementary stability analysis for discrete-time Markovian systems applied in interaction control. The control strategy considers an internal loop force control for Series Elastic Actuators based on the Robust Regulator for Discrete-time Markov Jump Linear systems and extern loop impedance control to regulate the interaction between the human and robot. Two examples show the complementary stability analysis of the human-robot interaction system, where is regarded as different values of the virtual impedances, different levels for the robustness of the force control, uncertainties, and abrupt changes for human parameters. Simulation results show the capacity of control strategy to deal with active interaction models, where the Mean Square Stability guarantees safe for the user. Additionally, from results it is understood the transmissibility of impedance achieved is a consequence of the robust performance given by the force controller.

A Detector-Based Approach for the Constrained Quadratic Control of Discrete-Time Markovian Jump Linear Systems

This paper considers the quadratic control problem of discrete-time Markov jump linear systems with constraints on the norm of the state and control variables. We assume that the Markov chain parameter is not available, and instead, there is a detector, which emits signals providing information on this parameter. It is desired to derive a feedback linear control using the information provided by this detector in order to stochastically stabilize the closed-loop system, satisfy the constraints whenever the initial conditions belong to an invariant set, and minimize an upper bound for the quadratic cost. We show that a linear matrix inequality (LMI) optimization problem can be formulated in order to obtain a solution for this problem. Two other related problems, one for minimizing the guaranteed quadratic cost considering fixed initial conditions and the other for maximizing an estimate of the domain of an invariant set, can also be formulated using our LMI approach. This paper is concluded with some numerical simulations.

Robust Model Predictive Control for a Class of Discrete-Time Markovian Jump Linear Systems With Operation Mode Disordering

For a class of discrete-time Markovian jump linear systems subject to operation mode disordering, a robust model predictive control method can be proposed to solve this issue. A bijective mapping scheme between the original random process and a new random process is studied to cope with the problem of operation mode disordering. At each sampling time, the original “min-max” optimization problem is transformed into a convex optimization problem with linear matrix inequalities so that the complexity of solving the optimization problem can be greatly reduced. The sufficient stability condition of the Markovian jump linear systems can be achieved by using the Lyapunov stability theory. Moreover, a state feedback control law is obtained, which minimizes an infinite prediction horizon performance cost. Furthermore, the cases of uncertain and unknown transition probabilities are also considered in this paper. The simulation results show that the proposed method can guarantee the optimal control performance and the stability of Markovian jump linear systems.

Derandomisation-based multiple frequency control for stochastic Markov jump systems

ABSTRACTUnder the framework of derandomisation approach, the state feedback control issues for both continuous-time and discrete-time Markov jump linear systems (MJLSs) are investigated to meet multiple performance objectives over multiple frequency ranges. Because of the stochastic jumping among different modes, the generalised Kalman–Yakubovic–Popov lemma-based finite-frequency controller design approach cannot be directly applied to MJLSs. To overcome this limitation, a derandomisation approach is established by transforming the original stochastic multiple modes systems to deterministic ones. Then the multiple frequency controllers for both discrete-time and continuous-time MJLSs are designed to guarantee the multiple performances of the closed-loop systems. To verify the effectiveness of the developed algorithms, examples are presented, where the performance requirements include specifications in low-frequency and high-frequency ranges, respectively.

Recursive Robust Regulator for Discrete-Time Markovian Jump Linear Systems

In this paper, we propose a robust regulator for discrete-time Markovian jump linear systems (DMJLS) subject to structured parameter uncertainties. States of Markov chain are considered known at each instant and the transition probabilities matrix is time-varying. The framework proposed to deal with this problem is developed through a regularization approach based on robust least-squares and penalty functions. Recursive solutions in terms of coupled Riccati equations, which do not depend on any auxiliary parameter to be tuned, are provided. These equations resemble the recursive solutions of DMJLS standard regulators when the system is not subject to uncertainties. Conditions for convergence and stability of this robust regulator are established. Numerical examples present comparative studies considering the proposed robust regulator, the standard linear quadratic regulator, and an $H_{\infty }$ -controller for DMJLS.

Stable receding-horizon scenario predictive control for Markov-jump linear systems

This paper discusses predictive control for constrained discrete-time Markov-jump linear systems (MJLS) which jump between a finite set of modes according to a Markov probabilistic transition/observation model, minimising an average cost. Due to the exponential explosion of the number of possible realisations as horizon grows, scenario approaches consider only a subset of them. Prior works cast the problem as a tree-based optimisation one, but enforce stability and feasibility via artificial Lyapunov-related constraints. The proposed approach avoids this route, proposing instead ‘terminal ingredients’ and tree properties (trim-contained, strictly-complete) properly generalising the stability/feasibility ideas in linear and MJLS literature.

Recursive Robust Regulator for Discrete-time Markovian Jump Linear Systems: Control of Series Elastic Actuators

Impedance control for human-robot interaction allows a more secure and controllable robotic rehabilitation. High-performance force control is required to achieve a more precise impedance-controlled robotic therapy using series elastic actuators. However, the performance of these controllers is sensitive to the highly uncertain and time-varying human dynamics. In this paper, we develop a force controller of a series elastic actuator using a Recursive Robust Regulator for discrete-time Markov jump linear systems subject to parametric uncertainties. We define three Markovian modes of operation depending on the human-robot interaction dynamics. Also, we include an integral action to eliminate steady-state errors, and propose a methodology for robust tracking based on the analysis of the frequency response of the system under parametric uncertainties. Experimental results of the impedance and force controllers for a robotic platform for ankle rehabilitation are presented. These results indicate that the proposed controller maintains similar performance levels despite the transitions between different modes of operation and even subject to the uncertainties introduced by human dynamics.

Reduced-order dynamic output feedback control of uncertain discrete-time Markov jump linear systems

ABSTRACTThis paper deals with the problem of designing reduced-order robust dynamic output feedback controllers for discrete-time Markov jump linear systems (MJLS) with polytopic state space matrices and uncertain transition probabilities. Starting from a full order, mode-dependent and polynomially parameter-dependent dynamic output feedback controller, sufficient linear matrix inequality based conditions are provided for the existence of a robust reduced-order dynamic output feedback stabilising controller with complete, partial or none mode dependency assuring an upper bound to the or the norm of the closed-loop system. The main advantage of the proposed method when compared to the existing approaches is the fact that the dynamic controllers are exclusively expressed in terms of the decision variables of the problem. In other words, the matrices that define the controller realisation do not depend explicitly on the state space matrices associated with the modes of the MJLS. As a consequence, the method is specially suitable to handle order reduction or cluster availability constraints in the context of or dynamic output feedback control of discrete-time MJLS. Additionally, as illustrated by means of numerical examples, the proposed approach can provide less conservative results than other conditions in the literature.

Networked Control Systems Application: Minimization of the global number of interactions, transmissions and receptions in Multi-Hop network using Discrete-Time Markovian Jump Linear Systems

This paper analyzes the Trade-Off between the global number of interactions, composed by the number of transmissions and receptions in the network, and the H-infinity norm degradation. Specifically on this study we implemented an optimum H-infinity filter in a semi-reliable Multi-Hop network modeled by Discrete-Time Markovian Jump Linear Systems. For this particular case, it was possible guarantee the filter stability with a global number the interaction of only 17.93% of packet that would be necessary for the same task using the classic filter.

Robust fault-tolerant control for power systems against mixed actuator failures

This paper employs linear matrix inequality (LMI) based optimization algorithm to develop a method for designing fault-tolerant state feedback controller with mixed actuator failures for power systems subject to random changes. Meanwhile, the random abrupt changes are determined by a finite set Markov chain so the considered system is equivalently represented as a discrete-time Markov jump linear system (MJLS). Further, due to the variations of loading conditions in power system, an uncertainty term is incorporated to MJLS. For the proposed system, we construct a novel actuator fault model containing both linear and nonlinear terms which is more general than the conventional actuator fault models. The main purpose of this paper is to design the robust fault-tolerant controller such that for all possible actuator failures, time-varying delays and admissible parameter uncertainties, the closed-loop uncertain discrete-time MJLS is robustly stochastically stable. Based on free-weighting matrix approach and linear matrix inequality theory, a new set of sufficient conditions that guaranteeing the robust stochastic stability is presented by choosing an appropriate Lyapunov–Krasovskii functional candidate. In addition, a single-machine infinite-bus (SMIB) power system is considered as an application example and its simulation results demonstrate the effectiveness of the proposed design techniques.

Quadratic control with partial information for discrete-time jump systems with the Markov chain in a general Borel space

This paper deals with the finite horizon quadratic optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space S. It is assumed that the controller has access to an output variable as well as the jump parameter. The goal is to design a dynamic Markov jump controller such that the closed loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. It is shown that an optimal controller can be obtained from two S-coupled difference Riccati equations, one associated to a filtering problem and the other one associated to a control problem in which the state variable is fully available. By S-coupled it is meant that the difference Riccati equations are coupled via an integral over S. This result, which can be seen as a separation principle for discrete-time MJLS, generalizes previous ones that were restricted to the Markov chain taking values in a finite set.

A Detector-Based Approach for the <inline-formula> <tex-math notation="TeX">$H_{2} $</tex-math></inline-formula> Control of Markov Jump Linear Systems With Partial Information

In this paper, we study the $H_{2} $ -control for discrete-time Markov Jump Linear Systems (MJLS) with partial information . We consider the case in which we do not have access to the Markov jump parameter but, instead, there is a detector that emits signals which provides information on this parameter. A salient feature of our formulation is that it encompasses, for instance, the cases with perfect information, no information and cluster observations of the Markov parameter, which were previously analyzed in the Markov jump control literature. The goal is to derive a feedback linear control using the information provided by the detector in order to stochastically stabilize the closed loop system. We present two Lyapunov like equations for the stochastic stability of the system. In addition, we show that a Linear Matrix Inequalities (LMI) formulation can be obtained in order to design a stochastically stabilizing feedback control. In the sequel we deal with the $H_{2} $ control problem and we show that, again, an LMI optimization problem can be formulated in order to design a stochastically stabilizing feedback control with guaranteed $H_{2} $ -cost. We also present two special cases, one of them always satisfied for the limit case in which the detector provides perfect information on the Markov parameter, and the Bernoulli jump case, under which LMI conditions become necessary and sufficient for the stochastic stabilizability of the system and the LMI optimization problems provide the optimal $H_{2} $ cost. For the Bernoulli jump case we show that our formulation generalizes previous ones. The case with convex polytopic uncertainty on the parameters of the system and on the transition probability matrix is also considered. The paper is concluded with some numerical examples.

Further Results on Stability Analysis of Discrete-Time Markov Jump Linear Systems with Time-Varying Delay and Partly Known Transition Probabilities

This paper is concerned with the stability analysis of discrete-time Markov jump linear systems (MJLSs) with time-varying delay and partly known transition probabilities. The time delay is varying between lower and upper bounds, and the partly known transition probabilities cover the cases of known, uncertain with known lower and upper bounds, and completely unknown, which is more general than the existing result. Via constructing an appropriate Lyapunov function and employing a new technique to separate Lyapunov variables from unknown transition probabilities, a novel stability criterion is obtained in the framework of linear matrix inequality. A numerical example is given to show the effectiveness of the proposed approach.

Co-design of controller and routing redundancy over a wireless network

In this paper we investigate the exploitation of redundancy when routing actuation data to a discrete-time LTI system connected to the controller via a wireless network affected by packet drops. We assume that actuation packets can be delivered from the controller to the actuator via multiple paths, each associated with a delay and a packet loss probability. We show that the joint design of controller gain and routing redundancy exploitation can tremendously improve the control performance. To achieve this goal we set up and solve a LQR problem for a class of systems that extends discrete-time Markov Jump Linear Systems, in that both continuous and discrete control signals can be actuated.

RobustH∞Filtering for Discrete-Time Markov Jump Linear System with Missing Measurements

The problem of robustH∞filtering is investigated for discrete-time Markov jump linear system (DMJLS) with uncertain parameters and missing measurements. The missing measurements process is modelled as a Bernoulli distributed sequence. A robustH∞filter is designed and sufficient conditions are established in terms of linear matrix inequalities via a mode-dependent Lyapunov function approach, such that, for all admissible uncertain parameters and missing measurements, the resulting filtering error system is robustly stochastically stable and a guaranteedH∞performance constraint is achieved. Furthermore, the optimalH∞performance index is subsequently obtained by solving a convex optimisation problem and the missing measurements effects on theH∞performance are evaluated. A numerical example is given to illustrate the feasibility and effectiveness of the proposed filter.