Markov Jump Linear Systems Research Articles

Realization Theory for a Class of Stochastic Bilinear Systems

This paper presents a complete realization theory for a class of discrete-time, bilinear systems with observed stochastic inputs, which we call generalized bilinear systems . This class of systems includes subclasses of bilinear systems, linear parameter varying (LPV) systems, and jump-Markov linear systems. We present necessary and sufficient conditions for the existence of a realization of generalized bilinear systems, along with a characterization of minimality in terms of rank conditions. We also formulate a realization algorithm and a minimization algorithm and we show that minimality can be checked algorithmically.

Robust Fault Detection H∞ Filter for Markovian Jump Linear Systems with Partial Information on the Jump Parameter

Abstract The present work focus on the Robust Fault Detection (RFD) problem in the Markovian Jump Linear System framework for the discrete-time domain, in which the Markov parameter θ(k) is considered not accessible. The assumption that the Markov Chain is not accessible brings a challenge where the filter designed for the RFD should not be dependent on the Markov Chain parameter. In order to represent this kind of situation, the implementation of a Hidden Markov Chain to model the system mode θ(k) and the estimated mode θˆ(k) is used. The main result presented in this work is the design of a H∞ MJLS Robust Fault Detection filter that depends only on the estimated mode θˆ(k) obtained through LMI formulation. In order to illustrate the feasibility of the proposed solution a numerical example is also included.

Optimal filter for MJL system with delayed modes and observations

This study investigates the optimal filter for Markov jump linear (MJL) system with multiple delayed modes and observations, in which the data losses are introduced naturally and the Markov chains are assumed to be known up to the present time. To deal with the mode and observation delays, a reorganised observation sequence is defined. Moreover, the optimal filter problem is transformed into one of the delay-free system which is just with jumping parameters and multiplicative noises. An optimal MJL filter is presented based on the mean squared method, where the filter gains are determined by solving a set of generalised coupled Riccati difference equations based on a set of coupled Lyapunov equations. Alternatively, an optimal stationary MJL filter is developed, where the filter gains are derived by solving a set of generalised coupled algebraic Riccati equations based on coupled algebraic Lyapunov equations. It can be shown that the difference Riccati equations developed in the former filter converge to the stationary ones under appropriate conditions.

Model predictive control synthesis for constrained Markovian jump linear systems with bounded disturbance

For controller design, exogenous disturbance, which commonly exists in real applications, is an important factor to be considered. However, the existing results on constrained model predictive control (MPC) design for Markovian jump linear systems (MJLS) mainly focused on the disturbance-free case. This motivated to study constrained MPC synthesis for MJLS with bounded additive disturbance. By projecting the desired closed loop trajectories onto a set of predetermined trajectories, the adopted predictive control law is a sum of these corresponding stabilising control laws. Constraints are guaranteed by restricting the augmented state into a maximal output admissible set. Then the MPC algorithm is given by optimising the infinite horizon sum of deviation of the quadratic cost from its steady-state value. Under the MPC algorithm, the long run average cost is proved to be less than or equal to its steady-state value, which equals to the optimal one when one of the constituent control laws is chosen to be the optimal control law. The optimisation and analysis of determination of projection coefficients have also been discussed. Finally, a numerical example is given to illustrate the proposed results.

Static Output-Feedback Control of Markov Jump Linear Systems Without Mode Observation

In this paper, we address infinite-horizon optimal control of Markov Jump Linear Systems (MJLS) via static output feedback. Because the jump parameter is assumed not to be observed, the optimal control law is nonlinear and intractable. Therefore, we assume the regulator to be linear. Under this assumption, we first present sufficient feasibility conditions for static output-feedback stabilization of MJLS with nonobserved mode in the mean square sense in terms of linear matrix inequalities (LMIs). However, these conditions depend on the particular state-space representation, i.e., a coordinate transform can make the LMIs feasible, while the original LMIs are infeasible. To avoid the issues with the ambiguity of the state-space representation, we, therefore, present an iterative algorithm for the computation of the regulator gain. The algorithm is shown to converge if the MJLS is stabilizable via mode-independent static output feedback. However, convergence of the algorithm is not sufficient for the stability of the closed loop, which requires an additional stability check after the regulator gains have been computed. A numerical example demonstrates the application of the presented results.

Stable receding-horizon scenario predictive control for Markov-jump linear systems

This paper discusses predictive control for constrained discrete-time Markov-jump linear systems (MJLS) which jump between a finite set of modes according to a Markov probabilistic transition/observation model, minimising an average cost. Due to the exponential explosion of the number of possible realisations as horizon grows, scenario approaches consider only a subset of them. Prior works cast the problem as a tree-based optimisation one, but enforce stability and feasibility via artificial Lyapunov-related constraints. The proposed approach avoids this route, proposing instead ‘terminal ingredients’ and tree properties (trim-contained, strictly-complete) properly generalising the stability/feasibility ideas in linear and MJLS literature.

Performance evaluation of sampled-data control of Markov jump linear systems

This technical communique extends the recent results of Geromel and Gabriel (2015) to H∞ sampled-data control design of Markov jump linear systems (MJLS). It fulfills a lack of a specific necessary and sufficient result in the literature of sampled-data control of MJLS in the context of H∞ performance. Mean square stabilizability and performance determination are addressed and discussed in a unified theoretical viewpoint. As a natural consequence, it is shown that the previous result of Geromel and Gabriel (2015) is obtained as a particular case. A globally uniformly convergent algorithm is proposed to solve the design conditions. The theory is illustrated by means of an example.

A Stochastic Stability Enhancement Method of Grid-Connected Distributed Energy Storage Systems

Integrating distributed energy storage systems (DESSs) into the distribution system can facilitate the high-level penetration of renewable energy source-based distributed generations (RES-DGs). To mitigate irregularly time-varying power outputs from RES-DGs, supervisory controllers of DESSs need to allocate corresponding power set points for DESS inverter primary controllers. Consequently, operating conditions of DESS inverters determined by allocated power set points could suddenly change over a wide range and seriously threaten the stability of DESSs. To guarantee DESSs operate stably over a wide range of conditions, this paper proposes a stochastic stability enhancement method embedded in the DESS grid-connected dc-ac inverter controllers. Specifically, by considering the stochastic change of DESS operating conditions as a Markov chain process, the small-signal expression of the distribution system is modeled as a Markov jump linear system (MJLS). Based on this MJLS model, a mode-dependent supplementary controller is developed for the DESS grid-connected dc-ac inverter by solving a set of linear matrix inequalities. This controller can adjust its control gains according to the measured real-time operating mode to enhance the stochastic stability of the distribution system. Studies of the Canadian urban benchmark distribution system have shown the proposed controller can enable the distribution system operate in a stable manner when its operating condition stochastically varies.

Robust Finite-Time Stabilization for Networked Control Systems via Static Output-Feedback Control: Markovian Jump Systems Approach

The problems of robust finite-time stochastic stability and stabilization of uncertain networked control systems (NCSs) in the presence of random delays are considered in this paper. First, network-induced random delays are modeled as a Markov chain, and the resulting closed-loop system is transformed into a Markovian jump linear system (MJLS). Since the accurate access to the transition probabilities (TPs) is hard or even impossible, some of the elements in the transition probability matrix (TPM) are considered as unknown parameters. Based on this model, the sufficient conditions for the robust finite-time stochastic stability and stabilization of the uncertain NCS are proposed. Moreover, a new linear matrix inequality (LMI) approach is employed to calculate the static output-feedback controllers. Additionally, the results of the proposed scheme are verified by two examples and simulations, which include a practical example.

$H_2$-Control of Continuous-Time Hidden Markov Jump Linear Systems

This technical note addresses the $H_2$ -control problem for a continuous-time Markov Jump Linear System (MJLS) with partial information on the mode of operation. The jumps of the system depend on a continuous-time hidden Markov model (CT-HMM) where the hidden process represents the dynamics of the real system, while the emitted signal represents the information available to the controller. We start by analyzing the stochastic stability control problem with the goal to design a state feedback linear controller that stochastically stabilizes the closed-loop system, relying on the information from the detector. Two dual formulations, based on LMI problems, are derived as a solution for this problem. These results are then extended to the $H_2$ guaranteed cost control problem. The technical note is concluded with a numerical example which illustrates the derived results.

Delay-dependent stochastic finite-time ℓ1-gain filtering for discrete-time positive Markov jump linear systems with time-delay

This paper deals with the stochastic finite-time ℓ1-gain filtering problem for discrete-time positive Markov jump linear systems (MJLSs) with time-delay. By using twice augmentation system approach of the Markovianized system variables and further analyzing the relationship of finite-time boundedness and finite-time ℓ1-gain performance between the positive MJLS with time-delay and the augmented systems, a delay-dependent sufficient condition in the form of linear programming (LP) is established such that the resulting filtering error system is positive, stochastically finite-time bounded and has prescribed finite-time ℓ1-gain performance. Then the design of the stochastic finite-time positive ℓ1-gain filter is converted into solving some LP-based conditions. A numerical example is given to validate the proposed filtering design method.

H2-control and the separation principle for discrete-time jump systems with the Markov chain in a general state space

ABSTRACTThis paper deals with the H2 optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space . It is assumed that the controller has access only to an output variable and to the jump parameter. The goal, in this case, is to design a dynamic Markov jump controller such that the H2-norm of the closed-loop system is minimised. It is shown that the H2-norm can be written as the sum of two H2-norms, such that one of them does not depend on the control, and the other one is obtained from the optimal filter for an infinite-horizon filtering problem. This result can be seen as a separation principle for MJLS with Markov chain in a Borel space considering the infinite time horizon case.

Robust stability of time-inhomogeneous Markov jump linear systems

In this work we derive necessary and sufficient conditions for robust mean square stability of discrete-time time-inhomogeneous Markov jump linear systems (MJLSs) affected by polytopic uncertainties on transition probabilities and bounded disturbances.

H2 and H∞ dynamic output feedback control of continuous-time Markov systems with uncertain rates using LMIs

The aim of this paper is to investigate the problems of H∞ and H2 dynamic output feedback control for continuous-time Markov jump linear systems (MJLS) subject to uncertain transition rates. Synthesis conditions are proposed in terms of parameter-dependent matrix inequalities with a scalar parameter and polynomial variables. The conditions become linear for fixed values of the scalar parameter, which provides an extra degree of freedom to search for feasible solutions and less conservative H∞ and H2 bounds. Differently from the existing approaches, the proposed method provides full order mode-dependent controllers constructed in terms of the state space matrices of the operation modes and partitions of the slack variables introduced in the design conditions. Therefore, the resulting controller does not depend explicitly on the Lyapunov matrix, being robust with respect to the uncertain transition rates. Moreover, the closed-loop MJLS stability and the bounds to the H∞ or H2 norm are certified by means of polynomially parameter-dependent Lyapunov matrices. Numerical examples illustrate the applicability and the advantages of the proposed technique when compared with other approaches from the literature.

Mixed H2/H∞ State Feedback Control for Markov Jump Linear Systems with Hidden Observations

In the present work, we study the mixed H2/H∞ state feedback control for Markov jump linear systems through a detector approach. It is considered that the Markov parameter θ(k) is not available to the controller, but instead there is an estimation θ^(k) given by a detector. Conditions for the synthesis of controllers that depend only on θ^(k) and stochastically stabilize the system while providing upper bounds for the H2 and H∞ norms are given in terms of linear matrix inequalities. The results are illustrated by a numerical example.

Recursive Robust Regulator for Discrete-time Markovian Jump Linear Systems: Control of Series Elastic Actuators

Impedance control for human-robot interaction allows a more secure and controllable robotic rehabilitation. High-performance force control is required to achieve a more precise impedance-controlled robotic therapy using series elastic actuators. However, the performance of these controllers is sensitive to the highly uncertain and time-varying human dynamics. In this paper, we develop a force controller of a series elastic actuator using a Recursive Robust Regulator for discrete-time Markov jump linear systems subject to parametric uncertainties. We define three Markovian modes of operation depending on the human-robot interaction dynamics. Also, we include an integral action to eliminate steady-state errors, and propose a methodology for robust tracking based on the analysis of the frequency response of the system under parametric uncertainties. Experimental results of the impedance and force controllers for a robotic platform for ankle rehabilitation are presented. These results indicate that the proposed controller maintains similar performance levels despite the transitions between different modes of operation and even subject to the uncertainties introduced by human dynamics.

Mean stability and stabilization of positive linear systems subject to mode-dependent Poisson jumps

This paper deals with the class of positive linear systems subject to Poisson jumps. The random jumps are described by an exponential distribution with mode-dependent parameter. Contrary to classical Markov Jump Linear Systems, no probabilistic information on the sequence of modes is assumed to be available. Mean stability of this class of systems is analyzed, showing that it is equivalent to stability under arbitrary switching of a suitably defined deterministic switching system. Sufficient conditions for mean stability are worked out by relying on common copositive (linear or quadratic) Lyapunov functions. Similar conditions are derived for designing a state-feedback control law guaranteeing closed-loop mean stability and positivity.

Global Kalman filter approaches to estimate absolute angles of lower limb segments

BackgroundIn this paper we propose the use of global Kalman filters (KFs) to estimate absolute angles of lower limb segments. Standard approaches adopt KFs to improve the performance of inertial sensors based on individual link configurations. In consequence, for a multi-body system like a lower limb exoskeleton, the inertial measurements of one link (e.g., the shank) are not taken into account in other link angle estimations (e.g., foot). Global KF approaches, on the other hand, correlate the collective contribution of all signals from lower limb segments observed in the state-space model through the filtering process. We present a novel global KF (matricial global KF) relying only on inertial sensor data, and validate both this KF and a previously presented global KF (Markov Jump Linear Systems, MJLS-based KF), which fuses data from inertial sensors and encoders from an exoskeleton. We furthermore compare both methods to the commonly used local KF.ResultsThe results indicate that the global KFs performed significantly better than the local KF, with an average root mean square error (RMSE) of respectively 0.942° for the MJLS-based KF, 1.167° for the matrical global KF, and 1.202° for the local KFs. Including the data from the exoskeleton encoders also resulted in a significant increase in performance.ConclusionThe results indicate that the current practice of using KFs based on local models is suboptimal. Both the presented KF based on inertial sensor data, as well our previously presented global approach fusing inertial sensor data with data from exoskeleton encoders, were superior to local KFs. We therefore recommend to use global KFs for gait analysis and exoskeleton control.

Stochastic Stability and Stabilization of Semi-Markov Jump Linear Systems with Uncertain Transition Rates

This paper investigates the problems of robust stochastic stability and stabilization for a general class of continuous-time semi-Markovian jump linear systems (S-MJLSs). The main contribution of the research is to eliminate the limitations of the traditional S-MJLSs with precisely available information by introducing a system with uncertain, time-varying transition rates (TRs) of the jump process, in addition to the imperfect information on the system dynamic matrices. The new system is called the general uncertain semi-Markov jump linear system (GUS-MJLS); it does not contain certain values of the transition rates, but includes nominal time-dependent values in addition to bounded deviations. It is suitable to describe a broader class of dynamical systems with estimated information and modeling errors and also covers the concepts of Markov jump linear system (MJLSs) with time-constant and certain TRs. For this system, the stability is firstly analyzed through the multiple stochastic Lyapunov function approach. Then, based on the stability results, a robust state-feedback controller is formulated. To deal with the time-dependent TRs, a sojourn-time fractionizing technique is used and numerically testable conditions are developed. Finally, discussions on reducing the conservativeness of the robust theorems are provided. The theoretical results are successfully tested on an industrial continuous stirred tank reactor (CSTR) subject to stochastically varying environmental conditions. Comparative simulations are also provided to show the superiority of the presented framework and design method to the existing ones.DOI: http://dx.doi.org/10.5755/j01.itc.46.1.13881

A Reduced-Order Fault Detection Filtering Approach for Continuous-Time Markovian Jump Systems with Polytopic Uncertainties

The fault detection (FD) reduced-order filtering problem is investigated for a family of continuous-time Markovian jump linear systems (MJLSs) with polytopic uncertain transition rates, which also include the totally known and partly unknown transition rates. Then, in accordance with the convexification techniques, a novel sufficient condition for the existence of FD reduced-order filter over MJLSs with deficient transition information is obtained in terms of linear matrix inequality (LMI), which can ensure the error augmented system with the FD reduced-order filter is randomly stable. In addition, a performance index is given to enhance the robustness of the residual system against deficient transition information and external disturbance, such that the error between the fault and the residual is made as small as possible to reinforce the faults sensitivity. Finally, the effectiveness of the proposed method is substantiated with two illustrative examples.