Markov Jump Linear Systems Research Articles

[formula omitted] detector-based state feedback control of continuous-time MJLS subject to uncertain transition rate matrices

This paper uses detector-based mode information to address the H2 state-feedback control problem for continuous-time Markov Jump Linear Systems (MJLS). The focus is on designing robust controllers capable of handling uncertainties affecting both the Markov process and its detector, a less explored scenario in the literature. The main contribution lies in presenting a new LMI-based algorithm within a suitable synthesis framework to deal with uncertain parameters. A key advantage is the absence of constraints on the optimization variables, even in cases involving structured controllers like decentralized control. To illustrate the effectiveness of the proposed technique, a numerical example based on a linearized model of an unmanned aircraft is provided, showcasing its superiority over existing approaches.

Strong Consistency and Rate of Convergence of Switched Least Squares System Identification for Autonomous Markov Jump Linear Systems

Strong Consistency and Rate of Convergence of Switched Least Squares System Identification for Autonomous Markov Jump Linear Systems

Stabilization and Optimal Control for Discrete-Time Markov Jump Linear System With Multiplicative Noises and Input Delays: A Complete Solution

Stabilization and Optimal Control for Discrete-Time Markov Jump Linear System With Multiplicative Noises and Input Delays: A Complete Solution

Robust Recursive Regulator for Polytopic Markovian Jump Linear Systems with Random State Delays

Robust Recursive Regulator for Polytopic Markovian Jump Linear Systems with Random State Delays

Stabilization and Optimal Control for Markov Jump Linear System With Two Controllers Over Multiple Erasure Channels

Stabilization and Optimal Control for Markov Jump Linear System With Two Controllers Over Multiple Erasure Channels

A New Approach to the Energy-to-Peak Performance Analysis of Continuous-Time Markov Jump Linear Systems

A New Approach to the Energy-to-Peak Performance Analysis of Continuous-Time Markov Jump Linear Systems

Stochastic Analysis of Control Systems Subject to Communication and Computation Faults

Control theory allows one to design controllers that are robust to external disturbances, model simplification, and modelling inaccuracy. Researchers have investigated whether the robustness carries on to the controller’s digital implementation, mostly looking at how the controller reacts to either communication or computational problems. Communication problems are typically modelled using random variables (i.e., estimating the probability that a fault will occur during a transmission), while computational problems are modelled using deterministic guarantees on the number of deadlines that the control task has to meet. These fault models allow the engineer to both design robust controllers and assess the controllers’ behaviour in the presence of isolated faults. Despite being very relevant for the real-world implementations of control system, the question of what happens when these faults occur simultaneously does not yet have a proper answer. In this paper, we answer this question in the stochastic setting, using the theory of Markov Jump Linear Systems to provide stability contracts with almost sure guarantees of convergence. For linear time-invariant Markov jump linear systems, mean square stability implies almost sure convergence – a property that is central to our investigation. Our research primarily emphasises the validation of this property for closed-loop systems that are subject to packet losses and computational overruns, potentially occurring simultaneously. We apply our method to two case studies from the recent literature and show their robustness to a comprehensive set of faults. We employ closed-loop system simulations to empirically derive performance metrics that elucidate the quality of the controller implementation, such as the system settling time and the integral absolute error.

Regulation of Uncertain Markov Jump Linear Systems With Application on Automotive Powertrain Control

Modeling and control of dynamic systems that experience abrupt changes are challenging and fundamental tasks in applications of different areas of engineering. Among these, control synthesis for autonomous heavy-duty vehicles has the potential to highly benefit transportation systems. That said, we propose a robust recursive regulator for discrete-time Markov jump linear systems with polytopic uncertain data. We assume the uncertainties affect the state-space parameters as well as the transition probability matrix. We formulate an optimization problem using the penalty function method and design a recursive solution, whereby we attain the robust state feedback gains. We impose no restrictions on how fast the uncertainties can change between two consecutive iterations. Conditions for convergence and mean-square stability are well defined in terms of Riccati recursive equations. We also set up a procedure for powertrain polytopic model identification based on CAN bus data of an autonomous heavy-duty ground vehicle. We provide two numerical examples to verify the effectiveness of the robust recursive regulator. Moreover, in a third example, we validate the obtained powertrain model on a trajectory tracking problem and compute acceleration and braking inputs using the robust regulator.

Asynchronous stabilization for hidden Markov jump linear systems with complex transition probabilities

This study is intended to investigate the asynchronous stabilization problem for discrete-time Markov jump linear systems (MJLSs) with complex mode transition probabilities (C-TPs). A practical scenario is considered in this study. First, the system mode may not always be precisely detected in practice, therefore a hidden Markov model (HMM) is used to characterize the mismatched mode phenomenon. Second, some mode TPs associated with the Markov process of the original system may be unknown or inaccurate, rendering the C-TPs scenario. This issue could also exist in the conditional TPs associated with the joint process of the HMM. A mode separation strategy is proposed to separate the Markov process and the joint process such that we can deal with the C-TPs associated with them, respectively. Based on the above considerations, a unified controller design framework is established for MJLSs with or without C-TPs. Moreover, the results developed in this study can cover some existing works as special cases without introducing additional conservativeness. A DC motor device is used to demonstrate the effectiveness of the proposed design method.

Quantized feedback stabilization of continuous‐time Markov jump systems under asynchronous control

AbstractThis paper addresses a new asynchronous control scheme for continuous‐time Markov jump linear systems (MJLSs). Both controlled system and quantizer are asynchronous with the controller due to the process by which the controller can accurately observe and emit the switching signal being stochastic. The random variable satisfying Bernoulli distribution is introduced to describe this observation. On this basis, two methods are proposed to obtain sufficient conditions for exponential almost sure stability and almost surely asymptotically stability, respectively from the perspective of the linear matrix inequality (LMI). The results are independent of the asynchronous time interval. Finally, a numerical example demonstrates the validity and feasibility of developed theoretical results.

Gain Scheduled Fault Detection Filter for Markovian Jump Linear System with Nonhomogeneous Markov Chain

In a networked control system scenario, the packet dropout is usually modeled by a time-invariant (homogeneous) Markov chain (MC) process. However, from a practical point of view, the probabilities of packet loss can vary in time and/or probability parameter dependency. Therefore, to design a fault detection filter (FDF) implemented in a semi-reliable communication network, it is important to consider the variation in time of the network parameters, by assuming the more accurate scenario provided by a nonhomogeneous jump system. Such a premise can be properly taken into account within the linear parameter varying (LPV) framework. In this sense, this paper proposes a new design method of H∞ gain-scheduled FDF for Markov jump linear systems under the assumption of a nonhomogeneous MC. To illustrate the applicability of the theoretical solution, a numerical simulation is presented.

Policy Optimization for Markovian Jump Linear Quadratic Control: Gradient Method and Global Convergence

Recently, policy optimization has received renewed attention from the control community due to various applications in reinforcement learning tasks. In this article, we investigate the global convergence of the gradient method for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS). First, we study the optimization landscape of direct policy optimization for MJLS, with static-state feedback controllers and quadratic performance costs. Despite the nonconvexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and smoothness. Based on these properties, we prove that the gradient method converges to the optimal-state feedback controller for MJLS at a linear rate if initialized at a controller, which is mean-square stabilizing. This article brings new insights for understanding the performance of the policy gradient method on the Markovian jump linear quadratic control problem.

A vehicle cloud control system considering communication quantization and stochastic delay

AbstractA novel kind of networked feedback controller is designed for the vehicle cloud control system in the presence of both sensor‐controller/controller‐actuator delay and communication quantization. Firstly, the vehicle cloud control system with delayed and quantized communication is modeled using the lateral/longitudinal vehicle dynamic and Markovian jump linear system (MJLS) theory. Then, some efficient stabilization conditions in matrix inequalities forms are derived for the considered cloud‐controlled vehicles. Furthermore, a cone complementary linearization method is utilized to solve the nonlinear matrix inequalities involved in the stabilization conditions. Simulation tests on connected vehicle lateral and longitudinal control are conducted to verify the effectiveness of the analytical results. Compared with the commonly used constant‐parameter controllers, the proposed method is practical to design vehicle cloud controllers under stochastic delay and communication quantization scenarios, with known delay distribution. Lastly, the stability of the vehicle cloud control system is guaranteed under the infection of unreliable communication factors with the proposed control method.

Asynchronous-Switching-Based Distributed Interval Filtering for Positive Markov Jump Linear Systems Under Denial-of-Service Attack

This article presents an asynchronous distributed interval filtering method for discrete-time positive Markov jump linear systems (MJLSs) with bounded disturbance under the denial-of-service attack. By designing an asynchronous switching-distributed mechanism that depends on a randomly occurred attack, the proposed positive filter can monitor the real-time range of estimated states against the attack. The filtering error systems are modeled as a class of MJLSs with two attack-dependent Markov chains. Some analysis and design results are derived such that the error systems are positive and stochastically internally stable with satisfied <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{1}$ </tex-math></inline-formula> -gain performance. The remote power monitoring in telecommunication systems is used for numerical verification.

Markovian-Jump Reinforcement Learning for Autonomous Underwater Vehicles under Disturbances with Abrupt Changes

This paper studies the position regulation problems of an Autonomous Underwater Vehicle (AUV) subject to external disturbances that may have abrupt variations due to some events, e.g., water flow hitting nearby underwater structures. The disturbing forces may frequently exceed the actuator capacities, necessitating a constrained optimization of control inputs over a future time horizon. However, the AUV dynamics and the parameters of the disturbance models are unknown. Estimating the Markovian processes of the disturbances is challenging since it is entangled with uncertainties from AUV dynamics. As opposed to a single-Markovian description, this paper formulates the disturbed AUV as an unknown Markovian-Jump Linear System (MJLS) by augmenting the AUV state with the unknown disturbance state. Based on an observer network and an embedded solver, this paper proposes a reinforcement learning approach, Disturbance-Attenuation-net (MDA–net), for attenuating Markovian-jump disturbances and stabilizing the disturbed AUV. MDA–net is trained based on the sensitivity analysis of the optimality conditions and is able to estimate the disturbance and its transition dynamics based on observations of AUV states and control inputs online. Extensive numerical simulations of position regulation problems and preliminary experiments in a tank testbed have shown that the proposed MDA–net outperforms the existing DOB–net and a classical approach, Robust Integral of Sign of Error (RISE).

Policy Synthesis for Switched Linear Systems With Markov Decision Process Switching

In this article, we study the synthesis of mode switching protocols for a class of discrete-time switched linear systems in which the mode jumps are governed by Markov decision processes (MDPs). We call such systems MDP-JLS for brevity. Each state of the MDP corresponds to a mode in the switched system. The probabilistic state transitions in the MDP represent the mode transitions. We focus on finding a policy that selects the switching actions at each mode such that the switched system is guaranteed to be stable. Given a policy in the MDP, the considered MDP-JLS reduces to a Markov jump linear system (MJLS). We consider both mean-square stability and stability with probability one. For mean-square stability, we leverage existing stability conditions for MJLSs and propose efficient semidefinite programming formulations to find a stabilizing policy in the MDP. For stability with probability one, we derive new sufficient conditions and compute a stabilizing policy using linear programming. We also extend the policy synthesis results to MDP-JLS with uncertain mode transition probabilities.

On the H2 Control of Hidden Markov Jump Linear Systems

This letter studies H2 control of linear systems with parameters driven by a hidden Markov chain, emphasizing a property of invariance of the norm to time shifts in the disturbance signal, which we call H2 time-invariance (H2TI). We show that some systems are not H2TI and that the lack of H2TI may deteriorate the performance of H2 control systems when they are subject to disturbances other than impulses at the time instant k = 0 (as in standard H2 control). This motivates us to propose a new variant of the H2-norm, based on a stochastic process having an impulse at a random time. We develop the formula for computing this new variant and apply the result to an LMI optimization problem. Numerical tests are performed, indicating the superiority of the proposed controller for systems subject to different types of disturbances.

Energy-to-Peak Static Output Control for Continuous-Time Hidden Markov Jump Linear Systems

We study the L2 — L∞ static output feedback control, also known as energy-to-peak control, for continuous-time Markov jump systems in a context of partial observation of the Markov chain. We consider that the controller has only access to an observed process related to the Markov chain of the plant by means of an exponential hidden Markov model. Sufficient conditions for the design of output feedback controllers satisfying an upper bound on the L2—L∞ ratio are given in terms of Linear Matrix Inequalities (LMIs). Based on this result a coordinate descent algorithm is presented to reduce the L2—L∞ upper bound ratio, starting from a feasible solution for the LMIs restrictions. The paper is conclude with a numerical example to illustrate the results.

Robust Regulator for Markov Jump Linear Systems with Random State Delays and Uncertain Transition Probabilities

The recursive robust regulation problem for Markov jump linear systems with state delay is investigated. The random delay belongs to a known interval and its maximum variation rate is considered. The transition probabilities matrix is subject to polytopic uncertainties. We model the delay by a Markov chain and obtain a delay-free Markovian system, through the augmented system approach. Based on combination of regularized least-squares with polytopic uncertainties and penalty function method, we propose a mode-dependent state-feedback control. The solution is given in terms of coupled Riccati equations presented in a symmetric matrix arrangement. With a numerical example, we compare our approach with other robust controllers from the literature.

Stabilization and Optimization of Discrete-Time Markovian Jump Linear Systems via Mode Feedback Control

This article is devoted to a specific kind of discrete-time switched linear systems, where the switching signal is governed by a Markov chain, i.e., Markovian jump linear systems. For such systems, a mode feedback control mechanism is adopted to adjust the mode transition probability matrix, which is referred to as the switching law design, and the optimal mode feedback controller is sought to minimize a quadratic performance index containing both the system state and the mode feedback control input. First, the admissible set of the mode feedback control is investigated, and a sufficient condition is derived to ensure the desired stochastic stability. Second, under the assumption that the concerned system is mode-cost sequential, the original performance index is approximated into a new tractable one and a suboptimal mode feedback controller is then derived within the admissible set. Third, we consider a general situation where no specific requirements are imposed on the Markovian jump linear systems’ dynamics, derive the optimal mode feedback controller via a value-iteration-based algorithm, and prove the convergence of the obtained optimal controller. Finally, simulation results are provided to illustrate the validity of the proposed mode feedback control mechanism.