Continuous-Time Markovian Jump Research Articles

[formula omitted] detector-based state feedback control of continuous-time MJLS subject to uncertain transition rate matrices

This paper uses detector-based mode information to address the H2 state-feedback control problem for continuous-time Markov Jump Linear Systems (MJLS). The focus is on designing robust controllers capable of handling uncertainties affecting both the Markov process and its detector, a less explored scenario in the literature. The main contribution lies in presenting a new LMI-based algorithm within a suitable synthesis framework to deal with uncertain parameters. A key advantage is the absence of constraints on the optimization variables, even in cases involving structured controllers like decentralized control. To illustrate the effectiveness of the proposed technique, a numerical example based on a linearized model of an unmanned aircraft is provided, showcasing its superiority over existing approaches.

An accelerated zeroing neural network for solving continuous coupled Lyapunov matrix equations

AbstractIn this paper, an improved zeroing neural network (ZNN) model is proposed to obtain the positive definite solutions of the continuous coupled Lyapunov matrix equations (CLMEs) associated with continuous‐time Markovian jump (CMJ) systems. To achieve this, a general ZNN model is established by constructing a matrix‐valued error function. Then, to accelerate the convergence rate of the proposed ZNN model, the latest estimation is introduced to obtain an improved ZNN model. Some convergence conditions have been derived for the presented improved ZNN model through Lyapunov theory. Comparisons among the improved ZNN model and the existing results are conducted to illustrate the advantages of the proposed improved ZNN model in numerical examples.

Stability and Tracking Recovery of Continuous-Time Markov Jump Piecewise Affine Systems Using Virtual-Sensor-Based Reconfiguration

Stability and Tracking Recovery of Continuous-Time Markov Jump Piecewise Affine Systems Using Virtual-Sensor-Based Reconfiguration

The Strictly Dissipative Condition of Continuous-Time Markovian Jump Systems with Uncertain Transition Rates

This study addresses the problem of strictly dissipative stabilization for continuous-time Markovian jump systems (MJSs) with external disturbances and generally uncertain transition rates that contain completely unknown transition rates and their bound values. A stabilization condition is obtained to guarantee strict dissipativity for the MJSs with partial knowledge in terms of the transition rates. To reduce the conservativity of the proposed condition, we used a boundary condition related to the bounds of the transition rate with slack variables. Finally, two simulation results are presented to describe the feasibility of the proposed controller.

A New Approach to the Energy-to-Peak Performance Analysis of Continuous-Time Markov Jump Linear Systems

A New Approach to the Energy-to-Peak Performance Analysis of Continuous-Time Markov Jump Linear Systems

Asynchronous Output Regulation Control for Continuous-Time Markovian Jump Systems with Colored-Noise

Asynchronous Output Regulation Control for Continuous-Time Markovian Jump Systems with Colored-Noise

Reinforcement Learning-Based Near Optimization for Continuous-Time Markov Jump Singularly Perturbed Systems

The design of a suboptimal controller for continuous-time Markov jump singularly perturbed systems with partially unknown dynamics is studied in this paper. With fast and slow decomposition technique, the original Markov jump singularly perturbed systems are decomposed into fast and slow subsystems as a new attempt. On this basis, an offline parallel Kleinman algorithm and an online parallel integral reinforcement learning algorithm are presented to cope with the different subsystems, respectively. Meanwhile, the controllers obtained by the above two algorithms are used to design the suboptimal controllers for original systems. Furthermore, the suboptimality of the proposed controllers is also discussed. Finally, an example of the electric circuit model is shown to illustrate the applicability of the proposed method.

Markov trajectories: Microcanonical Ensembles based on empirical observables as compared to Canonical Ensembles based on Markov generators

The Ensemble of trajectories $x(0 \leq t \leq T)$ produced by the Markov generator $M$ can be considered as 'Canonical' for the following reasons : (C1) the probability of the trajectory $x(0 \leq t \leq T)$ can be rewritten as the exponential of a linear combination of its relevant empirical time-averaged observables $E_n$, where the coefficients involving the Markov generator are their fixed conjugate parameters; (C2) the large deviations properties of these empirical observables $E_n$ for large $T$ are governed by the explicit rate function $I^{[2.5]}_M (E_.) $ at Level 2.5, while in the thermodynamic limit $T=+\infty$, they concentrate on their typical values $E_n^{typ[M]}$ determined by the Markov generator $M$. This concentration property in the thermodynamic limit $T=+\infty$ suggests to introduce the notion of the 'Microcanonical Ensemble' at Level 2.5 for stochastic trajectories $x(0 \leq t \leq T)$, where all the relevant empirical variables $E_n$ are fixed to some values $E^*_n$ and cannot fluctuate anymore for finite $T$. The goal of the present paper is to discuss its main properties : (MC1) when the long trajectory $x(0 \leq t \leq T) $ belongs the Microcanonical Ensemble with the fixed empirical observables $E_n^*$, the statistics of its subtrajectory $x(0 \leq t \leq \tau) $ for $1 \ll \tau \ll T $ is governed by the Canonical Ensemble associated to the Markov generator $M^*$ that would make the empirical observables $E_n^*$ typical ; (MC2) in the Microcanonical Ensemble, the central role is played by the number $\Omega^{[2.5]}_T(E^*_.) $ of stochastic trajectories of duration $T$ with the given empirical observables $E^*_n$, and by the corresponding explicit Boltzmann entropy $S^{[2.5]}( E^*_. ) = [\ln \Omega^{[2.5]}_T(E^*_.)]/T $. This general framework is applied to continuous-time Markov Jump processes and to discrete-time Markov chains with illustrative examples.

ℋ2 and ℋ∞ Filtering for Continuous-Time Markov Jump Lur'e Systems with Sector Bound Optimization

This paper is about the and filtering of Markov Jump Systems subject to a class of nonlinearities which include them on the Lur'e systems framework. The proposed filters are robust to nonlinearities belonging to a given sector with maximum slope. With regard to the availability of the Markov parameter, we present strategies which allow us to design either mode-dependent and modeindependent filters, with sector bound optimisation. The results are illustrated by two examples.

Energy-to-Peak Reduced Order Filtering for Continuous-Time Markov Jump Linear Systems With Partial Information on the Jump Parameter

This paper deals with the design of an energy-to-peak reduced-order filter (also called <i>L</i>₂ &#x2013; <i>L</i>_&#x221E; filtering) for continuous-time Markov jump linear systems, assuming that the filter has only access to an estimate of the Markov parameter, coming from the output of a detector device. To model this situation we consider that the joint process formed by the Markov parameter and the detector information follows an exponential hidden Markov model. The result is given in terms of Linear Matrix Inequalities (LMI) so that the available numerical package tools can be readily implemented to solve the problem. The paper is concluded with some numerical simulations.

Homogenized first-moment analysis of two-time-scale positive Markov jump linear systems

We address in this paper the mean stability and L∞ performance of continuous-time Positive Markov Jump Linear Systems (PMJLS). The distinguishing aspect of our approach vis-à-vis the existing literature is that the underlying Markov jump process is a two-time-scale Markov chain, and we consider the singular perturbation setup which arises when a small parameter (which determines the time-scale separation) goes to zero. The interest in this limiting scenario stems from large-scale situations, where complexity reduction is a central issue. To achieve this, we carry out a convergence analysis involving the semigroup that describes the first moment dynamics of the system state. This analysis allows us to subsequently characterize homogenized notions of stability and L∞ performance, and we show how these can be connected with linear programming methods. A numerical example, regarding a version of the Foschini-Miljanic algorithm for power allocation in a mobile communication system, illustrates the proposed results.

Mixed ℋ2/ℋ∞ State-Feedback Control of Continuous-Time Markov Jump Systems with Partial Observations of the Markov Chain

We study the mixed ℋ2/ℋ∞ state-feedback control of continuous-time Markov jump linear systems considering that the Markov chain is not observable, with the only information available to the controller coming from the output of a fault-detection and isolation device. We present sufficient design conditions given in terms of linear matrix inequalities so that the closed-loop system is stable and its ℋ2 and ℋ∞ norms are bounded. We present an illustrative example in which we investigate the behavior of the proposed algorithm.

Nash Tracking Game with Preview by State Feedback for Linear Continuous-Time Markovian Jump Systems

Nash Tracking Game with Preview by State Feedback for Linear Continuous-Time Markovian Jump Systems

A Reduced-order Fault Detection Filter Design for Polytopic Uncertain Continuous-time Markovian Jump Systems with Time-varying Delays

In this article, the fault detection (FD) reduced-order filtering problem is investigated for a family of polytopic uncertain continuous-time Markovian jump linear systems (MJLSs) with time-varying delays. Under meeting the the premise of fault detection accuracy, the reduced order fault detection filters are desirable in applications where fast data processing and saving computing space are necessary. Then, by the aid of the Markovian Lyapunov-Krasovskii functional and convex polyhedron techniques, some novel time-varying delays and polytopic uncertain sufficient conditions are proposed to insure the existence of the FD reduced-order filter. Finally, a simulated continuous stirred tank reactor process example is provided to verify the feasibility of the proposed methodology.

Robust Non-fragile Asynchronous Controller Design for Continuous-Time Markov Jump Linear Systems: Non-homogeneous Markov Process Approach

This paper proposes a robust asynchronous controller for continuous-time Markov jump linear systems (MJLSs). The asynchronous structure considers a case in which the Markov chain governing candidate controllers does not match the chain which manages the switching between system modes. This type of asynchronousy arises because the real-time, exact and precise detection of the system modes is not practical; therefore, the observed modes slightly differ with the actual modes. This paper models the asynchronousy through an additional, observed Markov chain which depends on the original Markov chain of the system according to uncertain transition probabilities. By this representation, the whole system is viewed as a non-homogeneous MJLS, and the sufficient stabilizability conditions as well as the controller gains are obtained through multiple mode-dependent Lyapunov functions. This approach leads to less conservative design results than the previous mode-independent schemes and proposes a much simple methodology to obtain the controller than the previous mode-dependent studies. For more generality, the proposed controller also takes the possible gain variations occurring in the implementation procedures into account and reports all the results in terms of linear matrix inequalities. Simulation results on a vertical takeoff and landing helicopter are presented and compared with the common mode-independent controller to illustrate the effectiveness of the developed method.

$H_2$-Control of Continuous-Time Hidden Markov Jump Linear Systems

This technical note addresses the $H_2$ -control problem for a continuous-time Markov Jump Linear System (MJLS) with partial information on the mode of operation. The jumps of the system depend on a continuous-time hidden Markov model (CT-HMM) where the hidden process represents the dynamics of the real system, while the emitted signal represents the information available to the controller. We start by analyzing the stochastic stability control problem with the goal to design a state feedback linear controller that stochastically stabilizes the closed-loop system, relying on the information from the detector. Two dual formulations, based on LMI problems, are derived as a solution for this problem. These results are then extended to the $H_2$ guaranteed cost control problem. The technical note is concluded with a numerical example which illustrates the derived results.

A Reduced-Order Fault Detection Filtering Approach for Continuous-Time Markovian Jump Systems with Polytopic Uncertainties

The fault detection (FD) reduced-order filtering problem is investigated for a family of continuous-time Markovian jump linear systems (MJLSs) with polytopic uncertain transition rates, which also include the totally known and partly unknown transition rates. Then, in accordance with the convexification techniques, a novel sufficient condition for the existence of FD reduced-order filter over MJLSs with deficient transition information is obtained in terms of linear matrix inequality (LMI), which can ensure the error augmented system with the FD reduced-order filter is randomly stable. In addition, a performance index is given to enhance the robustness of the residual system against deficient transition information and external disturbance, such that the error between the fault and the residual is made as small as possible to reinforce the faults sensitivity. Finally, the effectiveness of the proposed method is substantiated with two illustrative examples.

Stochastic Optimal Tracking with Preview for Linear Continuous-Time Markovian Jump Systems by Output Feedback

Stochastic Optimal Tracking with Preview for Linear Continuous-Time Markovian Jump Systems by Output Feedback

Average Reachability of Continuous-time Markov Jump Linear Systems and the Linear Minimum Mean Square Estimator

In this paper we study the average reachability gramian for continuous-time linear systems with additive noise and jump parameters driven by a general Markov chain. We define a rather natural reachability concept by requiring that the average reachability gramian be positive definite. Aiming at a testable condition, we introduce a set of reachability matrices for this class of systems and employ invariance properties of the null space of the noise coefficient matrices to show that the system is reachable if and only if these matrices are of full rank. We also show for reachable systems that the state second moment is positive definite. One consequence of this result in the context of linear minimum mean square state estimation for reachable systems is that the expectation of the error covariance matrix is positive definite. Moreover, the average boundedness of the error covariance matrix is invariant to a type of perturbation in the noise model, meaning that the estimates are not overly sensitive, which consists in a property that is desirable in applications and sometimes referred to as stability of the estimator.

Stochastic Optimal Tracking with Preview for Linear Continuous-Time Markovian Jump Systems by Output Feedback

The author has already presented stochastic optimal tracking control theory with preview for linear continuous-time Markovian jump systems on the finite time interval by state feedback ([21]). He has also presented optimal estimation theory for linear continuoustime Markovian jump systems on the finite time interval ([22]). In this paper we study stochastic optimal tracking control theory with preview for linear continuous-time Markovian jump systems on the finite time interval by output feedback. The necessary and sufficient conditions for the solvability of the stochastic optimal tracking problem with preview by state feedback are given by coupled Riccati differential equations with terminal conditions. In order to decide the gains of output feedback controllers, we need solutions of another type of coupled Riccati differential equations with initial conditions.