Control Of Stochastic Systems Research Articles

Modeling and Control of Stochastic Systems With Poorly Known Dynamics

This paper is concerned with controlling poorly known systems, for which only a simplified and rough model is available for control design. There are many systems that cannot be reasonably probed for the sake of identification, yet they are important for areas such as economy, populations, or medicine. The ideas are developed around an alternative way to account for the bare modeling in a stochastic-based setting, and to heighten the control features for such a modified model. The mathematical framework for the optimal control reveals important features such as the raising of a precautionary feedback policy of “keep the action unchanged” (inaction for short), on a certain state-space region. This feature is not seen in the robust approach, but has been pointed out and permeates part of the economics literature. The control problem relies on the viscosity solution for the Hamilton–Jacobi–Bellman equation, and the value of the problem is shown to be convex. When specialized to the quadratic problem with discounted cost, the exact solution inside the inaction region is given by a Lyapunov type of equation, and asymptotically, for large state values, by a Riccati-like equation. This scenario bridges to the stochastic stability analysis for the controlled model. The single control input is developed in full, part analytically, part numerically, for the scalar case, and an approximation is tested for the multidimensional case. The advantage of the precautionary policy is substantial in some situations.

Управление с прогнозирующей моделью стохастическими системами с марковскими скачками и сериально коррелированными параметрами при ограничениях

Управление с прогнозирующей моделью стохастическими системами с марковскими скачками и сериально коррелированными параметрами при ограничениях

Resilient dissipative based controller for stochastic systems with randomly occurring gain fluctuations

This paper addresses the dissipativity based resilient sampled-data control problem for a class of stochastic systems with actuator failures. In particular, the controller is assumed to posses probabilistic actuator faults with different failure rates. Moreover, the controller gain fluctuation appears in a random way which obeys certain Bernoulli distributed white noise sequences. A new set of sufficient conditions is obtained in terms of linear matrix inequalities for the derivation of resilient dissipative sampled-data controller by constructing an appropriate Lyapunov–Krasovskii functional together with free-weighting matrix technique. Simulation studies are performed to verify the performance and effectiveness of the proposed control design technique.

Modeling and Control of Itô Stochastic Networked Control Systems With Random Packet Dropouts Subject to Time-Varying Sampling

In this note, we discuss the modeling and control of a class of Ito stochastic networked control systems (NCSs) with packet dropouts that are subject to time-varying sampling. The system under consideration is first modeled as a continuous-time system with input delay that is subject to double randomness via the input-delay approach. Then, by assuming the packet drop rate and recurring to the celebrated formula of total probability, an equivalent model which takes full advantage of probability distribution characteristics of both packet dropouts and sampling periods is established. In particular, the probability distribution values of the stochastic delay taking values in two given intervals can be explicitly obtained, which is of crucial importance to model and analyze the actual problem in NCSs. Based on that, robustly exponentially mean-square stability of the system with an $H_\infty$ performance is guaranteed, an $H_\infty$ controller design procedure is then proposed. Finally, a numerical simulation example is exploited to show the effectiveness and applicability of the results derived and some less conservative results are obtained.

Optimal bounded control for nonlinear stochastic smart structure systems based on extended Kalman filter

An optimal bounded control strategy for smart structure systems as controlled Hamiltonian systems with random excitations and noised observations is proposed. The basic dynamic equations for a smart structure system with smart sensors and actuators are firstly given. The nonlinear stochastic control system with noised observations is then obtained from the simplified smart structure system, and the system is expressed by generalized Hamiltonian equations with control, random excitation and dissipative forces. The optimal control problem for nonlinear stochastic systems with noised observations includes two parts: optimal state estimation and optimal response control based on estimated states, which are coupled each other. The probability density of optimally estimated systems has generally infinite dimensions based on the separation theorem. The proposed optimal control strategy gives an approximate separate solution. First, the optimally estimated system state is determined by the observations based on the extended Kalman filter, and the estimated nonlinear system with controls and stochastic excitations is obtained which has finite-dimensional probability density. Second, the dynamical programming equation for the estimated system is determined based on the stochastic dynamical programming principle. The control boundedness due to actuator saturation is considered, and the optimal bounded control law is obtained by the programming equation with the bounded control constraint. The optimal control depends on the estimated system state which is determined by noised observations. The proposed optimal bounded control strategy is finally applied to a single-degree-of-freedom nonlinear stochastic system with control and noised observation. The remarkable vibration control effectiveness is illustrated with numerical results. Thus the proposed optimal bounded control strategy is promising for application to nonlinear stochastic smart structure systems with noised observations.

Stabilization of some composite stochastic control systems with nontrivial solutions

This paper deals with the global stability for some composite stochastic control systems with nontrivial solutions by means of dynamic feedback laws. In particular, we establish feedback law for global asymptotic stabilization of a control subsystem deduced from the composite stochastic system and apply the result to stabilize the original composite stochastic system. Under this framework, firstly we derive necessary and sufficient conditions for the existence of feedback law which renders a ball that contains all nontrivial equilibrium solutions of the stochastic control subsystem globally asymptotically stable in probability. Secondly, for the composite system, we give the sufficient conditions for practical global asymptotic stability of the composite system based upon the feedback law derived for the subsystem. Numerical examples are given to validate our results.

Explicit Boundary Controls for Finite Diffusion Process

Three types of boundary controls are considered: proportional control, feed-forward control and integrated-by-past-values control. They stabilise the diffusion process exponentially at the desired reference on the controlled boundary. The process is actuated through the Neumann boundary while on other boundaries it is given with the Dirichlet data and, in higher dimensions, also with the homogeneous Neumann boundary condition. Two systems, deterministic and stochastic are compared as an ideal and real description of a physical system. Disturbance in the stochastic system is originated by a white noise on the controlled boundary. Physically this noise represents different types of uncertainties like a side-reaction mass-flux and other uncertainties that are not part of a deterministic model. At first the proportional control is analysed on the deterministic system and then it is represented in the feedback integrated past controls form, and lastly the developed controls are modified as a version of stochastic control that applied on the stochastic system, which boundary is disturbed with an unmeasured noise. The regulation error that emerges in the stochastic system control is analysed and proved to be a zero-mean, bounded-variance Gaussian variable that is correlated spatially and temporarily. The process control is demonstrated on an electrodeposition process example.

Resource efficient stochastic predictive control under packet dropouts

This study presents a resource efficient framework for a class of stochastic control systems that utilises state dependent control strategies in order to reduce the online computational load. When the states are in a given neighbourhood of the desired operating point, the controller is switched off, and data from the feedback channel is not transmitted. Outside this neighbourhood, the authors pay close attention to the performance of the controller by adopting a stochastic predictive algorithm when the states are in a predefined comfort zone, and activate a recovery algorithm beyond the comfort zone that secures at least good qualitative properties. The authors demonstrate that the proposed controller leads to mean square boundedness of the closed loop states in the presence of stochastic noise, bounded control authority, and control channel erasures, while entailing a dramatic reduction in network traffic and computational resources.

Performance Analysis of Event-Triggered Control Systems with A Probabilistic Triggering Mechanism: The Scalar Case

This paper considers a linear stochastic control system whose feedback loop is closed over (possibly low-power wireless) communication channels. A probabilistic event-based scheduling rule is adopted to transmit measurements from the sensor to the controller. The controller only computes a new control command whenever a sensor packet arrives at the controller. The event-triggered strategy proposed in this paper aims at providing significant reductions in the usage of both the sensor-to-controller and controller-to-actuator channels. Within this setup, we investigate dead-beat control laws and compute the expected linear-quadratic loss of the closed-loop system. We also provide analytic expressions that quantify the trade-off between the communication cost and the control performance of such event-triggered control systems. This trade-off is demonstrated quantitatively via a numerical example.

Output‐feedback model predictive control for stochastic systems with multiplicative and additive uncertainty

SummaryThis paper studies the output‐feedback model predictive control (MPC) design problem for linear systems with multiplicative and additive random uncertainty. We first present an off‐line optimization algorithm to optimize feedback gains of the observer and the dual‐mode control policy. After that, by defining a cuboid tube whose center and boundary are both time‐varying variables, we develop a set sequence with increased freedom to contain stochastic system trajectories. A quadratic performance function with analytic upper and lower bounds is minimized such that it decreases exponentially to a finite range under the expectation. The resulting MPC algorithms are proved to guarantee practically stochastic input‐to‐state stability. A numerical example of the wind turbine model illustrates the properties of the MPC algorithms.

On the use of information quality in stochastic networked control systems

Networked control is challenged by stochastic delays that are caused by the communication networks as well as by the approach taken to exchange information about system state and set-points. Combined with stochastic changing information, there is a probability that information at the controller is not matching the true system observation, which we call mismatch probability (mmPr). The hypothesis is that the optimization of certain parameters of networked control systems targeting mmPr is equivalent to the optimization targeting control performance, while the former is practically much easier to conduct. This is first analyzed in simulation models for the example system of a wind-farm controller. As simulation analysis is subject to stochastic variability and requires large computational effort, the paper develops a Markov model of a simplified networked control system and uses numerical results from the Markov model analysis to demonstrate that mmPr based optimization can improve control performance.

Composite hierarchical anti-disturbance control for stochastic systems with multiple disturbances

A class of stochastic systems with multiple disturbances, which includes white noises and disturbances whose time derivative is bounded, is considered in this paper. To estimate the unknown bounded disturbance, a stochastic disturbance observer is proposed. Based on the observer, a disturbance observer-based disturbance control scheme is constructed such that the composite closed-loop system is asymptotically bounded. Finally, a simulation example is given to demonstrate the feasibility and effectiveness of the proposed scheme.

Data-injection attacks in stochastic control systems: Detectability and performance tradeoffs

Consider a stochastic process being controlled across a communication channel. The control signal that is transmitted across the control channel can be replaced by a malicious attacker. The controller is allowed to implement any arbitrary detection algorithm to detect if an attacker is present. This work characterizes some fundamental limitations of when such an attack can be detected, and quantifies the performance degradation that an attacker that seeks to be undetected or stealthy can introduce.

Minimum Entropy Active Fault Tolerant Control of the Non-Gaussian Stochastic Distribution System Subjected to Mean Constraint

Stochastic distribution control (SDC) systems are a group of systems where the outputs considered is the measured probability density function (PDF) of the system output whilst subjected to a normal crisp input. The purpose of the active fault tolerant control of such systems is to use the fault estimation information and other measured information to make the output PDF still track the given distribution when the objective PDF is known. However, if the target PDF is unavailable, the PDF tracking operation will be impossible. Minimum entropy control of the system output can be considered as an alternative strategy. The mean represents the center location of the stochastic variable, and it is reasonable that the minimum entropy fault tolerant controller can be designed subjected to mean constraint. In this paper, using the rational square-root B-spline model for the shape control of the system output probability density function (PDF), a nonlinear adaptive observer based fault diagnosis algorithm is proposed to diagnose the fault. Through the controller reconfiguration, the system entropy subjected to mean restriction can still be minimized when fault occurs. An illustrative example is utilized to demonstrate the use of the minimum entropy fault tolerant control algorithms.

Event-triggered sliding mode control of stochastic systems via output feedback

This paper is concerned with event-triggered sliding mode control (SMC) for uncertain stochastic systems subject to limited communication capacity. We consider the stochastic perturbation, exogenous disturbance and the network-induced communication constraints in a whole framework of SMC design. The measurable output is adequately sampled for periodic computation of a designed event trigger. Network-induced delays are characterized in the transmission over a shared network communication from sensor to controller. An event-triggered zero-order-hold (ZOH) is employed to keep receiving and sending the delayed measurement for control updating, and a state observer is designed to estimate the system state and to facilitate the design of sliding surface. Then, based on the measurement and observer’s outputs, a novel SMC law is presented. The stochastic stability for the overall closed-loop system is analyzed, and the exogenous disturbance is attenuated in an H∞ sense. Furthermore, the presented event-triggered SMC methods are successfully applied to multi-loop control case considering shared communication limitation. Simulation results are provided to verify the effectiveness of the proposed design scheme.

Disturbance Observer‐Based Elegant Anti‐Disturbance Control for Stochastic Systems with Multiple Disturbances

AbstractDisturbance observer‐based elegant anti‐disturbance control (DOBEADC) scheme is proposed for a class of stochastic systems with nonlinear dynamics and multiple disturbances. The stochastic disturbance observer based on pole placement is constructed to estimate disturbance which is generated by an exogenous system. Then, composite DOBC and controller is designed to guarantee the composite system is mean‐square stable and its performance satisfies a prescribed level. Finally, simulations on an A4D aircraft model show the effectiveness of the proposed approaches.

Infinite horizon optimal control of forward–backward stochastic system driven by Teugels martingales with Lévy processes

In this paper, we consider the infinite horizon nonlinear optimal control of forward–backward stochastic system governed by Teugels martingales associated with Lévy processes and one dimensional independent Brownian motion. Our aim is to establish the sufficient and necessary conditions for optimality of the above stochastic system under the convexity assumptions. Finally an application is given to illustrate the problem of optimal control of stochastic system.

Interval Type-2 Fuzzy GMC for Nonlinear Stochastic Process of Methane Production in the Anaerobic Digester System

The paper focused on the implementation of hybrid Interval Type-2 (IT2) Fuzzy with Generic Model Control (GMC) for the nonlinear stochastic waste treatment process in the anaerobic digester. Development of the deterministic methane process model has been extended to a set of stochastic nonlinear differential equations. The stochastic effect is introduced by adding white noise with unit covariance to give an interesting profile like physical plant dynamic. The IT2 Fuzzy based on Takagi-Sugeno-Mendel and GMC by Lee and Sullivan have been developed to control the holdup pH inside reactor. The pH value is being manipulated by the flowrate of Sodium hydroxide at optimal methane gas production condition. The process variables that need to be controlled and included into the controller are pH, error and change of error while the consequence fuzzy set output is from the GMC backpropagation law equations. As a result from several studies; servo and regulatory, the controller show significant improvement on the set point tracking and disturbance rejection over typical Fuzzy, Fuzzy-GMC and conventional Proportional-Integral-Derivative (PID) controllers. It shows the controller is suitable for stochastic process and nonlinear control system application.

Centralized Versus Decentralized Optimization of Distributed Stochastic Differential Decision Systems With Different Information Structures-Part I: A General Theory

Decentralized optimization of distributed stochastic dynamical systems with two or more controls of the decision makers (DMs) has been an active area of research for over half a century. Although, such decentralized optimization problems are often formulated utilizing static team and person-by-person (PbP) optimality criteria, the corresponding static team theory results have not been extended to dynamical systems. In this first part of the two-part paper, we derive team and PbP optimality conditions for distributed stochastic differential systems, when the controls of the DMs generate actions based on different information structures. The necessary conditions are given by a Hamiltonian System described by coupled backward and forward stochastic differential equations (SDEs) and a conditional Hamiltonian, conditioned on the information structures available to the controls of the DMs. The sufficient conditions state that PbP optimality implies team optimality, if the Hamiltonian is convex in the state and/or actions spaces of the controls of the DMs. We show existence of relaxed team optimal strategies, when the information structures are not affected by the controls of the DMs. Throughout the paper we discuss similarities to analogous optimality conditions of centralized decision or control of stochastic systems, and we note a connections to mean field stochastic optimal control problems.

Maximum Principle for Markov Regime-Switching Forward–Backward Stochastic Control System with Jumps and Relation to Dynamic Programming

This paper presents a sufficient stochastic maximum principle for a stochastic optimal control problem of Markov regime-switching forward–backward stochastic differential equations with jumps. The relationship between the stochastic maximum principle and the dynamic programming principle in a Markovian case is also established. Finally, applications of the main results to a recursive utility portfolio optimization problem in a financial market are discussed.