Control Problem For Stochastic Systems Research Articles

The mean‐field linear quadratic optimal control problem for stochastic systems controlled by impulses

AbstractIn the present note, we state and solve the linear quadratic (LQ) control problem for a class of McKean–Vlasov stochastic differential equations for which the control actions are of impulsive nature. After reformulating the original optimization problem using an orthogonal decomposition of the state variables, we introduce an adequately defined system of two coupled matrix Lyapunov‐type differential equations with jumps. Such equations are central for the definition of the optimal feedback gains corresponding to the closed‐loop representation of the optimal LQ control.

An improved iterative algorithm for solving optimal tracking control problems of stochastic systems

This paper proposes an improved iterative algorithm based on a Newton iterative algorithm, which is used to research the stochastic linear quadratic optimal tracking (SLQT) control for stochastic continuous-time systems. Firstly, a Newton iterative algorithm for solving stochastic algebraic Riccati equations (SAREs) is derived based on the Frechet derivative. Secondly, an improved iterative algorithm in an incremental form for indirectly solving SAREs is presented by combining the numerical iterative method with the incremental Newton iterative (INI) algorithm and introducing a tuning parameter. Under a suitable initial condition, the convergence of the improved iterative algorithm is talked over. Finally, numerical simulation experiments are carried out to compare our proposed improved iterative algorithm with some known works. The results show that the improved iterative algorithm has a faster convergence rate.

Linear quadratic optimal regulation for multiplicative noise systems with special terminal penalty

This paper focuses on the linear quadratic optimal control problem for stochastic systems with a special terminal penalty. The specialty lies in that the terminal penalty is a quadratic term over the conditional expectations of the terminal state. The essential difficulty lies in an additional measurability constraint on the input induced by the conditional expectation term, which is shared by the control problems for the stochastic system with delays, rational expectations problems, control problems involving conditional expectation terms, asymmetric information control and so on. A new idea, dynamic programming combined with orthogonal decomposition, is proposed to deal with this problem. The orthogonal decomposition eliminates the additional measurability constraint on the input. The solvability condition and the optimal controller are obtained by developing generalised Riccati equations. Numerical experiment is provided to support the achieved results.

Adaptive event‐triggered asynchronous fault‐tolerant control for stochastic systems with multiple‐types of failures

SummaryThis article studies a kind of asynchronous fault‐tolerant control problem for stochastic systems (Markov jump systems, MJSs) with multiple‐types of actuator failures. Both the loss of effectiveness and stuck fault of actuator are taken into consideration in a unified actuator failures model. The asynchronization phenomenon between the controller and the original system is described by a hidden Markov model. By coupling the unknown stuck fault of actuator and the external disturbances together, the stuck fault and disturbance can be estimated simultaneously. After this, a fault‐tolerant controller based on the estimated values is designed further to automatically compensate for the faults while preserving the uniformly ultimate boundedness of the solutions. Moreover, a mode‐dependent adaptive event‐triggered mechanism is introduced to limit the number of packets sent on the network. Finally, a numerical example is employed to illustrate the effectiveness of the proposed design scheme.

Optimal Control and Stabilization for Itô Systems with Input Delay

The paper considers the linear quadratic regulation (LQR) and stabilization problems for Ito stochastic systems with two input channels of which one has input delay. The underlying problem actually falls into the field of asymmetric information control because of the nonidentical measurability induced by the input delay. In contrast with single-channel single-delay problems, the challenge of the problems under study lies in the interaction between the two channels which are measurable with respect to different filtrations. The key techniques conquering such difficulty are the stochastic maximum principle and the orthogonal decomposition and reorganization technique proposed in a companion paper. The authors provide a way to solve the delayed forward backward stochastic differential equation (D-FBSDE) arising from the maximum principle. The necessary and sufficient solvability condition and the optimal controller for the LQR problem are given in terms of a new Riccati differential equation established herein. Further, the necessary and sufficient stabilization condition in the mean square sense is provided and the optimal controller is given. The idea proposed in the paper can be extended to solve related control problems for stochastic systems with multiple input channels and multiple delays.

First and second order necessary optimality conditions for stochastic distributed systems

An optimal control problem of stochastic system, which that dynamics is described by hyperbolic type stochastic partial differential equations with a multipoint type quality criterion, is considered. A stochastic analog of the Euler equation is obtained and singular controls in the classical sense are investigated.

Optimal Control for Stochastic Systems With Multiple Controllers of Different Information Structures

In this article, we investigate the optimal linear quadratic control problem for stochastic systems with multiple controllers, where each controller has its own information structure, which differs from each other. More specifically, we consider the optimal control problem for systems with multiple controllers of different delayed state information. First, the necessary and sufficient solvability conditions are given in terms of forward and backward difference equations (FBSDEs). Further, an innovation method is proposed to decouple the FBSDEs, and the optimal control strategies are derived based on a given nonsymmetric Riccati equation. Finally, a numerical example is provided to show the effectiveness of the main results. It is stressed that the proposed methods and results can be seen as an important addition to the optimal control theory with asymmetric-information-structure controllers.

A Simple Proof of Indefinite Linear-Quadratic Stochastic Optimal Control With Random Coefficients

We consider the indefinite linear-quadratic optimal control problem for stochastic systems with random coefficients, where the corresponding cost parameters need not be definite matrices. Although the solution to this problem was obtained previously via the stochastic maximum principle or by solving the stochastic Hamilton–Jacobi–Bellman (HJB) equation, our approach is simple and direct. Specifically, we develop a direct approach, also known as the completion of squares method, to characterize the explicit optimal solution and the optimal cost. The corresponding optimal solution is linear characterized in terms of the stochastic Riccati differential equation (SRDE) and the linear backward stochastic differential equation (BSDE). In our approach, the completion of squares method, together with the SRDE and the BSDE, allows us to construct an equivalent cost functional that is quadratic in the control $u$ , provided that an additional positive definiteness condition in terms of the SRDE holds. Then, the optimal control and the associated cost can be obtained by eliminating the quadratic term of $u$ in the equivalent cost functional. The additional positive definiteness condition is induced due to the indefiniteness of the cost parameters and the dependence of the control on the diffusion term. We verify the optimal solution by solving the stochastic HJB equation.

$H_\infty$ Control for Stochastic Systems with Disturbance Preview

The paper considers the $H_\infty$ control problem for stochastic systems with disturbance preview, which is very challenging since it involves the preview problem and multiplicative noise simultan...

Stochastic Point-to-Point Iterative Learning Tracking Without Prior Information on System Matrices

This paper contributes to a point-to-point iterative learning control problem for stochastic systems without prior information on system matrices. The stochastic approximation technique with gradient estimation by random difference is introduced to design the update law for input. It is strictly proved that the input sequence would converge almost surely to the optimal one, which minimizes the averaged tracking performance index. An illustrative simulation shows the effectiveness of the proposed algorithm.

A Donsker Delta Functional Approach to Optimal Insider Control and Applications to Finance

We study optimal insider control problems, i.e., optimal control problems of stochastic systems where the controller at any time t, in addition to knowledge about the history of the system up to this time, also has additional information related to a future value of the system. Since this puts the associated controlled systems outside the context of semimartingales, we apply anticipative white noise analysis, including forward integration and Hida–Malliavin calculus to study the problem. Combining this with Donsker delta functionals, we transform the insider control problem into a classical (but parametrised) adapted control system, albeit with a non-classical performance functional. We establish a sufficient and a necessary maximum principle for such systems. Then we apply the results to obtain explicit solutions for some optimal insider portfolio problems in financial markets described by Ito–Levy processes. Finally, in the Appendix, we give a brief survey of the concepts and results we need from the theory of white noise, forward integrals and Hida–Malliavin calculus.

Some remarks on infinite horizon stochastic [formula omitted] control with [formula omitted] noise and Markov jumps

This paper investigates the infinite horizon H2/H∞ control problem for stochastic systems with (x,u,v)-dependent noise and Markov jumps. Under the condition of exact detectability, a necessary and sufficient condition for the infinite horizon stochastic H2/H∞ control is derived in terms of four sets of coupled matrix-valued equations. The relationship between the infinite horizon H2/H∞ control and the Nash equilibrium point is discussed when disturbance does or does not enter into the diffusion term. Moreover, a heuristic iteration algorithm is proposed to solve coupled stochastic Riccati equations for the first time.

A maximum principle approach to stochastic H2/H∞ control with random jumps

A necessary maximum principle is given for nonzero-sum stochastic differential games with random jumps. The result is applied to solve the H2/H∞ control problem of stochastic systems with random jumps. A necessary and sufficient condition for the existence of a unique solution to the H2/H∞ control problem is derived. The resulting solution is given by the solution of an uncontrolled forward backward stochastic differential equation with random jumps.

Maximum principle for optimal control of anticipated forward–backward stochastic differential delayed systems with regime switching

SummaryThis paper is concerned with a Pontryagin maximum principle for optimal control problem of stochastic system, which is described by an anticipated forward–backward stochastic differential delayed equation and modulated by a continuous‐time finite‐state Markov chain. We establish a necessary maximum principle and sufficient verification theorem for the optimal control by virtue of the duality method and convex analysis. To illustrate the theoretical results, we apply them to a recursive utility investment‐consumption problem, and the optimal consumption rate is derived explicitly. Copyright © 2015 John Wiley & Sons, Ltd.

Algorithms to Solve StochasticH2/H∞Control with State-Dependent Noise

This paper is concerned with the algorithms which solveH2/H∞control problems of stochastic systems with state-dependent noise. Firstly, the algorithms for the finite and infinite horizonH2/H∞control of discrete-time stochastic systems are reviewed and studied. Secondly, two algorithms are proposed for the finite and infinite horizonH2/H∞control of continuous-time stochastic systems, respectively. Finally, several numerical examples are presented to show the effectiveness of the algorithms.

Robust Control, Optimization, and Applications to Markovian Jumping Systems

Robust Control, Optimization, and Applications to Markovian Jumping Systems

NonfragileH∞Control for Stochastic Systems with Markovian Jumping Parameters and Random Packet Losses

This paper is concerned with the nonfragileH∞control problem for stochastic systems with Markovian jumping parameters and random packet losses. The communication between the physical plant and controller is assumed to be imperfect, where random packet losses phenomenon occurs in a random way. Such a phenomenon is represented by a stochastic variable satisfying the Bernoulli distribution. The purpose is to design a nonfragile controller such that the resulting closed-loop system is stochastically mean square stable with a guaranteedH∞performance levelγ. By using the Lyapunov function approach, some sufficient conditions for the solvability of the previous problem are proposed in terms of linear matrix inequalities (LMIs), and a corresponding explicit parametrization of the desired controller is given. Finally, an example illustrating the effectiveness of the proposed approach is presented.

Temporal Difference-Based Policy Iteration for Optimal Control of Stochastic Systems

In this paper, a unified policy iteration approach is presented for the optimal control problem of stochastic system with discounted average cost and continuous state space. The approach consists o...

Statistical Information Based Single Neuron Adaptive Control for Non-Gaussian Stochastic Systems

Based on information theory, the single neuron adaptive control problem for stochastic systems with non-Gaussian noises is investigated in this paper. Here, the statistic information of the output within a receding window rather than the output value is used for the tracking problem. Firstly, the single neuron controller structure, which has the ability of self-learning and self-adaptation, is established. Then, an improved performance criterion is given to train the weights of the single neuron. Furthermore, the mean-square convergent condition of the proposed control algorithm is formulated. Finally, comparative simulation results are presented to show that the proposed algorithm is superior to the PID controller. The contributions of this work are twofold: (1) the optimal control algorithm is formulated in the data-driven framework, which needn’t the precise system model that is usually difficult to obtain; (2) the control problem of non-Gaussian systems can be effectively dealt with by the simple single neuron controller under improved minimum entropy criterion.

A Class of Stochastic Control Problem Governed by a Poisson Process

In this paper, we examine an optimal impulse control problem of stochastic system, whose state follows a Brownian motion. Here we want to maximum the objective function. The main feature of our model is that the controlled state process includes an impulse control governed by a Poisson process. In other words, the set of possible intervention times are discrete, random and determined by the signal process. Here we not only present a theorem giving a sufficient condition on the existence of an optimal control and its corresponding objective function, but also provide an explicit solution obtained under some simplified conditions.