Optimal Control Of Stochastic Systems Research Articles

Mixed H 2/H ∞ Control With Dynamic Event-Triggered Mechanism for Partially Unknown Nonlinear Stochastic Systems

This technical note discusses the design process of dynamic event-triggered control (DETC) with the mixed <inline-formula> <tex-math notation="LaTeX">$H_2/H_\infty$</tex-math> </inline-formula> for partially unknown nonlinear stochastic systems. The purpose of this problem is to design a controller to make the closed-loop system achieve the expected <inline-formula> <tex-math notation="LaTeX">$H_2$</tex-math> </inline-formula> performance under the condition that the <inline-formula> <tex-math notation="LaTeX">$H_\infty$</tex-math> </inline-formula> continuous attenuation level is protected. Firstly, a two-player non-zero-sum game for stochastic system is given. Then we prove the optimal control strategy and the worst interference, which constitute the Nash equilibrium solution and can be derived from the corresponding Hamiltonian functions. Furthermore, two neural networks (NNs) are used to realize Nash equilibrium. Under the condition of dynamic event-triggered mechanism (DETM), the control strategy only is updated at the trigger moment. In addition, the stability and weights convergence of the system are proved mathematically. Finally, two numerical example are given to prove it. <i>Note to Practitioners</i>&#x2014;In the practice of control engineering, mixed <inline-formula> <tex-math notation="LaTeX">$H_2/H_\infty$</tex-math> </inline-formula> control can not only evaluate the <inline-formula> <tex-math notation="LaTeX">$H_2$</tex-math> </inline-formula> performance index of the transient behavior of the system, but also measure the <inline-formula> <tex-math notation="LaTeX">$H_\infty$</tex-math> </inline-formula> performance index of the system&#x2019;s robustness to external disturbances and parameter uncertainty. And many useful signals and interference vary randomly. Therefore, the optimal control of stochastic systems and dynamics play an important role in the modern industry. Saving control resources is very important for actual production. Therefore DETC is considered in this paper. On the other hand, in practice, accurate system models are difficult to obtain. In order to tackle this difficulty, by designing a novel scheme via integral reinforcement learning technique, the system relaxes the requirement of drift dynamic and Zeno behavior is avoided.

Maximum principle for forward-backward partially observed optimal control of stochastic systems with delay

In this paper, we consider partially observed optimal control for forward-backward stochastic delay differential equations (FBSDDEs) where the control domain is non-convex and the control variable is allowed to enter into both diffusion and observation terms. We obtain a general stochastic maximum principle of these optimal control problems by using Girsanov?s theorem, the spike variational method and the filtering technique. We also derive the adjoint equations to the problem. Finally, we apply our results to study a linear-quadratic (LQ) optimal control with delay.

The maximum principle for stochastic control problem with Markov chain in progressive structure

In this paper, we obtain the stochastic maximum principle for progressive optimal control of the model driven by Brownian motions and Markov chain. The maximum principle in progressive structure is essentially different from that in the classical case. Our maximum principle has two parts, including continuous part and jump part, which are used to characterize the characteristics of control at continuous time and jump time respectively. This shows that the progressive structure can better describe the optimal control of stochastic system with jumps. Moreover, we give an LQ example to show that the cost functional can indeed get a smaller value in progressive structure. • Necessary and sufficient maximum principle in progressive structure. • Two maximum principle includes continuous part and jump part. • Progressive structure is essentially different from predictable structure. • Progressive structure accurately describes the feature of control for system with jump. • Progressive structure can find better control than predictable structure.

A general maximum principle for progressive optimal stochastic control problems with Markov regime-switching

In this paper, we give a general maximum principle for optimal controls of stochastic systems driven by Markov chains. The control is allowed to enter both diffusion and jump terms and the control domain is not necessarily convex. We apply a new spike variation and the stochastic integral of progressive processes to obtain the main result.

A diffusion wavelets-based multiscale framework for inverse optimal control of stochastic systems

This work presents a multiscale framework to solve a class of inverse optimal control (IOC) problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. An IOC problem constructed on a ‘abstract state’ is solved, which is much more tractable than using the original bases set; moreover, the solution can be obtained recursively in the ‘coarse to fine’ direction by utilizing the hierarchical structure of basis functions. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Illustrative numerical experiments on a robot path control in a complex environment and on a quadrotor ball-catching task are presented to verify the proposed method.

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Optimal control of stochastic system with Fractional Brownian Motion.

In this paper, we introduce a class of stochastic harvesting population system with Fractional Brownian Motion (FBM), which is still unclear when the stochastic noise has the character of memorability. Stochastic optimal control problems with FBM can not be studied using classical methods, because FBM is neither a Markov pocess nor a semi-martingale. When the external environment impact on the system of FBM, the necessary and sufficient conditions for the optimization are offered through the stochastic maximum principle, Hamilton function and ItÔ formula in our work. To illustrate our study, we provide an example to demonstrate the obtained theoretical results, which is the expansion of certainty population system.

Singular perturbations and optimal control of stochastic systems in infinite dimension: HJB equations and viscosity solutions

We study a stochastic optimal control problem for a two scale system driven by an infinite dimensional stochastic differential equation which consists of “slow” and “fast” components. We use the theory of viscosity solutions in Hilbert spaces to show that as the speed of the fast component goes to infinity, the value function of the optimal control problem converges to the viscosity solution of a reduced effective equation. We consider a rather general case where the evolution is given by an abstract semilinear stochastic differential equation with nonlinear dependence on the controls. The results of this paper generalize to the infinite dimensional case the finite dimensional results of Alvarez and Bardi [SIAM J. Control Optim.40(2001/02) 1159–1188] and complement the results in Hilbert spaces obtained recently in Guatteri and Tessitore [To appear in:Appl. Math. Optim.(2019)https://doi.org/10.1007/s00245-019-09577-y].

The Maximum Principle for Progressive Optimal Stochastic Control Problems with Random Jumps

In this paper, we obtain the maximum principle for optimal controls of stochastic systems with jumps by introducing a new method of variation. The control is allowed to enter both diffusion and jump terms and the control domain need not be convex.

Data-driven adaptive optimal control for stochastic systems with unmeasurable state

This paper proposes a computational data-driven adaptive optimal control strategy for a class of linear stochastic systems with unmeasurable state. First, a data-driven optimal observer is designed to obtain the optimal state estimation policy. On this basis, an off-policy data-driven ADP algorithm is further proposed, yielding the stochastic optimal control in the absence of system model. An application example of the learning mechanism of central nervous system in arm movement control is given to illustrate the effectiveness and practicality of the strategy.

Inverse optimal control of stochastic systems driven by Lévy processes

This paper formulates and solves new problems of inverse optimal stabilization and inverse optimal gain assignment for stochastic nonlinear systems driven by Lévy processes. First, a theorem is developed to design inverse optimal stabilizers based on inverse pre-optimal stabilization controllers for stochastic systems with known noise characteristics, where it does not require to solve a Hamilton–Jacobi–Bellman equation. Second, another theorem is developed to design inverse optimal gain assignment controllers for stochastic systems with unknown noise characteristics, where there is no need to solve a Hamilton–Jacobi–Isaacs equation. Third, the results are applied to design inverse optimal stabilizers for Euler–Lagrange systems.

Infinite horizon optimal control of forward–backward stochastic system driven by Teugels martingales with Lévy processes

In this paper, we consider the infinite horizon nonlinear optimal control of forward–backward stochastic system governed by Teugels martingales associated with Lévy processes and one dimensional independent Brownian motion. Our aim is to establish the sufficient and necessary conditions for optimality of the above stochastic system under the convexity assumptions. Finally an application is given to illustrate the problem of optimal control of stochastic system.

Maximum Principle for Forward-Backward Control System Driven by Itô-Lévy Processes under Initial-Terminal Constraints

This paper investigates a stochastic optimal control problem where the control system is driven by Itô-Lévy process. We prove the necessary condition about existence of optimal control for stochastic system by using traditional variational technique under the assumption that control domain is convex. We require that forward-backward stochastic differential equations (FBSDE) be fully coupled, and the control variable is allowed to enter both diffusion and jump coefficient. Moreover, we also require that the initial-terminal state be constrained. Finally, as an application to finance, we show an example of recursive consumption utility optimization problem to illustrate the practicability of our result.

SI “Deterministic and Stochastic Variational Principles and Applications”. December 2015

Variational methods are important tools for deriving optimality conditions and corresponding algorithms for solving optimization problems. Ekeland’s variational principle is a deep assertion about the existence of an exact solution of a perturbed optimization problem in a neighborhood of an approximate solution of the original problem. The aim of this special issue was to discuss new variational approaches from the theoretical as well as computational point of view. Furthermore, we are interested in applications of the variational methods for deterministic and stochastic models, especially in approximation theory and optimal control. In the special issue the importance of variational methods in optimization and stochastics is described.We present newdevelopments on the field of deterministic and stochastic variational methods including applications in mathematics and economics. Most of the papers in the special issue are dealing with optimal control and state estimates problems in the deterministic as well as stochastic case and maximum principles (Abdelmadjid Abba: On Mean-field Partial Information Maximum Principle of Optimal Control for Stochastic Systems with Levy Processes, Vo Anh: Least-Squares Estimation of Multifractional Random Fields in a Hilbert-Valued Context, Brigitte Breckner: Multiple solutions of Dirichlet problems on the Sierpinski gasket, Ioana Ciotir: A variational approach to Neumann stochastic semi-linear equations modelling the thermostatic control, CsabaVarga, Hannelore Lisei: Amultiplicity result for a class of elliptic problems on a compact Riemannian manifold, Tina Engler: On Investment Consumption Modeling with Jump Process Extensions for Productive Sectors, Diana

On the Separation Theorem of Stochastic Systems in the Case Of Continuous Observation Channels with Memory

The article proves the separation theorem for optimal control of stochastic systems in the case when an observed continuous-time process possess memory of arbitrary ration relating to a state vector.

On the Separation Theorem of Stochastic Systems in the Case Of Continuous- Discrete Observation Channels with Memory

The article proves the separation theorem for optimal control of stochastic systems in the case when an observed continuous-discrete-time process possess memory of arbitrary ration relating to a state vector.

On Mean-Field Partial Information Maximum Principle of Optimal Control for Stochastic Systems with Lévy Processes

In this paper, we study the mean-field-type partial information stochastic optimal control problem, where the system is governed by a controlled stochastic differential equation, driven by the Teugels martingales associated with some Levy processes and an independent Brownian motion. We derive necessary and sufficient conditions of the optimal control for these mean-field models in the form of a maximum principle. The control domain is assumed to be convex. As an application, the partial information linear quadratic control problem of the mean-field type is discussed.

Temporal Difference-Based Policy Iteration for Optimal Control of Stochastic Systems

In this paper, a unified policy iteration approach is presented for the optimal control problem of stochastic system with discounted average cost and continuous state space. The approach consists o...

Two levels of Bayesian model averaging for optimal control of stochastic systems

Bayesian model averaging provides the best possible estimate of a model, given the data. This article uses that approach twice: once to get a distribution of plausible models of the world, and again to find a distribution of plausible control functions. The resulting ensemble gives control instructions different from simply taking the single best-fitting model and using it to find a single lowest-error control function for that single model. The only drawback is, of course, the need for more computer time: this article demonstrates that the required computer time is feasible. The test problem here is from flood control and risk management.