Control Of Stochastic Differential Equations Research Articles

Optimal control of stochastic differential equations with random impulses and the Hamilton–Jacobi–Bellman equation

AbstractIn this article, we study the optimal control of stochastic differential equations with random impulses. We optimize the performance index and add the influence of random impulses to the performance index with a random compensation function. Using the idea of stochastic analysis and dynamic programming principle, a new Hamilton–Jacobi–Bellman (HJB) equation is obtained, and the existence and uniqueness of its viscosity solution are proved.

Model reduction for stochastic systems with nonlinear drift

In this paper, we study dimension reduction techniques for large-scale controlled stochastic differential equations (SDEs). The drift of the considered SDEs contains a polynomial term satisfying a one-sided growth condition. Such nonlinearities in high dimensional settings occur, e.g., when stochastic reaction diffusion equations are discretized in space. We provide a brief discussion around existence, uniqueness and stability of solutions. (Almost) stability then is the basis for new concepts of Gramians that we introduce and study in this work. With the help of these Gramians, dominant subspace is identified leading to a balancing related highly accurate reduced order SDE. We provide an algebraic error criterion and an error analysis of the propose model reduction schemes. The paper is concluded by applying our method to spatially discretized reaction diffusion equations.

On pointwise second‐order maximum principle for optimal stochastic controls of general mean‐field type

AbstractIn this paper, we establish a second‐order stochastic maximum principle for optimal stochastic control of stochastic differential equations of general mean‐field type. The coefficients of the system are nonlinear and depend on the state process as well as of its probability law. The control variable is allowed to enter into both drift and diffusion terms. We establish a set of second‐order necessary conditions for the optimal control in integral form. The control domain is assumed to be convex. The proof of our main result is based on the first‐ and second‐order derivatives with respect to the probability law and by using a convex perturbation with some appropriate estimates.

MAXIMUM PRINCIPLE FOR SINGULAR CONTROL PROBLEMS OF SYSTEMS DRIVEN BY MARTINGALE MEASURES

We provide necessary optimality conditions for singular controlled stochastic differential equations driven by an orthogonal continuous martingale measure. The control is allowed to enter both the drift and diffusion coefficient and has two components, the first being relaxed and the second singular, the domain of the first control does not need to be convex, and for the relaxing method, we show by a counter-example that replacing the drift and diffusion coefficients by their relaxed counterparts does not define a true relaxed control problem. The maximum principle for these systems is established by means of spike variation techniques on the relaxed part of the control and a convex perturbation on the singular one. Our result is a generalization of Peng's maximum principle to singular control problems.

Optimal control for stochastic differential equations and related Kolmogorov equations

<p style='text-indent:20px;'>This paper concerns a stochastic optimal control problem with feedback Markov inputs. The problem is reduced to a deterministic optimal control problem for a Kolmogorov equation where the control for the deterministic problem is of open-loop type. The existence of an optimal control is proved for the deterministic control problem in a particular case. A maximum principle and some first order necessary optimality conditions are derived. Some examples and comments are discussed.</p>

Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

We present a probabilistic formulation of risk aware optimal control problems for stochastic differential equations. Risk awareness is in our framework captured by objective functions in which the risk neutral expectation is replaced by a risk function, a nonlinear functional of random variables that accounts for the controller's risk preferences. We state and prove a risk aware minimum principle that gives necessary and sufficient conditions for optimality of generalized control processes taking values on probability measures defined on a given action space. We show that going from the risk neutral to the risk aware case, the minimum principle is modified by the introduction of one additional real-valued stochastic process that acts as a risk adjustment factor. This adjustment process is explicitly given as the expectation, conditional on the filtration at the given time, of an appropriately defined functional derivative of the risk function evaluated at the random total cost. The control model we employ differs from standard relaxed controls, and we elaborate on the differences, and benefits and drawbacks, of the control types; we further give conditions under which the generalized control can be realized using a strict control process. We present an application of the results for a portfolio allocation problem and show that the risk awareness of the objective function gives rise to a risk premium term that is characterized by the risk adjustment process described above. This suggests uses of our results in e.g. pricing of risk modeled by generic risk functions in financial applications.

Stabilisation in distribution by delay feedback control for stochastic differential equations with Markovian switching and Lévy noise

Stabilisation in distribution by delay feedback control for stochastic differential equations with Markovian switching and Lévy noise

Assessing non-convex value functions for the optimal control of stochastic differential equations

Solving the optimal control of stochastic differential equations (SDEs) using the dynamic programming method requires writing the problem in terms of the so-called value function. This paper presents conditions to assure that the value function is convex away from the origin, a concept that allows the value function be non-convex in a region close to the origin. In contrast, for regions away from the origin, the value function remains convex under some mild conditions. Stochastic ordering is used to prove this result. A numerical example illustrates the potential benefits of our approach.

Optimal Control of Stochastic Differential Equations via Fokker–Planck Equations

Optimal Control of Stochastic Differential Equations via Fokker–Planck Equations

Exact controllability of stochastic differential equations with memory

In this work, we mainly study the exact controllability of stochastic differential equations with memory. For time invariant systems, by applying forward backward stochastic differential equations, some rank criteria ensuring systems’ exact controllability are provided. For time variant systems, equivalent conditions guaranteeing systems’ exact controllability are given. For deterministic systems, as a special case of stochastic ones, more easily checkable rank criteria (for time invariant systems) and alternative conditions (for time variant systems) are proposed.

On the partial controllability of SDEs and the exact controllability of FBSDES

A notion of partial controllability (also can be called directional controllability or output controllability) is proposed for linear controlled (forward) stochastic differential equations (SDEs), which characterizes the ability of the state to reach some given random hyperplane. It generalizes the classical notion of exact controllability. For time-invariant system, checkable rank conditions ensuring SDEs’ partial controllability are provided. With some special setting, the partial controllability for SDEs is proved to be equivalent to the exact controllability for linear controlled forward-backward stochastic differential equations (FBSDEs). Moreover, we obtain some equivalent conclusions to partial controllability for SDEs or exact controllability for FBSDEs, including the validity of observability inequalities for the adjoint equations, the solvability of some optimal control problems, the solvability of norm optimal control problems, and the non-singularity of a random version of Gramian matrix.

Exact controllability of stochastic differential equations with multiplicative noise

One proves that the n-D stochastic controlled equation dX(t)+A(t)X(t)dt=σ(X(t))dW(t)+B(t)u(t)dt, where σ∈Lip(Rn,ℒ(Rd,Rn)), A(t)∈ℒ(Rn) and B(t)∈ℒ(Rn,Rn) is invertible, is exactly controllable with high probability in each y∈Rn such that σ(y)=0 on each finite interval (0,T). An application to approximate controllability of the stochastic heat equation is given. The case where B∈ℒ(Rm,Rn), 1≤m<n and the pair (A,B) satisfies the Kalman rank condition is also studied.

The optimal control of stochastic differential equations arising in biology, economy and finance

The optimal control of stochastic differential equations arising in biology, economy and finance

On Relaxed Stochastic Optimal Control for Stochastic Differential Equations Driven by G-Brownian Motion

In the G-framework, we establish existence of an optimal stochastic relaxed control for stochastic differential equations driven by a G-Brownian motion.

Infinite horizon mean-field type relaxed optimal control with Lévy processes

This paper investigates the problem of infinite horizon optimal control of stochastic differential equation with Teugels martingales associated with Lévy processes under non convex control domain. Proposed model of mean-field dynamical system is considered with the expectation values of state processes which are included explicitly in drift, diffusion, jump kernel and cost functional terms. In general, the assumption of non convex control domain does not guarantee the existence of optimal control. Therefore, the concerned system is transformed into relaxed control model, where set of all relaxed controls forms a convex set and exhibits the existence of optimal control. Moreover, stochastic maximum principle and necessary condition for optimality are established under convex perturbation technique for the proposed relaxed model. Finally, an application of the theoretical study is demonstrated by an example of portfolio optimization problem in financial market.

Minimal truncation error constants for Runge–Kutta method for stochastic optimal control problems

In this work, we obtain strong order-1 conditions with minimal truncation error constants of Runge–Kutta method for the optimal control of stochastic differential equations (SDEs). We match Stratonovich–Taylor expansion of the exact solution with Stratonovich–Taylor expansion of our approximation method that is defined by the Runge–Kutta scheme, term by term, in order to get the strong order-1 conditions. By a conclusion and an outlook to future research, the paper ends.

Self-triggered sampling for second-moment stability of state-feedback controlled SDE systems

Event-triggered and self-triggered control, whereby the times for controller updates are computed from sampled data, have recently been shown to reduce the computational load or increase task periods for real-time embedded control systems. In this work, we propose a self-triggered scheme for nonlinear controlled stochastic differential equations with additive noise terms. We find that the family of trajectories generated by these processes demands a departure from the standard deterministic approach to event- and self-triggering, and, for that reason, we use the statistics of the sampled-data system to derive a self-triggering update condition that guarantees second-moment stability. We show that the length of the times between controller updates as computed from the proposed scheme is strictly positive and provide related examples.

On Near-Optimal Mean-Field Stochastic Singular Controls: Necessary and Sufficient Conditions for Near-Optimality

Near-optimization is as sensible and important as optimization for both theory and applications. This paper deals with necessary and sufficient conditions for near-optimal singular stochastic controls for nonlinear controlled stochastic differential equations of mean-field type, which is also called McKean–Vlasov-type equations. The proof of our main result is based on Ekeland’s variational principle and some estimates of the state and adjoint processes. It is shown that optimal singular control may fail to exist even in simple cases, while near-optimal singular controls always exist. This justifies the use of near-optimal stochastic controls, which exist under minimal hypotheses and are sufficient in most practical cases. Moreover, since there are many near-optimal singular controls, it is possible to select among them appropriate ones that are easier for analysis and implementation. Under an additional assumptions, we prove that the near-maximum condition on the Hamiltonian function is a sufficient condition for near-optimality. This paper extends the results obtained in (Zhou, X.Y.: SIAM J. Control Optim. 36(3), 929–947, 1998) to a class of singular stochastic control problems involving stochastic differential equations of mean-field type. An example is given to illustrate the theoretical results.

Representation of Itô integrals by Lebesgue/Bochner integrals

In [Yong 2004], it was proved that as long as the integrand has certain properties, the corresponding It\^o integral can be written as a (parameterized) Lebesgue integral (or a Bochner integral). In this paper, we show that such a question can be answered in a more positive and refined way. To do this, we need to characterize the dual of the Banach space of some vector-valued stochastic processes having different integrability with respect to the time variable and the probability measure. The later can be regarded as a variant of the classical Riesz Representation Theorem, and therefore it will be useful in studying other problems. Some remarkable consequences are presented as well, including a reasonable definition of exact controllability for stochastic differential equations and a condition which implies a Black–Scholes market to be complete.

A General Stochastic Maximum Principle for SDEs of Mean-field Type

We study the optimal control for stochastic differential equations (SDEs) of mean-field type, in which the coefficients depend on the state of the solution process as well as of its expected value. Moreover, the cost functional is also of mean-field type. This makes the control problem time inconsistent in the sense that the Bellman optimality principle does not hold. For a general action space a Peng’s-type stochastic maximum principle (Peng, S.: SIAM J. Control Optim. 2(4), 966–979, 1990) is derived, specifying the necessary conditions for optimality. This maximum principle differs from the classical one in the sense that here the first order adjoint equation turns out to be a linear mean-field backward SDE, while the second order adjoint equation remains the same as in Peng’s stochastic maximum principle.