Risk-sensitive Stochastic Control Problem Research Articles

Partially Observed Risk-Sensitive Stochastic Control Problems with Non-Convexity Restriction

Partially Observed Risk-Sensitive Stochastic Control Problems with Non-Convexity Restriction

On the singular risk-sensitive stochastic maximum principle

We consider a risk-sensitive stochastic control problem of a nonlinear system in which the variable control has two components, the first being absolutely continuous and the second is singular. The author generalizes the result obtained in risk neutral into risk-sensitive performance cost. The auxiliary process has been introduced to solve the problem of risk sensitive. The relationship between the expected exponential utility and the quadratic backward stochastic differential equation has been given and proved. Actually, I have proved in details two results, the first is necessary optimality condition, while the second is sufficient optimality condition for risk sensitive performance cost. Two examples to carried out to illustrate our main results of risk-sensitive control problem under nonlinear stochastic dynamics with exponential quadratic cost function with numerical investigation, while the second will be in the exponential utility optimization problem.

Maximum Principle for Risk-Sensitive Stochastic Optimal Control Problem and Applications to Finance

This article is concerned with a risk-sensitive stochastic optimal control problem motivated by a kind of optimal portfolio choice problem in the financial market. The maximum principle for this kind of problem is obtained, which is similar in form to its risk-neutral counterpart. But the adjoint equations and maximum condition heavily depend on the risk-sensitive parameter. This result is used to solve a kind of optimal portfolio choice problem and the optimal portfolio choice strategy is obtained. Computational results and figures explicitly illustrate the optimal solution and the sensitivity to the volatility rate parameter.

Downside risk minimization via a large deviations approach

We consider minimizing the probability of falling below a target growth rate of the wealth process up to a time horizon T in an incomplete market model, and then study the asymptotic behavior of minimizing probability as T → ∞. This problem can be closely related to an ergodic risk-sensitive stochastic control problem in the risk-averse case. Indeed, in our main theorem, we relate the former problem concerning the asymptotics for risk minimization to the latter as its dual. As a result, we obtain an expression of the limit value of the probability as the Legendre transform of the value of the control problem, which is characterized as the solution to an H-J-B equation of ergodic type, in the case of a Markovian incomplete market model.

Opitmal Platform Strategies in the Smartphone Market

In a smartphone market, smartphone makers encourage smartphone application providers (AP) to create more popular smartphone applications through making a revenue-sharing contract with AP and providing application-purchasing support to end users. In this paper, we study revenue-sharing and application-purchasing support problem between a risk-averse smartphone maker and a smartphone application provider. The problem is formulated as the smartphone makers's risk-sensitive stochastic control problem. The sufficient conditions for the existence of the optimal revenue-sharing strategy, the optimal application-purchasing support strategy and the incentive compatible effort recommended to AP are obtained. The effects of the smartphone makers's risk-sensitivity on the optimal strategies are also discussed. A numerical example is solved to show the computation aspects of the problem.

Asymptotics of the probability of minimizing ‘down-side’ risk under partial information

We consider minimizing the probability of falling below a target growth rate of the wealth process up to a time horizon T in an incomplete market model under partial information and then study the asymptotic behavior of the minimizing probability as T → ∞. This problem is closely related to an ergodic risk-sensitive stochastic control problem under partial information in the risk-averse case. Indeed, in our main theorem we relate the former problem to the latter as its dual. As a result we obtain an explicit expression for the limit value of the former problem in the case of linear Gaussian models.

Explicit Solution to a Certain Non-ELQG Risk-sensitive Stochastic Control Problem

A risk-sensitive stochastic control problem with finite/infinite horizon is studied with a 1-dimensional controlled process defined by a linear SDE with a linear control-term in the drift. In the criterion function, a non-linear/quadratic term is introduced by using the solution to a Riccati differential equation, and hence, the problem is not ELQG (Exponential Linear Quadratic Gaussian) in general. For the problem, optimal value and control are calculated in explicit forms and the set of admissible risk-sensitive parameters is given in a concrete form. As applications, two types of large deviations control problems, i.e., maximizing an upside large deviations probability and minimizing a downside large deviations probability, are mentioned.

Max-Plus Stochastic Control and Risk-Sensitivity

In the Maslov idempotent probability calculus, expectations of random variables are defined so as to be linear with respect to max-plus addition and scalar multiplication. This paper considers control problems in which the objective is to minimize the max-plus expectation of some max-plus additive running cost. Such problems arise naturally as limits of some types of risk sensitive stochastic control problems. The value function is a viscosity solution to a quasivariational inequality (QVI) of dynamic programming. Equivalence of this QVI to a nonlinear parabolic PDE with discontinuous Hamiltonian is used to prove a comparison theorem for viscosity sub- and super-solutions. An example from mathematical finance is given, and an application in nonlinear H-infinity control is sketched.

Asymptotics of the probability minimizing a “down-side” risk

We consider a long-term optimal investment problem where an investor tries to minimize the probability of falling below a target growth rate. From a mathematical viewpoint, this is a large deviation control problem. This problem will be shown to relate to a risk-sensitive stochastic control problem for a sufficiently large time horizon. Indeed, in our theorem we state a duality in the relation between the above two problems. Furthermore, under a multidimensional linear Gaussian model we obtain explicit solutions for the primal problem.

A risk-sensitive stochastic control approach to an optimal investment problem with partial information

We consider an infinite time horizon optimal investment problem where an investor tries to maximize the probability of beating a given index. From a mathematical viewpoint, this is a large deviation probability control problem. As shown by Pham (in Syst. Control Lett. 49: 295–309, 2003; Financ. Stoch. 7: 169–195, 2003), its dual problem can be regarded as an ergodic risk-sensitive stochastic control problem. We discuss the partial information counterpart of Pham (in Syst. Control Lett. 49: 295–309, 2003; Financ. Stoch. 7: 169–195, 2003). The optimal strategy and the value function for the dual problem are constructed by using the solution of an algebraic Riccati equation. This equation is the limit equation of a time inhomogeneous Riccati equation derived from a finite time horizon problem with partial information. As a result, we obtain explicit representations of the value function and the optimal strategy for the problem. Furthermore we compare the optimal strategies and the value functions in both full and partial information cases.

Partially observed non-linear risk-sensitive optimal stopping control for non-linear discrete-time systems

In this paper we introduce and solve the partially observed optimal stopping non-linear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle established that leads to a forward dynamic programming equation and a backward dynamic programming inequality equation (both infinite dimensional). A verification theorem is given that establishes the optimal control and optimal stopping time. The risk-neutral optimal stopping stochastic control problem is also discussed.

Risk-Sinsitive Dynamic Portfolio Optimization with Partial Information on Infinite Time Horizon

We consider an optimal investment problem for a factor model treated by Bielecki and Pliska (Appl. Math. Optim. 39 337–360) as a risk-sensitive stochastic control problem, where the mean returns of individual securities are explicitly affected by economic factors defined as Gaussian processes. We relax the measurability condition assumed as Bielecki and Pliska for the investment strategies to select. Our investment strategies are supposed to be chosen without using information of factor processes but by using only past information of security prices. Then our problem is formulated as a kind of stochastic control problem with partial information. The case on a finite time horizon is discussed by Nagai (Stochastics in Finite and Infinite Dimension 321–340. Birkhauser, Boston). Here we discuss the problem on infinite time horizon.

Minimax LQG Control of Stochastic Partially Observed Uncertain Systems

We consider an infinite-horizon linear-quadratic minimax optimal control problem for stochastic uncertain systems with output measurement. A new description of stochastic uncertainty is introduced using a relative entropy constraint. For the stochastic uncertain system under consideration, a connection between the worst-case control design problem and a specially parametrized risk-sensitive stochastic control problem is established. Using this connection, a minimax optimal LQG controller is constructed which is based on a pair of algebraic matrix Riccati equations arising in risk-sensitive control. It is shown that this minimax optimal controller absolutely stabilizes the stochastic uncertain system.

Robust output feedback stabilization via risk-sensitive control

We consider a problem of robust linear quadratic Gaussian (LQG) control for discrete-time stochastic uncertain systems with partial state measurements. For a finite-horizon case, the problem was recently introduced by Petersen et al. (IEEE Trans. Automat. Control 45 (2000) 398). In this paper, an infinite horizon extension of the results of Petersen et al. (IEEE Trans. Automat. Control 45 (2000) 398) is discussed. We show that for a broad class of uncertain systems under consideration, a controller constructed in terms of the solution to a specially parameterized risk-sensitive stochastic control problem absolutely stabilizes the stochastic uncertain system.

Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint.

Nash Equilibria of Risk-Sensitive Nonlinear Stochastic Differential Games

This paper considers a class of risk-sensitive stochastic nonzero-sum differential games with parametrized nonlinear dynamics and parametrized cost functions. The parametrization is such that, if all or some of the parameters are set equal to some nominal values, then the differential game either becomes equivalent to a risk-sensitive stochastic control (RSSC) problem or decouples into several independent RSSC problems, which in turn are equivalent to a class of stochastic zero-sum differential games. This framework allows us to study the sensitivity of the Nash equilibrium (NE) of the original stochastic game to changes in the values of these parameters, and to relate the NE (generally difficult to compute and to establish existence and uniqueness, at least directly) to solutions of RSSC problems, which are relatively easier to obtain. It also allows us to quantify the sensitivity of solutions to RSSC problems (and thereby nonlinear H∞-control problems) to unmodeled subsystem dynamics controlled by multiple players.

Risk-Sensitive Control of Finite State Machines on an Infinite Horizon I

In this paper we consider robust and risk-sensitive control of discrete time finite state systems on an infinite horizon. The solution of the state feedback robust control problem is characterized in terms of the value of an average cost dynamic game. The risk-sensitive stochastic optimal control problem is solved using the policy iteration algorithm, and the optimal rate is expressed in terms of the value of a stochastic dynamic game with average cost per unit time criterion. By taking a small noise limit, a deterministic dynamic game which is closely related to the robust control problem is obtained.

Connections between stochastic control and dynamic games

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.

H/sup ∞/-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach

One of the major concentrated activities of the past decade in control theory has been the development of the so-called control theory, which addresses the issue of worst-case controller design for linear plants subject to unknown disturbances and plant uncertainties. Among the different time-domain approaches to this class of worst-case design problems, the one that uses the framework of dynamic, differential game theory stands out to be the most natural. This is so because the original H-infinity control problem (in its equivalent time-domain formulation) is in fact a minimax optimization problem, and hence a zero-sum game, where the controller can be viewed as the minimizing player and disturbance as the maximizing player. Using this framework, the authors present in this book a complete theory that encompasses continuous-time as well as discrete-time systems, finite as well as infinite horizons, and several different measurement schemes, including closed loop perfect state, delayed perfect state, samples state, closed-loop imperfect state, delayed imperfect state and sampled imperfect state information patterns. They also discuss extensions of the linear theory to nonlinear systems, and derivation of the lower dimensional controller for systems with regularly and singularly perturbed dynamics. This is the second edition of a 1991 book with the same title, which, besides featuring a more streamlined presentation of the results included in the first edition, and at places under more refined conditions, also contains substantial new material, reflecting new developments in the field since 1991. Among these are the nonlinear theory; connections between H-infinity-optimal control and risk sensitive stochastic control problems; H-infinity filtering for linear and nonlinear systems; and robustness considerations in the presence of regular and singular perturbations. Also included are a rather detailed description of the relationship between frequency-and time-domain approaches to robust controller design, and a complete set of results on the existence of value and characterization of optimal policies in finite- and infinite-horizon LQ differential games. The authors believe that the theory is now at a stage where it can easily be incorporated into a second-level graduate course in a control curriculum, that would follow a basic course in linear control theory covering LQ and LQG designs. The framework adopted in this book makes such an ambitious plan possible. For the most part, the only prerequisite for the book is a basic knowledge of linear control theory. No background in differential games, or game theory in general, is required, as the requisite concepts and results have been developed in the book at the appropriate level. The book is written in such a way that makes it possible to follow the theory for the continuous- and discrete-time systems independently (and also in parallel).

Uniqueness for Viscosity Solutions of Nonstationary Hamilton–Jacobi–Bellman Equations under Some a Priori Conditions (with Applications)

This paper extends the uniqueness results for viscosity solutions of nonstationary Hamilton--Jacobi--Bellman equations. The conditions for uniqueness which are obtained can involve a trade-off between the growth of the solution and the growth of the Hamiltonian. In particular, the result is valid for solutions which grow quadratically in the space variable and are associated with Hamiltonians which also grow quadratically. This particular class arises in the robust control limit of risk-sensitive stochastic control problems.