Risk-sensitive Problem Research Articles

An optimal control of a risk-sensitive problem for backward doubly stochastic differential equations with applications

Abstract In this paper, we are concerned with an optimal control problem where the system is driven by a backward doubly stochastic differential equation with risk-sensitive performance functional. We generalized the result of Chala [A. Chala, Pontryagin’s risk-sensitive stochastic maximum principle for backward stochastic differential equations with application, Bull. Braz. Math. Soc. (N. S.) 48 2017, 3, 399–411] to a backward doubly stochastic differential equation by using the same contribution of Djehiche, Tembine and Tempone in [B. Djehiche, H. Tembine and R. Tempone, A stochastic maximum principle for risk-sensitive mean-field type control, IEEE Trans. Automat. Control 60 2015, 10, 2640–2649]. We use the risk-neutral model for which an optimal solution exists as a preliminary step. This is an extension of an initial control system in this type of problem, where an admissible controls set is convex. We establish necessary as well as sufficient optimality conditions for the risk-sensitive performance functional control problem. We illustrate the paper by giving two different examples for a linear quadratic system, and a numerical application as second example.

Long-Run Risk Sensitive Dyadic Impulse Control

In this paper long-run risk sensitive optimisation problem is studied with dyadic impulse control applied to continuous-time Feller–Markov process. In contrast to the existing literature, focus is put on unbounded and non-uniformly ergodic case by adapting the weight norm approach. In particular, it is shown how to combine geometric drift with local minorisation property in order to extend local span-contraction approach when the process as well as the linked reward/cost functions are unbounded. For any predefined risk-aversion parameter, the existence of solution to suitable Bellman equation is shown and linked to the underlying stochastic control problem. For completeness, examples of uncontrolled processes that satisfy the geometric drift assumption are provided.

Affine Monotonic and Risk-Sensitive Models in Dynamic Programming

In this paper, we consider a broad class of infinite horizon discrete-time optimal control models that involve a nonnegative cost function and an affine mapping in their dynamic programming equation. They include as special cases several classical models, such as stochastic undiscounted nonnegative cost problems, stochastic multiplicative cost problems, and risk-sensitive problems with exponential cost. We focus on the case where the state space is finite and the control space has some compactness properties, and we emphasize shortest path-type models. We assume that the affine mapping has a semicontractive character, whereby for some policies it is a contraction, whereas for others it is not. In one line of analysis, we impose assumptions guaranteeing that the noncontractive policies cannot be optimal. Under these assumptions, we prove strong results that resemble those for discounted Markovian decision problems, such as the uniqueness of solution of Bellman's equation, and the validity of forms of value and policy iteration. In the absence of these assumptions, the results are weaker and unusual in character: the optimal cost function need not be a solution of Bellman's equation, and may not be found by value or policy iteration. Instead the optimal cost function over just the contractive policies is the largest solution of Bellman's equation, and can be computed by a variety of algorithms.

Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions

We consider the infinite horizon risk-sensitive problem for nondegenerate diffusions with a compact action space, and controlled through the drift. We only impose a structural assumption on the running cost function, namely near-monotonicity, and show that there always exists a solution to the risk-sensitive Hamilton–Jacobi–Bellman (HJB) equation, and that any minimizer in the Hamiltonian is optimal in the class of stationary Markov controls. Under the additional hypothesis that the coefficients of the diffusion are bounded, and satisfy a condition that limits (even though it still allows) transient behavior, we show that any minimizer in the Hamiltonian is optimal in the class of all admissible controls. In addition, we present a sufficient condition, under which the solution of the HJB is unique (up to a multiplicative constant), and establish the usual verification result. We also present some new results concerning the multiplicative Poisson equation for elliptic operators in Rd.

Optimal investment with stopping in finite horizon

Abstract In this paper, we investigate dynamic optimization problems featuring both stochastic control and optimal stopping in a finite time horizon. The paper aims to develop new methodologies, which are significantly different from those of mixed dynamic optimal control and stopping problems in the existing literature. We formulate our model to a free boundary problem of a fully nonlinear equation. Furthermore, by means of a dual transformation for the above problem, we convert the above problem to a new free boundary problem of a linear equation. Finally, we apply the theoretical results to some challenging, yet practically relevant and important, risk-sensitive problems in wealth management to obtain the properties of the optimal strategy and the right time to achieve a certain level over a finite time investment horizon. MSC:35R35, 91B28, 93E20.

Risk-sensitive filtering for discrete-time systems with time-varying delay

In this paper, the risk-sensitive filtering problem with time-varying delay is investigated. The problem is transformed into Krein space as an equivalent optimisation problem. The observations with time-varying delays are restructured as ones with multiple constant delays by defining a binary variable model with respect to the arrival process of observations, containing the same state information as the original. Finally, the reorganised innovation analysis approach in Krein space allows the solution to the proposed risk-sensitive filtering in terms of the solutions to Riccati and matrix difference equations.

Risk sensitive and LEG filtering problems are not equivalent

Filtering problems with general exponential quadratic criteria are investigated for Gauss–Markov processes. In this setting, the linear exponential Gaussian and risk sensitive filtering problems are solved and it is shown that they may have different solutions.

Erratum

On page 119 of “The Relationship Between Perfect and Imperfect Information in a Two-Action Risk-Sensitive Problem,” by J. Eric Bickel (Decision Analysis, Vol. 5, No. 3, September 2008, pp. 116–128) , the text in the last paragraph was corrected to read: Without loss of generality, we assume that the information signal is unit normal and therefore the conditional mean of [Formula: see text] given θ is [Formula: see text], which will exceed v as long as [Formula: see text].

From the Editor…

All of the papers in this issue focus on the value of obtaining information about uncertainties. We begin with an article by J. Eric Bickel on “The Relationship Between Perfect and Imperfect Information in a Two-Action Risk-Sensitive Problem.” Next, is an article by Philippe Delquié on “The Value of Information and Intensity of Preference.” Our third article, which uses an example of an oilfield decision to demonstrate an approach to “Valuing Future Information Under Uncertainty Using Polynomial Chaos,” is by Michael Prange, William J. Bailey, Benoit Couët, Hugues Djikpesse, Margaret Armstrong, Alain Galli, and David Wilkinson. In the final article, Thomas Eppel and Detlof von Winterfeldt describe the calculation of value of information about the contents of waste storage tanks in “Value-of-Information Analysis For Nuclear Waste Storage Tanks.” Since information is the topic of this issue, our trivia question is about the late Jacob Marschak, from whom I learned about the economics of information and about clarity of thought.

The Relationship Between Perfect and Imperfect Information in a Two-Action Risk-Sensitive Problem

The ability to value information is a central feature of decision analysis and one of its most interesting areas of application. Unfortunately, general assertions regarding the drivers of information value or its properties have been difficult to formulate or have been disproved by counterexample. In this paper, we investigate the value of imperfect information relative to perfect information (RVOI). Within the context of a two-action decision problem with normal priors and exponential utility, we derive a closed-form solution for the value of information and demonstrate that the RVOI is maximal when the decision maker is indifferent between the two alternatives. In addition, we determine when the value of an information system providing a normally distributed signal with correlation coefficient ρ is equal to ρ × 100% or ρ2 × 100% of the value of perfect information. These results deepen our understanding of information value and enable practitioners to estimate the value of imperfect information in particular settings.

Optimal Estimation and Regulator: Risk-Sensitive Method for Systems of First Degree

The algorithms for the optimal filter and control have been obtained for polynomial systems of first grade. For the filter, two cases are presented: systems with disturbances in L2 and systems with Brownian motion and parameter ∈ multiplying the diffusion term, in state and observations equations. The performance of this algorithms is verified and compared with the optimal Kalman-Bucy filter through an example. Besides the solution to the optimal control Risk-Sensitive problem for stochastic system, taking quadratic value function as solution of PDE HJB is obtained. These Risk-Sensitive control algorithms are compared with the L-Q control algorithms through a numerical example, using quadratic-exponential cost function to be minimized. The optimal risk-sensitive filter and control algorithms show better performance for large values of the parameter ∈.

Robust Multi-Product Pricing

This paper concerns dynamic pricing of multiple perishable products when there is model uncertainty, which we formulate as a worst-case stochastic intensity control problem where ambiguity is modeled using the notion of relative entropy. One feature of our formulation is that the demand models for different products can have different levels of ambiguity, a situation that arises (for instance) if a new product is being sold along side an established one. We show that this multiple-ambiguity multi-product robust pricing problem is equivalent to another (non-standard) risk-sensitive pricing problem, and show that it can be decentralized under additional assumptions on the demand rate model. The risk-sensitive problem has several unusual features: (i) the net income from sales of each product is valued by its certainty equivalent under an exponential utility function where the aversion parameter is determined by the level of ambiguity of its demand model, (ii) the overall goal is to maximize the sum of the certainty equivalents over all products, and (iii) products making sales are required to compensate other products for the use of commonresources according to a revenue sharing rule. We characterize the revenue sharing rule which leads to an equivalence between the risk-sensitive problem we have just described and the original robust pricing problem. This generalizes risk-sensitive/robust control duality to the case where different components of the model have different levels of model uncertainty. Finally, we show that the robust multi-product problem can be decentralized and solved in terms of modified robust/risk-sensitive single-product problems, if the demand rate functions satisfy certain independence assumptions. The modification of the single product problems involves the introduction of a cost to account for the value of inventory that is used at each sale. This cost is closely related to the revenue sharing rule associated with the robust/risk-sensitive control equivalence.

A new risk-sensitive maximum principle

In this paper, a new maximum principle for the risk-sensitive control problem is established. One important feature of this result is that it applies to systems in which the diffusion term may depend on the control. Such control dependence gives rise to interesting phenomena not observed in the usual setting where control independence of the diffusion term is assumed. In particular, there is an additional second order adjoint equation and additional terms in the maximum condition that involve this second order process as well as the risk-sensitive parameter. Moreover, contrary to a conventional maximum principle, the first-order adjoint equation involved in our maximum principle is a nonlinear equation. An advantage of considering this new type of adjoint equation is that the risk-sensitive maximum principle derived is similar in form to its risk-neutral counterpart. The approach is based on the logarithmic transformation and the relationship between the adjoint variables and the value function. As an example, a linear-quadratic risk-sensitive problem is solved using the maximum principle derived.

An Escape-Time Criterion for Queueing Networks: Asymptotic Risk-Sensitive Control via Differential Games

We consider the problem of risk-sensitive control of a stochastic network. In controlling such a network, an escape-time criterion can be useful if one wishes to regulate the occurrence of large buffers and buffer overflow. In this paper a risk-sensitive escape time criterion is formulated, which in comparison to the ordinary escape-time criteria penalizes exits that occur on short time intervals more heavily. The properties of the risk-sensitive problem are studied in the large buffer limit and related to the value of a deterministic differential game with constrained dynamics. We prove that the game has value and that the value is the (viscosity) solution of a PDE. For a simple network, the value is computed, demonstrating the applicability of the approach.

Multiple-objective risk-sensitive control and its small noise limit

This paper is concerned with a (minimizing) multiple-objective risk-sensitive control problem. Asymptotic analysis leads to the introduction of a new class of two-player, zero-sum, deterministic differential games. The distinguishing feature of this class of games is that the cost functional is multiple-objective in nature, being composed of the risk-neutral integral costs associated with the original risk-sensitive problem. More precisely, the opposing player in such a game seeks to maximize the most ‘vulnerable’ member of a given set of cost functionals while the original controller seeks to minimize the worst ‘damage’ that the opponent can do over this set. It is then shown that the problem of finding an efficient risk-sensitive controller is equivalent, asymptotically, to solving this differential game. Surprisingly, this differential game is proved to be independent of the weights on the different objectives in the original multiple-objective risk-sensitive problem. As a by-product, our results generalize the existing results for the single-objective risk-sensitive control problem to a substantially larger class of nonlinear systems, including those with control-dependent diffusion terms.

Finite Time--Horizon Risk-Sensitive Control and the Robust Limit under a Quadratic Growth Assumption

The finite time--horizon risk-sensitive limit problem for continuous nonlinear systems is considered. Previous results are extended to cover more typical examples. In particular, the cost may grow quadratically, and the diffusion coefficient may depend on the state. It is shown that the risk-sensitive value function is the solution of the corresponding dynamic programming equation. It is also shown that this value converges to the value of the robust control problem as the cost becomes infinitely risk averse, with corresponding scaling of the diffusion coefficient.

On asymptotic stability of continuous-time risk-sensitive filters with respect to initial conditions

In this paper, we consider the problem of risk-sensitive filtering for continuous-time stochastic linear Gaussian time-invariant systems. In particular, we address the problem of forgetting of initial conditions. Our results show that suboptimal risk-sensitive filters initialized with arbitrary Gaussian initial conditions asymptotically approach the optimal risk-sensitive filter for a linear Gaussian system with Gaussian but unknown initial conditions in the mean square sense at an exponential rate, provided the arbitrary initial covariance matrix results in a stabilizing solution of the ( H ∞-like) Riccati equation associated with the risk-sensitive problem. More importantly, in the case of non-Gaussian initial conditions, a suboptimal risk-sensitive filter asymptotically approaches the optimal risk-sensitive filter in the mean square sense under a boundedness condition satisfied by the fourth order absolute moment of the initial non-Gaussian density and a slow growth condition satisfied by a certain Radon–Nikodym derivative.

Lie algebraic methods in optimal control of stochastic systems with exponential-of-integral cost

The purpose of this paper is to formulate and study the optimal control of partially observed stochastic systems with exponential-of-integral-sample cost, known as risk-sensitive problems, using Lie algebraic tools. This leads to the introduction of the sufficient statistic algebra, L s , through which one can determine á priori the maximum order of the controller. When dim( L s)<∞ , the construction of the control laws is addressed through extensions of the Wei–Norman method, as in nonlinear filtering problems. Aside from specific known finite-dimensional examples which are studied in order to delineate the application of the Lie algebraic tools, new classes of finite-dimensional controllers are identified as well. In addition, relations with minimax dynamic games are explored to best assess the importance and generality of the finite-dimensional control systems.

Risk-sensitive and risk-neutral control for continuous-time hidden Markov models

In this paper the optimal control of a continuous-time hidden Markov model is discussed. The risk-sensitive problem involves a cost function which has an exponential form and a risk parameter, and is solved by defining an appropriate information state and dynamic programming. As the risk parameter tends to zero, the classical risk-neutral optimal control problem is recovered. The limits are proved using viscosity solution methods.

General finite-dimensional risk-sensitive problems and small noise limits

For a risk-sensitive, partially observed stochastic control problem, the modified Zakai equation includes an extra term related to the exponential running cost. The finite-dimensional solutions of this modified Zakai equation are obtained. These are analogs of the Kalman and Benes filters. The small noise limits of the finite-dimensional risk-sensitive problems are then obtained. These lead to differential games with deterministic disturbances.