Risk-sensitive Control Problem Research Articles

<i>G</i>-stochastic maximum principle for risk-sensitive control problem and its applications

<i>G</i>-stochastic maximum principle for risk-sensitive control problem and its applications

Ergodic risk-sensitive control for regime-switching diffusions

In this article, we study the ergodic risk-sensitive control problem for controlled regime-switching diffusions. Under a blanket stability hypothesis, we solve the associated nonlinear eigenvalue problem for weakly coupled systems and characterize the optimal stationary Markov controls via a suitable verification theorem. We also consider the near-monotone case and obtain the existence of principal eigenfunction and optimal stationary Markov controls.

Risk-sensitive control for a class of diffusions with jumps

We consider a class of diffusions controlled through the drift and jump size, and driven by a jump Lévy process and a nondegenerate Wiener process, and we study infinite horizon (ergodic) risk-sensitive control problems for this model. We start with the controlled Dirichlet eigenvalue problem in smooth bounded domains, which also allows us to generalize current results in the literature on exit rate control problems. Then we consider the infinite horizon average risk-sensitive minimization and maximization problems on the whole domain. Under suitable hypotheses, we establish existence and uniqueness of a principal eigenfunction for the Hamilton–Jacobi–Bellman (HJB) operator on the whole space, and fully characterize stationary Markov optimal controls as the measurable selectors of this HJB equation.

Indefinite risk-sensitive control

We consider a risk-sensitive control problem with indefinite cost matrices. All solutions to such an optimal control problem are obtained in an explicit closed-form by the completion of squares and the change of measure methods. Both the finite and infinite horizon cases are considered. As an application, we find all solutions to an optimal investment problem in a market with a stochastic interest rate.

Hybrid [formula omitted] sub-optimal stochastic risk-sensitive control for polynomial systems of first degree

This paper presents the novelty RS (Risk-Sensitive) H2 optimal control problem with criterion J2, showing the energy of the control spent into the process, H∞ RS control approach with level of attenuation λ1 is obtained, with cost criterion J∞. Robustness is added when hybridized H2∕H∞ RS sub-optimal control problem is solved, as optimization approach, taking J2 as cost function criterion, with H∞ inequality constraint. The sub-optimal solution is obtained imposing that the H∞ constraint holds with the equality sign. Values of the criterion J2 versus exponential quadratic cost criterion J, (RS traditional criterion), illustrate advantages of exponential quadratic cost criterion J for different values of diffusion parameter ϵ into the state equation in the final time through an application to the model of the immune system. The experiments show the values of level of attenuation λ1 in H∞ RS control problem and hybrid H2∕H∞ RS sub-optimal control problem which holds with equality sign. The results of the different experiments that were carried out show that the proposed methodology is feasible and meets the conditions required in H2, and H∞ approaches.

On the policy improvement algorithm for ergodic risk-sensitive control

In this article we consider the ergodic risk-sensitive control problem for a large class of multidimensional controlled diffusions on the whole space. We study the minimization and maximization problems under either a blanket stability hypothesis, or a near-monotone assumption on the running cost. We establish the convergence of the policy improvement algorithm for these models. We also present a more general result concerning the region of attraction of the equilibrium of the algorithm.

Risk-Sensitive Asset Management with Lognormal Interest Rates

Risk-sensitive asset management on both finite and infinite time horizons are treated on a market with a bank account and a risky stock. The risk-free interest rate is formulated as a geometric Brownian motion, and affects the return of the risky stock. The problems become standard risk-sensitive control problems. We derive the Hamilton–Jacobi–Bellman equations and study these solutions. Using solutions, we construct optimal strategies and optimal values.

Time-Inconsistent Risk-Sensitive Equilibrium for Countable-Stated Markov Decision Processes

This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is time-inconsistent, it is impossible to find a global optimal strategy for both cases. Instead, for each case, we will prove the existence of time-inconstant equilibrium strategies which verify the so-called step-optimality. Moreover, we prove the convergence of $\e$-equilibriums and the corresponding value functions as $\e\rightarrow0^+$.

On the singular risk-sensitive stochastic maximum principle

We consider a risk-sensitive stochastic control problem of a nonlinear system in which the variable control has two components, the first being absolutely continuous and the second is singular. The author generalizes the result obtained in risk neutral into risk-sensitive performance cost. The auxiliary process has been introduced to solve the problem of risk sensitive. The relationship between the expected exponential utility and the quadratic backward stochastic differential equation has been given and proved. Actually, I have proved in details two results, the first is necessary optimality condition, while the second is sufficient optimality condition for risk sensitive performance cost. Two examples to carried out to illustrate our main results of risk-sensitive control problem under nonlinear stochastic dynamics with exponential quadratic cost function with numerical investigation, while the second will be in the exponential utility optimization problem.

Behavioral Benchmarked Investment Management with Expert Forecasts

We propose a continuous-time portfolio selection model that explains the active-passive continuum. Our model illuminates the pivotal role of expert opinions and factors in the asset allocation process. In the model, investors aim to outperform a benchmark. As securities and benchmark's drift depends on unobservable factors, portfolio selection becomes a partial observation risk-sensitive control problem. We find that the optimal investment policy combines Kelly, benchmark-tracking, and intertemporal hedging portfolios. The Kelly portfolio sums two alpha-generating strategies. Our simulation reveals the importance of debiasing expert opinions and shows that factor choice is crucial at every stage of the investment process.

On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application

In this paper, we are concerned with an optimal control problem where the system is driven by fully coupled forward-backward stochastic differential equation of mean-field type with risk-sensitive performance functional. We study the risk-neutral model for which an optimal solution exists as a preliminary step. This is an extension of the initial stochastic control problem in this type of risk-sensitive performance problem, where an admissible set of controls are convex. We establish necessary as well as sufficient optimality conditions for the risk-sensitive performance functional control problem. Finally, we illustrate our main result of this paper by giving two examples of risk-sensitive control problem under linear stochastic dynamics with exponential quadratic cost function, the second example will be a mean-variance portfolio with a recursive utility functional optimization problem involving optimal control. The explicit expression of the optimal portfolio selection strategy is obtained in the state feedback.

On the relative value iteration with a risk-sensitive criterion

A multiplicative relative value iteration algorithm for solving the dynamic programming equation for the risk-sensitive control problem is studied for discrete time controlled Markov chains with a compact Polish state space, and controlled diffusions in on the whole Euclidean space. The main result is a proof of convergence to the desired limit in each case.

Risk-sensitive control of branching processes

This article solves the risk-sensitive control problem for branching processes where the one-period progeny of an individual can take values from a finite set. The decision maker is assumed to maximize the expected risk-averse exponential utility (or to minimize the expected risk-averse exponential disutility) of the rewards earned in an infinite horizon. Individuals are assumed to produce progeny independently, and with the same probability mass function if they take the same action. This article characterizes the expected disutility of stationary policies, identifies necessary and sufficient conditions for the existence of a stationary optimal policy that assigns the same action to all individuals in all periods, and discusses computational methods to obtain such a policy. Supplementary materials are available for this article. See the publisher’s online edition of IIE Transactions, datasets, additional tables, detailed proofs, etc.

A convergent hierarchy of non-linear eigenproblems to compute the joint spectral radius of nonnegative matrices

We show that the joint spectral radius of a finite collection of nonnegative matrices can be bounded by the eigenvalue of a non-linear operator. This eigenvalue coincides with the ergodic constant of a risk-sensitive control problem, or of an entropy game, in which the state space consists of all switching sequences of a given length. We show that, by increasing this length, we arrive at a convergent approximation scheme to compute the joint spectral radius. The complexity of this method is exponential in the length of the switching sequences, but it is quite insensitive to the size of the matrices, allowing us to solve very large scale instances (several matrices in dimensions of order 1000 within a minute). An idea of this method is to replace a hierarchy of optimization problems, introduced by Ahmadi, Jungers, Parrilo and Roozbehani, by a hierarchy of nonlinear eigenproblems. To solve the latter eigenproblems, we introduce a projective version of Krasnoselskii-Mann iteration. This method is of independent interest as it applies more generally to the nonlinear eigenproblem for a monotone positively homogeneous map. Here, this method allows for scalability by avoiding the recourse to linear or semidefinite programming techniques.

Multi-point Gaussian States, Quadratic–Exponential Cost Functionals, and Large Deviations Estimates for Linear Quantum Stochastic Systems

This paper is concerned with risk-sensitive performance analysis for linear quantum stochastic systems interacting with external bosonic fields. We consider a cost functional in the form of the exponential moment of the integral of a quadratic polynomial of the system variables over a bounded time interval. Such functionals are related to more conservative behaviour and robustness of systems with respect to statistical uncertainty, which makes the challenging problems of their computation and minimization practically important. To this end, we obtain an integro-differential equation for the time evolution of the quadratic–exponential functional, which is different from the original quantum risk-sensitive performance criterion employed previously for measurement-based quantum control and filtering problems. Using multi-point Gaussian quantum states for the past history of the system variables and their first four moments, we discuss a quartic approximation of the cost functional and its infinite-horizon asymptotic behaviour. The computation of the asymptotic growth rate of this approximation is reduced to solving two algebraic Lyapunov equations. Further approximations of the cost functional, based on higher-order cumulants and their growth rates, are applied to large deviations estimates in the form of upper bounds for tail distributions. We discuss an auxiliary classical Gaussian–Markov diffusion process in a complex Euclidean space which reproduces the quantum system variables at the level of covariances but has different fourth-order cumulants, thus showing that the risk-sensitive criteria are not reducible to quadratic–exponential moments of classical Gaussian processes. The results of the paper are illustrated by a numerical example and may find applications to coherent quantum risk-sensitive control problems, where the plant and controller form a fully quantum closed-loop system, and other settings with nonquadratic cost functionals.

Strict monotonicity of principal eigenvalues of elliptic operators in [formula omitted] and risk-sensitive control

This paper studies the eigenvalue problem on Rd for a class of second order, elliptic operators of the form Lf=aij∂xi∂xj+bi∂xi+f, associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function f fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton–Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.

The maximum principles for partially observed risk-sensitive optimal controls of Markov regime-switching jump-diffusion system

ABSTRACTThis paper studies partially observed risk-sensitive optimal control problems with correlated noises between the system and the observation. It is assumed that the state process is governed by a continuous-time Markov regime-switching jump-diffusion process and the cost functional is of an exponential-of-integral type. By virtue of a classical spike variational approach, we obtain two general maximum principles for the aforementioned problems. Moreover, under certain convexity assumptions on both the control domain and the Hamiltonian, we give a sufficient condition for the optimality. For illustration, a linear-quadratic risk-sensitive control problem is proposed and solved using the main results. As a natural deduction, a fully observed risk-sensitive maximum principle is also obtained and applied to study a risk-sensitive portfolio optimization problem. Closed-form expressions for both the optimal portfolio and the corresponding optimal cost functional are obtained.

A risk-sensitive maximum principle for a Markov regime-switching jump-diffusion system and applications

In this paper, we derive a general stochastic maximum principle for a risk-sensitive type optimal control problem of Markov regime-switching jump-diffusion model. The results are obtained via a logarithmic transformation and the relationship between adjoint variables and the value function. We apply the results to study both a linear-quadratic optimal control problem and a risk-sensitive benchmarked asset management problem for Markov regime-switching models. In the latter case, the optimal control is of feedback form and is given in terms of solutions to a Markov regime-switching Riccatti equation and an ordinary Markov regime-switching differential equation.

Variance-Optimal Martingale Measures for Diffusion Processes with Stochastic Coefficients

In this paper we present the solution of the optimal variance optimal martingale measure for stochastic volatility models, when the noises are correlated. It is proved that the value function of the dual problem is a classical solution of the corresponding Hamilton-Jacobi-Bellman equation. The method to develop our results is based on a Bernstein’s type of argument. The dual problem of the quadratic hedging problem is studied analyzing the expression obtained after a change of measure, which corresponds to some class of risk-sensitive control problems.

Risk-sensitive asset management in a general diffusion factor model: risk-seeking case

We consider risk-sensitive asset management on both finite and infinite time horizons. In particular, we treat the risk-seeking case. The returns and volatilities of the assets are random and affected by some economic factors, modeled as a diffusion process. The problems become standard risk-sensitive control problems. We derive the Hamilton–Jacobi–Bellman equations and study these solutions. Using solutions, we construct optimal strategies and optimal values. Moreover, we solve an upside large deviations probability maximization problem by conducting the duality relation with the infinite time horizon problem.