Risk-sensitive Control Research Articles

Bellman Equations of Risk-Sensitive Control

Risk-sensitive control problems are considered. Existence of a nonnegative solution to the Bellman equation of risk-sensitive control is shown. The result is applied to prove that no breaking down occurs. Asymptotic behaviour of the nonnegative solution is studied in relation to ergodic control problems and the relationship between the asymptotics and the large deviation principle is noted.

Risk-Sensitive and Risk-Neutral Control for Continuous-Time Hidden Markov Models

Risk-Sensitive and Risk-Neutral Control for Continuous-Time Hidden Markov Models

Finite-dimensional optimal controllers for nonlinear plants

Optimal risk sensitive feedback controllers are now available for very general stochastic nonlinear plants and performance indices. They consist of nonlinear static feedback of so called information states from an information state filter. In general, these filters are linear, but infinite dimensional, and the information state feedback gains are derived from (doubly) infinite dimensional dynamic programming. The challenge is to achieve optimal finite dimensional controllers using finite dimensional calculations for practical implementation. This paper derives risk sensitive optimality results for finite-dimensional controllers. The controllers can be conveniently derived for ‘linearized’ (approximate) models (applied to nonlinear stochastic systems). Performance indices for which the controllers are optimal for the nonlinear plants are revealed. That is, inverse risk-sensitive optimal control results for nonlinear stochastic systems with finite dimensional linear controllers are generated. It is instructive to see from these results that as the nonlinear plants approach linearity, the risk sensitive finite dimensional controllers designed using linearized plant models and risk sensitive indices with quadratic cost kernels, are optimal for a risk sensitive cost index which approaches one with a quadratic cost kernel. Also even far from plant linearity, as the linearized model noise variance becomes suitably large, the index optimized is dominated by terms which can have an interesting and practical interpretation. Limiting versions of the results as the noise variances approach zero apply in a purely deterministic nonlinear H ∞ setting. Risk neutral and continuous-time results are summarized. More general indices than risk sensitive indices are introduced with the view to giving useful inverse optimal control results in non-Gaussian noise environments.

Finite Dimensional Risk Sensitive Information States

In risk sensitive control the information state satisfies a modified Zakai equation which includes an extra term related to the exponential running cost. We show that a wide class of finite dimensional solutions related to the Beneš filter is possible as long as nonlinearities in the drift are cancelled in an appropriate way by terms in the running cost.

Deposit insurance subsidies, moral hazard, and bank regulation

The model developed in the paper separates deposit insurance subsidies into two components: a premium-linked subsidy which arises from an ex-ante mispricing of the deposit insurance premium, and an asset-linked subsidy which arises from a lack of ex-post monitoring of the bank's actions. The identification of these two subsidies provides important insight into the relation between deposit-insurance subsidies and bank risk. The asset-linked subsidy is higher for banks of average risk and lower for very-high and very-low risk banks. The premium-linked subsidy behaves differently under risk-adjusted and fixed-rate premiums. The model also indicates that the implementation of a risk-adjusted insurance-rate schedule alone would not be sufficient to eliminate the bank's excessive risk-taking behavior. Thus, some combination of risk-sensitive deposit-insurance pricing and regulatory control is necessary to reduce the moral hazard problem.

Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems

Solves a finite-horizon partially observed risk-sensitive stochastic optimal control problem for discrete-time nonlinear systems and obtains small noise and small risk limits. The small noise limit is interpreted as a deterministic partially observed dynamic game, and new insights into the optimal solution of such game problems are obtained. Both the risk-sensitive stochastic control problem and the deterministic dynamic game problem are solved using information states, dynamic programming, and associated separated policies. A certainty equivalence principle is also discussed. The authors' results have implications for the nonlinear robust stabilization problem. The small risk limit is a standard partially observed risk-neutral stochastic optimal control problem. >

A Tribute to Wendell H. Fleming

Next article A Tribute to Wendell H. FlemingL. D. Berkovitz, Steven E. Shreve, and William P. ZiemerL. D. Berkovitz, Steven E. Shreve, and William P. Ziemerhttps://doi.org/10.1137/0331017PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout Next article FiguresRelatedReferencesCited ByDetails On the role of Föllmer-Schweizer minimal martingale measure in risk-sensitive control asset managementJournal of Applied Probability, Vol. 52, No. 03 | 30 March 2016 Cross Ref On the role of Föllmer-Schweizer minimal martingale measure in risk-sensitive control asset managementJournal of Applied Probability, Vol. 52, No. 3 | 30 March 2016 Cross Ref Volume 31, Issue 2| 1993SIAM Journal on Control and Optimization273-538 History Published online:14 July 2006 InformationCopyright © 1993 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0331017Article page range:pp. 273-281ISSN (print):0363-0129ISSN (online):1095-7138Publisher:Society for Industrial and Applied Mathematics

Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games

In this paper we consider a finite horizon, nonlinear, stochastic, risk-sensitive optimal control problem with complete state information, and show that it is equivalent to a stochastic differential game. Risk-sensitivity and small noise parameters are introduced, and the limits are analyzed as these parameters tend to zero. First-order expansions are obtained which show that the risk-sensitive controller consists of a standard deterministic controller, plus terms due to stochastic and game-theoretic methods of controller design. The results of this paper relate to the design of robust controllers for nonlinear systems.

Tractable risk sensitive control based on approximate expected utility

This paper presents a tractable method of risk sensitive optimal control based on approximate Neumann-Morgenstern expected utility. The technique is to first write any expected loss function (defined as the negative of expected utility) exactly in terms of all relevant moments of the additive shocks, and then to approximate expected loss with an expression quadratic in the first moment but also involving all relevant higher moments. Then the atandard quadratic linear control technique can be used to obtain risk sensitive closed loop policy decisions. Requirements for preference revelation by the policy maker are minimal.

Entropy-minimising and risk-sensitive control rules

Glover and Doyle [4] have shown that the entropy-minimising criterion is identical with the infinite-horizon form of the LEQG-optimality criterion (LEQG = linear/exponential-of-quadratic/Gaussian). Whittle and Kuhn [11] have given a Hamiltonian formulation which provides a natural treatment of LEQG-optimisation. This paper contains a direct proof that the control yielded by these Hamiltonian methods is entropy-minimising. The conclusion would follow from the two assertions above, but a simple adjoining of the proofs of these two assertions yields an argument so circuitous that a direct proof is of interest.

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion

Controlled Branching Processes

Controlled Branching Processes

Wiener Integrals Associated with Diffusion Processes

Previous article Next article Wiener Integrals Associated with Diffusion ProcessesV. E. Beněs and L. A. SheppV. E. Beněs and L. A. Shepphttps://doi.org/10.1137/1113057PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] M. Kac, On some connections between probability theory and differential and integral equations, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950, University of California Press, Berkeley and Los Angeles, 1951, 189–215 MR0045333 0045.07002 Google Scholar[2] Yu V. Prokhorov, Convergence of random processes and limit theorems in probability theory, Theory Prob. Applications, 1 (1956), 157–214 10.1137/1101016 LinkGoogle Scholar[3] A. V. Skorohod, On the existence and uniqueness of solutions of stochastic differential equations, Sibirsk. Mat. Ž., 2 (1961), 129–137, (In Russian.) MR0132595 Google Scholar[4] L. A. Shepp, Radon-Nikodym derivatives of Gaussian measures, Ann. Math. Statist., 37 (1966), 321–354 MR0190999 0142.13901 CrossrefGoogle Scholar[5] E. B. Dynkin, Theory of Markov processes, Translated from the Russian by D. E. Brown; edited by T. Köváry, Prentice-Hall Inc., Englewood Cliffs, N.J., 1961ix+210 MR0131900 0091.13605 Google Scholar[6] Kiyosi Itô, Stationary random distributions, Mem. Coll. Sci. Univ. Kyoto. Ser. A. Math., 28 (1954), 209–223 MR0065060 0059.11505 CrossrefGoogle Scholar[7] J. L. Doob, Stochastic processes, John Wiley & Sons Inc., New York, 1953viii+654 MR0058896 0053.26802 Google Scholar[8] Kiyosi Ito, On stochastic differential equations, Mem. Amer. Math. Soc., 1951 (1951), 51– MR0040618 0054.05803 Google Scholar[9] A. V. Skorohod, Constructive methods of defining random processes, Uspehi Mat. Nauk, 20 (1965), 67–87, (In Russian.) MR0185656 0138.40308 Google Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Partially observable nonlinear risk-sensitive control problems: dynamic programming and verification theoremsIEEE Transactions on Automatic Control, Vol. 42, No. 8 Cross Ref Certain nonlinear partially observable stochastic optimal control problems with explicit control laws equivalent to LEQG/LQG problemsIEEE Transactions on Automatic Control, Vol. 42, No. 4 Cross Ref Variational path-integral representations for the density of a diffusion process4 April 2007 | Stochastics and Stochastic Reports, Vol. 26, No. 4 Cross Ref A rigorous Onsager-Machlup formulation of nonequilibrium thermodynamics Cross Ref A simple derivation of the Onsager–Machlup formula for one‐dimensional nonlinear diffusion processJournal of Mathematical Physics, Vol. 19, No. 8 Cross Ref Detailed Time-Dependent Description of Tunneling Phenomena Arising from Stochastic Quantization13 March 1978 | Physical Review Letters, Vol. 40, No. 11 Cross Ref Composition and invariance methods for solving some stochastic control problems1 July 2016 | Advances in Applied Probability, Vol. 7, No. 02 Cross Ref Composition and invariance methods for solving some stochastic control problems1 July 2016 | Advances in Applied Probability, Vol. 7, No. 2 Cross Ref A path space picture for Feynman-Kac averagesAnnals of Physics, Vol. 88, No. 2 Cross Ref A stochastic primer Cross Ref Volume 13, Issue 3| 1968Theory of Probability & Its Applications History Submitted:25 January 1967Published online:17 July 2006 InformationCopyright © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1113057Article page range:pp. 475-478ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics