Entropic Risk Measure Research Articles

AoI-aware transmission control in real-time mmwave energy harvesting systems: a risk-sensitive reinforcement learning approach

The evolution of enabling technologies in wireless communications has paved the way for supporting novel applications with more demanding QoS requirements, but at the cost of increasing the complexity of optimizing the digital communication chain. In particular, Millimeter Wave (mmWave) communications provide an abundance of bandwidth, and energy harvesting supplies the network with a continual source of energy to facilitate self-sustainability; however, harnessing these technologies is challenging due to the stochastic dynamics of the mmWave channel as well as the random sporadic nature of the harvested energy. In this paper, we aim at the dynamic optimization of update transmissions in mmWave energy harvesting systems in terms of Age of Information (AoI). AoI has recently been introduced to quantify information freshness and is a more stringent QoS metric compared to conventional delay and throughput. However, most prior art has only addressed average-based AoI metrics, which can be insufficient to capture the occurrence of rare but high-impact freshness violation events in time-critical scenarios. We formulate a control problem that aims to minimize the long-term entropic risk measure of AoI samples by configuring the “sense & transmit” of updates. Due to the high complexity of the exponential cost function, we reformulate the problem with an approximated mean-variance risk measure as the new objective. Under unknown system statistics, we propose a two-timescale model-free risk-sensitive reinforcement learning algorithm to compute a control policy that adapts to the trio of channel, energy, and AoI states. We evaluate the efficiency of the proposed scheme through extensive simulations.

Markov decision processes with risk-sensitive criteria: an overview

AbstractThe paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term ’risk-sensitive’ refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk. This comprises the well-known entropic risk measure and Conditional Value-at-Risk. We restrict our considerations to stationary problems with an infinite time horizon. Conditions are given under which optimal policies exist and solution procedures are explained. We present both the theory when the Optimized Certainty Equivalent is applied recursively as well as the case where it is applied to the cumulated reward. Discounted as well as non-discounted models are reviewed.

Parabolic PDE-constrained optimal control under uncertainty with entropic risk measure using quasi-Monte Carlo integration

We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem—and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.

Optimal Investment with Risk Controlled by Weighted Entropic Risk Measures

Optimal Investment with Risk Controlled by Weighted Entropic Risk Measures

Properties of the entropic risk measure EVaR in relation to selected distributions

Entropic Value-at-Risk (EVaR) measure is a convenient coherent risk measure. Due to certain difficulties in finding its analytical representation, it was previously calculated explicitly only for the normal distribution. We succeeded to overcome these difficulties and to calculate Entropic Value-at-Risk (EVaR) measure for Poisson, compound Poisson, Gamma, Laplace, exponential, chi-squared, inverse Gaussian distribution and normal inverse Gaussian distribution with the help of Lambert function that is a special function, generally speaking, with two branches.

Algorithms for a risk-averse Stackelberg game with multiple adversaries

We consider a Stackelberg game that arises in a security domain (SSG), where a defender can simultaneously protect m out of n targets from an adversary that observes the defense strategy before deciding on an utility maximizing attack. Given the high stakes in security settings, it is reasonable that the defender in this game is risk averse with respect to the attacker’s decisions.Here we focus on developing efficient solution algorithms for a specific SSG, where the defender uses an entropic risk measure to model risk aversion to the attacker’s strategies, and where multiple attackers select targets following logit quantal response equilibrium models. This problem can be formulated as a nonconvex nonlinear optimization problem. We propose two solution methods: (1) approximate the problem through convex mixed integer nonlinear programs (MINR) and (2) a general purpose methodology (CELL) to optimize nonconvex and nonseparable fractional problems through mixed integer linear programming approximations. Both methods provide arbitrarily good incumbents and lower bounds on SSG. We present cutting plane methods to solve these problems for large instances. Our computational experiments illustrate the advantages of introducing risk aversion into the defender’s behavior and show that MINR dominates CELL, producing in 2 h solutions that are within 2% of optimal on average.

Non-stationary Risk-Sensitive Reinforcement Learning: Near-Optimal Dynamic Regret, Adaptive Detection, and Separation Design

We study risk-sensitive reinforcement learning (RL) based on an entropic risk measure in episodic non-stationary Markov decision processes (MDPs). Both the reward functions and the state transition kernels are unknown and allowed to vary arbitrarily over time with a budget on their cumulative variations. When this variation budget is known a prior, we propose two restart-based algorithms, namely Restart-RSMB and Restart-RSQ, and establish their dynamic regrets. Based on these results, we further present a meta-algorithm that does not require any prior knowledge of the variation budget and can adaptively detect the non-stationarity on the exponential value functions. A dynamic regret lower bound is then established for non-stationary risk-sensitive RL to certify the near-optimality of the proposed algorithms. Our results also show that the risk control and the handling of the non-stationarity can be separately designed in the algorithm if the variation budget is known a prior, while the non-stationary detection mechanism in the adaptive algorithm depends on the risk parameter. This work offers the first non-asymptotic theoretical analyses for the non-stationary risk-sensitive RL in the literature.

Capital allocation with multivariate convex risk measures

Capital allocation is of fundamental importance in risk management. Capital allocation problems with univariate risk measures have been extensively studied in the finance literature. In contrast, the studies about capital allocations with multivariate risk measures are quite insufficient in the literature. In this paper, we study capital allocations with multivariate coherent and convex risk measures. First, we propose the definitions of capital allocation problem and principle for general multivariate risk measures. Then based on the dual representations proved by [41], we provide the capital allocation principles with multivariate coherent and convex risk measures. Their properties are also investigated. For the case of multivariate convex risk measures, we also suggest a reasonable and effective capital allocation scheme. Finally, examples are given to illustrate the effectiveness of the proposed capital allocation scheme, where the involved convex risk measures are multivariate and univariate entropic risk measures, respectively.

Optimized Data Sampling and Energy Consumption in IIoT: A Federated Learning Approach

Real-time environment monitoring is a key application in Industrial Internet of Things, where sensors proactively collect and transmit environmental data to the controller. However, due to limited wireless resources, keeping sensors’ sampled data fresh at the controller is critical. This work aims to investigate the trade-off between the sensor’s data-sampling frequency and long-term data transmission energy consumption while maintaining information freshness. Leveraging the entropic risk measure (ERM), we jointly minimize the global transmission energy’s mean and variance subject to probabilistic constraints on information freshness. Furthermore, while jointly saving the model training energy, we adopt the federated learning (FL) paradigm and propose an FL-based two-stage iterative optimization framework to optimize the aforementioned objective. Specifically, we iteratively learn the sampling frequency via Bayesian optimization and minimize the long-term ERM of the global energy consumption via Lyapunov optimization. Numerical results show that the proposed FL-based scheme saves substantial executing energy with less performance loss. Quantitatively, compared with the centralized learning baseline, the proposed FL-based framework saves up to 69% model training energy at the expense of a mere increased objective outcome, i.e., 6.3% in the global data transmission energy consumption ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$9.936\times 10^{-5}$ </tex-math></inline-formula> in ERM) under 0.4% bias from the global optimal data-sampling frequency.

Stochastic Control of Optimized Certainty Equivalents

Optimized certainty equivalents (OCEs) are a family of risk measures widely used by both practitioners and academics. This is mostly due to its tractability and the fact that it encompasses important examples, including entropic risk measures and average value-at-risk. In this work we consider stochastic optimal control problems where the objective criterion is given by an OCE risk measure or, in other words, a risk minimization problem for controlled diffusions. A major difficulty arises since OCEs are often time-inconsistent. Nevertheless, via an enlargement of state space we achieve a substitute of sorts for time-consistency in fair generality. This allows us to derive a dynamic programming principle and thus recover central results of (risk-neutral) stochastic control theory. In particular, we show that the value of our risk minimization problem can be characterized as a viscosity solution of a Hamilton--Jacobi--Bellman--Isaacs equation. We further establish a comparison principle and uniqueness of the latter under suitable technical conditions.

Convergence rates of large-time sensitivities with the Hansen–Scheinkman decomposition

This paper investigates the large-time asymptotic behavior of the sensitivities of cash flows. In quantitative finance, the price of a cash flow is expressed in terms of a pricing operator of a Markov diffusion process. We study the extent to which the pricing operator is affected by small changes of the underlying Markov diffusion. The main idea is a partial differential equation (PDE) representation of the pricing operator by incorporating the Hansen–Scheinkman decomposition method. The sensitivities of the cash flows and their large-time convergence rates can be represented via simple expressions in terms of eigenvalues and eigenfunctions of the pricing operator. Furthermore, compared to the work of Park (Finance Stoch 4:773–825, 2018), more detailed convergence rates are provided. In addition, we discuss the application of our results to three practical problems: utility maximization, entropic risk measures, and bond prices. Finally, as examples, explicit results for several market models such as the Cox–Ingersoll–Ross (CIR) model, 3/2 model and constant elasticity of variance (CEV) model are presented.

AN ERGODIC BSDE RISK REPRESENTATION IN A JUMP-DIFFUSION FRAMEWORK

We consider the representation of forward entropic risk measures using the theory of ergodic backward stochastic differential equations in a jump-diffusion framework. Our paper can be viewed as an extension of the work considered by Chong et al. (2019) in the diffusion case. We also study the behavior of a forward entropic risk measure under jumps when a financial position is held for a longer maturity.

Large deviations built on max-stability

In this paper, we show that the basic results in large deviations theory hold for general monetary risk measures, which satisfy the crucial property of max-stability. A max-stable monetary risk measure fulfills a lattice homomorphism property, and satisfies under a suitable tightness condition the Laplace Principle (LP), that is, admits a dual representation with affine convex conjugate. By replacing asymptotic concentration of probability by concentration of risk, we formulate a Large Deviation Principle (LDP) for max-stable monetary risk measures, and show its equivalence to the LP. In particular, the special case of the asymptotic entropic risk measure corresponds to the classical Varadhan–Bryc equivalence between the LDP and LP. The main results are illustrated by the asymptotic shortfall risk measure.

Bayes risk, elicitability, and the Expected Shortfall

AbstractMotivated by recent advances on elicitability of risk measures and practical considerations of risk optimization, we introduce the notions of Bayes pairs and Bayes risk measures. Bayes risk measures are the counterpart of elicitable risk measures, extensively studied in the recent literature. The Expected Shortfall (ES) is the most important coherent risk measure in both industry practice and academic research in finance, insurance, risk management, and engineering. One of our central results is that under a continuity condition, ES is the only class of coherent Bayes risk measures. We further show that entropic risk measures are the only risk measures which are both elicitable and Bayes. Several other theoretical properties and open questions on Bayes risk measures are discussed.

Markov decision processes with recursive risk measures

In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces and unbounded cost. We treat both finite and infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This is motivated by recursive utilities in the economic literature. It has been studied before for the entropic risk measure and is extended here to general static risk measures. Under direct assumptions on the model data we derive a Bellman equation and prove the existence of optimal Markov policies. For an infinite planning horizon, the model is shown to be contractive and the optimal policy to be stationary. Our approach unifies results for a number of well-known risk measures. Moreover, we establish a connection to distributionally robust MDPs, which provides a global interpretation of the recursively defined objective function. Monotone models are studied in particular.

Generalized entropic risk measures and related BSDEs

In this paper, we consider the generalized entropic risk measures and related BSDEs. We establish a variational representation for the generalized relative entropy. By the variational representation result, (conditional) generalized entropic risk measures are defined and their features are studied. We also build the relationship between conditional generalized entropic risk measures and a class of BSDEs.

Bayes Risk, Elicitability, and the Expected Shortfall

Motivated by recent advances on elicitability of risk measures and practical considerations of risk optimization, we introduce the notions of Bayes pairs and Bayes risk measures. Bayes risk measures are the counterpart of elicitable risk measures, extensively studied in the recent literature. The Expected Shortfall (ES) is the most important coherent risk measure in both industry practice and academic research in finance, insurance, risk management, and engineering. One of our central results is that under a continuity condition, ES is the only class of coherent Bayes risk measures. We further show that entropic risk measures are the only risk measures which are both elicitable and Bayes. Several other theoretical properties and open questions on Bayes risk measures are discussed.

A Critical Comparison of Three Notions of Fractional Stochastic Dominance

Two notions of fractional stochastic dominance are recently proposed by Muller et al. (2017) and Huang et al. (2020), respectively. Our main objective is to understand the comparative advantages of the two notions, as well as their suitability in different contexts, by establishing several new technical results. For a more comprehensive comparison, we further include a third natural notion of fractional stochastic dominance based on the coefficient of relative risk aversion. Among the three notions, it turns out that one can be seen as a logarithmic version of second-order stochastic dominance (SSD), another one can be seen as a power version of SSD, whereas there does not exist a transformation to associate the last one with SSD. We find that these notions of fractional stochastic dominance are naturally connected to five classes of risk measures, including Value-at-Risk, Expected Shortfall, expectiles, entropic risk measures, and loss certainty equivalents. The three notions are further characterized in the contexts of the rank-dependent utility model and the cumulative prospect theory. We make some recommendations on which notion to use in specific situations, as they all have their own merits.

Asymptotic behavior of estimators of entropic risk measures under asymmetric Laplace law

In this paper, we study several asymptotic behaviors of the estimators of convex and coherent entropic risk measures. First, the moderate deviation principles of the estimators are given. Second, the central limit theorems of the estimators are given. Finally, several simulation results are given to support our main conclusions.

The risk-averse ultimate pit problem

In this work, we consider a risk-averse ultimate pit problem where the grade of the mineral is uncertain. We derive conditions under which we can generate a set of nested pits by varying the risk level instead of using revenue factors. We propose two properties that we believe are desirable for the problem: risk nestedness, which means the pits generated for different risk aversion levels should be contained in one another, and additive consistency, which states that preferences in terms of order of extraction should not change if independent sectors of the mine are added as precedences. We show that only an entropic risk measure satisfies these properties and propose a two-stage stochastic programming formulation of the problem, including an efficient approximation scheme to solve it. We illustrate our approach in a small self-constructed example, and apply our approximation scheme to a real-world section of the Andina mine, in Chile.