Measure-valued Differentiation Research Articles

Monte Carlo Sensitivities Using the Absolute Measure-Valued Derivative Method

Measure-valued differentiation (MVD) is a relatively new method for computing Monte Carlo sensitivities, relying on a decomposition of the derivative of transition densities of the underlying process into a linear combination of probability measures. In computing the sensitivities, additional paths are generated for each constituent distribution and the payoffs from these paths are combined to produce sample estimates. The method generally produces sensitivity estimates with lower variance than the finite difference and likelihood ratio methods, and can be applied to discontinuous payoffs in contrast to the pathwise differentiation method. However, these benefits come at the expense of an additional computational burden. In this paper, we propose an alternative approach, called the absolute measure-valued differentiation (AMVD) method, which expresses the derivative of the transition density at each simulation step as a single density rather than a linear combination. It is computationally more efficient than the MVD method and can result in sensitivity estimates with lower variance. Analytic and numerical examples are provided to compare the variance in the sensitivity estimates of the AMVD method against alternative methods.

A simultaneous perturbation weak derivative estimator for stochastic neural networks

In this paper we study gradient estimation for a network of nonlinear stochastic units known as the Little model. Many machine learning systems can be described as networks of homogeneous units, and the Little model is of a particularly general form, which includes as special cases several popular machine learning architectures. However, since a closed form solution for the stationary distribution is not known, gradient methods which work for similar models such as the Boltzmann machine or sigmoid belief network cannot be used. To address this we introduce a method to calculate derivatives for this system based on measure-valued differentiation and simultaneous perturbation. This extends previous works in which gradient estimation algorithm’s were presented for networks with restrictive features like symmetry or acyclic connectivity.

A Measure-Valued Differentiation Approach to Sensitivities of Quantiles

Quantiles play an important role in modelling quality of service in the service industry and in modelling risk in the financial industry. The recent discovery that efficient simulation-based estimators can be obtained for quantile sensitivities has led to an intensive search for sample-path differentiation-based estimators for quantile sensitivities. In this paper, we present a novel approach to quantile sensitivity estimation. Our approach elaborates on the concept of measure-valued differentiation. Thereby, we overcome the main obstacle of the sample-path approach, which is the requirement that the sample cost have to be Lipschitz continuous with respect to the parameter of interest. Specifically, we perform a sensitivity analysis of the value at risk in financial models. In addition, we discuss an application of our sensitivity estimator to queueing networks.

Efficient calculation of the Greeks for exponential Lévy processes: an application of measure valued differentiation

Monte Carlo simulation methods have become more and more important in the financial sector in the past years. In this paper, we introduce a new simulation method for the estimation of the derivatives of prices of financial contracts with respect to (w.r.t.) certain distributional parameters called the ‘Greeks’. In particular, we assume that the underlying financial process is a Lévy-type process in discrete time. Our method is based on the Measure-Valued Differentiation (MVD) approach, which allows representation of derivatives as differences of two processes, called the phantoms. We discuss the applicability of MVD for different types of option pay-offs in combination with different types of models of the underlying and provide a framework for the applicability of MVD for path-dependent pay-off functions, as Lookback Options or Asian Options.

Gradient Estimation for Multicomponent Maintenance Systems with Age-Replacement Policy

We consider multicomponent maintenance systems with an F-failure group age-replacement policy: it keeps failed components idling until F components are failed and then replaces all failed components together with the nonfailed components whose age has passed the critical threshold age θn for components of type n. With each maintenance action, costs are associated. We derive various unbiased gradient estimators based on the measure-valued differentiation approach for the gradient of the average cost. Each estimator has its own domain of applicability. We also compare the performance of our gradient estimators when applied to stochastic optimization with other general gradient-free methods.

Gradient estimation for discrete-event systems by measure-valued differentiation

In simulation of complex stochastic systems, such as Discrete-Event Systems (DES), statistical distributions are used to model the underlying randomness in the system. A sensitivity analysis of the simulation output with respect to parameters of the input distributions, such as the mean and the variance, is therefore of great value. The focus of this article is to provide a practical guide for robust sensitivity, respectively, gradient estimation that can be easily implemented along the simulation of a DES. We study the Measure-Valued Differentiation (MVD) approach to sensitivity estimation. Specifically, we will exploit the “modular” structure of the MVD approach, by firstly providing measure-valued derivatives for input distributions that are of importance in practice, and subsequently, by showing that if an input distribution possesses a measure-valued derivative, then so does the overall Markov kernel modeling the system transitions. This simplifies the complexity of applying MVD drastically: one only has to study the measure-valued derivative of the input distribution, a measure-valued derivative of the associated Markov kernel is then given through a simple formula in canonical form. The derivative representations of the underlying simple distributions derived in this article can be stored in a computer library. Combined with the generic MVD estimator, this yields an automated gradient estimation procedure. The challenge in automating MVD so that it can be included into a simulation package is the verification of the integrability condition to guarantee that the estimators are unbiased. The key contribution of the article is that we establish a general condition for unbiasedness which is easily checked in applications. Gradient estimators obtained by MVD are typically phantom estimators and we discuss the numerical efficiency of phantom estimators with the example of waiting times in the G/G/1 queue.

Gradient estimation for a class of systems with bulk services

This article presents a comparison of different gradient estimators for the sensitivity of waiting times in a bulk server system. Inspired by a transportation network, our model is that of a bursty arrival process that waits at a “platform” until the server is available (representing a train or bus ready for departure). At the departure epochs, all waiting passengers leave at once. The departure process is assumed to be a renewal process and, based on a limiting result, the interdeparture times are approximated by truncated normal random variables. The interarrival times are assumed to be identically and independently distributed (i.i.d.), with a general distribution of bounded density. We are interested in calculating the sensitivities of the total cumulative waiting time of all passengers with respect to the interdeparture times. For this general model where neither the interarrival times nor the interdeparture times are exponential, there is no analytical formula available. However, the estimation of such sensitivities is an important problem for flow control in such networks. We establish a Smoothed Perturbation Analysis (SPA), a Measure-Valued Differentiation (MVD), and a Score Function (SF) estimator, including numerical experiments.

A Note on Gradient Estimation for Maintenance Systems

This technical note addresses gradient estimation for the cost performance of a two-component maintenance system with respect to the threshold parameter of an age replacement policy. We derive a new gradient estimator based on the measure-valued differentiation (MVD) approach. The performance of the phantom estimator is compared with that of the known smoothed perturbation analysis (SPA) estimator. We show that the phantom estimator has a lower variance, and requires less computational effort than the SPA estimator.

Measure-Valued Differentiation for Markov Chains

This paper addresses the problem of sensitivity analysis for finite-horizon performance measures of general Markov chains. We derive closed-form expressions and associated unbiased gradient estimators for the derivatives of finite products of Markov kernels by measure-valued differentiation (MVD). In the MVD setting, the derivatives of Markov kernels, called $\mathcal{D}$ -derivatives, are defined with respect to a class of performance functions $\mathcal{D}$ such that, for any performance measure $g\in\mathcal{D}$ , the derivative of the integral of g with respect to the one-step transition probability of the Markov chain exists. The MVD approach (i) yields results that can be applied to performance functions out of a predefined class, (ii) allows for a product rule of differentiation, that is, analyzing the derivative of the transition kernel immediately yields finite-horizon results, (iii) provides an operator language approach to the differentiation of Markov chains and (iv) clearly identifies the trade-off between the generality of the performance classes that can be analyzed and the generality of the classes of measures (Markov kernels). The $\mathcal{D}$ -derivative of a measure can be interpreted in terms of various (unbiased) gradient estimators and the product rule for $\mathcal {D}$ -differentiation yields a product-rule for various gradient estimators.

Measure-Valued Differentiation for Stationary Markov Chains

We study general state-space Markov chains that depend on a parameter, say, θ. Sufficient conditions are established for the stationary performance of such a Markov chain to be differentiable with respect to θ. Specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of Markov chains to unbounded mappings. First, a closed-form formula for the derivative of the stationary performance of a general state-space Markov chain is given using an operator-theoretic approach. In a second step, we translate the derivative formula into unbiased gradient estimators. Specifically, we establish phantom-type estimators and score function estimators. We illustrate our results with examples from queueing theory.

Single-Run Gradient Estimation Via Measure-Valued Differentiation

We show how single-run-based measure-valued differentiation gradient estimators can be obtained. The key idea is to apply a change of measure a posterior to the mathematical analysis of the derivative. From the point of view of the likelihood ratio method, we show that likelihood ratio type gradient estimators can be applied in situations where the mathematical conditions needed for applying a likelihood ratio analysis are not met.

Gradient Estimation for a Class of Systems with Bulk Services: A Problem in Public Transportation

This paper deals with a system where batch arrivals wait in a station until a server (a train) is available, at which moment it services all customers in waiting. This is an example of a bulk server, which has many applications in public transportation, telecommunications, computer resource allocation, and multiple access telecommuncation networks, among others. We consider a subway model and focus on a metro line serving a particular metro station. Denote the planned inter-departure time of this line by x. The metro station is served by several other lines and passengers change trains at the station. Traveling times of trains are assumed to be given by fixed times and an additional stochastic noise. We perform a sensitivity analysis of the total delay of passengers waiting for the x line with respect to x. We establish a smoothed perturbation analysis (SPA), a measure-valued differentiation (MVD), and a score function (SF) estimator. Numerical experiments are performed to compare the ensuing estimators. It turns out that the SPA and MVD estimators are intrinsically different and the model presented in this paper may serve as a counter-example to the widespread belief that SPA and MVD yield similar estimators.