State-dependent Importance Sampling Research Articles

State-dependent importance sampling for estimating expectations of functionals of sums of independent random variables

Estimating the expectations of functionals applied to sums of random variables (RVs) is a well-known problem encountered in many challenging applications. Generally, closed-form expressions of these quantities are out of reach. A naive Monte Carlo simulation is an alternative approach. However, this method requires numerous samples for rare event problems. Therefore, it is paramount to use variance reduction techniques to develop fast and efficient estimation methods. In this work, we use importance sampling (IS), known for its efficiency in requiring fewer computations to achieve the same accuracy requirements. We propose a state-dependent IS scheme based on a stochastic optimal control formulation, where the control is dependent on state and time. We aim to calculate rare event quantities that could be written as an expectation of a functional of the sums of independent RVs. The proposed algorithm is generic and can be applied without restrictions on the univariate distributions of RVs or the functional applied to the sum. We apply this approach to the log-normal distribution to compute the left tail and cumulative distribution of the ratio of independent RVs. For each case, we numerically demonstrate that the proposed state-dependent IS algorithm compares favorably to most well-known estimators dealing with similar problems.

Importance sampling of heavy-tailed iterated random functions

AbstractWe consider the stationary solutionZof the Markov chain {Zn}n∈ℕdefined byZn+1=ψn+1(Zn), where {ψn}n∈ℕis a sequence of independent and identically distributed random Lipschitz functions. We estimate the probability of the event {Z>x} whenxis large, and develop a state-dependent importance sampling estimator under a set of assumptions on ψnsuch that, for largex, the event {Z>x} is governed by a single large jump. Under natural conditions, we show that our estimator is strongly efficient. Special attention is paid to a class of perpetuities with heavy tails.

Automated State-Dependent Importance Sampling for Markov Jump Processes via Sampling from the Zero-Variance Distribution

Many complex systems can be modeled via Markov jump processes. Applications include chemical reactions, population dynamics, and telecommunication networks. Rare-event estimation for such models can be difficult and is often computationally expensive, because typically many (or very long) paths of the Markov jump process need to be simulated in order to observe the rare event. We present a state-dependent importance sampling approach to this problem that is adaptive and uses Markov chain Monte Carlo to sample from the zero-variance importance sampling distribution. The method is applicable to a wide range of Markov jump processes and achieves high accuracy, while requiring only a small sample to obtain the importance parameters. We demonstrate its efficiency through benchmark examples in queueing theory and stochastic chemical kinetics.

Automated State-Dependent Importance Sampling for Markov Jump Processes via Sampling from the Zero-Variance Distribution

Many complex systems can be modeled via Markov jump processes. Applications include chemical reactions, population dynamics, and telecommunication networks. Rare-event estimation for such models can be difficult and is often computationally expensive, because typically many (or very long) paths of the Markov jump process need to be simulated in order to observe the rare event. We present a state-dependent importance sampling approach to this problem that is adaptive and uses Markov chain Monte Carlo to sample from the zero-variance importance sampling distribution. The method is applicable to a wide range of Markov jump processes and achieves high accuracy, while requiring only a small sample to obtain the importance parameters. We demonstrate its efficiency through benchmark examples in queueing theory and stochastic chemical kinetics.

On Lyapunov Inequalities and Subsolutions for Efficient Importance Sampling

In this article we explain some connections between Lyapunov methods and subsolutions of an associated Isaacs equation for the design of efficient importance sampling schemes. As we shall see, subsolutions can be derived by taking an appropriate limit of an associated Lyapunov inequality. They have been recently proposed in several works of Dupuis, Wang, and others and applied to address several important problems in rare-event simulation. Lyapunov inequalities have been used for testing the efficiency of state-dependent importance sampling schemes in heavy-tailed or discrete settings in a variety of works by Blanchet, Glynn, and others. While subsolutions provide an analytic criterion for the construction of efficient samplers, Lyapunov inequalities are useful for finding more precise information, in the form of bounds, for the behavior of the coefficient of variation of the associated importance sampling estimator in the prelimit. In addition, Lyapunov inequalities provide insight into the various mollification procedures that are often required in constructing associated subsolutions. Our aim is to demonstrate that applying Lyapunov inequalities for verification of efficiency can help both guide the selection of various mollification parameters and sharpen the information on the efficiency gain induced by the sampler.

Efficient rare-event simulation for perpetuities

We consider perpetuities of the form D=B1exp(Y1)+B2exp(Y1+Y2)+⋯, where the Yj’s and Bj’s might be i.i.d. or jointly driven by a suitable Markov chain. We assume that the Yj’s satisfy the so-called Cramér condition with associated root θ∗∈(0,∞) and that the tails of the Bj’s are appropriately behaved so that D is regularly varying with index θ∗. We illustrate by means of an example that the natural state-independent importance sampling estimator obtained by exponentially tilting the Yj’s according to θ∗ fails to provide an efficient estimator (in the sense of appropriately controlling the relative mean squared error as the tail probability of interest gets smaller). Then, we construct estimators based on state-dependent importance sampling that are rigorously shown to be efficient.

On importance sampling with mixtures for random walks with heavy tails

State-dependent importance sampling algorithms based on mixtures are considered. The algorithms are designed to compute tail probabilities of a heavy-tailed random walk. The increments of the random walk are assumed to have a regularly varying distribution. Sufficient conditions for obtaining bounded relative error are presented for rather general mixture algorithms. Two new examples, called the generalized Pareto mixture and the scaling mixture, are introduced. Both examples have good asymptotic properties and, in contrast to some of the existing algorithms, they are very easy to implement. Their performance is illustrated by numerical experiments. Finally, it is proved that mixture algorithms of this kind can be designed to have vanishing relative error.

State-dependent doubly weighted stochastic simulation algorithm for automatic characterization of stochastic biochemical rare events

In recent years there has been substantial growth in the development of algorithms for characterizing rare events in stochastic biochemical systems. Two such algorithms, the state-dependent weighted stochastic simulation algorithm (swSSA) and the doubly weighted SSA (dwSSA) are extensions of the weighted SSA (wSSA) by H. Kuwahara and I. Mura [J. Chem. Phys. 129, 165101 (2008)]. The swSSA substantially reduces estimator variance by implementing system state-dependent importance sampling (IS) parameters, but lacks an automatic parameter identification strategy. In contrast, the dwSSA provides for the automatic determination of state-independent IS parameters, thus it is inefficient for systems whose states vary widely in time. We present a novel modification of the dwSSA--the state-dependent doubly weighted SSA (sdwSSA)--that combines the strengths of the swSSA and the dwSSA without inheriting their weaknesses. The sdwSSA automatically computes state-dependent IS parameters via the multilevel cross-entropy method. We apply the method to three examples: a reversible isomerization process, a yeast polarization model, and a lac operon model. Our results demonstrate that the sdwSSA offers substantial improvements over previous methods in terms of both accuracy and efficiency.

Efficient rare event simulation for heavy-tailed compound sums

We develop an efficient importance sampling algorithm for estimating the tail distribution of heavy-tailed compound sums, that is, random variables of the form S M = Z 1 +…+ Z M where the Z i 's are independently and identically distributed (i.i.d.) random variables in R and M is a nonnegative, integer-valued random variable independent of the Z i 's. We construct the first estimator that can be rigorously shown to be strongly efficient only under the assumption that the Z i 's are subexponential and M is light-tailed. Our estimator is based on state-dependent importance sampling and we use Lyapunov-type inequalities to control its second moment. The performance of our estimator is empirically illustrated in various instances involving popular heavy-tailed models.

The cross-entropy method with patching for rare-event simulation of large Markov chains

There are various importance sampling schemes to estimate rare event probabilities in Markovian systems such as Markovian reliability models and Jackson networks. In this work, we present a general state-dependent importance sampling method which partitions the state space and applies the cross-entropy method to each partition. We investigate two versions of our algorithm and apply them to several examples of reliability and queueing models. In all these examples we compare our method with other importance sampling schemes. The performance of the importance sampling schemes is measured by the relative error of the estimator and by the efficiency of the algorithm. The results from experiments show considerable improvements both in running time of the algorithm and the variance of the estimator.

Efficient importance sampling in ruin problems for multidimensional regularly varying random walks

We consider the problem of efficient estimation via simulation of first passage time probabilities for a multidimensional random walk with heavy-tailed increments. In addition to being a natural generalization to the problem of computing ruin probabilities in insurance - in which the focus is the maximum of a one-dimensional random walk with negative drift - this problem captures important features of large deviations for multidimensional heavy-tailed processes (such as the role played by the mean of the process in connection to the location of the target set). We develop a state-dependent importance sampling estimator for this class of multidimensional problems. Then, using techniques based on Lyapunov inequalities, we argue that our estimator is strongly efficient in the sense that therelativemean squared error of our estimator can be made arbitrarily small by increasing the number of replications, uniformly as the probability of interest approaches 0.

Efficient importance sampling in ruin problems for multidimensional regularly varying random walks

We consider the problem of efficient estimation via simulation of first passage time probabilities for a multidimensional random walk with heavy-tailed increments. In addition to being a natural generalization to the problem of computing ruin probabilities in insurance - in which the focus is the maximum of a one-dimensional random walk with negative drift - this problem captures important features of large deviations for multidimensional heavy-tailed processes (such as the role played by the mean of the process in connection to the location of the target set). We develop a state-dependent importance sampling estimator for this class of multidimensional problems. Then, using techniques based on Lyapunov inequalities, we argue that our estimator is strongly efficient in the sense that the relative mean squared error of our estimator can be made arbitrarily small by increasing the number of replications, uniformly as the probability of interest approaches 0.

Asymptotic robustness of estimators in rare-event simulation

The asymptotic robustness of estimators as a function of a rarity parameter, in the context of rare-event simulation, is often qualified by properties such as bounded relative error (BRE) and logarithmic efficiency (LE), also called asymptotic optimality. However, these properties do not suffice to ensure that moments of order higher than one are well estimated. For example, they do not guarantee that the variance of the empirical variance remains under control as a function of the rarity parameter. We study generalizations of the BRE and LE properties that take care of this limitation. They are named bounded relative moment of order k (BRM- k ) and logarithmic efficiency of order k (LE- k ), where k ≥ 1 is an arbitrary real number. We also introduce and examine a stronger notion called vanishing relative centered moment of order k , and exhibit examples where it holds. These properties are of interest for various estimators, including the empirical mean and the empirical variance. We develop (sufficient) Lyapunov-type conditions for these properties in a setting where state-dependent importance sampling (IS) is used to estimate first-passage time probabilities. We show how these conditions can guide us in the design of good IS schemes, that enjoy convenient asymptotic robustness properties, in the context of random walks with light-tailed and heavy-tailed increments. As another illustration, we study the hierarchy between these robustness properties (and a few others) for a model of highly reliable Markovian system (HRMS) where the goal is to estimate the failure probability of the system. In this setting, for a popular class of IS schemes, we show that BRM- k and LE- k are equivalent and that these properties become strictly stronger when k increases. We also obtain a necessary and sufficient condition for BRM- k in terms of quantities that can be readily computed from the parameters of the model.

The Cross-Entropy Method with Patching for Rare-Event Simulation of Large Markov Chains

There are various importance sampling schemes to estimate rare event probabilities in Markovian systems such as Markovian reliability models and Jackson networks. In this work, we present a general state dependent importance sampling method which partitions the state space and applies the cross-entropy method to each partition. We investigate two versions of our algorithm and apply them to several examples of reliability and queueing models. In all these examples we compare our method with other importance sampling schemes. The performance of the importance sampling schemes is measured by the relative error of the estimator and by the efficiency of the algorithm. The results from experiments show considerable improvements both in running time of the algorithm and the variance of the estimator.

State-dependent importance sampling schemes via minimum cross-entropy

We present a method to obtain state- and time-dependent importance sampling estimators by repeatedly solving a minimum cross-entropy (MCE) program as the simulation progresses. This MCE-based approach lends a foundation to the natural notion to stop changing the measure when it is no longer needed. We use this method to obtain a state- and time-dependent estimator for the one-tailed probability of a light-tailed i.i.d. sum that is logarithmically efficient in general and strongly efficient when the jumps are Gaussian. We go on to construct an estimator for the two-tailed problem which is shown to be similarly efficient. We consider minor variants of the algorithm obtained via MCE, and present some numerical comparisons between our algorithms and others from the literature.

Importance sampling for Jackson networks

Rare event simulation in the context of queueing networks has been an active area of research for more than two decades. A commonly used technique to increase the efficiency of Monte Carlo simulation is importance sampling. However, there are few rigorous results on the design of efficient or asymptotically optimal importance sampling schemes for queueing networks. Using a recently developed game/subsolution approach, we construct simple and efficient state-dependent importance sampling schemes for simulating buffer overflows in stable open Jackson networks. The sampling distributions do not depend on the particular event of interest, and hence overflow probabilities for different events can be estimated simultaneously. A by-product of the analysis is the identification of the minimizing trajectory for the calculus of variation problem that is associated with the sample-path large deviation rate function.

State-dependent importance sampling for regularly varying random walks

Consider a sequence (Xk:k≥ 0) of regularly varying independent and identically distributed random variables with mean 0 and finite variance. We develop efficient rare-event simulation methodology associated with large deviation probabilities for the random walk (Sn:n≥ 0). Our techniques are illustrated by examples, including large deviations for the empirical mean and path-dependent events. In particular, we describe two efficient state-dependent importance sampling algorithms for estimating the tail ofSnin a large deviation regime asn↗ ∞. The first algorithm takes advantage of large deviation approximations that are used to mimic the zero-variance change of measure. The second algorithm uses a parametric family of changes of measure based on mixtures. Lyapunov-type inequalities are used to appropriately select the mixture parameters in order to guarantee bounded relative error (or efficiency) of the estimator. The second example involves a path-dependent event related to a so-called knock-in financial option under heavy-tailed log returns. Again, the importance sampling algorithm is based on a parametric family of mixtures which is selected using Lyapunov bounds. In addition to the theoretical analysis of the algorithms, numerical experiments are provided in order to test their empirical performance.

State-dependent importance sampling for regularly varying random walks

Consider a sequence (X k : k ≥ 0) of regularly varying independent and identically distributed random variables with mean 0 and finite variance. We develop efficient rare-event simulation methodology associated with large deviation probabilities for the random walk (S n : n ≥ 0). Our techniques are illustrated by examples, including large deviations for the empirical mean and path-dependent events. In particular, we describe two efficient state-dependent importance sampling algorithms for estimating the tail of S n in a large deviation regime as n ↗ ∞. The first algorithm takes advantage of large deviation approximations that are used to mimic the zero-variance change of measure. The second algorithm uses a parametric family of changes of measure based on mixtures. Lyapunov-type inequalities are used to appropriately select the mixture parameters in order to guarantee bounded relative error (or efficiency) of the estimator. The second example involves a path-dependent event related to a so-called knock-in financial option under heavy-tailed log returns. Again, the importance sampling algorithm is based on a parametric family of mixtures which is selected using Lyapunov bounds. In addition to the theoretical analysis of the algorithms, numerical experiments are provided in order to test their empirical performance.

Improving Adaptive Importance Sampling Simulation of Markovian Queueing Models using Non-parametric Smoothing

Previous work on state-dependent adaptive importance sampling techniques for the simulation of rare events in Markovian queueing models used either no smoothing or a parametric smoothing technique, which was known to be non-optimal. In this paper, we introduce the use of kernel smoothing in this conteXt. We derive eXpressions for the smoothed transition probabilities, compare several variations of the technique, and eXplore the choice of kernel width. We provide some eXamples, demonstrating that the technique significantly improves convergence and estimator variance.

Fluid heuristics, Lyapunov bounds and efficient importance sampling for a heavy-tailed G/G/1 queue

We develop a strongly efficient rare-event simulation algorithm for computing the tail of the steady-state waiting time in a single server queue with regularly varying service times. Our algorithm is based on a state-dependent importance sampling strategy that is constructed so as to be straightforward to implement. The construction of the algorithm and its asymptotic optimality rely on a Lyapunov-type inequality that is used to bound the second moment of the estimator. The solution to the Lyapunov inequality is constructed using fluid heuristics. Our approach takes advantage of the regenerative ratio formula for the steady-state distribution--and does not use the first passage time representation that is particular to the delay in the G/G/1 queue. Hence, the strategy has the potential to be applied in more general queueing models.