Large Deviation Analysis Research Articles

Large Deviations Analysis for Distributed Algorithms in an Ergodic Markovian Environment

We provide a large deviations analysis of deadlock phenomena occurring in distributed systems sharing common resources. In our model transition probabilities of resource allocation and deallocation are time and space dependent. The process is driven by an ergodic Markov chain and is reflected on the boundary of the d-dimensional cube. In the large resource limit, we prove Freidlin-Wentzell estimates, we study the asymptotic of the deadlock time and we show that the quasi-potential is a viscosity solution of a Hamilton-Jacobi equation with a Neumann boundary condition. We give a complete analysis of the colliding 2-stacks problem and show an example where the system has a stable attractor which is a limit cycle.

Losses in Investment-Grade Tranches of Synthetic CDO's: A Large Deviations Analysis

Rare events are an important part of the financial world. For instance, investment-grade financial products by design should suffer losses only rarely. In many cases, this is accomplished by pooling a large number of assets and trancheing the losses. This often leads to a very complex system. We the theory of large deviations, a collection of tools for studying the origination and transformations of rare events, to address some of these issues in losses in senior tranches of synthetic collateralized debt obligations, which in some sense is an archetype of structured finance. Our calculations are fairly self-contained.

Large deviations for infinite dimensional stochastic dynamical systems

The large deviations analysis of solutions to stochastic differential equations and related processes is often based on approximation. The construction and justification of the approximations can be onerous, especially in the case where the process state is infinite dimensional. In this paper we show how such approximations can be avoided for a variety of infinite dimensional models driven by some form of Brownian noise. The approach is based on a variational representation for functionals of Brownian motion. Proofs of large deviations properties are reduced to demonstrating basic qualitative properties (existence, uniqueness and tightness) of certain perturbations of the original process.

Large-Deviation Approximations for General Occupancy Models

We obtain large-deviation approximations for the empirical distribution for a general family of occupancy problems. In the general setting, balls are allowed to fall in a given urn depending on the urn's contents prior to the throw. We discuss a parametric family of statistical models that includes Maxwell–Boltzmann, Bose–Einstein and Fermi–Dirac statistics as special cases. A process-level large-deviation analysis is conducted and the rate function for the original problem is then characterized, via the contraction principle, by the solution to a calculus of variations problem. The solution to this variational problem is shown to coincide with that of a simple finite-dimensional minimization problem. As a consequence, the large-deviation approximations and related qualitative information are available in more-or-less explicit form.

Large Deviations Analysis of Extinction in Branching Models

Cramer's classical theorem is applied to obtain large deviations in branching processes. This is a new avenue for analysis of models in discrete and continuous time. For the Galton-Watson process a new formula for the rate function in terms of the Legendre transform of its offspring distribution is derived. Further analysis of the approximate path to extinction produces a new interesting formula.

Dynamic importance sampling for queueing networks

Abstract : Importance sampling is a technique that is commonly used to speed up Monte Carlo simulation of rare events. However, little is known regarding the design of efficient importance sampling algorithms in the context of queueing networks. The standard approach, which simulates the system using an a priori fixed change of measure suggested by large deviation analysis, has been shown to fail in even the simplest network setting (e.g., a two-node tandem network). Exploiting connections between importance sampling, differential games, and classical subsolutions of the corresponding Isaacs equation, we show how to design and analyze simple and efficient dynamic importance sampling schemes for general classes of networks. The models used to illustrate the approach include d-node tandem Jackson networks and a two node network with feedback, and the rare events studied are those of large queueing backlogs, including total population overflow and the overflow of individual buffers.

A fluid system with coupled input and output, and its application to bottlenecks in ad hoc networks

This paper studies a fluid queue with coupled input and output. Flows arrive according to a Poisson process, and when n flows are present, each of them transmits traffic into the queue at a rate c/(n+1), where the remaining c/(n+1) is used to serve the queue. We assume exponentially distributed flow sizes, so that the queue under consideration can be regarded as a system with Markov fluid input. The rationale behind studying this queue lies in ad hoc networks: bottleneck links have roughly this type of sharing policy. We consider four performance metrics: (i) the stationary workload of the queue, (ii) the queueing delay, i.e., the delay of a ‘packet’ (a fluid particle) that arrives at the queue at an arbitrary point in time, (iii) the flow transfer delay, i.e., the time elapsed between arrival of a flow and the epoch that all its traffic has been put into the queue, and (iv) the sojourn time, i.e., the flow transfer time increased by the time it takes before the last fluid particle of the flow is served. For each of these random variables we compute the Laplace transform. The corresponding tail probabilities decay exponentially, as is shown by a large-deviations analysis.

Crystallization in Large Wireless Networks

We analyze fading interference relay networks where M single-antenna source-destination terminal pairs communicate concurrently and in the same frequency band through a set of K single-antenna relays using half-duplex two-hop relaying. Assuming that the relays have channel state information (CSI), it is shown that in the large-M limit, provided K grows fast enough as a function of M, the network "decouples" in the sense that the individual source-destination terminal pair capacities are strictly positive. The corresponding required rate of growth of K as a function of M is found to be sufficient to also make the individual source-destination fading links converge to nonfading links. We say that the network "crystallizes" as it breaks up into a set of effectively isolated "wires in the air". A large-deviations analysis is performed to characterize the "crystallization" rate, i.e., the rate (as a function of M,K) at which the decoupled links converge to nonfading links. In the course of this analysis, we develop a new technique for characterizing the large-deviations behavior of certain sums of dependent random variables. For the case of no CSI at the relay level, assuming amplify-and-forward relaying, we compute the per source-destination terminal pair capacity for M,K converging to infinity, with K/M staying fixed, using tools from large random matrix theory.

A Large Deviations Analysis of Scheduling in Wireless Networks

In this correspondence, we consider a cellular network consisting of a base station and N receivers. The channel states of the receivers are assumed to be identical and independent of each other. The goal is to compare the throughput of two different scheduling policies (a queue-length-based (QLB) policy and a greedy policy) given an upper bound on the queue overflow probability or the delay violation probability. We consider a multistate channel model, where each channel is assumed to be in one of L states. Given an upper bound on the queue overflow probability or an upper bound on the delay violation probability, we show that the total network throughput of the (QLB) policy is no less than the throughput of the greedy policy for all N. We also obtain a lower bound on the throughput of the (QLB) policy. For sufficiently large N, the lower bound is shown to be tight, strictly increasing with N, and strictly larger than the throughput of the greedy policy. Further, for a simple multistate channel model-ON-OFF channel, we prove that the lower bound is tight for all N

A large-deviations analysis of the GI/GI/1 SRPT queue

We consider a GI/GI/1 queue with the shortest remaining processing time discipline (SRPT) and light-tailed service times. Our interest is focused on the tail behavior of the sojourn-time distribution. We obtain a general expression for its large-deviations decay rate. The value of this decay rate critically depends on whether there is mass in the endpoint of the service-time distribution or not. An auxiliary priority queue, for which we obtain some new results, plays an important role in our analysis. We apply our SRPT results to compare SRPT with FIFO from a large-deviations point of view.

Large deviations of sojourn times in processor sharing queues

This paper presents a large deviation analysis of the steady-state sojourn time distribution in the GI/G/1 PS queue. Logarithmic estimates are obtained under the assumption of the service time distribution having a light tail, thus supplementing recent results for the heavy-tailed setting. Our proof gives insight into the way a large sojourn time occurs, enabling the construction of an (asymptotically efficient) importance sampling algorithm. Finally our results for PS are compared to a number of other service disciplines, such as FCFS, LCFS, and SRPT.

Power-law distributions in random multiplicative processes with non-Gaussian colored multipliers

One class of universal mechanisms that generate power-law probability distributions is that of random multiplicative processes. In this paper, we consider a multiplicative Langevin equation driven by non-Gaussian colored multipliers. We analytically derive a formula that relates the power-law exponent to the statistics of the multipliers and numerically confirm its validity using multiplicative noise generated by chaotic dynamical systems and by a two-valued Markov process. We also investigate the relationship between our treatment and the large deviation analysis of time series, and demonstrate the appearance of log-periodic fluctuations superimposed on the power-law distribution due to the non-Gaussian nature of the multipliers.

An analytical probabilistic approach to the ground state of lattice quantum systems: exactresults in terms of a cumulant expansion

We present a large deviation analysis of a recently proposed probabilistic approach to thestudy of the ground-state properties of lattice quantum systems. The ground-state energyas well as the correlation functions in the ground state are exactly determinedas series expansions in the cumulants of the multiplicities of the potential andhopping energies assumed by the system during its long-time evolution. Once thesecumulants are known, even at a finite order, our approach provides the groundstate analytically as a function of the Hamiltonian parameters. A scenario ofpossible applications of the analytic character of the present approach is discussed.

MDP for integral functionals of fast and slow processes with averaging

We establish the moderate deviation principle (MDP) for the family of X t ɛ = 1 ɛ κ ∫ 0 t H ( ξ s ɛ , Y s ɛ ) d s , ɛ ↓ 0 , where 0 < κ < 0.5 and ( ξ t ɛ , Y t ɛ ) are slow and fast diffusion processes. We embed the original problem in the MDP study for the pair ( X t ɛ , Y t ɛ ) . The main tool for the MDP analysis is the Poisson equation technique, borrowed from the recent papers of Pardoux and Veretennikov, (Ann. Probab. 29 (3) (2001) 1061; Ann. Probab. 31 (3) (2003) 1166), and a new approach to the large deviation analysis, proposed by Puhalskii, (Large Deviations and Idempotent Probability, 2001), which exploits “fast homogenization” of the drift and diffusion parameters instead of the traditional Laplace transform technique. The obtained MDP for ( X t ɛ , Y t ɛ ) has a typical structure of the Freidlin–Wentzell-type large deviation principle.

Bridges and networks: Exact asymptotics

We extend the Markov additive methodology developed in [Ann. Appl. Probab. 9 (1999) 110–145, Ann. Appl. Probab. 11 (2001) 596–607] to obtain the sharp asymptotics of the steady state probability of a queueing network when one of the nodes gets large. We focus on a new phenomenon we call a bridge. The bridge cases occur when the Markovian part of the twisted Markov additive process is one null recurrent or one transient, while the jitter cases treated in [Ann. Appl. Probab. 9 (1999) 110–145, Ann. Appl. Probab. 11 (2001) 596–607] occur when the Markovian part is (one) positive recurrent. The asymptotics of the steady state is an exponential times a polynomial term in the bridge case, but is purely exponential in the jitter case. We apply this theory to a modified, stable, two node Jackson network where server two helps server one when server two is idle. We derive the sharp asymptotics of the steady state distribution of the number of customers queued at each node as the number of customers queued at the server one grows large. In so doing we get an intuitive understanding of the companion paper [Ann. Appl. Probab. 15 (2005) 519–541] which gives a large deviation analysis of this problem using the flat boundary theory in the book by Shwartz and Weiss. Unlike the (unscaled) large deviation path of a Jackson network which jitters along the boundary, the unscaled large deviation path of the modified network tries to avoid the boundary where server two helps server one (and forms a bridge). In the fluid limit this bridge does collapse to a straight line, but the proportion of time spent on the flat boundary tends to zero. This bridge phenomenon is ubiquitous. We also treated the bathroom problem described in the Shwartz and Weiss book and found the bridge case is present. Here we derive the sharp asymptotics of the steady state of the bridge case and we obtain the results consistent with those obtained in [SIAM J. Appl. Math. (1984) 44 1041–1053] using complex variable methods.

Restarts and Exponential Acceleration of the Davis?Putnam?Loveland?Logemann Algorithm: A Large Deviation Analysis of the Generalized Unit Clause Heuristic for Random 3-SAT

An analysis of the hardness of resolution of random 3-SAT instances using the Davis–Putnam–Loveland–Logemann (DPLL) algorithm slightly below threshold is presented. While finding a solution for such instances demands exponential effort with high probability, we show that an exponentially small fraction of resolutions require a computation scaling linearly in the size of the instance only. We compute analytically this exponentially small probability of easy resolutions from a large deviation analysis of DPLL with the Generalized Unit Clause search heuristic, and show that the corresponding exponent is smaller (in absolute value) than the growth exponent of the typical resolution time. Our study therefore gives some quantitative basis to heuristic restart solving procedures, and suggests a natural cut-off cost (the size of the instance) for the restart.

Conditional limit theorems for queues with Gaussian input, a weak convergence approach

We consider a buffered queueing system that is fed by a Gaussian source and drained at a constant rate. The fluid offered to the system in a time interval ( 0 , t ] is given by a separable continuous Gaussian process Y with stationary increments. The variance function σ 2 : t ↦ V ar Y t of Y is assumed to be regularly varying with index 2 H , for some 0 < H < 1 . By proving conditional limit theorems, we investigate how a high buffer level is typically achieved. The underlying large deviation analysis also enables us to establish the logarithmic asymptotics for the probability that the buffer content exceeds u as u → ∞ . In addition, we study how a busy period longer than T typically occurs as T → ∞ , and we find the logarithmic asymptotics for the probability of such a long busy period. The study relies on the weak convergence in an appropriate space of { Y α t / σ ( α ) : t ∈ R } to a fractional Brownian motion with Hurst parameter H as α → ∞ . We prove this weak convergence under a fairly general condition on σ 2 , sharpening recent results of Kozachenko et al. (Queueing Systems Theory Appl. 42 (2002) 113). The core of the proof consists of a new type of uniform convergence theorem for regularly varying functions with positive index.

Importance Sampling, Large Deviations, and Differential Games

A standard heuristic in the area of importance sampling is that the changes of measure used to prove large deviation lower bounds give good performance when used for importance sampling. Recent work, however, has shown that a naive implementation of the heuristic is incorrect in many situations. The present paper considers the simple setting of sums of independent and identically distributed (iid) random variables, and shows that under mild conditions asymptotically optimal changes of measure can be found if one considers dynamic importance sampling schemes. For such schemes, the change of measure applied to an individual summand can depend on the historical empirical mean of the simulation (i.e. the system state). In analyzing the asymptotic performance of importance sampling, we show that the value function of a differential game characterizes the optimal performance, with player A corresponding to the choice of change of measure and player B arising due to a large deviations analysis. From this perspective, the traditional implementation of the heuristic is shown to correspond to player A choosing a control that is fixed, regardless of the system state or player B's choice of control. This leads to an "open loop" control for player A, which is suboptimal except in special cases. Our final contribution is a method, based on the Isaacs equation associated with the differential game, for constructing dynamic schemes. Numerical examples are presented to illustrate the results.

Performance Prediction Methodology for Biometric Systems Using a Large Deviations Approach

In the selection of biometrics for use in a recognition system and in the subsequent design of the system, the predicted performance is a key consideration. The realizations of the biometric signatures or vectors of features extracted from the signatures can be modeled as realizations of random processes. These random processes and the resulting distributions on the measurements determine bounds on the performance, regardless of the implementation of the recognition system. Given the underlying random processes, two techniques are applied to determine performance bounds. First, large deviation analysis is applied to bound performance in (M+1)-ary hypothesis testing problems, where the number of hypotheses is fixed as a measure of the fidelity of the measurements increases. Second, the capacity of a recognition system as a function of the desired error rate is explored. The recognition system is directly analogous to a communication problem with random coding. The recognition reliability function then determines the error rate as a function of the exponential growth rate of the number of hypotheses. Gaussian and binary examples are considered in detail. Performance prediction in the binary example is compared with performance prediction results for iris recognition systems using Daugman's IrisCode.

Large deviation asymptotics for occupancy problems

In the standard formulation of the occupancy problem one considers the distribution of r balls in n cells, with each ball assigned independently to a given cell with probability 1/n. Although closed form expressions can be given for the distribution of various interesting quantities (such as the fraction of cells that contain a given number of balls), these expressions are often of limited practical use. Approximations provide an attractive alternative, and in the present paper we consider a large deviation approximation as r and n tend to infinity. In order to analyze the problem we first consider a dynamical model, where the balls are placed in the cells sequentially and “time” corresponds to the number of balls that have already been thrown. A complete large deviation analysis of this “process level” problem is carried out, and the rate function for the original problem is then obtained via the contraction principle. The variational problem that characterizes this rate function is analyzed, and a fairly complete and explicit solution is obtained. The minimizing trajectories and minimal cost are identified up to two constants, and the constants are characterized as the unique solution to an elementary fixed point problem. These results are then used to solve a number of interesting problems, including an overflow problem and the partial coupon collector’s problem.