Sums Of Independent Random Variables Research Articles

A fast and accurate numerical method for the left tail of sums of independent random variables

We present a flexible, deterministic numerical method for computing left-tail rare events of sums of non-negative, independent random variables. The method is based on iterative numerical integration of linear convolutions by means of Newtons–Cotes rules. The periodicity properties of convoluted densities combined with the Trapezoidal rule are exploited to produce a robust and efficient method, and the method is flexible in the sense that it can be applied to all kinds of non-negative continuous RVs. We present an error analysis and study the benefits of utilizing Newton–Cotes rules versus the fast Fourier transform (FFT) for numerical integration, showing that although there can be efficiency benefits to using FFT, Newton–Cotes rules tend to preserve the relative error better, and indeed do so at an acceptable computational cost. Numerical studies on problems with both known and unknown rare-event probabilities showcase the method’s performance and support our theoretical findings.

Poissonization Inequalities for Sums of Independent Random Variables in Banach Spaces with Applications to Empirical Processes

Poissonization Inequalities for Sums of Independent Random Variables in Banach Spaces with Applications to Empirical Processes

Limit theorems for a class of processes generalizing the U-empirical process

In this paper, we develop theory and tools for studying U-processes, a natural higher-order generalization of the empirical processes. We introduce a class of random discrete U-measures that generalize the empirical U-measure. We establish a Glivenko-Cantelli and a Donsker theorem under conditions on entropy numbers prevalent in the theory of empirical processes. These results are proved under some standard structural conditions on the Vapnik-Chervonenkis classes of functions and some mild conditions on the model. The uniform limit theorems discussed in this paper are key tools for many further developments involving empirical process techniques. Our results are applied to prove the asymptotic normality of Liu's simplicial median. We conclude this paper by extending Anscombe's central limit theorem to encompass randomly stopped U-processes, building upon its application to randomly stopped sums of independent random variables.

Total variation distance and compound poisson approximations for random sums

We derive new upper bounds for the total variation distance between compound Poisson distributions as well as between a random sum and a compound Poisson distribution, and as a result we also obtain upper bounds for the total variation distance between compound Poisson distributions and a sum of independent random variables. These bounds are generalizations and refinements of some well-known bounds in the literature. We also derive upper bounds for the total variation distance between negative binomial distributions of order k and between the negative binomial distributions of order k and compound Poisson distributions. Upper bounds for the total variation distances between the number of success runs of length k in binary Markovian trials and its limiting distributions for several enumeration schemes, are also given.

On Poisson Approximation

The problem of evaluating the accuracy of Poisson approximation to the distribution of a sum of independent integer-valued random variables has attracted a lot of attention in the past six decades. Among authors who contributed to the topic are Prokhorov, Kolmogorov, LeCam, Shorgin, Barbour, Hall, Deheuvels, Pfeifer, Roos, and many others. From a practical point of view, the problem has important applications in insurance, reliability theory, extreme value theory, etc. From a theoretical point of view, the topic provides insights into Kolmogorov’s problem concerning the accuracy of approximation of the distribution of a sum of independent random variables by infinitely divisible laws. The task of establishing an estimate of the accuracy of Poisson approximation with a correct (the best possible) constant at the leading term remained open for decades. We present a solution to that problem in the case where the accuracy of approximation is evaluated in terms of the point metric. We generalise and sharpen the corresponding inequalities established by preceding authors. A new result is established for the intensively studied topic of compound Poisson approximation to the distribution of a sum of integer-valued r.v.s.

Random Sampling of Mellin Band-Limited Signals

In this paper, we address the random sampling problem for the class of functions in the space of Mellin band-limited functions B T , which are concentrated on a bounded cube. It is established that any Mellin band-limited function can be approximated by an element in a finite-dimensional subspace of B T . Utilizing the notion of covering number and Bernstein’s inequality to the sum of independent random variables, we prove that the probabilistic sampling inequality holds for the set of concentrated signals in B T with an overwhelming probability provided the sampling size is large enough.

The Emergence of the Normal Distribution in Deterministic Chaotic Maps.

The central limit theorem states that, in the limits of a large number of terms, an appropriately scaled sum of independent random variables yields another random variable whose probability distribution tends to attain a stable distribution. The condition of independence, however, only holds in real systems as an approximation. To extend the theorem to more general situations, previous studies have derived a version of the central limit theorem that also holds for variables that are not independent. Here, we present numerical results that characterize how convergence is attained when the variables being summed are deterministically related to one another through the recurrent application of an ergodic mapping. In all the explored cases, the convergence to the limit distribution is slower than for random sampling. Yet, the speed at which convergence is attained varies substantially from system to system, and these variations imply differences in the way information about the deterministic nature of the dynamics is progressively lost as the number of summands increases. Some of the identified factors in shaping the convergence process are the strength of mixing induced by the mapping and the shape of the marginal distribution of each variable, most particularly, the presence of divergences or fat tails.

F-Divergences and Their Applications in Lossy Compression and Bounding Generalization Error

In this paper, we provide three applications for f-divergences: (i) we introduce Sanov’s upper bound on the tail probability of the sum of independent random variables based on super-modular f-divergence and show that our generalized Sanov’s bound strictly improves over ordinary one, (ii) we consider the lossy compression problem which studies the set of achievable rates for a given distortion and code length. We extend the rate-distortion function using mutual f-information and provide new and strictly better bounds on achievable rates in the finite blocklength regime using super-modular f-divergences, and (iii) we provide a connection between the generalization error of algorithms with bounded input/output mutual f-information and a generalized rate-distortion problem. This connection allows us to bound the generalization error of learning algorithms using lower bounds on the f-rate-distortion function. Our bound is based on a new lower bound on the rate-distortion function that (for some examples) strictly improves over previously best-known bounds.

Laws of Large Numbers for Weighted Sums of Independent Random Variables: A Game of Mass

We consider weighted sums of independent random variables regulated by an increment sequence and provide operative conditions that ensure a strong law of large numbers for such sums in both the centred and non-centred case. The existing criteria for the strong law are either implicit or based on restrictions on the increment sequence. In our setup we allow for an arbitrary sequence of increments, possibly random, provided the random variables regulated by such increments satisfy some mild concentration conditions. In the non-centred case, convergence can be translated into the behaviour of a deterministic sequence and it becomes a game of mass when the expectation of the random variables is a function of the increment sizes. We identify various classes of increments and illustrate them with a variety of concrete examples.

Netolygūs įverčiai puasoninėje aproksimacijoje

Poisson approximation for the sum of independent random variables is investigates in this paper.

Upper-bound estimates for weighted sums satisfying Cramer’s condition

Let S = ω1S1 + ω2S2 + ⋯ + ωNSN. Here Sj is the sum of identically distributed random variables and ωj > 0 denotes weight. We consider the case, when Sj is the sum of independent random variables satisfying Cramer’s condition. Upper-bounds for the accuracy of compound Poisson first and second order approximations in uniformmetric are established.

On the Probabilities of Large Deviations of Combinatorial Sums of Independent Random Variables That Satisfy the Linnik Condition

On the Probabilities of Large Deviations of Combinatorial Sums of Independent Random Variables That Satisfy the Linnik Condition

On approximation and estimation of distribution function of sum of independent random variables

On approximation and estimation of distribution function of sum of independent random variables

Sharp concentration results for heavy-tailed distributions

Abstract We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $P(X>t) \leq{\text{ e}}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow \alpha \in [0, \infty )$ as $t \rightarrow \infty $. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of sub-Weibull random variables, but it can also produce new results for the sum of random variables with heavier tails. We show that the concentration inequalities we obtain are sharp enough to offer large deviation results for the sums of independent random variables as well. Our analyses which are based on standard truncation arguments simplify, unify and generalize the existing results on the concentration and large deviation of heavy-tailed random variables.

ON THE ACCURACY BE THE METRIC OF THE DENSITY ESTIMATION CONSTRUCTED BY DEPENDENT OBSERVATIONS

ON THE ACCURACY BE THE METRIC OF THE DENSITY ESTIMATION CONSTRUCTED BY DEPENDENT OBSERVATIONS

State-dependent importance sampling for estimating expectations of functionals of sums of independent random variables

Estimating the expectations of functionals applied to sums of random variables (RVs) is a well-known problem encountered in many challenging applications. Generally, closed-form expressions of these quantities are out of reach. A naive Monte Carlo simulation is an alternative approach. However, this method requires numerous samples for rare event problems. Therefore, it is paramount to use variance reduction techniques to develop fast and efficient estimation methods. In this work, we use importance sampling (IS), known for its efficiency in requiring fewer computations to achieve the same accuracy requirements. We propose a state-dependent IS scheme based on a stochastic optimal control formulation, where the control is dependent on state and time. We aim to calculate rare event quantities that could be written as an expectation of a functional of the sums of independent RVs. The proposed algorithm is generic and can be applied without restrictions on the univariate distributions of RVs or the functional applied to the sum. We apply this approach to the log-normal distribution to compute the left tail and cumulative distribution of the ratio of independent RVs. For each case, we numerically demonstrate that the proposed state-dependent IS algorithm compares favorably to most well-known estimators dealing with similar problems.

On probabilities of large deviations of combinatorial sums of independent random variables satisfying Linnik’s condition

We derive new results on asymptotic behaviour for probabilities of large deviations of combinatorial sums of independent random variables satisfying Linnik’s condition. We find zones in which these probabilities are equivalent to the tail of the standard normal law. The author earlier obtained such results under Bernstein’s condition. The truncations method is applied in proofs of results.

A Mutual Information Inequality and Some Applications

In this paper we derive an inequality relating linear combinations of mutual information between subsets of mutually independent random variables and an auxiliary random variable. One choice of a family of auxiliary variables leads to a new proof of a Stam-type inequality regarding the Fisher Information of sums of independent random variables. Another choice of a family of auxiliary random variables leads to new results as well as new proofs of results relating to strong data processing constants and maximal correlation between sums of independent random variables. Other results obtained include convexity of Kullback–Leibler divergence over a parameterized path along pairs of binomial and Poisson distributions, as well as a new duality-based argument relating the Stam-type inequality and entropy power inequality.

On asymptotic behaviour for probabilities of moderate deviations of combinatorial sums

We investigate an asymptotic behaviour for probabilities of moderate deviations of combinatorial sums of independent random variables having moments of order p > 2. We find zones in which these probabilities are equivalent to the tail of the standard normal law. The width of the zone is a function from the logarithm of a combinatorial variant for Lyapunov’s ratio. The author earlier obtained similar results under Bernstein’s and Linnik’s conditions. The truncations method is used in proofs of the results.

Sharp Rosenthal‐type inequalities for mixtures and log‐concave variables

AbstractWe obtain Rosenthal‐type inequalities with sharp constants for moments of sums of independent random variables which are mixtures of a fixed distribution. We also identify extremizers in log‐concave settings when the moments of summands are individually constrained.