Sums of independent random variables

Series representations for several density functions are obtained as mixtures of generalized gamma distributions with discrete mass probability weights, by using the exponential expansion and the binomial theorem. Based on these results, approximations based on mixtures of generalized gamma distributions are proposed to approximate the distribution of the sum of independent random variables, which may not be identically distributed. The applicability of the proposed approximations are illustrated for the sum of independent Rayleigh random variables, the sum of independent gamma random variables, and the sum of independent Weibull random variables. Numerical studies are presented to assess the precision of these approximations.

: It is shown that the distribution of a sum of independent beta random variables is often well approximated by a properly scaled beta distribution. The relationship between the type of Pearson curve which best fits a sum of independent random variables and the types of the Pearson curves which best fit the sum and random variables is also investigated. The best fitting Pearson curve for a distribution is defined here to be the unique Pearson curve with the same first four moments.

We study the learnability of sums of independent integer random variables given a bound on the size of the union of their supports. For $\mathcal{A} \subset \mathbf{Z}_{+}$, a sum of independent random variables with collective support $\mathcal{A}$} (called an $\mathcal{A}$-sum in this paper) is a distribution $\mathbf{S} = \mathbf{X}_1 + \cdots + \mathbf{X}_N$ where the $\mathbf{X}_i$'s are mutually independent (but not necessarily identically distributed) integer random variables with $\cup_i \mathsf{supp}(\mathbf{X}_i) \subseteq \mathcal{A}.$ We give two main algorithmic results for learning such distributions: 1. For the case $| \mathcal{A} | = 3$, we give an algorithm for learning $\mathcal{A}$-sums to accuracy $\epsilon$ that uses $\mathsf{poly}(1/\epsilon)$ samples and runs in time $\mathsf{poly}(1/\epsilon)$, independent of $N$ and of the elements of $\mathcal{A}$. 2. For an arbitrary constant $k \geq 4$, if $\mathcal{A} = \{ a_1,...,a_k\}$ with $0 \leq a_1 < ... < a_k$, we give an algorithm that uses $\mathsf{poly}(1/\epsilon) \cdot \log \log a_k$ samples (independent of $N$) and runs in time $\mathsf{poly}(1/\epsilon, \log a_k).$ We prove an essentially matching lower bound: if $|\mathcal{A}| = 4$, then any algorithm must use $\Omega(\log \log a_4) $ samples even for learning to constant accuracy. We also give similar-in-spirit (but quantitatively very different) algorithmic results, and essentially matching lower bounds, for the case in which $\mathcal{A}$ is not known to the learner.

In a 1957 paper F. Spitzer published, among other things, one estimate of the distribution function of sums of independent symmetric random variables with a common absolutely continuous distribution. This estimation was constructed with the help of one identity for the distribution of a sum of the above-mentioned random variables, the proof of which was not given. In this paper we prove the Spitzer identity for any independent random variables, and by using it we construct an estimate of the distribution of a sum of independent identically distributed random variables. The Spitzer estimate can be derived as a particular case of the proposed estimation.

On the Rate of Approach of the Distributions of Sums of Independent Random Variables to Accompanying Distributions

February 27, 2007, was the 75th birthday of the eminent mathematician, Professor Vladimir Mikhailovich Zolotarev. In 1949, Vladimir entered the faculty of mechanics and mathematics of Moscow State University. As his specialization field he chose probability theory and began his studies under the supervision of Eugene Borisovich Dynkin. After graduating from the university he was recommended to graduate studies, where his advisor was Andrei Nikolaevich Kolmogorov. Other distinguished mathematicians also have had a potent effect on Zolotarev's mathematical talent. Later, he mentions more than once not only his teachers E. B. Dynkin and A. N. Kolmogorov, but also B. V. Gnedenko and Yu. V. Linnik. In his graduate studies, Vladimir begins to study the properties of stable distributions. He continues to be interested in this theme even today. At first he was dealing with the stable distributions in the scheme of summation of independent identically distributed random variables. Later, he extended the concept of a stable law to the schemes of maximum and multiplication of random variables. His studies of random variables are summarized in the monograph [1]. It has gained widespread recognition and has been translated into English. As it saw the light, the nomenclature, such as Zolotarev's theorem, Zolotarev's formula, and Zolotarev's transformation, became quite conventional. Contemporaneously with studying the properties of stable laws, Zolotarev began to work in the field of limit theorems for sums of independent random variables. His results obtained in this direction can be conventionally divided into three groups. The first group concerns refining classical theorems and convergence rate estimates. As an example, we mention the convergence rate estimate in the central limit theorem in terms of pseudomoments, which for some time remained the best estimate known. But it is even more important that the method of pseudomoments used by Zolotarev to obtain this estimate led to a new structure of convergence rate estimates in limit theorems for sums of independent random variables. His disciples have made this a powerful tool which provides us with even more sharp estimates. The principal novelty resides in that a pseudomoment allows us to recognize the contribution of every individual summand to the whole estimate. The notions of a center and a scatter (spread) introduced by Zolotarev turn out to be indispensable in establishing the weak compactness of sequences of sums of independent random variables which have no finite moments and allowed us to extend limit theorems, previously known to be valid, under quite strict moment conditions to such variables. Zolotarev's second group of results gathers together the results whose essence reduces to weakening the condition of independence of random variables so that the limit theorems remain true. The third group concerns the so-called nonclassical scheme of summation. The cornerstone of this scheme consists of breaking the habitual pattern, where an individual summand does not influence the form of the limit distribution. In the nonclassical summation theory an individual summand is allowed to play a discernible part. It is fair to say that Vladimir Zolotarev is one of the fathers of this direction in the theory of summation of random variables. He has generalized the results of his predecessors, P. Lévy and Yu. V. Linnik, who on the heuristic level of reasoning pointed to the possibility of a new approach to limit theorems for sums of independent random variables, and developed a self-sufficient theory of summation of random variables, now referred to as nonclassical. The theory of probability metrics built by Zolotarev underlies the novel viewpoint of limit theorems of probability theory as stability theorems. Zolotarev summarizes his studies in this field in the monograph [3], which immediately became a widely used source for new investigations. In 1997, a revised and enlarged version of this monograph was published in English [4]. Along with sums of random variables, Vladimir Zolotarev deals with more general asymptotic schemes. In particular, together with his Hungarian colleague L. Szeidl, Zolotarev has made an essential contribution to the asymptotic theory of random polynomials. Their joint results are presented in the monograph [6]. Investigating the asymptotic properties of sums of independent random variables, Vladimir Zolotarev came to related studies in the theory of stochastic processes and queueing theory. He and his colleagues analyze the stability and continuity of queueing systems with the use of related concepts of the theory of summation of random variables. The quantitative characteristics of these properties suggested by Zolotarev led to a deeper understanding of these phenomena. Speaking of the results obtained by Vladimir Zolotarev, we must say that he masterfully manages to use both modern and classical technique of analysis. Zolotarev enriches the store of probability theory by a series of new analytic methods and tools; it suffices to mention the thorough study of theorems of classical analysis, harmonic analysis, and the theory of special functions. Zolotarev also clears up how to use the Mellin–Stieltjes transforms in probability theory. Modern probability theory plays an important role in many natural sciences. For example, modern statistical physics becomes infeasible without using the fundamental concepts of probability theory. Vladimir Zolotarev has given many talks on how to apply probability theory to explaining various phenomena in physics, genetics, and geology. Together with V. V. Uchaikin he wrote the monograph [5], where one finds plenty of applications of stable laws explaining a series of physical and economical phenomena. In 1956, A. N. Kolmogorov founded the journal Theory of Probability and Its Applications. From the very beginning, Vladimir Zolotarev took an active part in the operation of the journal. From the day of foundation to 1966 he was the executive secretary; from 1967 to 1990 the deputy editor-in-chief; and from 1991 to the present he is a member of the advisory board. Vladimir Zolotarev founded the series of issues Stability Problems for Stochastic Models of the Journal of Mathematical Sciences, being its editor-in-chief. He also heads the editorial board of the series of monographs Modern Probability and Statistics published by VSP/Brill (The Netherlands). Vladimir Zolotarev is the initiator and continuous leader of the international scientific seminar on stability problems of stochastic models, widely known as the Zolotarev seminar. From 1973, twenty-six sessions of this seminar have been held in various countries. These sessions take place almost every year and attract about a hundred participants from diverse countries. Vladimir Zolotarev does much to popularize science. In the brochure [2] he presented an exciting story about stable laws and their applications. At the same time he created two educational films about fundamental limit theorems of probability theory. A crowd of youthful science enthusiasts is always gathering around him. Many of his pupils have become reputable specialists in various fields of mathematics. The scientific school of Vladimir Zolotarev is highly regarded. Friends and colleagues of Zolotarev love and appreciate him because he is a man of principle, well-wishing and tenderhearted, ready to stand by and assist. Devoting heart and soul to science, he demands the same from his colleagues and students. The scientific society highly appreciates the contributions of Vladimir Zolotarev. In 1971, for a series of works on limit theorems for sums of independent random variables, the Presidium of the Academy of Sciences of the USSR awarded him the Markov prize. We wish Vladimir Zolotarev many years of good health and continued activity.

We investigate the asymptotic behavior of weighted sums of independent standardized random variables with uniformly bounded third moments. The sequence of weights is given by a family of rectangular matrices with uniformly small entries and approximately orthogonal rows. We prove that the empirical CDF of the resulting partial sums converges to the normal CDF with probability one. This result implies almost sure convergence of empirical periodograms, almost sure convergence of spectral distribution of circulant and reverse circulant matrices, and almost sure convergence of the CDF generated from independent random variables by independent random orthogonal matrices. In the special case of trigonometric weights, the speed of the almost sure convergence is described by a normal approximation as well as a large deviation principle.

Previous article Next article A Problem on Large Deviations in a Space of TrajectoriesI. F. PinelisI. F. Pinelishttps://doi.org/10.1137/1126006PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] V. V. Godovanchuk, Probabilities of large deviations for sums of independent random variables attracted to a stable law, Theory Prob. Appl., 23 (1978), 602–608 0422.60020 LinkGoogle Scholar[2] A. A. Borovkov, Analysis of large deviations in boundary-value problems with arbitrary boundaries, Sibirsk. Mat. Ž., 5 (1964), 253–289 29:645 0166.14303 A. A. Borovkov, Analysis of large deviations in boundary-value problems with arbitrary boundaries. II, Sibirsk. Mat. Ž., 5 (1964), 750–767 29:4117 0202.48502 Google Scholar[3] A. A. Borovkov, Boundary-value problems for random walks and large deviations in function spaces, Theory Prob. Appl., 12 (1967), 575–595 0178.20004 LinkGoogle Scholar[4] A. A. Mogul'skii, Large deviations for trajectories of multi-dimensional random walks, Theory Prob. Appl., 21 (1976), 300–315 0366.60031 LinkGoogle Scholar[5] A. V. Nagaev, Limit theorems that take into account large deviations when Cramér's condition is violated, Izv. Akad. Nauk UzSSR Ser. Fiz.-Mat. Nauk, 13 (1969), 17–22, (In Russian.) 43:8108 0226.60043 Google Scholar[6] I. A. Ibragimov and , Yu. V. Linnik, Independent and stationary sequences of random variables, Wolters-Noordhoff Publishing, Groningen, 1971, 443– 48:1287 0219.60027 Google Scholar[7] D. Kh. Fuk and , S. V. Nagaev, Probability inequalities for sums of independent random variables, Theory Prob. Appl., 16 (1971), 643–660 0259.60024 LinkGoogle Scholar[8] A. V. Nagaev, doctoral dissertation, Tashkent, 1970. (In Russian.) Google Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Hidden regular variation for point processes and the single/multiple large point heuristicThe Annals of Applied Probability, Vol. 32, No. 1 Cross Ref On the Nonuniform Berry–Esseen Bound Cross Ref Tail asymptotics for delay in a half-loaded GI/GI/2 queue with heavy-tailed job sizes21 June 2015 | Queueing Systems, Vol. 81, No. 4 Cross Ref An asymptotically Gaussian bound on the Rademacher tailsElectronic Journal of Probability, Vol. 17, No. none Cross Ref Large deviations for random walks under subexponentiality: The big-jump domainThe Annals of Probability, Vol. 36, No. 5 Cross Ref An integro-local theorem applicable on the whole half-axis to the sums of random variables with regularly varying distributions11 September 2008 | Siberian Mathematical Journal, Vol. 49, No. 4 Cross Ref Integro-local and integral theorems for sums of random variables with semiexponential distributionsSiberian Mathematical Journal, Vol. 47, No. 6 Cross Ref Large deviations of the first passage time for a random walk with semiexponentially distributed jumpsSiberian Mathematical Journal, Vol. 47, No. 6 Cross Ref On Probabilities of Large Deviations for Random Walks. I. Regularly Varying Distribution TailsA. A. Borovkov and K. A. Borovkov25 July 2006 | Theory of Probability & Its Applications, Vol. 46, No. 2AbstractPDF (215 KB)Large-Deviation Probabilities for Maxima of Sums of Independent Random Variables with Negative Mean and Subexponential DistributionD. A. Korshunov25 July 2006 | Theory of Probability & Its Applications, Vol. 46, No. 2AbstractPDF (155 KB)Exact Asymptotics for Large Deviation Probabilities, with Applications Cross Ref Sample path large deviations in finer topologies4 April 2007 | Stochastics and Stochastic Reports, Vol. 67, No. 3-4 Cross Ref The time until the final zero crossing of random sums with application to nonparametric bandit theoryApplied Mathematics and Computation, Vol. 63, No. 2-3 Cross Ref Probabilities of Large Deviations on the Whole AxisL. V. Rozovskii17 July 2006 | Theory of Probability & Its Applications, Vol. 38, No. 1AbstractPDF (1937 KB)Sharp Exponential Inequalities for the Martingales in the 2-Smooth Banach Spaces and Applications to “Scalarizing” Decoupling Cross Ref Large Deviations of Sums of Independent Random Variables without Several Maximal SummandsV. V. Vinogradov and V. V. Godovan’chuk17 July 2006 | Theory of Probability & Its Applications, Vol. 34, No. 3AbstractPDF (383 KB)Probabilities of Large Deviations of Sums of Independent Random Variables with Common Distribution Function in the Domain of Attraction of the Normal LawL. V. Rozovskii17 July 2006 | Theory of Probability & Its Applications, Vol. 34, No. 4AbstractPDF (1377 KB) Volume 26, Issue 1| 1981Theory of Probability & Its Applications History Submitted:05 February 1979Published online:17 July 2006 InformationCopyright © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1126006Article page range:pp. 69-84ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics

Under the assumption that the distribution of a nonnegative random variable $$X$$ admits a bounded coupling with its size biased version, we prove simple and strong concentration bounds. In particular the upper tail probability is shown to decay at least as fast as the reciprocal of a Gamma function, guaranteeing a moment generating function that converges everywhere. The class of infinitely divisible distributions with finite mean, whose Levy measure is supported on an interval contained in $$[0,c]$$ for some $$c < \infty $$ , forms a special case in which this upper bound is logarithmically sharp. In particular the asymptotic estimate for the Dickman function, that $$\rho (u) \approx u^{-u}$$ for large $$u$$ , is shown to be universal for this class. A special case of our bounds arises when $$X$$ is a sum of independent random variables, each admitting a 1-bounded size bias coupling. In this case, our bounds are comparable to Chernoff–Hoeffding bounds; however, ours are broader in scope, sharper for the upper tail, and equal for the lower tail. We discuss bounded and monotone couplings, give a sandwich principle, and show how this gives an easy conceptual proof that any finite positive mean sum of independent Bernoulli random variables admits a 1-bounded coupling with the same conditioned to be nonzero.

On using Lehmann alternatives with nonresponders

The Ewens sampling formula is well known as the probability for a partition of a positive integer. Here, we discuss the asymptotic and approximate discrete distributions of the length of the formula. We give a sufficient condition for the length to converge in distribution to the shifted Poisson distribution. This condition is proved using two methods: One is based on the sum of independent Bernoulli random variables, and the other is based on an expression of the length that is not the sum of independent random variables. As discrete approximations of the length, we give those based on the Poisson distribution and the binomial distribution. The results show that the first two moments of the approximation based on the binomial distribution are almost equal to those of the length. Two applications of this approximation are given.

We show that the differential entropy of a sum of independent symmetric random variables supported on [-1,1] is maximal when one is uniformly distributed and the remaining are Bernoulli /spl plusmn/1.

In this article a relationship between the Littlewood–Offord problem and the problem of estimating the distribution of sums of independent vector-valued random variables is developed. A bound is given for the concentration function for the sum of independent random variables taking three values.

We provide an optimal Berry-Esseen type inequality for Zolotarev’s ideal ζ3-metric measuring the difference between expectations of sufficiently smooth functions, like |·|3, of a sum of independent random variables X 1,..., X n with finite third-order moments and a sum of independent symmetric two-point random variables, isoscedastic to the X i . In the homoscedastic case of equal variances, and in particular, in case of identically distributed X 1,..., X n the approximating law is a standardized symmetric binomial one. As a corollary, we improve an already optimal estimate of the accuracy of the normal approximation due to Tyurin (2009).

Sums of independent random variables already appeared in the preceding chapters in some concrete situations (Gaussian and Rademacher averages, representation of stable random variables). On the intuitive basis of central limit theorems which approximate normalized sums of independent random variables by smooth limiting distributions (Gaussian, stable), one would expect that results similar to those presented previously should hold in a sense or in another for sums of independent random variables. The results presented in this chapter go in this direction and the reader will recognize in this general setting the topics covered before: integrability properties, equivalence of moments, concentration, tail behavior, etc. We will mainly describe ideas and techniques which go from simple but powerful observations such as symmetrization (randomization) techniques to more elaborate results like those obtained from the isoperimetric inequality for product measures of Theorem 1.4. Section 6.1 is concerned with symmetrization, Section 6.2 with Hoffmann-Jorgensen’s inequalities and the equivalence of moments of sums of independent random variables. In the last and main section, martingale and isoperimetric methods are developed in this context. Many results presented in this chapter will be of basic use in the study of limit theorems later.