Donsker Classes Research Articles

About the Bootstrap Weak Convergence for the Foster-Greer-Thorbecke Poverty Index

We assume the Foster-Greer-Thorbecke (FGT) poverty index as a centered and normalized empirical process indexed by a particular Donsker class or collection of functions and define this poverty index as a bootstrapped empirical process, to show that the weak convergence of the FGT empirical process centered and normalized is a necessary and sufficient condition for the weak convergence of the FGT bootstrap empirical process centered and normalized. Thus, this result reflects that under certain conditions, the consistency in weak convergence of the FGT empirical process considered as a classical estimator of poverty (statistics) and the consistency in weak convergence of the FGT bootstrap empirical process considered as a bootstrap estimator of poverty (bootstrap statistics) are asymptotically equivalents for random samples of incomes statistically large and representative of a statistical universe of households.

Performance Guarantees for Policy Learning.

This article gives performance guarantees for the regret decay in optimal policy estimation. We give a margin-free result showing that the regret decay for estimating a within-class optimal policy is second-order for empirical risk minimizers over Donsker classes when the data are generated from a fixed data distribution that does not change with sample size, with regret decaying at a faster rate than the standard error of an efficient estimator of the value of an optimal policy. We also present a result giving guarantees on the regret decay of policy estimators for the case that the policy falls within a restricted class and the data are generated from local perturbations of a fixed distribution, where this guarantee is uniform in the direction of the local perturbation. Finally, we give a result from the classification literature that shows that faster regret decay is possible via plug-in estimation provided a margin condition holds. Three examples are considered. In these examples, the regret is expressed in terms of either the mean value or the median value, and the number of possible actions is either two or finitely many.

Directional differentiability for supremum-type functionals: Statistical applications

We show that various functionals related to the supremum of a real function defined on an arbitrary set or a measure space are Hadamard directionally differentiable. We specifically consider the supremum norm, the supremum, the infimum, and the amplitude of a function. The (usually non-linear) derivatives of these maps adopt simple expressions under suitable assumptions on the underlying space. As an application, we improve and extend to the multidimensional case the results in Raghavachari (Ann. Statist. 1 (1973) 67–73) regarding the limiting distributions of Kolmogorov–Smirnov type statistics under the alternative hypothesis. Similar results are obtained for analogous statistics associated with copulas. We additionally solve an open problem about the Berk–Jones statistic proposed by Jager and Wellner (In A Festschrift for Herman Rubin (2004) 319–331 IMS). Finally, the asymptotic distribution of maximum mean discrepancies over Donsker classes of functions is derived.

Stability Properties of Empirical Risk Minimization over Donsker Classes

We study some stability properties of algorithms which minimize (or almost-minimize) empirical error over Donsker classes of functions. We show that, as the number n of samples grows, the L2-diamet...

Empirical likelihood confidence tubes for functional parameters in plug-in estimation

We consider the infinite-dimensional inference problem in which the parameter of interest is a multivariate trajectory that can be written as an explicit functional T of a number of probability distributions. We propose an empirical likelihood procedure to build simultaneous confidence regions for these trajectories. Our main assumption is the Hadamard differentiability of T under norms adapted to empirical measures, i.e., supremum norms indexed by Donsker classes of functions. In order to handle practical computational issues, the proposed method, which we prove to be consistent, is based on a first order expansion of T . We also prove a general result of independent interest in empirical likelihood theory. Three applications are provided.

Consistency of the Subsample Bootstrap empirical process

In the classical Bootstrap approach the number of distinct observation in the resample is random. To overcome this hitch Rao et al. [Bootstrap by sequential resampling, J. Statist. Plan. Inference 64 (1997), pp. 257–281] have proposed a modified resampling procedure – the so-called Sequential Bootstrap or 0.632-Bootstrap – in which each resample has exactly the same number m ∼eq ⌊0.632 n⌋ of distinct observations. Motivated by this idea we introduce an akin procedure, the Subsample Bootstrap, where additionally even the size of each resample is equal. It will turn out that the Subsample Bootstrap empirical process is consistent for a wide class of Donsker classes.

Bracketing Metric Entropy Rates and Empirical Central Limit Theorems for Function Classes of Besov- and Sobolev-Type

We derive ℒ r (μ)-bracketing metric and sup-norm metric entropy rates of bounded subsets of general function spaces defined over ℝ d or, more generally, over Borel subsets thereof, by adapting results of Haroske and Triebel (Math. Nachr. 167, 131–156, 1994; 278, 108–132, 2005). The function spaces covered are of (weighted) Besov, Sobolev, Holder, and Triebel type. Applications to the theory of empirical processes are discussed. In particular, we show that (norm-)bounded subsets of the above mentioned spaces are Donsker classes uniformly in various sets of probability measures.

Donsker-type theorems for nonparametric maximum likelihood estimators

Let \({\mathcal{P}}\) be a nonparametric probability model consisting of smooth probability densities and let \({\hat{p}_{n}}\) be the corresponding maximum likelihood estimator based on n independent observations each distributed according to the law \({\mathbb{P}}\) . With \(\hat{\mathbb{P}}_{n}\) denoting the measure induced by the density \({\hat{p}_{n}}\) , define the stochastic process \({\hat{\nu}}_{n}: f\longmapsto \sqrt{n} \int fd({\hat{\mathbb{P}}}_{n} -\mathbb{P})\) where f ranges over some function class \({\mathcal{F}}\) . We give a general condition for Donsker classes \({\mathcal{F}}\) implying that the stochastic process \(\hat{\nu}_{n}\) is asymptotically equivalent to the empirical process in the space \({\ell ^{\infty }(\mathcal{F})}\) of bounded functions on \({ \mathcal{F}}\) . This implies in particular that \(\hat{\nu}_{n}\) converges in law in \({\ell ^{\infty }(\mathcal{F})}\) to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes \({\mathcal{ F}}\) . We give a number of applications: convergence of the probability measure \({\hat{\mathbb{P}}_{n}}\) to \({\mathbb{P}}\) at rate \({\sqrt{n}}\) in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; \({\sqrt{n}}\) -efficient estimation of nonlinear functionals defined on \({\mathcal{P}}\) ; limit theorems at rate \({\sqrt{n}}\) for the maximum likelihood estimator of the convolution product \({\mathbb{P\ast P}}\) .

Ergodic Regularity of Some Donsker Classes

We use a weak decoupling inequality in ergodic theory for maximal operators. We apply this inequality to the study of the property for a set of functions to be a Donsker class. The sets we examine are built from a sequence of $L^2$-operators and naturally appear in the study of the almost sure regularity properties of these. We obtain new individual necessary conditions (for a given $f\in L^2(\mu)$) and new global necessary conditions. The latter conditions are of uniform type and have a natural translation on the regularity properties of the canonical Gaussian process Z defined on $L^2(\mu)$.

Ergodic regularity of some Donsker classes

Ergodic regularity of some Donsker classes

Uniform Donsker Classes of Functions

A class $\mathscr{F}$ of measurable functions on a probability space $(A, \mathbb{A}, P)$ is called a $P$-Donsker class and we also write $\mathscr{F} \in \operatorname{CLT}(P)$, if the empirical processes $\mathbb{X}^P_n \equiv \sqrt{n}(\mathbb{P}_n - P)$ converge weakly to a $P$-Brownian bridge $G_P$ having bounded uniformly continuous sample paths almost surely. If this convergence holds for every probability measure $P$ on $(A, \mathbb{A})$, then $\mathscr{F}$ is called a universal Donsker class and we write $\mathscr{F} \in \operatorname{CLT}(\mathbf{M})$, where $\mathbf{M} \equiv \{$all probability measures on $(A, \mathbb{A})\}$. If the convergence holds uniformly in all $P$, then $\mathscr{F}$ is called a uniform Donsker class and we write $\mathscr{F} \in \operatorname{CLT}_u(\mathbf{M})$. For many applications the latter concept is too restrictive and it is useful to focus instead on a fixed subcollection $\mathscr{P}$ of the collection $\mathbf{M}$ of all probability measures on $(A, \mathbb{A})$. If the empirical processes converge weakly to $G_P$ uniformly for all $P \in \mathscr{P}$, then we say that $\mathscr{F}$ is a $\mathscr{P}$-uniform Donsker class and write $\mathscr{F} \in \operatorname{CLT}_u(\mathscr{P})$. We give general sufficient conditions for the $\mathscr{P}$-uniform Donsker property and establish basic equivalences in the uniform (in $P \in \mathscr{P}$) central limit theorem for $\mathbf{X}_n$, including a detailed study of the equivalences to the "functional" or "process in $n$" formulations of the $\operatorname{CLT}$. We give applications of our uniform convergence results to sequences of measures $\{P_n\}$ and to bootstrap resampling methods.

Frechet Differentiability, $p$-Variation and Uniform Donsker Classes

Differentiability of functionals of the empirical distribution function is extended. The supremum norm is replaced by $p$-variation seminorms, which are the $p$th roots of suprema of sums of $p$th powers of absolute increments of a function over nonoverlapping intervals. Frechet derivatives often exist for such norms when they do not for the supremum norm. For $1 < q < 2$, classes of functions uniformly bounded in $q$-variation are universal and uniform Donsker classes: The central limit theorem for empirical measures holds with respect to uniform convergence over such a class, also uniformly over all probability laws on the line. The integral $\int F dG$ was defined by L. C. Young if $F$ and $G$ are of bounded $p$- and $q$-variation respectively, where $p^{-1} + q^{-1} > 1$. Thus the normalized empirical distribution function $n^{1/2}(F_n - F)$ is with high probability in sets of uniformly bounded $p$-variation for any $p > 2$, uniformly in $n$.

Gaussian Characterization of Uniform Donsker Classes of Functions

It is proved that, for classes of functions $\mathscr{F}$ satisfying some measurability, the empirical processes indexed by $\mathscr{F}$ and based on $P \in \mathscr{P}(S)$ satisfy the central limit theorem uniformly in $P \in \mathscr{P}(S)$ if and only if the $P$-Brownian bridges $G_p$ indexed by $\mathscr{F}$ are sample bounded and $\rho_p$ uniformly continuous uniformly in $P \in \mathscr{P}(S)$. Uniform exponential bounds for empirical processes indexed by universal bounded Donsker and uniform Donsker classes of functions are also obtained.

Donsker classes of sets

We study the central limit theorem (CLT) and the law of large numbers (LLN) for empirical processes indexed by a (countable) class of sets C. The main result, of purely measure-theoretical nature, relates different ways to measure the “size” of C. It relies on a new rearrangement inequality that has been inspired by techniques used in the local theory of Banach spaces. As an application, we give sharp necessary conditions for the CLT, that are in some sense the best possible. We also obtain a way to compute the rate of convergence in the LLN.

Measurability Problems for Empirical Processes

To a class $\mathscr{F}$ of bounded functions on a probability space we associate two classes $\mathscr{F}_r$ and $\mathscr{F}_s$. The class $\mathscr{F}$ is a Donsker class if and only if $\mathscr{F}_r$ and $\mathscr{F}_s$ are Donsker classes. The class $\mathscr{F}_r$ corresponds to a separable version of the empirical process. It is obtained by applying a special type of lifting to $\mathscr{F}$. The class $\mathscr{F}_s$ consists of positive functions that are zero almost surely. It concentrates the pathology of $\mathscr{F}$ with respect to measurability. We use this method to prove without any measurability assumption a general contraction principle for processes that satisfy the central limit theorem.

Empirical Processes Indexed by Lipschitz Functions

Necessary and sufficient conditions on $P$ for the unit balls of $BL(\mathbf{R})$ and $\operatorname{Lip}(\mathbf{R})$ to be functional $P$-Donsker classes are obtained.

Relationships between Donsker classes and Sobolev spaces

Let H be a separable real Hubert space. Let X be a second countable topological space and (X, ℬ, P) be a probability space where ℬ is the Borel sets. Let T: H → C b (X) be linear and continuous. Then the image of the unit ball of H is a Donsker class. So, if k −1∈L 2 then the unit ball of the Sobolev space $$\{ f \in L^2 (\mathbb{R}^n )|(\smallint |\hat fk|^2 )^{\tfrac{1}{2}} < \infty \} $$ is a Donsker class for any P. For most other k it is not a Donsker class for any P with a bounded density.

Log Log Laws for Empirical Measures

Let $(X, \mathscr{A}, P)$ be a probability space and $\mathscr{C}$ a collection of measurable sets. Suppose $\mathscr{C}$ is a Donsker class, i.e., the central limit theorem for empirical measures holds uniformly on $\mathscr{C}$, in a suitable sense. Suppose also that suitable ($P\varepsilon$-Suslin) measurability conditions hold. Then we show that the $\log\log$ law for empirical measures, in the Strassen-Finkelstein form, holds uniformly on $\mathscr{C}$.

General chi-square goodness-of-fit tests with data-dependent cells

The goodness-of-fit of a parametric model for non-categorical data can be tested using the x2 statistic calculated after grouping the data into a finite number of disjoint cells. Work of Watson, Cebysev, Moore and others shows that the classical limit distributions still hold even for certain methods of grouping that depend on the data themselves. These results are generalised to cover a much wider class of methods of grouping; the parameters can be estimated from either the grouped or the ungrouped data. The proofs use a Central Limit Theorem for Empirical Measures due to Dudley. The grouping cells are allowed to come from any Donsker class for the underlying sampling distribution.

Central Limit Theorems for Empirical Measures

Let $(X, \mathscr{A}, P)$ be a probability space. Let $X_1, X_2,\cdots,$ be independent $X$-valued random variables with distribution $P$. Let $P_n := n^{-1}(\delta_{X_1} + \cdots + \delta_{X_n})$ be the empirical measure and let $\nu_n := n^\frac{1}{2}(P_n - P)$. Given a class $\mathscr{C} \subset \mathscr{a}$, we study the convergence in law of $\nu_n$, as a stochastic process indexed by $\mathscr{C}$, to a certain Gaussian process indexed by $\mathscr{C}$. If convergence holds with respect to the supremum norm $\sup_{C \in \mathscr{C}}|f(C)|$, in a suitable (usually nonseparable) function space, we call $\mathscr{C}$ a Donsker class. For measurability, $X$ may be a complete separable metric space, $\mathscr{a} =$ Borel sets, and $\mathscr{C}$ a suitable collection of closed sets or open sets. Then for the Donsker property it suffices that for some $m$, and every set $F \subset X$ with $m$ elements, $\mathscr{C}$ does not cut all subsets of $F$ (Vapnik-Cervonenkis classes). Another sufficient condition is based on metric entropy with inclusion. If $\mathscr{C}$ is a sequence $\{C_m\}$ independent for $P$, then $\mathscr{C}$ is a Donsker class if and only if for some $r, \sigma_m(P(C_m)(1 - P(C_m)))^r < \infty$.