Uniform Central Limit Theorem Research Articles

Uniform in number of neighbors consistency and weak convergence of $ k $NN empirical conditional processes and $ k $NN conditional $ U $-processes involving functional mixing data

<abstract><p>$ U $-statistics represent a fundamental class of statistics arising from modeling quantities of interest defined by multi-subject responses. $ U $-statistics generalize the empirical mean of a random variable $ X $ to sums over every $ m $-tuple of distinct observations of $ X $. Stute [182] introduced a class of so-called conditional $ U $-statistics, which may be viewed as a generalization of the Nadaraya-Watson estimates of a regression function. Stute proved their strong pointwise consistency to: $ r^{(m)}(\varphi, \mathbf{t}): = \mathbb{E}[\varphi(Y_{1}, \ldots, Y_{m})|(X_{1}, \ldots, X_{m}) = \mathbf{t}], \; \mbox{for}\; \mathbf{ t}\in \mathcal{X}^{m}. $ In this paper, we are mainly interested in the study of the $ k $NN conditional $ U $-processes in a functional mixing data framework. More precisely, we investigate the weak convergence of the conditional empirical process indexed by a suitable class of functions and of the $ k $NN conditional $ U $-processes when the explicative variable is functional. We treat the uniform central limit theorem in both cases when the class of functions is bounded or unbounded satisfying some moment conditions. The second main contribution of this study is the establishment of a sharp almost complete Uniform consistency in the Number of Neighbors of the constructed estimator. Such a result allows the number of neighbors to vary within a complete range for which the estimator is consistent. Consequently, it represents an interesting guideline in practice to select the optimal bandwidth in nonparametric functional data analysis. These results are proved under some standard structural conditions on the Vapnik-Chervonenkis classes of functions and some mild conditions on the model. The theoretical results established in this paper are (or will be) key tools for further functional data analysis developments. Potential applications include the set indexed conditional <italic>U</italic>-statistics, Kendall rank correlation coefficient, the discrimination problems and the time series prediction from a continuous set of past values.</p></abstract>

Some impossibility results for inference with cluster dependence with large clusters

This paper focuses on a setting with observations having a cluster dependence structure and presents two main impossibility results. First, we show that when there is only one large cluster, i.e., the researcher does not have any knowledge on the dependence structure of the observations, it is not possible to consistently discriminate the mean. When within-cluster observations satisfy the uniform central limit theorem, we also show that a sufficient condition for consistent n-discrimination of the mean is that we have at least two large clusters. This result shows some limitations for inference when we lack information on the dependence structure of observations. Our second result provides a necessary and sufficient condition for the cluster structure that the long run variance is consistently estimable. Our result implies that when there is at least one large cluster, the long run variance is not consistently estimable.

Uniform Limit Theorems under length-biased sampling and type I censoring

In recent years, in view of theory of empirical processes, authors have become more interested in the uniform analogue of the three fundamental theorems: the uniform law of large numbers of Glivenko-Cantelli type, the uniform central limit theorem for Donsker type and the functional law of the iterated logarithm (LIL). In this paper, under the bracketing entropy conditions, the uniform law of large numbers, uniform central limit theorem and the uniform LIL of Strassen type have been investigated in the case of length-biased and type I censoring.

Uniform limit theorems for a class of conditional [formula omitted]-estimators when covariates are functions

This paper considers nonparametric estimation of a parameter, which is a zero of a certain estimating equation, indexed by a class of functions and depending on an infinite-dimensional covariate. This problem is known in the literature as Z-estimation, where a rich theory, in the case of the finite-dimensional covariates, exists. However, in certain applications, covariates may belong to an infinite-dimensional space, which makes the estimation as well as the study of the asymptotic properties challenging topics. This paper aims to bring several contributions to the existing functional data analysis literature. First, we introduce nonparametric conditional Z-estimators whenever functional stationary ergodic data are observed under right-censorship model. A sharp uniform-in-bandwidth limit law for the proposed estimator is presented. Such result allows the bandwidth to vary within a complete range for which the estimator is consistent. Consequently, it represents an interesting guideline in practice to select the optimal bandwidth in nonparametric functional data analysis. Moreover, a uniform consistency rate, over Vapnik–Chervonenkis classes of functions, is provided for the process of conditional Z-estimators. Finally, a uniform central limit theorem for the process indexed by functions is established. The last result is of particular interest since it is obtained for dependent functional data. The uniform limit theorems, discussed in this paper, are key tools for many further developments in functional data analysis involving empirical process techniques. These results are proved under some standard structural conditions on the Vapnik–Chervonenkis classes of functions and some mild conditions on the model.

Central Limit Theorem in high dimensions: The optimal bound on dimension growth rate

In this article, we try to give an answer to the simple question: “What is the optimal growth rate of the dimensionppas a function of the sample sizennfor which the Central Limit Theorem (CLT) holds uniformly over the collection ofpp-dimensional hyper-rectangles ?”. Specifically, we are interested in the normal approximation of suitably scaled versions of the sum∑i=1nXi\sum _{i=1}^{n}X_iinRp\mathcal {R}^puniformly over the class of hyper-rectanglesAre={∏j=1p[aj,bj]∩R:−∞≤aj≤bj≤∞,j=1,…,p}\mathcal {A}^{re}=\{\prod _{j=1}^{p}[a_j,b_j]\cap \mathcal {R}:-\infty \leq a_j\leq b_j \leq \infty , j=1,\ldots ,p\}, whereX1,…,XnX_1,\dots ,X_nare independentp−p-dimensional random vectors with each having independent and identically distributed (iid) components. We investigate the optimal cut-off rate oflog⁡p\log pbelow which the uniform CLT holds and above which it fails. According to some recent results of Chernozukov et al. [Ann. Probab. 45 (2017), pp. 2309–2352], it is well known that the CLT holds uniformly overAre\mathcal {A}^{re}iflog⁡p=o(n1/7)\log p=o\big (n^{1/7}\big ). They also conjectured that for CLT to hold uniformly overAre\mathcal {A}^{re}, the optimal rate islog⁡p=o(n1/3)\log p = o\big (n^{1/3}\big ). We show instead that under some suitable conditions on the even moments and under vanishing odd moments, the CLT holds uniformly overAre\mathcal {A}^{re}, whenlog⁡p=o(n1/2)\log p=o\big (n^{1/2}\big ). More precisely, we show that iflog⁡p=ϵn\log p =\epsilon \sqrt {n}for some sufficiently smallϵ>0\epsilon >0, the normal approximation is valid with an errorϵ\epsilon, uniformly overAre\mathcal {A}^{re}. Further, we show by an example that the uniform CLT overAre\mathcal {A}^{re}fails iflim supn→∞n−(1/2+δ)log⁡p>0\limsup _{ n\rightarrow \infty } n^{-(1/2+\delta )} \log p >0for someδ>0\delta >0. Therefore, with some moment conditions the optimal rate of the growth ofppfor the validity of the CLT is given bylog⁡p=o(n1/2)\log p=o\big (n^{1/2}\big ).

Central Limit Theorems for Conditional Empirical and Conditional U-Processes of Stationary Mixing Sequences

In this paper we are concerned with the weak convergence to Gaussian processes of conditional empirical processes and conditional U-processes from stationary β-mixing sequences indexed by classes of functions satisfying some entropy conditions. We obtain uniform central limit theorems for conditional empirical processes and conditional U-processes when the classes of functions are uniformly bounded or unbounded with envelope functions satisfying some moment conditions. We apply our results to introduce statistical tests for conditional independence that are multivariate conditional versions of the Kendall statistics.

Large sample theory for merged data from multiple sources

We develop large sample theory for merged data from multiple sources. Main statistical issues treated in this paper are (1) the same unit potentially appears in multiple datasets from overlapping data sources, (2) duplicated items are not identified and (3) a sample from the same data source is dependent due to sampling without replacement. We propose and study a new weighted empirical process and extend empirical process theory to a dependent and biased sample with duplication. Specifically, we establish the uniform law of large numbers and uniform central limit theorem over a class of functions along with several empirical process results under conditions identical to those in the i.i.d. setting. As applications, we study infinite-dimensional $M$-estimation and develop its consistency, rates of convergence and asymptotic normality. Our theoretical results are illustrated with simulation studies and a real data example.

A Conversation with Dick Dudley

Richard Mansfield Dudley (Dick Dudley) was born in 1938. He received the A.B. from Harvard in 1952 and the Ph.D. from Princeton in 1962 (under the supervision of Gilbert Hunt and Edward Nelson). Following an appointment at UC Berkeley as an assistant professor, he joined the Department of Mathematics at MIT in 1967. Dick Dudley has made fundamental contributions to the theory of Gaussian processes and Probability in Banach Spaces. Among his major achievements is the development of a general framework for empirical processes theory, in particular, for uniform central limit theorems. These results have had and continue having tremendous impact in contemporary statistics and in mathematical foundations of machine learning. A more extensive biographical sketch is contained in the preface to the Selected works of R. M. Dudley (editors: E. Gine, V. Koltchinskii and R. Norvaisa) published in 2010. This conversation took place (mostly, via email) in the fall of 2017.

UNIFORMLY VALID POST-REGULARIZATION CONFIDENCE REGIONS FOR MANY FUNCTIONAL PARAMETERS IN Z-ESTIMATION FRAMEWORK.

In this paper, we develop procedures to construct simultaneous confidence bands for potentially infinite-dimensional parameters after model selection for general moment condition models where is potentially much larger than the sample size of available data, n. This allows us to cover settings with functional response data where each of the parameters is a function. The procedure is based on the construction of score functions that satisfy Neyman orthogonality condition approximately. The proposed simultaneous confidence bands rely on uniform central limit theorems for high-dimensional vectors (and not on Donsker arguments as we allow for ). To construct the bands, we employ a multiplier bootstrap procedure which is computationally efficient as it only involves resampling the estimated score functions (and does not require resolving the high-dimensional optimization problems). We formally apply the general theory to inference on regression coefficient process in the distribution regression model with a logistic link, where two implementations are analyzed in detail. Simulations and an application to real data are provided to help illustrate the applicability of the results.

On the optimal estimation of probability measures in weak and strong topologies

Given random samples drawn i.i.d. from a probability measure $\mathbb{P}$ (defined on say, $\mathbb{R}^d$), it is well-known that the empirical estimator is an optimal estimator of $\mathbb{P}$ in weak topology but not even a consistent estimator of its density (if it exists) in the strong topology (induced by the total variation distance). On the other hand, various popular density estimators such as kernel and wavelet density estimators are optimal in the strong topology in the sense of achieving the minimax rate over all estimators for a Sobolev ball of densities. Recently, it has been shown in a series of papers by Gin\'{e} and Nickl that these density estimators on $\mathbb{R}$ that are optimal in strong topology are also optimal in $\|\cdot\|_{\mathcal{F}}$ for certain choices of $\mathcal{F}$ such that $\|\cdot\|_{\mathcal{F}}$ metrizes the weak topology, where $\|\mathbb{P}\|_{\mathcal{F}}:=\sup\{\int f\,\mathrm{d}\mathbb{P}: f\in\mathcal{F}\}$. In this paper, we investigate this problem of optimal estimation in weak and strong topologies by choosing $\mathcal{F}$ to be a unit ball in a reproducing kernel Hilbert space (say $\mathcal{F}_H$ defined over $\mathbb{R}^d$), where this choice is both of theoretical and computational interest. Under some mild conditions on the reproducing kernel, we show that $\|\cdot\|_{\mathcal{F}_H}$ metrizes the weak topology and the kernel density estimator (with $L^1$ optimal bandwidth) estimates $\mathbb{P}$ at dimension independent optimal rate of $n^{-1/2}$ in $\|\cdot\|_{\mathcal{F}_H}$ along with providing a uniform central limit theorem for the kernel density estimator.

Expectile asymptotics

We discuss in detail the asymptotic distribution of sample expectiles. First, we show uniform consistency under the assumption of a finite mean. In case of a finite second moment, we show that for expectiles other then the mean, only the additional assumption of continuity of the distribution function at the expectile implies asymptotic normality, otherwise, the limit is non-normal. For a continuous distribution function we show the uniform central limit theorem for the expectile process. If, in contrast, the distribution is heavy-tailed, and contained in the domain of attraction of a stable law with $1 < \alpha < 2$, then we show that the expectile is also asymptotically stable distributed. Our findings are illustrated in a simulation section.

Bootstrap uniform central limit theorems for Harris recurrent Markov chains

The main objective of this paper is to establish bootstrap uniform functional central limit theorem for Harris recurrent Markov chains over uniformly bounded classes of functions. We show that the result can be generalized also to the unbounded case. To avoid some complicated mixing conditions, we make use of the well-known regeneration properties of Markov chains. We show that in the atomic case the proof of the bootstrap uniform central limit theorem for Markov chains for functions dominated by a function in $L^{2}$ space proposed by Radulović (2004) can be significantly simplified. Finally, we prove bootstrap uniform central limit theorems for Fréchet differentiable functionals in a Markovian setting.

On the asymptotics of random forests

The last decade has witnessed a growing interest in random forest models which are recognized to exhibit good practical performance, especially in high-dimensional settings. On the theoretical side, however, their predictive power remains largely unexplained, thereby creating a gap between theory and practice. In this paper, we present some asymptotic results on random forests in a regression framework. Firstly, we provide theoretical guarantees to link finite forests used in practice (with a finite number M of trees) to their asymptotic counterparts (with M = ∞ ). Using empirical process theory, we prove a uniform central limit theorem for a large class of random forest estimates, which holds in particular for Breiman’s (2001) original forests. Secondly, we show that infinite forest consistency implies finite forest consistency and thus, we state the consistency of several infinite forests. In particular, we prove that q quantile forests–close in spirit to Breiman’s (2001) forests but easier to study–are able to combine inconsistent trees to obtain a final consistent prediction, thus highlighting the benefits of random forests compared to single trees.

Uniform central limit theorems for the Grenander estimator

We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness $s>1/2$. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric case. Nevertheless, our proofs rely on the fact that the derivative of the likelihood can be shown to be small at the maximum likelihood estimator.

A uniform central limit theorem for neural network-based autoregressive processes with applications to change-point analysis

We consider an autoregressive process with a nonlinear regression function that is modelled by a feedforward neural network. First, we derive a uniform central limit theorem which is useful in the context of change-point analysis. Then, we propose a test for a change in the autoregression function which – by the uniform central limit theorem – has asymptotic power one for a large class of alternatives including local alternatives not restricted to the correctly specified model.

The uniform central limit theorem for the tent map

This paper considers the asymptotic behaviors of the processes generated by the classical ergodic tent map that is defined on the unit interval. We get the uniform version of central limit theorem for the tent map by using the method of uniformly integrable entropy. An application to Kolmogorov–Smirnov type result is provided.

A uniform central limit theorem and efficiency for deconvolution estimators

We estimate linear functionals in the classical deconvolution problem by kernel estimators. We obtain a uniform central limit theorem with $\sqrt{n}$-rate on the assumption that the smoothness of the functionals is larger than the ill-posedness of the problem, which is given by the polynomial decay rate of the characteristic function of the error. The limit distribution is a generalized Brownian bridge with a covariance structure that depends on the characteristic function of the error and on the functionals. The proposed estimators are optimal in the sense of semiparametric efficiency. The class of linear functionals is wide enough to incorporate the estimation of distribution functions. The proofs are based on smoothed empirical processes and mapping properties of the deconvolution operator.

ON SET-INDEXED RESIDUAL PARTIAL SUM LIMIT PROCESS OF SPATIAL LINEAR REGRESSION MODELS

In this paper we derive the limit process of the sequence of set-indexedleast-squares residual partial sum processes of observations obtained form a spatiallinear regression model. For the proof of the result we apply the uniform central limittheorem of Alexander and Pyke (1986) and generalize the geometrical approach ofBischo (2002) and Bischo and Somayasa (2009). It is shown that the limit processis a projection of the set-indexed Brownian sheet onto the reproducing kernel Hilbertspace of this process. For that we dene the projection via Choquet integral of theregression function with respect to the set-indexed Brownian sheet.DOI : http://dx.doi.org/10.22342/jims.17.2.4.73-83

On the area of excursion sets of spherical Gaussian eigenfunctions

The high frequency behaviour for random eigenfunctions of the spherical Laplacian has been recently the object of considerable interest, also because of strong motivation arising from physics and cosmology. In this paper, we are concerned with the high frequency behaviour of excursion sets; in particular, we establish a uniform central limit theorem for the empirical measure, i.e., the proportion of spherical surface, where spherical Gaussian eigenfunctions lie below a level z. Our proofs borrow some techniques from the literature on stationary long memory processes; in particular, we expand the empirical measure into Hermite polynomials, and establish a uniform weak reduction principle, entailing that the asymptotic behaviour is asymptotically dominated by a single term in the expansion. As a result, we establish a functional central limit theorem; the limiting process is fully degenerate.

Central limit theorems for local empirical processes near boundaries of sets

We define the local empirical process, based on $n$ i.i.d. random vectors in dimension $d$, in the neighborhood of the boundary of a fixed set. Under natural conditions on the shrinking neighborhood, we show that, for these local empirical processes, indexed by classes of sets that vary with $n$ and satisfy certain conditions, an appropriately defined uniform central limit theorem holds. The concept of differentiation of sets in measure is very convenient for developing the results. Some examples and statistical applications are also presented.