Uniform Convergence Rate Research Articles

Logical aspects of rates of convergence in metric spaces

AbstractIn this paper we develop a method for finding, under general conditions, explicit and highly uniform rates of convergence for the Picard iteration sequences for selfmaps on bounded metric spaces from ineffective proofs of convergence to a unique fixed point. We are able to extract full rates of convergence by extending the use of a logical metatheorem recently proved by Kohlenbach. In recent case studies we were able to find such explicit rates of convergence in two concrete cases. Our novel method now provides an explanation in logical terms for these findings. This amounts, loosely speaking, to general conditions under which we in this specific setting can transform a ∀∃∀-sentence into a ∀∃-sentence via an argument involving product spaces. This reduction in logical complexity allows us to use the existing machinery to extract quantitative bounds of the sort we need.

The imaging performance of a LaBr3-based PET scanner

A prototype time-of-flight (TOF) PET scanner based on cerium-doped lanthanum bromide [LaBr3 (5% Ce)] has been developed. LaBr3 has a high light output, excellent energy resolution and fast timing properties that have been predicted to lead to good image quality. Intrinsic performance measurements of spatial resolution, sensitivity and scatter fraction demonstrate good conventional PET performance; the results agree with previous simulation studies. Phantom measurements show the excellent image quality achievable with the prototype system. Phantom measurements and corresponding simulations show a faster and more uniform convergence rate, as well as more uniform quantification, for TOF reconstruction of the data, which have 375 ps intrinsic timing resolution, compared to non-TOF images. Measurements and simulations of a hot and cold sphere phantom show that the 7% energy resolution helps to mitigate residual errors in the scatter estimate because a high energy threshold (>480 keV) can be used to restrict the amount of scatter accepted without a loss of true events. Preliminary results with incorporation of a model of detector blurring in the iterative reconstruction algorithm not only show improved contrast recovery but also point out the importance of an accurate resolution model of the tails of LaBr3's point spread function. The LaBr3 TOF-PET scanner demonstrated the impact of superior timing and energy resolutions on image quality.

Nonparametric covariate adjustment for receiver operating characteristic curves

AbstractThe accuracy of a diagnostic test is typically characterized using the receiver operating characteristic (ROC) curve. Summarizing indexes such as the area under the ROC curve (AUC) are used to compare different tests as well as to measure the difference between two populations. Often additional information is available on some of the covariates which are known to influence the accuracy of such measures. The authors propose nonparametric methods for covariate adjustment of the AUC. Models with normal errors and possibly non‐normal errors are discussed and analyzed separately. Nonparametric regression is used for estimating mean and variance functions in both scenarios. In the model that relaxes the assumption of normality, the authors propose a covariate‐adjusted Mann–Whitney estimator for AUC estimation which effectively uses available data to construct working samples at any covariate value of interest and is computationally efficient for implementation. This provides a generalization of the Mann–Whitney approach for comparing two populations by taking covariate effects into account. The authors derive asymptotic properties for the AUC estimators in both settings, including asymptotic normality, optimal strong uniform convergence rates and mean squared error (MSE) consistency. The MSE of the AUC estimators was also assessed in smaller samples by simulation. Data from an agricultural study were used to illustrate the methods of analysis. The Canadian Journal of Statistics 38:27–46; 2010 © 2009 Statistical Society of Canada

Rates of strong uniform convergence of the k T -occupation time density estimator

In this paper, we study an extension of the k n nearest neighbor density estimators in continuous time. We obtain the optimal and the superoptimal uniform convergence rates up to a logarithm term on a compact set, in the context of strongly mixing processes.

The Key Theorem and the Bounds on the Rate of Uniform Convergence of Statistical Learning Theory on Quasi-Probability Spaces

Some properties of quasi-probability are further discussed.The definitions and properties of quasi-random variable and its distribution function,expected value and variance are then presented.Markov inequality,Chebyshev's inequality and the Khinchine's law of large numbers on quasi-probability spaces are also proved.Then the key theorem of learning theory on quasi-probability spaces is proved,and the bounds on the rate of uniform convergence of learning process on quasi-probability spaces are constructed.The investigations will help lay essential theoretical foundations for the systematic and comprehensive development of the quasi-statistical learning theory.

Uniform superconvergence analysis of the discontinuous Galerkin method for a singularly perturbed problem in 1-D

It has been observed from the authors' numerical experiments (2007) that the Local Discontinuous Galerkin (LDG) method converges uniformly under the Shishkin mesh for singularly perturbed two-point boundary problems of the convection-diffusion type. Especially when using a piecewise polynomial space of degree k, the LDG solution achieves the optimal convergence rate k+1 1 under the L 2 -norm, and a superconvergence rate 2k+1 1 for the one-sided flux uniformly with respect to the singular perturbation parameter ∈. In this paper, we investigate the theoretical aspect of this phenomenon under a simplified ODE model. In particular, we establish uniform convergence rates √∈ (1nN/N) k+1 for the L 2 -norm and ( ln N / N ) 2k+1 for the one-sided flux inside the boundary layer region. Here N (even) is the number of elements.

On the properties of systems of root functions of a second-order discontinuous operator

In [1], a second-order discontinuous operator was studied and constructive necessary and sufficient conditions were established for the Riesz basis property of the system of root functions of this operator. These works played an important role in the study of the absolute and uniform convergence of biorthogonal expansions of -functions in terms of the root functions of the Schrodinger operator with multipoint boundary conditions [2, 3]. For a second-order discontinuous operator, we establish a necessary and sufficient condition for the Riesz basis property of its systems of root functions and analyze the absolute and uniform convergence and the rate of uniform convergence of biorthogonal expansions of discontinuous functions in terms of root functions.

Estimating monotone convex functions via sequential shape modification

We propose a sequential method to estimate monotone convex functions that consists of: (i) monotone regression via solving a constrained least square (LS) problem and (ii) convexification of the monotone regression estimate via solving a uniform approximation problem with associated constraints. We show that this method is faster than the constrained LS method. The ratio of computation time increases as data size increases. Moreover, we show that, under an appropriate smoothness condition, the uniform convergence rate achieved by the proposed method is nearly comparable to the best achievable rate for a non-parametric estimate which ignores the shape constraint. Simulation studies show that our method is comparable to the constrained LS method in estimation error. We illustrate our method by analysing ground water level data of wells in Korea.

On optimal estimation of a non-smooth mode in a nonparametric regression model with [formula omitted]-mixing errors

We consider the problem of mode estimation in the fixed-design regression model, the regression function having a unique non-smooth mode. We estimate the mode by maximization over the curve estimator, which is given as a weighted mean of the observations, including most of the common kernel estimators, such as Gasser–Müller, Priestley–Chao and Nadaraya–Watson. To obtain optimal rates of convergence of the mode estimator, we first derive upper bounds, where we benefit from the contrast of the curve at the mode rather than taking into account the rate of uniform convergence of the curve estimator. In a next step we show that these rates are also optimal. We prove our results for α -mixing observations, and a non-smooth regression function that is only assumed to be Hölder continuous. Our method consists in a rather direct evaluation of the mean squared error of the empirical mode, using a recent moment inequality of Rosenthal type due to Yang [2007. Maximal moment inequality for partial sums of strong mixing sequences and application. Acta Math. Sinica 23, 1013–1024] for mixing random variables.

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n − 1 / 2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.

Limit distribution theory for maximum likelihood estimation of a log-concave density

We find limiting distributions of the nonparametric maximum likelihood estimator (MLE) of a log-concave density, i.e. a density of the form f(0) = exp varphi(0) where varphi(0) is a concave function on R. Existence, form, characterizations and uniform rates of convergence of the MLE are given by Rufibach (2006) and Dümbgen and Rufibach (2007). The characterization of the log-concave MLE in terms of distribution functions is the same (up to sign) as the characterization of the least squares estimator of a convex density on [0, infinity) as studied by Groeneboom, Jongbloed and Wellner (2001b). We use this connection to show that the limiting distributions of the MLE and its derivative are, under comparable smoothness assumptions, the same (up to sign) as in the convex density estimation problem. In particular, changing the smoothness assumptions of Groeneboom, Jongbloed and Wellner (2001b) slightly by allowing some higher derivatives to vanish at the point of interest, we find that the pointwise limiting distributions depend on the second and third derivatives at 0 of H(k), the "lower invelope" of an integrated Brownian motion process minus a drift term depending on the number of vanishing derivatives of varphi(0) = log f(0) at the point of interest. We also establish the limiting distribution of the resulting estimator of the mode M(f(0)) and establish a new local asymptotic minimax lower bound which shows the optimality of our mode estimator in terms of both rate of convergence and dependence of constants on population values.

Asymptotic properties of conditional quantile estimator for censored dependent observations

In this paper, we establish strong uniform convergence and asymptotic normality of the conditional quantile estimator for the censorship model when the data exhibit some kind of dependence. It is assumed that the observations form a stationary α-mixing sequence. The strong uniform convergence in iid framework has recently been discussed by Ould-Said (Stat Probab Lett 76:579–586, 2006). As a by-product, we also obtain a uniform weak convergence rate for the product-limit estimator of the lifetime and censoring distributions under dependence, which is interesting independently.

Univariate modified Fourier methods for second order boundary value problems

We develop and analyse a new spectral-Galerkin method for the numerical solution of linear, second order differential equations with homogeneous Neumann boundary conditions. The basis functions for this method are the eigenfunctions of the Laplace operator subject to these boundary conditions. Due to this property this method has a number of beneficial features, including an $\mathcal{O}(N^{2})$ condition number and the availability of an optimal, diagonal preconditioner. This method offers a uniform convergence rate of $\mathcal{O}(N^{-3})$ , however we show that by the inclusion of an additional 2M basis functions, this figure can be increased to $\mathcal{O}(N^{-2M-3})$ for any positive integer M.

Some theoretical results of learning theory based on random sets in set‐valued probability space

PurposeThe purpose of this paper is to introduce some basic knowledge of statistical learning theory (SLT) based on random set samples in set‐valued probability space for the first time and generalize the key theorem and bounds on the rate of uniform convergence of learning theory in Vapnik, to the key theorem and bounds on the rate of uniform convergence for random sets in set‐valued probability space. SLT based on random samples formed in probability space is considered, at present, as one of the fundamental theories about small samples statistical learning. It has become a novel and important field of machine learning, along with other concepts and architectures such as neural networks. However, the theory hardly handles statistical learning problems for samples that involve random set samples.Design/methodology/approachBeing motivated by some applications, in this paper a SLT is developed based on random set samples. First, a certain law of large numbers for random sets is proved. Second, the definitions of the distribution function and the expectation of random sets are introduced, and the concepts of the expected risk functional and the empirical risk functional are discussed. A notion of the strict consistency of the principle of empirical risk minimization is presented.FindingsThe paper formulates and proves the key theorem and presents the bounds on the rate of uniform convergence of learning theory based on random sets in set‐valued probability space, which become cornerstones of the theoretical fundamentals of the SLT for random set samples.Originality/valueThe paper provides a studied analysis of some theoretical results of learning theory.

The saturation of convergence on the interval [0,1] for the q-Bernstein polynomials in the case q>1

In the note, we consider saturation of convergence on the interval [ 0 , 1 ] for the q -Bernstein polynomials of a continuous function f for arbitrary fixed q > 1 . We show that the rate of uniform convergence on [ 0 , 1 ] is o ( q − n ) if and only if f is linear. The result is sharp in the following sense: it ceases to be true if we replace “ o ” by “ O ”.

The generalization performance of ERM algorithm with strongly mixing observations

The generalization performance is the main concern of machine learning theoretical research. The previous main bounds describing the generalization ability of the Empirical Risk Minimization (ERM) algorithm are based on independent and identically distributed (i.i.d.) samples. In order to study the generalization performance of the ERM algorithm with dependent observations, we first establish the exponential bound on the rate of relative uniform convergence of the ERM algorithm with exponentially strongly mixing observations, and then we obtain the generalization bounds and prove that the ERM algorithm with exponentially strongly mixing observations is consistent. The main results obtained in this paper not only extend the previously known results for i.i.d. observations to the case of exponentially strongly mixing observations, but also improve the previous results for strongly mixing samples. Because the ERM algorithm is usually very time-consuming and overfitting may happen when the complexity of the hypothesis space is high, as an application of our main results we also explore a new strategy to implement the ERM algorithm in high complexity hypothesis space.

Empirical spectral processes for locally stationary time series

A time-varying empirical spectral process indexed by classes of functions is defined for locally stationary time series. We derive weak convergence in a function space, and prove a maximal exponential inequality and a Glivenko–Cantelli-type convergence result. The results use conditions based on the metric entropy of the index class. In contrast to related earlier work, no Gaussian assumption is made. As applications, quasi-likelihood estimation, goodness-of-fit testing and inference under model misspecification are discussed. In an extended application, uniform rates of convergence are derived for local Whittle estimates of the parameter curves of locally stationary time series models.

On the distribution of the adaptive LASSO estimator

We study the distribution of the adaptive LASSO estimator [Zou, H., 2006. The adaptive LASSO and its oracle properties. J. Amer. Statist. Assoc. 101, 1418–1429] in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly nonnormal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than n - 1 / 2 in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the ‘oracle’ property of the adaptive LASSO estimator established in Zou [2006. The adaptive LASSO and its oracle properties. J. Amer. Statist. Assoc. 101, 1418–1429]. Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator. The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using nonorthogonal regressors.

A strong uniform convergence rate of a kernel conditional quantile estimator under random left-truncation and dependent data

In this paper we study some asymptotic properties of the kernel conditional quantile estimator with randomly left-truncated data which exhibit some kind of dependence. We extend the result obtained by Lemdani, Ould-Saïd and Poulin [16] in the iid case. The uniform strong convergence rate of the estimator under strong mixing hypothesis is obtained.

Uniform Rate of Weak Convergence of the Minimum Contrast Estimator in the Ornstein–Uhlenbeck Process

The paper shows that the distribution of the normalized minimum contrast estimator of the drift parameter in the Ornstein–Uhlenbeck process observed over [0, T] converges to the standard normal distribution with an uniform error rate of the order O (T − 1/2). A precise estimate of the constant in the upper bound is also given.