Convergence Rate Of Estimator Research Articles

Asymptotic properties of penalized splines for functional data

Penalized spline methods are popular for functional data analysis but their asymptotic properties have not been established. We present a theoretic study of the $L_{2}$ and uniform convergence of penalized splines for estimating the mean and covariance functions of functional data under general settings. The established convergence rates for the mean function estimation are mini-max rate optimal and the rates for the covariance function estimation are comparable to those using other smoothing methods.

Estimation of autocovariance matrices for high dimensional linear processes

In this paper under some mild restrictions upper bounds on the rate of convergence for estimators of ptimes p autocovariance and precision matrices for high dimensional linear processes are given. We show that these estimators are consistent in the operator norm in the sub-Gaussian case when p={mathcal {O}}left( n^{gamma /2}right) for some gamma >1, and in the general case when p^{2/beta }(n^{-1} log p)^{1/2}rightarrow 0 for some beta >2 as p=p(n)rightarrow infty and the sample size nrightarrow infty . In particular our results hold for multivariate AR processes. We compare our results with those previously obtained in the literature for independent and dependent data. We also present non-asymptotic bounds for the error probability of these estimators.

Improving covariance-regularized discriminant analysis for EHR-based predictive analytics of diseases

Linear Discriminant Analysis (LDA) is a well-known technique for feature extraction and dimension reduction. The performance of classical LDA however, significantly degrades on the High Dimension Low Sample Size (HDLSS) data for the ill-posed inverse problem. Existing approaches for HDLSS data classification typically assume the data in question are with Gaussian distribution and deal the HDLSS classification problem with regularization. However, these assumptions are too strict to hold in many emerging real-life applications, such as enabling personalized predictive analysis using Electronic Health Records (EHRs) data collected from an extremely limited number of patients who have been diagnosed with or without the target disease for prediction. In this paper, we revised the problem of predictive analysis of disease using personal EHR data and LDA classifier. To fill the gap, in this paper, we first studied an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. The model gives a theoretical upper bound of LDA error rate that is controlled by two factors: (1) the statistical convergence rate of (inverse) covariance matrix estimators and (2) the divergence of the training/testing datasets to fitted distributions. To this end, we could lower the error rate by balancing the two factors for better classification performance. Hereby, we further proposed a novel LDA classifier De-Sparse that leverages De-sparsified Graphical Lasso to improve the estimation of LDA, which outperforms state-of-the-art LDA approaches developed for HDLSS data. Such advances and effectiveness are further demonstrated by both theoretical analysis and extensive experiments on EHR datasets https://www.overleaf.com/project/5d2728c718f6ff3b2bcf5991 .

On Complete Consistency for the Estimator of Nonparametric Regression Model Based on Asymptotically Almost Negatively Associated Errors

In this paper, we mainly study the consistency for the estimator of nonparametric regression model based on asymptotically almost negatively associated (AANA, in short) errors. Firstly, the Bernstein type inequality for AANA random variables is established. By using the Bernstein type inequality and moment inequalities, we investigate the complete consistency and convergence rate for the estimator of nonparametric regression model based on AANA errors. As applications, the complete consistency and convergence rate for the nearest neighbor estimator are obtained.

Convergence Rates of Wavelet Density Estimators for Strongly Mixing Samples

This paper considers wavelet estimations for a multivariate density function based on strongly mixing data. We first construct a linear wavelet estimator and provide a convergence rate over $$L^{p} (1\le p<\infty )$$ risk in Besov space $$B^{s}_{r,q}(\mathbb {R}^{d})$$ . However, this estimator depends on the smoothness of density function, which means that the estimator is not adaptive. A nonlinear adaptive wavelet estimator is proposed by thresholding method. Moreover, the convergence rate of nonlinear estimator is better than the linear one in the case of $$r\le p$$ .

Modified Kalman filtering based multi-step-length gradient iterative algorithm for ARX models with random missing outputs

This study presents a modified Kalman filtering-based multi-step-length gradient iterative algorithm to identify ARX models with missing outputs. The Kalman filtering method is modified to enhance the estimation of unmeasurable outputs, laying the foundation for enabling the multi-step-length gradient iterative algorithm to update effectively the ARX model parameter estimation through the estimated outputs. Compared to the classical gradient iterative algorithm, this study improves the estimation accuracy of the missing outputs by introducing a modified Kalman filter, and the parameter estimation convergence rate by deriving a new multi-step-length formulation. To validate the framework and the algorithm developed, a series of bench tests were conducted with computational experiments. The simulated numerical results are consistent with the analytically derived results in terms of the feasibility and effectiveness of the proposed procedure.

Asymptotic behavior of cross spectral density estimator at the zero frequency in the presence of degeneracy

We analyze the asymptotic properties of nonparametric cross spectral density estimators when the time series process has a degenerate spectrum at the origin in the frequency domain. Degeneracy at the zero frequency caused by over-differencing the series invalidates inferences based on cross spectral methods. We derive the asymptotic mean squared errors of kernel-based cross spectral density estimators at the zero frequency. It is found that degeneracy entails different asymptotics including convergence rates for nonparametric cross spectral density estimators from existing results in time series econometrics literature.

The consistency and convergence rate for the nearest neighbor density estimator based on -mixing random samples

In this work, we mainly investigate the consistency and strong convergence rate for the nearest neighbor density estimator based on -mixing random samples. The weak consistency, complete consistency, the rates of complete consistency and strong consistency for the nearest neighbor estimator of density function based on -mixing random samples are established. The results obtained in the article extend some corresponding ones for independent samples.

A Bi-Invariant Statistical Model Parametrized by Mean and Covariance on Rigid Motions.

This paper aims to describe a statistical model of wrapped densities for bi-invariant statistics on the group of rigid motions of a Euclidean space. Probability distributions on the group are constructed from distributions on tangent spaces and pushed to the group by the exponential map. We provide an expression of the Jacobian determinant of the exponential map of which enables the obtaining of explicit expressions of the densities on the group. Besides having explicit expressions, the strengths of this statistical model are that densities are parametrized by their moments and are easy to sample from. Unfortunately, we are not able to provide convergence rates for density estimation. We provide instead a numerical comparison between the moment-matching estimators on and , which shows similar behaviors.

On mean derivative estimation of longitudinal and functional data: from sparse to dense

Derivative estimation of the mean of longitudinal and functional data is useful, because it provides a quantitative measure of changes in the mean function that can be used for modeling of the data. We propose a general method for estimation of the derivative of the mean function that allows us to make inference about both longitudinal and functional data regardless of the sparsity of data. The $$L^2$$ and uniform convergence rates of the local linear estimator for the true derivative of the mean function are derived. Then the optimal weighting scheme under the $$L^2$$ rate of convergence is obtained. The performance of the proposed method is evaluated by a simulation study, and additionally compared with another existing method. The method is used to analyse a real data set involving children weight growth failure.

Wavelet regression estimation over Lp risk based on negatively associated sample

This paper considers wavelet estimations of a regression function based on negatively associated sample. We provide upper bound estimations over [Formula: see text] risk of linear and nonlinear wavelet estimators in Besov space, respectively. When the random sample reduces to the independent case, our convergence rates coincide with the optimal convergence rates of classical nonparametric regression estimation.

Partial deconvolution estimation in nonparametric regression

AbstractIn this article, we propose a class of partial deconvolution kernel estimators for the nonparametric regression function when some covariates are measured with error and some are not. The estimation procedure combines the classical kernel methodology and the deconvolution kernel technique. According to whether the measurement error is ordinarily smooth or supersmooth, we establish the optimal local and global convergence rates for these proposed estimators, and the optimal bandwidths are also identified. Furthermore, lower bounds for the convergence rates of all possible estimators for the nonparametric regression functions are developed. It is shown that, in both the super and ordinarily smooth cases, the convergence rates of the proposed partial deconvolution kernel estimators attain the lower bound. The Canadian Journal of Statistics 48: 535–560; 2020 © 2020 Statistical Society of Canada

Nonparametric Quantile Regression Estimation With Mixed Discrete and Continuous Data

In this article, we investigate the problem of nonparametrically estimating a conditional quantile function with mixed discrete and continuous covariates. A local linear smoothing technique combining both continuous and discrete kernel functions is introduced to estimate the conditional quantile function. We propose using a fully data-driven cross-validation approach to choose the bandwidths, and further derive the asymptotic optimality theory. In addition, we also establish the asymptotic distribution and uniform consistency (with convergence rates) for the local linear conditional quantile estimators with the data-dependent optimal bandwidths. Simulations show that the proposed approach compares well with some existing methods. Finally, an empirical application with the data taken from the IMDb website is presented to analyze the relationship between box office revenues and online rating scores. Supplementary materials for this article are available online.

Wavelet thresholding estimation of density derivatives from a negatively associated size-biased sample

This paper considers wavelet estimation for density derivatives based on negatively associated and size-biased data. We provide upper bounds of nonlinear wavelet estimator on [Formula: see text] risk. It turns out that the convergence rate of the nonlinear estimator is better than that of the linear one.

Nonparametric Estimation of Conditional Expectation with Auxiliary Information and Dimension Reduction

Nonparametric estimation of the conditional expectation of an outcome Y given a covariate vector U is of primary importance in many statistical applications such as prediction and personalized medicine. In some problems, there is an additional auxiliary variable Z in the training dataset used to construct estimators, but Z is not available for future prediction or selecting patient treatment in personalized medicine. For example, in the training dataset longitudinal outcomes are observed, but only the last outcome Y is concerned in the future prediction or analysis. The longitudinal outcomes other than the last point is then the variable Z that is observed and related with both Y and U. Previous work on how to make use of Z in the estimation of mainly focused on using Z in the construction of a linear function of U to reduce covariate dimension for better estimation. Using , we propose a two-step estimation of inner and outer expectations, respectively, with sufficient dimension reduction for kernel estimation in both steps. The information from Z is utilized not only in dimension reduction, but also directly in the estimation. Because of the existence of different ways for dimension reduction, we construct two estimators that may improve the estimator without using Z. The improvements are shown in the convergence rate of estimators as the sample size increases to infinity as well as in the finite sample simulation performance. A real data analysis about the selection of mammography intervention is presented for illustration.

Robust estimation of mixing measures in finite mixture models

In finite mixture models, apart from underlying mixing measure, true kernel density function of each subpopulation in the data is, in many scenarios, unknown. Perhaps the most popular approach is to choose some kernel functions that we empirically believe our data are generated from and use these kernels to fit our models. Nevertheless, as long as the chosen kernel and the true kernel are different, statistical inference of mixing measure under this setting will be highly unstable. To overcome this challenge, we propose flexible and efficient robust estimators of the mixing measure in these models, which are inspired by the idea of minimum Hellinger distance estimator, model selection criteria, and superefficiency phenomenon. We demonstrate that our estimators consistently recover the true number of components and achieve the optimal convergence rates of parameter estimation under both the well- and misspecified kernel settings for any fixed bandwidth. These desirable asymptotic properties are illustrated via careful simulation studies with both synthetic and real data.

Rank Determination in Tensor Factor Model

Factor model is an appealing and effective analytic tool for high-dimensional time series, with a wide range of applications in economics, finance and statistics. One of the fundamental issues in using factor model for time series in practice is the determination of the number of factors to use. This paper develops two criteria for such a task for tensor factor models where the signal part of an observed time series in tensor form assumes a Tucker decomposition with the core tensor as the factor tensor. The task is to determine the dimensions of the core tensor. One of the proposed criteria is similar to information based criteria of model selection, and the other is an extension of the approaches based on the ratios of consecutive eigenvalues often used in factor analysis for panel time series. The new criteria are designed to locate the gap between the true smallest non-zero eigenvalue and the zero eigenvalues of a functional of the population version of the auto-ross-covariances of the tensor time series using their sample versions. As sample size and tensor dimension increase, such a gap increases under regularity conditions, resulting in consistency of the rank estimator. The criteria are built upon the existing non-iterative and iterative estimation procedures of tensor factor model, yielding different performances. We provide sufficient conditions and convergence rate for the consistency of the criteria as the sample size $T$ and the dimensions of the observed tensor time series go to infinity. The results include the vector factor models as special cases, with an additional convergence rates. The results also include the cases when there exist factors with different signal strength. In addition, the convergence rates of the eigenvalue estimators are established. Simulation studies provide promising finite sample performance for the two criteria.

A Consensus-Based Diffusion Levenberg-Marquardt Method for Collaborative Localization With Extension to Distributed Optimization

Non-linear least squares problems arise from data fitting have received recently a lot of attention, particularly for the estimates of the model parameters over networked systems. Although the diffusion Gauss-Newton method offers many advantages for solving the non-linear least squares problem in wireless sensor network to estimate target position parameter, there are some key challenges when applying it to practice, including singularity of Gauss-Newton Hessian, selection to constant step sizes and steady state oscillation. These remaining issues lead to obvious performance degradation such as high computational cost, vulnerability to step size change and resulting instability on estimation.In this paper, to eliminate the singularity, we develop a diffusion Levenberg-Marquardt method such that the problem of constant step size is also addressed together. Then, to reach agreement on estimated vector, a consensus implementation is further proposed, thus eliminating the oscillation during steady state. Consequently, the proposed consensus-based diffusion Levenberg-Marquardt method provides a general solution for the non-linear least squares problems with an objective that takes the form of a sum of squared residual terms. By applying to collaborative localization and distributed optimization arise in large scale machine learning, simulation results confirm the effectiveness and wide applicability of proposed method in terms of convergence rate, accuracy and consistency of estimates.

Function-on-Partially Linear Functional Additive Models

We consider a functional partially linear additive model that predicts a functional response by a scalar predictor and functional predictors. The B-spline and eigenbasis least squares estimator for both the parametric and the nonparametric components proposed. In the final of this paper, as a result, we got the variance decomposition of the model and establish the asymptotic convergence rate for estimator.

On aggregation of strongly dependent time series

AbstractWe consider cross‐sectional aggregation of time series with long‐range dependence. This question arises for instance from the statistical analysis of networks where aggregation is defined via routing matrices. Asymptotically, aggregation turns out to increase dependence substantially, transforming a hyperbolic decay of autocorrelations to a slowly varying rate. This effect has direct consequences for statistical inference. For instance, unusually slow rates of convergence for nonparametric trend estimators and nonstandard formulas for optimal bandwidths are obtained. The situation changes, when time‐dependent aggregation is applied. Suitably chosen time‐dependent aggregation schemes can preserve a hyperbolic rate or even eliminate autocorrelations completely.