Convergence Rate Of Estimator Research Articles

ON THE CONVERGENCE RATE OF POTENTIALS OF BRENIER MAPS

The theory of optimal transportation has experienced a sharp increase in interest in many areas of economic research such as optimal matching theory and econometric identification. A particularly valuable tool, due to its convenient representation as the gradient of a convex function, has been the Brenier map: the matching obtained as the optimizer of the Monge–Kantorovich optimal transportation problem with the euclidean distance as the cost function. Despite its popularity, the statistical properties of the Brenier map have yet to be fully established, which impedes its practical use for estimation and inference. This article takes a first step in this direction by deriving a convergence rate for the simple plug-in estimator of the potential of the Brenier map via the semi-dual Monge–Kantorovich problem. Relying on classical results for the convergence of smoothed empirical processes, it is shown that this plug-in estimator converges in standard deviation to its population counterpart under the minimax rate of convergence of kernel density estimators if one of the probability measures satisfies the Poincaré inequality. Under a normalization of the potential, the result extends to convergence in the $L^2$ norm, while the Poincaré inequality is automatically satisfied. The main mathematical contribution of this article is an analysis of the second variation of the semi-dual Monge–Kantorovich problem, which is of independent interest.

Robust estimation of superhedging prices

We consider statistical estimation of superhedging prices using historical stock returns in a frictionless market with $d$ traded assets. We introduce a plug-in estimator based on empirical measures and show it is consistent but lacks suitable robustness. To address this, we propose novel estimators which use a larger set of martingale measures defined through a tradeoff between the radius of Wasserstein balls around the empirical measure and the allowed norm of martingale densities. We then extend our study, in part, to estimation of risk measures, to the case of markets with traded options, to a multi-period setting and to settings with model uncertainty. We also study convergence rates of estimators and convergence of super-hedging strategies.

Sequential Shrinkage Estimate for COX Regression Models with Uncertain Number of Effective Variables

In the applications of COX regression models, we always encounter data sets that contain too many variables that only a few of them contribute to the model. Therefore, it will waste much more samples to estimate the “noneffective” variables in the inference. In this paper, we use a sequential procedure for constructing the fixed size confidence set for the “effective” parameters to the model based on an adaptive shrinkage estimate such that the “effective” coefficients can be efficiently identified with the minimum sample size. Fixed design is considered for numerical simulation. The strong consistency, asymptotic distributions and convergence rates of estimates under the fixed design are obtained. In addition, the sequential procedure is shown to be asymptotically optimal in the sense of Chow and Robbins (1965).

Uniform Convergence Rates for Nonparametric Estimators of a Density Function and its Derivatives when the Density has a Known Pole

Uniform Convergence Rates for Nonparametric Estimators of a Density Function and its Derivatives when the Density has a Known Pole

Parameter estimation for SPDEs based on discrete observations in time and space

Parameter estimation for a parabolic linear stochastic partial differential equation in one space dimension is studied observing the solution field on a discrete grid in a fixed bounded domain. Considering an infill asymptotic regime in both coordinates, we prove central limit theorems for realized quadratic variations based on temporal and spatial increments as well as on double increments in time and space. Resulting method of moments estimators for the diffusivity and the volatility parameter inherit the asymptotic normality and can be constructed robustly with respect to the sampling frequencies in time and space. Upper and lower bounds reveal that in general the optimal convergence rate for joint estimation of the parameters is slower than the usual parametric rate. The theoretical results are illustrated in a numerical example.

Convergence rates for Bayesian estimation and testing in monotone regression

Shape restrictions such as monotonicity on functions often arise naturally in statistical modeling. We consider a Bayesian approach to the estimation of a monotone regression function and testing for monotonicity. We construct a prior distribution using piecewise constant functions. For estimation, a prior imposing monotonicity of the heights of these steps is sensible, but the resulting posterior is harder to analyze theoretically. We consider a “projection-posterior” approach, where a conjugate normal prior is used, but the monotonicity constraint is imposed on posterior samples by a projection map onto the space of monotone functions. We show that the resulting posterior contracts at the optimal rate n−1∕3 under the L1-metric and at a nearly optimal rate under the empirical Lp-metrics for 0<p≤2. The projection-posterior approach is also computationally more convenient. We also construct a Bayesian test for the hypothesis of monotonicity using the posterior probability of a shrinking neighborhood of the set of monotone functions. We show that the resulting test has a universal consistency property and obtain the separation rate which ensures that the resulting power function approaches one.

Complete consistency and convergence rate of the nearest neighbor estimator of the density function based on WOD samples

Complete consistency and convergence rate of the nearest neighbor estimator of the density function based on WOD samples

Antithetical stratified sampling estimator for filtering signals with discontinuities

A novel approach to signal filtering using digital alias-free signal processing (DASP) is presented in this paper. We propose an unbiased, fast-converging estimator of the output of a finite impulse response (FIR) continuous-time filter. The estimator processes 2N signal samples collected with the use of random antithetical stratified (AnSt) sampling technique. To assess the estimator convergence rate as a function of N, we consider various forms of smoothness of the input signal, filter impulse response and windowing function. The cases are piecewise-continuous second-order derivative (SOD), piecewise-continuous first-order derivative (FOD) and piecewise-continuous zero-order derivative (ZOD). In each case we assume that the respective derivative has a finite number of bounded discontinuities. We prove that the proposed estimator converges to the true filter output at the rate of N−5 in the first case. But for the other two the rate drops to N−4 and N−2 respectively.

Invariant density adaptive estimation for ergodic jump–diffusion processes over anisotropic classes

We consider the solution X=(Xt)t≥0 of a multivariate stochastic differential equation with Levy-type jumps and with unique invariant probability measure with density μ. We assume that a continuous record of observations XT=(Xt)0≤t≤T is available.In the case without jumps, Dalalyan and Reiss (2007) and Strauch (2018) have found convergence rates of invariant density estimators, under respectively isotropic and anisotropic Hölder smoothness constraints, which are considerably faster than those known from standard multivariate density estimation.We extend the previous works by obtaining, in presence of jumps, some estimators which have the same convergence rates they had in the case without jumps for d≥2 and a rate which depends on the degree of the jumps in the one-dimensional setting. We propose moreover a data driven bandwidth selection procedure based on the Goldenshluger and Lepski method (Goldenshluger and Lepski, 2011) which leads us to an adaptive non-parametric kernel estimator of the stationary density μ of the jump–diffusion X.

Multiple block sizes and overlapping blocks for multivariate time series extremes

Block maxima methods constitute a fundamental part of the statistical toolbox in extreme value analysis. However, most of the corresponding theory is derived under the simplifying assumption that block maxima are independent observations from a genuine extreme value distribution. In practice, however, block sizes are finite and observations from different blocks are dependent. Theory respecting the latter complications is not well developed, and, in the multivariate case, has only recently been established for disjoint blocks of a single block size. We show that using overlapping blocks instead of disjoint blocks leads to a uniform improvement in the asymptotic variance of the multivariate empirical distribution function of rescaled block maxima and any smooth functionals thereof (such as the empirical copula), without any sacrifice in the asymptotic bias. We further derive functional central limit theorems for multivariate empirical distribution functions and empirical copulas that are uniform in the block size parameter, which seems to be the first result of this kind for estimators based on block maxima in general. The theory allows for various aggregation schemes over multiple block sizes, leading to substantial improvements over the single block length case and opens the door to further methodology developments. In particular, we consider bias correction procedures that can improve the convergence rates of extreme-value estimators and shed some new light on estimation of the second-order parameter when the main purpose is bias correction.

Block bootstrap optimality and empirical block selection for sample quantiles with dependent data

Summary We establish a general theory of optimality for block bootstrap distribution estimation for sample quantiles under mild strong mixing conditions. In contrast to existing results, we study the block bootstrap for varying numbers of blocks. This corresponds to a hybrid between the sub- sampling bootstrap and the moving block bootstrap, in which the number of blocks is between 1 and the ratio of sample size to block length. The hybrid block bootstrap is shown to give theoretical benefits, and startling improvements in accuracy in distribution estimation in important practical settings. The conclusion that bootstrap samples should be of smaller size than the original sample has significant implications for computational efficiency and scalability of bootstrap methodologies with dependent data. Our main theorem determines the optimal number of blocks and block length to achieve the best possible convergence rate for the block bootstrap distribution estimator for sample quantiles. We propose an intuitive method for empirical selection of the optimal number and length of blocks, and demonstrate its value in a nontrivial example.

Adaptive elastic-net selection in a quantile model with diverging number of variable groups

In real applications of the linear model, the explanatory variables are very often naturally grouped, the most common example being the multivariate variance analysis. In the present paper, a quantile model with group structure is considered, where the number of groups can diverge with the sample size. We introduce and study the adaptive elastic-net group quantile estimator, for improving the parameter estimation accuracy. This method allows automatic selection, with a probability converging to 1, of the non-zero coefficient vectors and further, the asymptotic normality of the non-zero parameter estimators. We also give the convergence rate of the adaptive elastic-net group quantile estimator, rate which depends on the number of the groups. In order to put the estimation method into practice, an algorithm based on the subgradient method is proposed and implemented. The performed Monte Carlo simulations show that the adaptive elastic-net group quantile estimations are more accurate than other existing group estimations in the literature. Moreover, the numerical study confirms the theoretical results and the usefulness of the proposed estimation method.

Uniform convergence rate of the nonparametric maximum likelihood estimator for current status data with competing risks

ABSTRACT We study the uniform convergence rate of the nonparametric maximum likelihood estimator (MLE) for the sub-distribution functions of failure times in the current status data with competing risks model. It is known that the MLE have -norm convergence rate in the absolutely continuous case, but no agreement has emerged on a uniform convergence rate. We specify conditions under which is the uniform convergence rate for the MLE of the sub-distribution functions on finite intervals. The obtained result refines known uniform convergence rate in the particular case of current status data. The main result is applied in order to get the uniform convergence rate of the MLE for the survival function of failure time in the current status right-censored data model.

Asymptotic properties of penalized splines for functional data

Penalized spline methods are popular for functional data analysis but their asymptotic properties have not been established. We present a theoretic study of the $L_{2}$ and uniform convergence of penalized splines for estimating the mean and covariance functions of functional data under general settings. The established convergence rates for the mean function estimation are mini-max rate optimal and the rates for the covariance function estimation are comparable to those using other smoothing methods.

Estimation of autocovariance matrices for high dimensional linear processes

In this paper under some mild restrictions upper bounds on the rate of convergence for estimators of ptimes p autocovariance and precision matrices for high dimensional linear processes are given. We show that these estimators are consistent in the operator norm in the sub-Gaussian case when p={mathcal {O}}left( n^{gamma /2}right) for some gamma >1, and in the general case when p^{2/beta }(n^{-1} log p)^{1/2}rightarrow 0 for some beta >2 as p=p(n)rightarrow infty and the sample size nrightarrow infty . In particular our results hold for multivariate AR processes. We compare our results with those previously obtained in the literature for independent and dependent data. We also present non-asymptotic bounds for the error probability of these estimators.

Improving covariance-regularized discriminant analysis for EHR-based predictive analytics of diseases

Linear Discriminant Analysis (LDA) is a well-known technique for feature extraction and dimension reduction. The performance of classical LDA however, significantly degrades on the High Dimension Low Sample Size (HDLSS) data for the ill-posed inverse problem. Existing approaches for HDLSS data classification typically assume the data in question are with Gaussian distribution and deal the HDLSS classification problem with regularization. However, these assumptions are too strict to hold in many emerging real-life applications, such as enabling personalized predictive analysis using Electronic Health Records (EHRs) data collected from an extremely limited number of patients who have been diagnosed with or without the target disease for prediction. In this paper, we revised the problem of predictive analysis of disease using personal EHR data and LDA classifier. To fill the gap, in this paper, we first studied an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. The model gives a theoretical upper bound of LDA error rate that is controlled by two factors: (1) the statistical convergence rate of (inverse) covariance matrix estimators and (2) the divergence of the training/testing datasets to fitted distributions. To this end, we could lower the error rate by balancing the two factors for better classification performance. Hereby, we further proposed a novel LDA classifier De-Sparse that leverages De-sparsified Graphical Lasso to improve the estimation of LDA, which outperforms state-of-the-art LDA approaches developed for HDLSS data. Such advances and effectiveness are further demonstrated by both theoretical analysis and extensive experiments on EHR datasets https://www.overleaf.com/project/5d2728c718f6ff3b2bcf5991 .

On Complete Consistency for the Estimator of Nonparametric Regression Model Based on Asymptotically Almost Negatively Associated Errors

In this paper, we mainly study the consistency for the estimator of nonparametric regression model based on asymptotically almost negatively associated (AANA, in short) errors. Firstly, the Bernstein type inequality for AANA random variables is established. By using the Bernstein type inequality and moment inequalities, we investigate the complete consistency and convergence rate for the estimator of nonparametric regression model based on AANA errors. As applications, the complete consistency and convergence rate for the nearest neighbor estimator are obtained.

Convergence Rates of Wavelet Density Estimators for Strongly Mixing Samples

This paper considers wavelet estimations for a multivariate density function based on strongly mixing data. We first construct a linear wavelet estimator and provide a convergence rate over $$L^{p} (1\le p<\infty )$$ risk in Besov space $$B^{s}_{r,q}(\mathbb {R}^{d})$$ . However, this estimator depends on the smoothness of density function, which means that the estimator is not adaptive. A nonlinear adaptive wavelet estimator is proposed by thresholding method. Moreover, the convergence rate of nonlinear estimator is better than the linear one in the case of $$r\le p$$ .

Modified Kalman filtering based multi-step-length gradient iterative algorithm for ARX models with random missing outputs

This study presents a modified Kalman filtering-based multi-step-length gradient iterative algorithm to identify ARX models with missing outputs. The Kalman filtering method is modified to enhance the estimation of unmeasurable outputs, laying the foundation for enabling the multi-step-length gradient iterative algorithm to update effectively the ARX model parameter estimation through the estimated outputs. Compared to the classical gradient iterative algorithm, this study improves the estimation accuracy of the missing outputs by introducing a modified Kalman filter, and the parameter estimation convergence rate by deriving a new multi-step-length formulation. To validate the framework and the algorithm developed, a series of bench tests were conducted with computational experiments. The simulated numerical results are consistent with the analytically derived results in terms of the feasibility and effectiveness of the proposed procedure.

Asymptotic behavior of cross spectral density estimator at the zero frequency in the presence of degeneracy

We analyze the asymptotic properties of nonparametric cross spectral density estimators when the time series process has a degenerate spectrum at the origin in the frequency domain. Degeneracy at the zero frequency caused by over-differencing the series invalidates inferences based on cross spectral methods. We derive the asymptotic mean squared errors of kernel-based cross spectral density estimators at the zero frequency. It is found that degeneracy entails different asymptotics including convergence rates for nonparametric cross spectral density estimators from existing results in time series econometrics literature.