Difference-based Estimator Research Articles

Using Difference-Based Methods for Inference in Nonparametric Regression with Time Series Errors

SummaryWe show that difference-based methods can be used to construct simple and explicit estimators of error covariance and autoregressive parameters in nonparametric regression with time series errors. When the error process is Gaussian our estimators are efficient, but they are available well beyond the Gaussian case. As an illustration of their usefulness we show that difference-based estimators can be used to produce a simplified version of time series cross-validation. This new approach produces a bandwidth selector that is equivalent, to both first and second orders, to that given by the full time series cross-validation algorithm. Other applications of difference-based methods are to variance estimation and construction of confidence bands in nonparametric regression.

Estimating the error variance in nonparametric regression by a covariate-matched u-statistic

For nonparametric regression models with fixed and random design, two classes of estimators for the error variance have been introduced: second sample moments based on residuals from a nonparametric fit, and difference-based estimators. The former are asymptotically optimal but require estimating the regression function; the latter are simple but have larger asymptotic variance. For nonparametric regression models with random covariates, we introduce a class of estimators for the error variance that are related to difference-based estimators: covariate-matched U-statistics. We give conditions on the random weights involved that lead to asymptotically optimal estimators of the error variance. Our explicit construction of the weights uses a kernel estimator for the covariate density.

Estimating the Variance In Nonparametric Regression—What is a Reasonable Choice?

Summary The exact mean-squared error (MSE) of estimators of the variance in nonparametric regression based on quadratic forms is investigated. In particular, two classes of estimators are compared: Hall, Kay and Titterington's optimal difference-based estimators and a class of ordinary difference-based estimators which generalize methods proposed by Rice and Gasser, Sroka and Jennen-Steinmetz. For small sample sizes the MSE of the first estimator is essentially increased by the magnitude of the integrated first two squared derivatives of the regression function. It is shown that in many situations ordinary difference-based estimators are more appropriate for estimating the variance, because they control the bias much better and hence have a much better overall performance. It is also demonstrated that Rice's estimator does not always behave well. Data-driven guidelines are given to select the estimator with the smallest MSE.

Nonparametric estimation of residual variance revisited

SUMMARY Several difference-based estimators of residual variance are compared for finite sample size. Since the introduction of a rather simple estimator by Gasser, Sroka & JennenSteinmetz (1986) other proposals have been made. Here the one given by Hall, Kay & Titterington (1990) is of particular interest. It minimizes the asymptotic variance. Unfortunately it has severe problems with finite sample bias, and the estimator of Gasser et al. (1986) proves still to be a good choice. A new estimator is introduced, compromising between bias and variance.

On estimation of noise variance in two-dimensional signal processing

Estimation of noise variance is an important component of digital signal processing, in particular of image processing. In this paper we develop methods for estimating the variance of white noise in a two-dimensional degraded signal. We discuss optimal configurations of pixels for difference-based estimation, and describe asymptotically optimal selection of weights for the component pixels. After extensive analysis of possible configurations we recommend averaging linear configurations over a variety of different orientations (usually two or four). This approach produces estimators with properties of both statistical and numerical efficiency.

On estimation of noise variance in two-dimensional signal processing

Estimation of noise variance is an important component of digital signal processing, in particular of image processing. In this paper we develop methods for estimating the variance of white noise in a two-dimensional degraded signal. We discuss optimal configurations of pixels for difference-based estimation, and describe asymptotically optimal selection of weights for the component pixels. After extensive analysis of possible configurations we recommend averaging linear configurations over a variety of different orientations (usually two or four). This approach produces estimators with properties of both statistical and numerical efficiency.

Asymptotically optimal difference-based estimation of variance in nonparametric regression

Abstract We define and compute asymptotically optimal difference sequences for estimating error variance in homoscedastic nonparametric regression. Our optimal difference sequences do not depend on unknowns, such as the mean function, and provide substantial improvements over the suboptimal sequences commonly used in practice. For example, in the case of normal data the usual variance estimator based on symmetric second-order differences is only 64% efficient relative to the estimator based on optimal second-order differences. The efficiency of an optimal mth-order difference estimator relative to the error sample variance is 2m/(2m + 1). Again this is for normal data, and increases as the tails of the error distribution become heavier.