Residual-based Estimator Research Articles

Strong uniform consistency and asymptotic normality of a kernel based error density estimator in functional autoregressive models

Estimating the innovation probability density is an important issue in any regression analysis. This paper focuses on functional autoregressive models. A residual-based kernel estimator is proposed for the innovation density. Asymptotic properties of this estimator depend on the average prediction error of the functional autoregressive function. Sufficient conditions are studied to provide strong uniform consistency and asymptotic normality of the kernel density estimator.

A note on residual-based empirical likelihood kernel density estimation

In general the empirical likelihood method can improve the performance of estimators by including additional information about the underlying data distribution. Application of the method to kernel density estimation based on independent and identically distributed data always improves the estimation in second order. In this paper we consider estimation of the error density in nonparametric regression by residual-based kernel estimation. We investigate whether the estimator is improved when additional information is included by the empirical likelihood method. We show that the convergence rate is not effected, but in comparison to the residual-based kernel estimator we observe a change in the asymptotic bias of the empirical likelihood estimator in first order and in the asymptotic variance in second order. Those changes do not result in a general uniform improvement of the estimation, but in typical examples we demonstrate the good performance of the residual-based empirical likelihood estimator in asymptotic theory as well as in simulations.

Flux Recovery and A Posteriori Error Estimators: Conforming Elements for Scalar Elliptic Equations

In this paper, we first study two flux recovery procedures for the conforming finite element approximation to general second-order elliptic partial differential equations. One is accurate in a weighted $L^2$ norm studied in [Z. Cai and S. Zhang, SIAM J. Numer. Anal., 47 (2009), pp. 2132-2156] for linear elements, and the other is accurate in a weighted $H(\mathrm{div})$ norm, up to the accuracy of the current finite element approximation. For the $L^2$ recovered flux, we introduce and analyze an a posteriori error estimator that is more accurate than the explicit residual-based estimator. Based on the $H(\mathrm{div})$ recovered flux, we introduce two a posteriori error estimators. One estimator may be regarded as an extension of the recovery-based estimator studied in [Z. Cai and S. Zhang, SIAM J. Numer. Anal., 47 (2009), pp. 2132-2156] to higher-order conforming elements. The global reliability and the local efficiency bounds for this estimator are established provided that the underlying problem is neither convection- nor reaction-dominant. The other is proved to be exact locally and globally on any given mesh with no regularity assumptions with respect to a norm depending on the underlying problem. Numerical results on test problems for these estimators are also presented.

Residual-Based Estimation of Peer and Link Lifetimes in P2P Networks

Existing methods of measuring lifetimes in P2P systems usually rely on the so-called <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Create-Based</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Method</i> (CBM), which divides a given observation window into two halves and samples users ldquocreatedrdquo in the first half every Delta time units until they die or the observation period ends. Despite its frequent use, this approach has no rigorous accuracy or overhead analysis in the literature. To shed more light on its performance, we first derive a model for CBM and show that small window size or large Delta may lead to highly inaccurate lifetime distributions. We then show that create-based sampling exhibits an inherent tradeoff between overhead and accuracy, which does not allow any fundamental improvement to the method. Instead, we propose a completely different approach for sampling user dynamics that keeps track of only <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">residual</i> lifetimes of peers and uses a simple renewal-process model to recover the actual lifetimes from the observed residuals. Our analysis indicates that for reasonably large systems, the proposed method can reduce bandwidth consumption by several orders of magnitude compared to prior approaches while simultaneously achieving higher accuracy. We finish the paper by implementing a two-tier Gnutella network crawler equipped with the proposed sampling method and obtain the distribution of ultrapeer lifetimes in a network of 6.4 million users and 60 million links. Our experimental results show that ultrapeer lifetimes are Pareto with shape alpha ap 1.1; however, link lifetimes exhibit much lighter tails with alpha ap 1.8.

A covariate-matched estimator of the error variance in nonparametric regression

There are two classes of estimators for the error variance in nonparametric regression: residual-based estimators and difference-based estimators. Residual-based estimators require an estimator of the regression function and are asymptotically equivalent to the sample variance based on the actual errors. Difference-based estimators avoid estimating the regression function and are thus simpler to calculate. They also possess superior bias properties at the expense of larger variances. Müller et al. [U.U. Müller, A. Schick, and W. Wefelmeyer, Estimating the error variance in nonparametric regression by a covariate-matched U-statistics, Statistics 37 (2003), pp. 179–188.] suggested improving difference-based estimators using covariate matching. They showed that a covariate-matched version of Rice's [J. Rice, Bandwidth choice for nonparametric regression, Ann. Statist. 12 (1984), pp. 1215–1230.] difference-based estimator matches the asymptotic performance of residual-based estimators, yet still possesses the good bias properties of Rice's estimator. Here we prove a similar result for a covariate-matched version of the difference-based estimator of Gasser et al. [T. Gasser, L. Sroka, and C. Jennen-Steinmetz, Residual variance and residual pattern in nonlinear regression, Biometrika 73 (1986), pp. 625–633.] as their estimator has even better bias properties than Rice's estimator.

Relative Errors of Difference-Based Variance Estimators in Nonparametric Regression

Difference-based estimators for the error variance are popular since they do not require the estimation of the mean function. Unlike most existing difference-based estimators, new estimators proposed by Müller et al. (2003) and Tong and Wang (2005) achieved the asymptotic optimal rate as residual-based estimators. In this article, we study the relative errors of these difference-based estimators which lead to better understanding of the differences between them and residual-based estimators. To compute the relative error of the covariate-matched U-statistic estimator proposed by Müller et al. (2003), we develop a modified version by using simpler weights. We further investigate its asymptotic property for both equidistant and random designs and show that our modified estimator is asymptotically efficient.

Estimating the innovation distribution in nonparametric autoregression

We prove a Bahadur representation for a residual-based estimator of the innovation distribution function in a nonparametric autoregressive model. The residuals are based on a local linear smoother for the autoregression function. Our result implies a functional central limit theorem for the residual-based estimator.

Difference-based estimation for error variances in repeated measurement regression models

Consider a repeated measurement regression model y ij = g ( x i ) + ε ij where i = 1 , … , n , j = 1 , … , m , y ij 's are responses, g ( · ) is an unknown function, x i 's are design points, ε ij 's are random errors with a one-way error component structure, i.e. ε ij = μ i + ν ij , μ i and ν ij 's are i.i.d random variables with mean zero, variance σ μ 2 and σ ν 2 , respectively. This paper focuses on estimating σ μ 2 and σ ν 2 . It is well known that although the residual-based estimator of σ ν 2 works very well the residual-based estimator of σ μ 2 works poorly, especially when the sample size is small. We here propose a difference-based estimation and show the resulted estimator of σ μ 2 performs much better than the residual-based one. In addition, we show the difference-based estimator of σ ν 2 is equal to the residual-based one. This explains why the residual-based estimator of σ ν 2 works very well even when the sample size is small. Another advantage of the difference-based estimation is that it does not need to estimate the unknown function g ( · ) . The asymptotic normalities of the difference-based estimators are established.

Estimating the error distribution function in semiparametric regression

We prove a stochastic expansion for a residual-based estimator of the error distribution function in a partly linear regression model. It implies a functional central limit theorem. As special cases we cover nonparametric, nonlinear and linear regression models.

Truncated Estimator of Asymptotic Covariance Matrix in Partially Linear Models with Heteroscedastic Errors

A partially linear regression model with heteroscedastic and/or serially correlated errors is studied here. It is well known that in order to apply the semiparametric least squares estimation (SLSE) to make statistical inference a consistent estimator of the asymptotic covariance matrix is needed. The traditional residual-based estimator of the asymptotic covariance matrix is not consistent when the errors are heteroscedastic and/or serially correlated. In this paper we propose a new estimator by truncating, which is an extension of the procedure in White (32) . This estimator is shown to be consistent when the truncating parameter converges to infinity with some rate.

A Posteriori Error Estimates for the Mortar Mixed Finite Element Method

Several a posteriori error estimators for mortar mixed finite element discretizations of elliptic equations are derived. A residual-based estimator provides optimal upper and lower bounds for the pressure error. An efficient and reliable estimator for the velocity and mortar pressure error is also derived, which is based on solving local (element) problems in a higher-order space. The interface flux-jump term that appears in the estimators can be used as an indicator for driving an adaptive process for the mortar grids only.

A theory for local, a posteriori, pointwise, residual-based estimation of the finite element error

Residual based, a posteriori FEM error estimation is based on the formulation and solution of local, boundary value problems for the error. The error problem is inherently global, but it is split into local problems utilizing a recovery of the local boundary conditions for the error on single elements or patches of elements: “Recovery of the fluxes” or “splitting of the jumps”. Approximation decisions are involved in the splitting of the global problem into local problems, and the actual estimator depends heavily on the type of approximations performed. To provide a foundation for the decision process and to understand the various versions of residual estimators in use today, it is essential to know the underlying mathematical theory behind the error estimation problem, and the restrictions that is put on the problems. In this work, such a theory is presented, and in light of the theory, various error estimators are developed.

Adaptive Galerkin finite element methods for partial differential equations

We present a general method for error control and mesh adaptivity in Galerkin finite element discretizations of partial differential equations. Our approach is based on the variational framework of projection methods and uses concepts from optimal control and model reduction. By employing global duality arguments and Galerkin orthogonality, we derive a posteriori error estimates for quantities of physical interest. These residual-based estimates contain the dual solution and provide the basis of a feed-back process for successive mesh adaptation. This approach is developed within an abstract setting and illustrated by examples for its application to different types of differential equations including also an optimal control problem.

Semiparametric Inference in the Proportional Odds Regression Model

For fitting the proportional odds regression model with right-censored survival times, we introduce some weighted empirical odds functions. These functions are solutions of some self-consistency equations and have a nice martingale representation. From these functions, several classes of new regression estimators, such as the pseudo–maximum likelihood estimator, martingale residual-based estimators, and minimum distance estimators, are derived. These estimators have desirable properties such as easy computation, asymptotic normality via a martingale analysis, and reliable asymptotic covariance estimation in closed form. Extensive numerical studies show that the minimum L 2 distance estimators have very good finite-sample behaviors compared to existing methods. Results of some simulation studies and applications to a real dataset are given. The weighted odds function–based approach also provides inference on the baseline odds function and some measures for lack-of-fit analysis.

A residual-baseda posteriori error estimator for the Ciarlet-Raviart formulation of the first biharmonic problem

In this article we propose a residual-based estimator for the psi-omega formulation of the biharmonic problem. We show how an appropriate modification of Verfurth methodology gives the proper scaling of the residuals leading to both lower and upper estimation. Numerical examples confirm the viability of the method. © 1997 John Wiley & Sons, Inc.