Local Polynomial Estimator Research Articles

Goodness-of-fit tests in semiparametric transformation models using the integrated regression function

Consider the following semiparametric transformation model Λ θ ( Y ) = m ( X ) + ε , where X is a d -dimensional covariate, Y is a univariate dependent variable and ε is an error term with zero mean and which is independent of X . We assume that m is an unknown regression function and that { Λ θ : θ ∈ Θ } is a parametric family of strictly increasing functions. We use a profile likelihood estimator for the parameter θ and a local polynomial estimator for m . Our goal is to develop a new test for the parametric form of the regression function m , which has power against local directional alternatives that converge to the null model at parametric rate, and to compare its performance to that of the test proposed by Colling and Van Keilegom (2016). The idea of the new test is to compare the integrated regression function estimated in a semiparametric way to the integrated regressionfunction estimated under the null hypothesis. We consider two different test statistics, a Kolmogorov–Smirnov and a Cramér–von Mises type statistic, and establish the limiting distributions of these two test statistics under the null hypothesis and under a local alternative. We use a bootstrap procedure to approximate the critical values of the test statistics under the null hypothesis. Finally, a simulation study is carried out to illustrate the performance of our testing procedure, to compare this new test to the previous one and to see under which model conditions which test behaves the best. We also apply both methods on a real data set.

Estimating conditional means with heavy tails

When a conditional distribution has an infinite variance, commonly employed kernel smoothing methods such as local polynomial estimators for the conditional mean admit non-normal limiting distributions (Hall et al., 2002). This complicates the related inference as the conventional tests and confidence intervals based on asymptotic normality are no longer applicable, and the standard bootstrap method often fails. By utilizing the middle part of data nonparametrically and the tail parts parametrically based on extreme value theory, this paper proposes a new estimation method for conditional means, resulting in asymptotically normal estimators even when the conditional distribution has infinite variance. Consequently the standard bootstrap method could be employed to construct, for example, confidence intervals regardless of the tail heaviness. The same idea can be applied to estimating the difference between a conditional mean and a conditional median, which is a useful measure in data exploratory analysis.

An alternative local polynomial estimator for the error-in-variables problem

ABSTRACTWe consider the problem of estimating a regression function when a covariate is measured with error. Using the local polynomial estimator of Delaigle et al. [(2009), ‘A Design-adaptive Local Polynomial Estimator for the Errors-in-variables Problem’, Journal of the American Statistical Association, 104, 348–359] as a benchmark, we propose an alternative way of solving the problem without transforming the kernel function. The asymptotic properties of the alternative estimator are rigorously studied. A detailed implementing algorithm and a computationally efficient bandwidth selection procedure are also provided. The proposed estimator is compared with the existing local polynomial estimator via extensive simulations and an application to the motorcycle crash data. The results show that the new estimator can be less biased than the existing estimator and is numerically more stable.

Asymptotic theory for varying coefficient regression models with dependent data

The varying coefficient models (VCMs) are extremely important tools in the statistical literature and are widely used in many subject areas for data modeling and exploration. In linear VCMs, typically the errors are assumed to be independent. However, in many situations, especially in spatial or spatiotemporal settings, this is not a viable assumption. In this article, we consider nonparametric VCMs with a general dependent error structure which allows for both spatially autoregressive and spatial moving average models as special cases. We investigate asymptotic properties of local polynomial estimators of the model components. Specifically, we show that the estimates of the unknown functions and their derivatives are consistent and asymptotically normally distributed. We show that the rate of convergence and the asymptotic covariance matrix depend on the error dependence structure and we derive the explicit formula for the convergence results.

Improved estimation of fixed effects panel data partially linear models with heteroscedastic errors

Fixed effects panel data regression models are useful tools in econometric and microarray analysis. In this paper, we consider statistical inferences under the setting of fixed effects panel data partially linear regression models with heteroscedastic errors. We find that the usual local polynomial estimator of the error variance function based on residuals is inconsistent, and develop a consistent estimator. Applying this consistent estimator of error variance and spline series approximation of the nonparametric component, we further construct a weighted semiparametric least squares dummy variables estimator for the parametric and nonparametric components. Asymptotic normality of the proposed estimator is derived and its asymptotic covariance matrix estimator is provided. The proposed estimator is shown to be asymptotically more efficient than those ignoring heteroscedasticity. Simulation studies are conducted to demonstrate the finite sample performances of the proposed procedure. As an application, a set of economic data is analyzed by the proposed method.

Statistical inference on seemingly unrelated single-index regression models

In this article, we consider a class of seemingly unrelated single-index regression models. By taking the contemporaneous correlation among equations into account we construct the weighted estimators (WEs) for unknown parameters of the coefficients and the improved local polynomial estimators for the unknown functions, respectively. We establish the asymptotic normalities of these estimators, and show both of them are more asymptotically efficient than those ignoring the contemporaneous correlation. The performances of the proposed procedures are evaluated through simulation studies.

Asymptotically optimal differenced estimators of error variance in nonparametric regression

The existing differenced estimators of error variance in nonparametric regression are interpreted as kernel estimators, and some requirements for a “good” estimator of error variance are specified. A new differenced method is then proposed that estimates the errors as the intercepts in a sequence of simple linear regressions and constructs a variance estimator based on estimated errors. The new estimator satisfies the requirements for a “good” estimator and achieves the asymptotically optimal mean square error. A feasible difference order is also derived, which makes the estimator more applicable. To improve the finite-sample performance, two bias-corrected versions are further proposed. All three estimators are equivalent to some local polynomial estimators and thus can be interpreted as kernel estimators. To determine which of the three estimators to be used in practice, a rule of thumb is provided by analysis of the mean square error, which solves an open problem in error variance estimation which difference sequence to be used in finite samples. Simulation studies and a real data application corroborate the theoretical results and illustrate the advantages of the new method compared with the existing methods.

Regression estimation by local polynomial fitting for multivariate data streams

In this paper we study a local polynomial estimator of the regression function and its derivatives. We propose a sequential technique based on a multivariate counterpart of the stochastic approximation method for successive experiments for the local polynomial estimation problem. We present our results in a more general context by considering the weakly dependent sequence of stream data, for which we provide an asymptotic bias-variance decomposition of the considered estimator. Additionally, we study the asymptotic normality of the estimator and we provide algorithms for the practical use of the method in data streams framework.

Simple and Honest Confidence Intervals in Nonparametric Regression

We consider the problem of constructing honest confidence intervals (CIs) for a scalar parameter of interest, such as the regression discontinuity parameter, in nonparametric regression based on kernel or local polynomial estimators. To ensure that our CIs are honest, we derive and tabulate novel critical values that take into account the possible bias of the estimator upon which the CIs are based. We give sharp efficiency bounds of using different kernels, and derive the optimal bandwidth for constructing honest CIs. We show that using the bandwidth that minimizes the maximum mean-squared error results in CIs that are nearly efficient and that in this case, the critical value depends only on the rate of convergence. For the common case in which the rate of convergence is n^{−4/5}, the appropriate critical value for 95% CIs is 2.18, rather than the usual 1.96 critical value. We illustrate our results in an empirical application.

More efficient local polynomial regression with random-effects panel data models

ABSTRACTWe propose a modification on the local polynomial estimation procedure to account for the “within-subject” correlation presented in panel data. The proposed procedure is rather simple to compute and has a closed-form expression. We study the asymptotic bias and variance of the proposed procedure and show that it outperforms the working independence estimator uniformly up to the first order. Simulation study shows that the gains in efficiency with the proposed method in the presence of “within-subject” correlation can be significant in small samples. For illustration purposes, the procedure is applied to explore the impact of market concentration on airfare.

The non parametric regression estimate with dependent measurement errors

ABSTRACTNon parametric regression estimation with measurement errors data has received great attention, and deconvolution local polynomial estimators can be used to deal with the problem that the errors are independent of other variables in the literature. In this article, the copula method is applied to tackle the case that the errors may depend on covariates, and the asymptotic properties of the resulting estimators are derived. Two simulations are conducted to illustrate the performance of the proposed estimators.

Continuous Chain-Ladder with Paid Data: The Theoretical Foundation

In survival analysis one is usually interested in making inference on the transition time. Recent literature explains how claims data in the non-life insurance context can be embedded in this framework and it is used to calculate future claim numbers. However, when reserves are to be calculated there is a cost associated to every claim, depending on the transition time. We introduce a local polynomial estimator of the cost weighted density of the survival time. This enables one to forecast the cost of future claims. This is done without the use of more complex marked point process theory. Consistency and a central limit theorems for the normalized estimation errors are provided.

Bias-corrected inference for multivariate nonparametric regression: Model selection and oracle property

The local polynomial estimator is particularly affected by the curse of dimensionality, which reduces the potential of this tool for large-dimensional applications. We propose an estimation procedure based on the local linear estimator and a sparseness condition that focuses on nonlinearities in the model. Our procedure, called BID (bias inflation–deflation), is automatic and easily applicable to models with many covariates without requiring any additivity assumption. It is an extension of the RODEO method, and introduces important new contributions: consistent estimation of the multivariate optimal bandwidth (the tuning parameter of the estimator); consistent estimation of the multivariate bias-corrected regression function and confidence bands; and automatic identification and separation of nonlinear and linear effects. Some theoretical properties of the method are discussed. In particular, we show the nonparametric oracle property. For linear models, BID automatically reaches the optimal rate O p ( n − 1 / 2 ) , equivalent to the parametric case. A simulation study shows the performance of the procedure for finite samples.

Local Polynomial Regression Estimator of the Finite Population Total under Stratified Random Sampling: A Model-Based Approach

In this paper, auxiliary information is used to determine an estimator of finite population total using nonparametric regression under stratified random sampling. To achieve this, a model-based approach is adopted by making use of the local polynomial regression estimation to predict the nonsampled values of the survey variable y. The performance of the proposed estimator is investigated against some design-based and model-based regression estimators. The simulation experiments show that the resulting estimator exhibits good properties. Generally, good confidence intervals are seen for the nonparametric regression estimators, and use of the proposed estimator leads to relatively smaller values of RE compared to other estimators.

Local polynomial regression with an ordinal covariate

We are interested in fitting a nonparametric regression model to data when the covariate is an ordered categorical variable. We extend the local polynomial estimator, which normally requires continuous covariates, to a local polynomial estimator that allows for ordered categorical covariates. We derive the asymptotic conditional bias and variance under the assumption that the categories correspond to quantiles of an unobserved continuous latent variable. We conduct a simulation study with two patterns of ordinal data to evaluate our estimator.

Fast and accurate computation for kernel estimators

Standard kernel density and regression estimators are well-known to be computationally very slow when analyzing large data sets, and algorithms that achieve considerable computational savings are highly desirable. With this goal in mind, two fast and accurate computational methods are proposed in this paper for computation of univariate and multivariate local polynomial estimators defined on an equally spaced grid. Compared to direct implementation, the proposed algorithms lead to substantial computational savings. For the case in which the evaluation points are not the grid points, a fast algorithm is also proposed. It is shown that the proposed method is asymptotically superior to the general binning method. Numerical comparisons are also made to compare computational speeds and error bounds of the proposed methods with those of the linear binning and the direct implementation.

Curve of Correlation for Time Series

The curve of correlation is a measure of local correlation between two random variables X and Y at the point X = x of the support of this variable. This article studies this local measure using the theory of time series for bivariate and univariate stationary stochastic process. We suggest local polynomial estimators for time series observing their consistency both theoretically and through simulations. For this, different sizes of series, bandwidths, and kernels, besides lags and models’ configurations were used. Applications have also been made using the daily returns of two financial series.

Goodness-of-fit tests in semiparametric transformation models

Consider a semiparametric transformation model of the form \(\varLambda _{\theta }(Y)\) \(= m(X) + \varepsilon \), where \(Y\) is a univariate dependent variable, \(X\) is a \(d\)-dimensional covariate, and \(\varepsilon \) is independent of \(X\) and has mean zero. We assume that \(\{ \varLambda _{\theta } : \theta \in \varTheta \}\) is a parametric family of strictly increasing functions, while \(m\) is an unknown regression function. The goal of the paper is to develop tests for the null hypothesis that \(m(\cdot )\) belongs to a certain parametric family of regression functions. We propose a Kolmogorov–Smirnov and a Cramér–von Mises type test statistic, which measure the distance between the distribution of \(\varepsilon \) estimated under the null hypothesis and the distribution of \(\varepsilon \) without making use of this null hypothesis. The estimated distributions are based on a profile likelihood estimator of \(\theta \) and a local polynomial estimator of \(m(\cdot )\). The limiting distributions of these two test statistics are established under the null hypothesis and under a local alternative. We use a bootstrap procedure to approximate the critical values of the test statistics under the null hypothesis. Finally, a simulation study is carried out to illustrate the performance of our testing procedures, and we apply our tests to data on the scattering of sunlight in the atmosphere.

Estimation of Population Total Using Spline Functions

This study sought to estimate finite population total using spline functions. The emerging patterns from spline smoother were compared with those that were obtained from the model-based, the model-assisted and the non-parametric estimators. To measure the performance of each estimator, three aspects were considered: the average bias, the efficiency by use of the average mean square error and the robustness using the rate of change of efficiency. We used six populations: four natural and two simulated. The findings showed that the model-based estimator works very well in terms of efficiency while the model-assisted is almost unbiased when the model is linear and homoscedastic. However, the estimators break down when the underlying model assumptions are violated. The Kernel Estimator (Nadaraya-Watson) is found to be the most robust of the five estimators considered. Between the two spline functions that we considered, the periodic spline was found to perform better. The spline functions were found to provide good results whether or not the design points were uniformly spaced. We also found out that, under certain conditions, a smoothing spline estimator and a Kernel estimator are equivalent. The study recommends that both the ratio estimator and the local polynomial estimator should be used within the confines of a linear homoscedastic model. The Nadaraya-Watson and the periodic spline estimators, both of which are non-parametric, are highly robust. The Nadaraya-Watson however is even more robust than the periodic spline.

A Design Unbiased Variance Estimator of the Systematic Sample Means

Systematic sampling is normally used in surveys of finite populations because of its appealing simplicity and efficiency. When properly applied, it can reflect stratification in the population and thus can be more precise than SRS. In systematic sampling technique, the sampling units are evenly spread over the whole population. This sampling scheme is very sensitive to correlation between units in the entire population. A positive autocorrelation reduces the precision while a negative autocorrelation will improve the precision compared to simple random sampling. The limitation of this sampling method is that, it is not possible to estimate the design variance that is unbiased. This study proposes an estimator for the design variance based on a non-parametric model for the population using local polynomial regression as the estimation technique. The non-parametric model is more flexible that it can hold for many practical situations. A simulation study is performed to enable the comparison of the efficiency of the proposed estimator to the existing ones. The performance measures used include: Relative Bias (RB) and Mean Square Error (MSE). From the simulation results, it can be seen that local polynomial estimator based on nonparametric model is consistent and design unbiased for the variance of systematic sample mean. The simulation study gave smaller values for the relative biases and mean squared errors for proposed estimator.