Local Polynomial Regression Estimator Research Articles

Nprobust: Nonparametric Kernel-Based Estimation and Robust Bias-Corrected Inference

Nonparametric kernel density and local polynomial regression estimators are very popular in statistics, economics, and many other disciplines. They are routinely employed in applied work, either as part of the main empirical analysis or as a preliminary ingredient entering some other estimation or inference procedure. This article describes the main methodological and numerical features of the software package nprobust, which offers an array of estimation and inference procedures for nonparametric kernel-based density and local polynomial regression methods, implemented in both the R and Stata statistical platforms. The package includes not only classical bandwidth selection, estimation, and inference methods (Wand and Jones 1995; Fan and Gijbels 1996), but also other recent developments in the statistics and econometrics literatures such as robust bias-corrected inference and coverage error optimal bandwidth selection (Calonico, Cattaneo, and Farrell 2018, 2019a). Furthermore, this article also proposes a simple way of estimating optimal bandwidths in practice that always delivers the optimal mean square error convergence rate regardless of the specific evaluation point, that is, no matter whether it is implemented at a boundary or interior point. Numerical performance is illustrated using an empirical application and simulated data, where a detailed numerical comparison with other R packages is given.

Nonparametric inference via bootstrapping the debiased estimator

In this paper, we propose to construct confidence bands by bootstrapping the debiased kernel density estimator (for density estimation) and the debiased local polynomial regression estimator (for regression analysis). The idea of using a debiased estimator was recently employed by Calonico et al. (2018b) to construct a confidence interval of the density function (and regression function) at a given point by explicitly estimating stochastic variations. We extend their ideas of using the debiased estimator and further propose a bootstrap approach for constructing simultaneous confidence bands. This modified method has an advantage that we can easily choose the smoothing bandwidth from conventional bandwidth selectors and the confidence band will be asymptotically valid. We prove the validity of the bootstrap confidence band and generalize it to density level sets and inverse regression problems. Simulation studies confirm the validity of the proposed confidence bands/sets. We apply our approach to an Astronomy dataset to show its applicability.

On Comparison of Local Polynomial Regression Estimators for P=0 and P=1 in a Model Based Framework

This article discusses the local polynomial regression estimator for  and the local polynomial regression estimator for  in a finite population. The performance criterion exploited in this study focuses on the efficiency of the finite population total estimators. Further, the discussion explores analytical comparisons between the two estimators with respect to asymptotic relative efficiency. In particular, asymptotic properties of the local polynomial regression estimator of finite population total for  are derived in a model based framework. The results of the local polynomial regression estimator for  are compared with those of the local polynomial regression estimator for  studied by Kikechi et al (2018). Variance comparisons are made using the local polynomial regression estimator  for  and the local polynomial regression estimator  for  which indicate that the estimators are asymptotically equivalently efficient. Simulation experiments carried out show that the local polynomial regression estimator  outperforms the local polynomial regression estimator  in the linear, quadratic and bump populations.

Approximation of Finite Population Totals Using Lagrange Polynomial

Approximation of finite population totals in the presence of auxiliary information is considered. A polynomial based on Lagrange polynomial is proposed. Like the local polynomial regression, Horvitz Thompson and ratio estimators, this approximation technique is based on annual population total in order to fit in the best approximating polynomial within a given period of time (years) in this study. This proposed technique has shown to be unbiased under a linear polynomial. The use of real data indicated that the polynomial is efficient and can approximate properly even when the data is unevenly spaced.

Statistical classification of early and late wood through the growth rings using thermogravimetric analysis

The aim of this study is to statistically identify and distinguish wood samples corresponding to different areas of annual rings in trees of temperate regions, using the corresponding thermogravimetric (TG) and their first TG derivative (DTG) curves, and specifically to verify whether late and early wood chestnut samples are different with statistical significance, taking into account their TG and DTG curves. These significant differences are sought by applying statistical procedures based on functional data analysis (FDA), such as the functional ANOVA and the FDA classification methods. Each TG curve is firstly smoothed using the local polynomial regression estimator, and its first derivative is estimated. Then, functional ANOVA based on random projections (RP) is used to identify significant differences between TG or DTG curves of early and late wood samples. In order to know the extent of the differences between early and late wood samples, they are discriminated (and the correct classification proportion obtained) by employing a kernel nonparametric functional data analysis technique, based on the Bayes' rule, as well as functional generalized linear models and functional generalized additive models, allowing to classify materials using more than one type of thermal curves simultaneously. The results are compared with those obtained using some classical multivariate supervised classification methods: linear discriminant analysis, naive Bayes (NBC) and quadratic classification (QDA). The partial least squares (PLS) dimension reduction procedure was previously applied to the TG curves in order to employ these multivariate methods. The application of RP ANOVA shows significant differences between late and early wood regarding mass loss and mass loss rate under combustion. The use of PLS multivariate methods or FDA classification approaches applied to the TG and DTG curves allows to distinguish very accurately between late and early wood. The proposed method could be applied to other species to identify thermooxidative differences, combined with other experimental methods to find their chemical and physical causes.

Local Polynomial Regression Estimator of the Finite Population Total under Stratified Random Sampling: A Model-Based Approach

In this paper, auxiliary information is used to determine an estimator of finite population total using nonparametric regression under stratified random sampling. To achieve this, a model-based approach is adopted by making use of the local polynomial regression estimation to predict the nonsampled values of the survey variable y. The performance of the proposed estimator is investigated against some design-based and model-based regression estimators. The simulation experiments show that the resulting estimator exhibits good properties. Generally, good confidence intervals are seen for the nonparametric regression estimators, and use of the proposed estimator leads to relatively smaller values of RE compared to other estimators.

Convergence rates for uniform confidence intervals based on local polynomial regression estimators

We investigate the convergence rates of uniform bias-corrected confidence intervals for a smooth curve using local polynomial regression for both the interior and boundary region. We discuss the cases when the degree of the polynomial is odd and even. The uniform confidence intervals are based on the volume-of-tube formula modified for biased estimators. We empirically show that the proposed uniform confidence intervals attain, at least approximately, nominal coverage. Finally, we investigate the performance of the volume-of-tube based confidence intervals for independent non-Gaussian errors.

Identification and estimation of single-index models with measurement error and endogeneity

Economic variables are often measured with an error and may be endogenous. In this paper, we give new identification results for the ratio of partial effects in linear index models with measurement error and endogeneity. The identification restrictions include independence of covariates and error terms, and the derivative of some conditional mean functions being nonzero. We propose a local polynomial regression estimator to estimate the single‐index parameters. We apply these tools to estimate the labour‐supply elasticity and find that the labour‐supply elasticity for married men is positive, while the coefficients for married women are negative for the full sample and positive for the working sample.

Theoretical investigation of an exploratory approach for log-density in scale-space

We develop an exploratory data analysis tool in scale-space to discover significant features of a log-density using a local likelihood approach with local polynomial regression estimators. We study its asymptotic properties at multiple locations and levels of resolution.

Classification of wood using differential thermogravimetric analysis

The aim of this study is to propose an alternative methodology to classify wood species using the first (DTG), second (2DTG), and third (3DTG) derivatives of the thermogravimetric curves (TG). Accordingly, the main contribution of this new procedure consists on classifying materials (wood) taking into account the mass loss rate and acceleration with respect to temperature. In our research, each TG curve is firstly smoothed using the local polynomial regression estimator, and the first, second, and third derivatives are estimated. The application of the local polynomial regression estimator provides a reliable way to obtain the TG derivatives, overcoming the noise problem in the TG derivative estimation. Then, using these estimated curves, the different wood classes are discriminated employing a nonparametric functional data analysis (NPFDA) technique, based on the Bayes rule and the Nadaraya-Watson regression estimator, and also novel functional generalized additive models (GAM). The latter allows to classify materials using simultaneously more than one type of thermal curves. The results are compared with those obtained using classical and machine learning multivariate supervised classification methods, such as Linear discriminant analysis, Quadratic classification, Naive Bayes, Logistic regression, \(k\) Nearest neighbors, Neural networks, and Support vector machines. A regression model consisting of the mixture of the first derivatives of four generalized logistic components, one per principal wood constituent (water, hemicellulose, cellulose, and lignin), is applied to fit the DTG curves. The resulting 16 parameters from this fit characterize each curve and are used as datasets to apply the multivariate supervised classification methods. The use of the TG derivatives jointly with the TG curves has proved to be an optimal discriminating feature, when the new functional GAM techniques are employed.

Local modal regression

A local modal estimation procedure is proposed for the regression function in a nonparametric regression model. A distinguishing characteristic of the proposed procedure is that it introduces an additional tuning parameter that is automatically selected using the observed data in order to achieve both robustness and efficiency of the resulting estimate. We demonstrate both theoretically and empirically that the resulting estimator is more efficient than the ordinary local polynomial regression (LPR) estimator in the presence of outliers or heavy-tail error distribution (such as t-distribution). Furthermore, we show that the proposed procedure is as asymptotically efficient as the LPR estimator when there are no outliers and the error distribution is a Gaussian distribution. We propose an expectation–maximisation-type algorithm for the proposed estimation procedure. A Monte Carlo simulation study is conducted to examine the finite sample performance of the proposed method. The simulation results confirm the theoretical findings. The proposed methodology is further illustrated via an analysis of a real data example.

On Asymptotic Normality of the Local Polynomial Regression Estimator with Stochastic Bandwidths

Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are non stochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result, nonparametric estimators are usually constructed using stochastic bandwidths. In this article, we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstochastic bandwidths. Our result extends previous work by Boente and Fraiman (1995) and Ziegler (2004).

On kernel nonparametric regression designed for complex survey data

In this article, we consider nonparametric regression analysis between two variables when data are sampled through a complex survey. While nonparametric regression analysis has been widely used with data that may be assumed to be generated from independently and identically distributed (iid) random variables, the methods and asymptotic analyses established for iid data need to be extended in the framework of complex survey designs. Local polynomial regression estimators are studied, which include as particular cases design-based versions of the Nadaraya–Watson estimator and of the local linear regression estimator. In this paper, special emphasis is given to the local linear regression estimator. Our estimators incorporate both the sampling weights and the kernel weights. We derive the asymptotic mean squared error (MSE) of the kernel estimators using a combined inference framework, and as a corollary consistency of the estimators is deduced. Selection of a bandwidth is necessary for the resulting estimators; an optimal bandwidth can be determined, according to the MSE criterion in the combined mode of inference. Simulation experiments are conducted to illustrate the proposed methodology and an application with the Canadian survey of labour and income dynamics is presented.

Robust nonparametric estimation of the intensity function of point data

Intensity functions—which describe the spatial distribution of the occurrences of point processes—are useful for risk assessment. This paper deals with the robust nonparametric estimation of the intensity function of space–time data from events such as earthquakes. The basic approach consists of smoothing the frequency histograms with the local polynomial regression (LPR) estimator. This method allows for automatic boundary corrections, and its jump-preserving ability can be improved with robustness. We derive a robust local smoother from the weighted-average approach to M-estimation and we select its bandwidths with robust cross-validation (RCV). Further, we develop a robust recursive algorithm for sequential processing of the data binned in time. An extensive application to the Northern California earthquake catalog in the San Francisco, CA, area illustrates the method and proves its validity.

Optimal Bandwidth Choice for Matching Estimators by Double Smoothing

This paper addresses the choice of an optimal smoothing parameter for local polynomial matching estimators. In order to estimate the MSE of the matching estimator as a bandwidth selection criterion, the Double Smoothing approach (Mýller (1985), Hýrdle, Hall, and Marron (1992)) is applied. The proposed bandwidth selector exhibits a faster rate of convergence than cross-validation selectors for local polynomial regression estimators. A Monte Carlo study reveals that using the proposed algorithm can yield efficiency gains compared to the MISE based cross-validation procedure. Furthermore, the estimated MSE nicely approximates the true MSE of the matching estimator.

Plug-in bandwidth selector for local polynomial regression estimator with correlated errors

Consider the fixed regression model where the error random variables are coming from a strictly stationary, non-white noise stochastic process. In a situation like this, automated bandwidth selection methods for non-parametric regression break down. We present a plug-in method for choosing the smoothing parameter for local least squares estimators of the regression function. The method takes the presence of correlated errors explicitly into account through a parametric correlation function specification. The theoretical performance for the local linear estimator of the regression function is obtained in the case of an AR(1) correlation function. These results can readily be extended to other settings, such as different parametric specifications of the correlation function, derivative estimation and multiple non-parametric regression. Estimators of regression functionals and the error correlation based on local least squares ideas are developed in this article. A simulation study and an analysis with real economic data illustrate the selection method proposed.

Local Likelihood Estimation in Generalized Additive Models

ABSTRACT.  Generalized additive models are a popular class of multivariate non‐parametric regression models, due in large part to the ease of use of the local scoring estimation algorithm. However, the theoretical properties of the local scoring estimator are poorly understood. In this article, we propose a local likelihood estimator for generalized additive models that is closely related to the local scoring estimator fitted by local polynomial regression. We derive the statistical properties of the estimator and show that it achieves the same asymptotic convergence rate as a one‐dimensional local polynomial regression estimator. We also propose a wild bootstrap estimator for calculating point‐wise confidence intervals for the additive component functions. The practical behaviour of the proposed estimator is illustrated through a simulation experiment.

Local Polynomial Regression: Optimal Kernels and Asymptotic Minimax Efficiency

We consider local polynomial fitting for estimating a regression function and its derivatives nonparametrically. This method possesses many nice features, among which automatic adaptation to the boundary and adaptation to various designs. A first contribution of this paper is the derivation of an optimal kernel for local polynomial regression, revealing that there is a universal optimal weighting scheme. Fan (1993, Ann. Statist., 21, 196-216) showed that the univariate local linear regression estimator is the best linear smoother, meaning that it attains the asymptotic linear minimax risk. Moreover, this smoother has high minimax risk. We show that this property also holds for the multivariate local linear regression estimator. In the univariate case we investigate minimax efficiency of local polynomial regression estimators, and find that the asymptotic minimax efficiency for commonly-used orders of fit is 100% among the class of all linear smoothers. Further, we quantify the loss in efficiency when going beyond this class.