Local Linear Regression Estimator Research Articles

Scalable aggregation predictive analytics

We introduce a predictive modeling solution that provides high quality predictive analytics over aggregation queries in Big Data environments. Our predictive methodology is generally applicable in environments in which large-scale data owners may or may not restrict access to their data and allow only aggregation operators like COUNT to be executed over their data. In this context, our methodology is based on historical queries and their answers to accurately predict ad-hoc queries’ answers. We focus on the widely used set-cardinality, i.e., COUNT, aggregation query, as COUNT is a fundamental operator for both internal data system optimizations and for aggregation-oriented data exploration and predictive analytics. We contribute a novel, query-driven Machine Learning (ML) model whose goals are to: (i) learn the query-answer space from past issued queries, (ii) associate the query space with local linear regression & associative function estimators, (iii) define query similarity, and (iv) predict the cardinality of the answer set of unseen incoming queries, referred to the Set Cardinality Prediction (SCP) problem. Our ML model incorporates incremental ML algorithms for ensuring high quality prediction results. The significance of contribution lies in that it (i) is the only query-driven solution applicable over general Big Data environments, which include restricted-access data, (ii) offers incremental learning adjusted for arriving ad-hoc queries, which is well suited for query-driven data exploration, and (iii) offers a performance (in terms of scalability, SCP accuracy, processing time, and memory requirements) that is superior to data-centric approaches. We provide a comprehensive performance evaluation of our model evaluating its sensitivity, scalability and efficiency for quality predictive analytics. In addition, we report on the development and incorporation of our ML model in Spark showing its superior performance compared to the Spark’s COUNT method.

Nonparametric estimations of the sea state bias for a radar altimeter

To estimate the sea state bias (SSB) for radar altimeter, two nonparametric models, including a Nadaraya-Watson (NW) kernel estimator and a local linear regression (LLR) estimator, are studied based on the Jason-2 altimeter data. Selecting from different combinations of the Gaussian kernel function, spherical Epanechnikov kernel function, a fixed bandwidth and a local adjustable bandwidth, it is observed that the LLR method with the spherical Epanechnikov kernel function and the local adjustable bandwidth is the optimal nonparametric model for the SSB estimation. The comparisons between the nonparametric and parametric models are conducted and the results show that the nonparametric model performs relatively better at high-latitudes of the Northern Hemisphere. This method has been applied to the HY-2A altimeter as well and the same conclusion can be obtained.

Nonparametric geostatistical risk mapping

In this work, a fully nonparametric geostatistical approach to estimate threshold exceeding probabilities is proposed. To estimate the large-scale variability (spatial trend) of the process, the nonparametric local linear regression estimator, with the bandwidth selected by a method that takes the spatial dependence into account, is used. A bias-corrected nonparametric estimator of the variogram, obtained from the nonparametric residuals, is proposed to estimate the small-scale variability. Finally, a bootstrap algorithm is designed to estimate the unconditional probabilities of exceeding a threshold value at any location. The behavior of this approach is evaluated through simulation and with an application to a real data set.

Functional-coefficient spatial autoregressive models with nonparametric spatial weights

We apply local linear regression and sieve estimation technique to estimate functional coefficients and an unknown spatial weighting function, respectively, via a nonparametric GMM estimation method, where we allow both exogenous and endogenous spatial covariates. A consistency result is derived to support the method. Moreover, a two-step estimator is constructed for the functional coefficients, and under certain conditions, we show that this estimator can be oracle efficient in the sense that its limiting distribution is the same regardless of whether or not the spatial weights are known. Both simulated and real data examples are used to illustrate our theory.

Local linear estimation for regression models with locally stationary long memory errors

In this paper we consider the local linear regression estimation for the nonparametric regression models with locally stationary long memory errors. The asymptotic behaviors of the regression estimators are established. It is shown that there is a multiple bandwidth dichotomy for the asymptotic distribution of the estimators of the regression function and its derivatives. The finite sample performance of the estimator is discussed through simulation studies.

Longitudinal Survey, Nonmonotone, Nonresponse, Imputation, Nonparametric Regression

The study focuses on the imputation for the longitudinal survey data which often has nonignorable nonrespondents. Local linear regression is used to impute the missing values and then the estimation of the time-dependent finite populations means. The asymptotic properties (unbiasedness and consistency) of the proposed estimator are investigated. Comparisons between different parametric and nonparametric estimators are performed based on the bootstrap standard deviation, mean square error and percentage relative bias. A simulation study is carried out to determine the best performing estimator of the time-dependent finite population means. The simulation results show that local linear regression estimator yields good properties.

A partially linear single‐index transformation model and its nonparametric estimation

AbstractIn this paper, we consider the nonparametric estimation of the partially linear single‐index transformation model, where the transformation function, single‐index function and error distribution are all completely unknown. We first use the minimum average variance estimation method to estimate the regression coefficients, and then propose a new incorporated local linear regression estimator for the derivative function of the single‐index function. Accordingly by integration we can obtain the estimator of the single‐index function. Finally we propose a constrained least square estimator for the transformation function, where basis function approximation is employed and cross validation method is proposed to select suitable sets of basis functions. Asymptotical properties of the estimators are established. Simulation studies show that our proposed estimators work well. A real‐world data analysis of total health care charges was used to illustrate the proposed procedure. The Canadian Journal of Statistics 43: 97–117; 2015 © 2015 Statistical Society of Canada

Nonlinearities in the Slovenian apple price transmission

Purpose – The purpose of this paper is to assess price linkages and patterns of transmission among producer and consumer markets for apple in Slovenia. Design/methodology/approach – Non-linear error correction models are applied. Non-linearities are allowed by means of threshold and multivariate local linear regression estimation techniques. Monthly prices over the period 2000-2011 are used in the empirical application. Findings – Both techniques provide evidence of non-linearities in price adjustments. Findings suggest that producer and consumer prices tend to increase rather than decrease. Results also indicate that parametric threshold approaches may have difficulties in adequately representing price behavior dynamics. Originality/value – The main contribution of this work to the literature relies on the fact that this is the first attempt to assess vertical price transmission in the apple sector in Central and Eastern European Country markets. Further, it is the first attempt to use multivariate local linear regression techniques in this context.

Differencing techniques in semi-parametric panel data varying coefficient models with fixed effects: a Monte Carlo study

Recently, some new techniques have been proposed for the estimation of semi-parametric fixed effects varying coefficient panel data models. These new techniques fall within the class of the so-called differencing estimators. In particular, we consider first-differences and within local linear regression estimators. Analyzing their asymptotic properties it turns out that, keeping the same order of magnitude for the bias term, these estimators exhibit different asymptotic bounds for the variance. In both cases, the consequences are suboptimal non-parametric rates of convergence. In order to solve this problem, by exploiting the additive structure of this model, a one-step backfitting algorithm is proposed. Under fairly general conditions, it turns out that the resulting estimators show optimal rates of convergence and exhibit the oracle efficiency property. Since both estimators are asymptotically equivalent, it is of interest to analyze their behavior in small sample sizes. In a fully parametric context, it is well-known that, under strict exogeneity assumptions the performance of both first-differences and within estimators is going to depend on the stochastic structure of the idiosyncratic random errors. However, in the non-parametric setting, apart from the previous issues other factors such as dimensionality or sample size are of great interest. In particular, we would be interested in learning about their relative average mean square error under different scenarios. The simulation results basically confirm the theoretical findings for both local linear regression and one-step backfitting estimators. However, we have found out that within estimators are rather sensitive to the size of number of time observations.

Reference Curves Estimation Using Conditional Quantile and Radial Basis Function Network with Mass Constraint

This paper focuses on the improvement of reference curves building \(Y=q(X)\) using a fast algorithm, robust against outliers. Our method consists in plugging a radial basis function neural network in the local linear quantile regression estimation proposed by Yu and Jones (QYJ). This neural network (QRBFc) is designed with a constructive algorithm, introducing a constraint on its integral over the input space. After explaining the different models and algorithms, we compare the QYJ and QRBFc estimators with the quantile regression neural network (QRNN) implemented by A. J. Cannon through simulations with a known underlying model using the R software. We observe that the QRBFc estimator reduces the mean absolute deviation error obtained with other estimators by about 16 %, the introduction of the constraint allowing to lower the number of neurons and therefore the computation time. Finally, using a database of 416 electroencephalograms recorded on preterm infants, we compare the QYJ, QRBFc and QRNN models for the building of brain maturation curves which are based on the dependence of the mean duration of interburst intervals (called IBIs—periods of quiescence between periods of normal electrical activity) with the age. The pathological infants represent 12 % of the total population. Denoting by \(S_A\) the set of individuals whose coordinates (age, mean IBI length) are above the \(90~\%\)-quantile curve, the QRBFc network improves by \(16.5~\%\) the number of pathological infants in \(S_A\) compared to QYJ, when QRNN proved to be too unstable.

Estimation and inference in regression discontinuity designs with asymmetric kernels

We study the behaviour of the Wald estimator of causal effects in regression discontinuity design when local linear regression (LLR) methods are combined with an asymmetric gamma kernel. We show that the resulting statistic is no more complex to implement than existing methods, remains consistent at the usual non-parametric rate, and maintains an asymptotic normal distribution but, crucially, has bias and variance that do not depend on kernel-related constants. As a result, the new estimator is more efficient and yields more reliable inference. A limited Monte Carlo experiment is used to illustrate the efficiency gains. As a by product of the main discussion, we extend previous published work by establishing the asymptotic normality of the LLR estimator with a gamma kernel. Finally, the new method is used in a substantive application.

Asymptotic normality for a local composite quantile regression estimator of regression function with truncated data

In this paper, we construct a local linear composite quantile regression (CQR) estimator of regression function for left-truncated data, which extends the CQR method to the left-truncated model. The asymptotic normality of the proposed estimator is also established. The estimator is much more efficient than the local linear regression estimator for commonly-used non-normal error distributions via simulations.

Does Education Matter for Economic Growth?

Empirical growth regressions typically include mean years of schooling as a proxy for human capital. However, empirical research often finds that the sign and significance of schooling depends on the sample of observations or the specification of the model. We use a nonparametric local-linear regression estimator and a nonparametric variable relevance test to conduct a rigorous and systematic search for significance of mean years of schooling by examining five of the most comprehensive schooling databases. Contrary to a few recent papers that have identified significant nonlinearities between education and growth, our results suggest that mean years of schooling is not a statistically relevant variable in growth regressions. However, we do find evidence (within a cross-sectional framework), that educational achievement, measured by mean test scores, may provide a more reliable measure of human capital than mean years of schooling.

On Projection‐type Estimators of Multivariate Isotonic Functions

Abstract. Let M be an isotonic real‐valued function on a compact subset of and let be an unconstrained estimator of M. A feasible monotonizing technique is to take the largest (smallest) monotone function that lies below (above) the estimator or any convex combination of these two envelope estimators. When the process is asymptotically equicontinuous for some sequence rn→∞, we show that these projection‐type estimators are rn‐equivalent in probability to the original unrestricted estimator. Our first motivating application involves a monotone estimator of the conditional distribution function that has the distributional properties of the local linear regression estimator. Applications also include the estimation of econometric (probability‐weighted moment, quantile) and biometric (mean remaining lifetime) functions.

Asymptotic distributions of two “synthetic data” estimators for censored single-index models

The censored single-index model provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored and the link function is unknown. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure time models for survival analysis. This paper proposes two methods for estimation of single-index models with randomly censored samples. We first transform the censored data into synthetic data or pseudo-responses unbiasedly, then obtain estimates of the index coefficients by the rOPG or rMAVE procedures of Xia (2006) [1]. Finally, we estimate the unknown nonparametric link function using techniques for univariate censored nonparametric regression. The estimators for the index coefficients are shown to be root- n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodologies.

On kernel nonparametric regression designed for complex survey data

In this article, we consider nonparametric regression analysis between two variables when data are sampled through a complex survey. While nonparametric regression analysis has been widely used with data that may be assumed to be generated from independently and identically distributed (iid) random variables, the methods and asymptotic analyses established for iid data need to be extended in the framework of complex survey designs. Local polynomial regression estimators are studied, which include as particular cases design-based versions of the Nadaraya–Watson estimator and of the local linear regression estimator. In this paper, special emphasis is given to the local linear regression estimator. Our estimators incorporate both the sampling weights and the kernel weights. We derive the asymptotic mean squared error (MSE) of the kernel estimators using a combined inference framework, and as a corollary consistency of the estimators is deduced. Selection of a bandwidth is necessary for the resulting estimators; an optimal bandwidth can be determined, according to the MSE criterion in the combined mode of inference. Simulation experiments are conducted to illustrate the proposed methodology and an application with the Canadian survey of labour and income dynamics is presented.

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara–Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.

Improved double kernel local linear quantile regression

As sample quantiles can be obtained as maximum likelihood estimates of location parameters in suitable asymmetric Laplace distributions, so kernel estimates of quantiles can be obtained as maximum likelihood estimates of location parameters in a general class of distributions with simple exponential tails. In this paper, this observation is applied to kernel quantile regression. In doing so, a new double kernel local linear quantile regression estimator is obtained which proves to be consistently superior in performance to the earlier double kernel local linear quantile regression estimator proposed by the authors. It is also straightforward to compute and more readily affords a first derivative estimate. An alternative method of selection for one of the two bandwidths involved also arises naturally but proves not to be so consistently successful.

선형보간법에 의한 자료 희소성 해결방안의 문제와 대안

국소선형회귀모형의 추정량은 좋은 특성을 가지고 있는 추정량으로서 가장 흔히 사용되는 비모수적 회귀모형의 추정량이라고 하겠다. 이러한 국소선형 추정량이 자료가 희박한 구간에서는 심하게 왜곡된 추정결과를 보이는 문제가 있으며, Hall과 Turlach(1997)이 제안한 선형보간법이 이러한 문제에 대한 매우 효과적인 해결방안이라는 것은 잘 알려진 사실이다. 그러나 Hall과 Turlach가 제안한 선형보간법이 이상값에 매우 취약하다는 사실은 아직 지적된 적이 없는 문제이다. 이 논문에서는 이상값의 영향력을 감소시킬 수 있는 수정된 선형보간법에 의한 유사자료의 생성방법을 제안하고, 그 특성을 모의실험을 통하여 기존의 방법과 비교하였다. Local linear regression estimator is the most widely used nonparametric regression estimator which has a number of advantages over the traditional kernel estimators. It is well known that local linear estimator can produce erratic result in sparse regions in the realization of the design and the interpolation method of Hall and Turlach (1997) is the very efficient way to resolve this problem. However, it has been never pointed out that Hall and Turlach's interpolation method is very sensitive to outliers. In this paper, we propose the robust version of the interpolation method for adapting to sparse design. The finite sample properties of the method is compared with Hall and Turlach's method by the simulation study.

Relative error prediction via kernel regression smoothers

In this article, we introduce and study local constant and local linear nonparametric regression estimators when it is appropriate to assess performance in terms of mean squared relative error of prediction. We give asymptotic results for both boundary and non-boundary cases. These are special cases of more general asymptotic results that we provide concerning the estimation of the ratio of conditional expectations of two functions of the response variable. We also provide a good bandwidth selection method for the estimators. Examples of application, limited simulation results and discussion of related problems and approaches are also given.