Kernel Regression Estimator Research Articles

Statistical inference for nonparametric censored regression

Nonparametric regression is of primary importance in many statistical applications. For the data with censored outcomes, how to construct a confidence band for a regression function is a basic issue but has limited research. We propose a procedure to construct the pointwise and simultaneous confidence bands for a regression function based on a debiased estimator, which is proposed by correcting the bias of the original kernel regression estimator. The corresponding theoretical properties such as consistency, convergence rates, and the asymptotic distribution of the proposed estimator are also derived. Furthermore, we carry out extensive simulation studies and a real data example from the German Breast Cancer study to evaluate the finite sample performance of the proposed method.

Bias reduction by projection on parametric models in Hilbertian nonparametric regression

In this paper we discuss a general way of improving the bias properties of nonparametric kernel regression estimators. The procedure involves choosing a parametric model and constructing a semiparametric estimator that consists of a parametric component and a nonparametric adjustment, where the parametric component is picked from the parametric model in such a way that the resulting estimator has the best bias performance. We study the method for response variables taking values in a general Hilbert space and for local linear smoother. We show that the procedure always improves the bias of the local linear estimator regardless of the choice of parametric model. We also illustrate the method via a real data example where the response variable is a random density.

Recursive kernel regression estimation under α – mixing data

In this paper, we consider an extension of the generalized class of recursive regression estimators to the case of strong mixing data. Then, we study the properties of these estimators and compare them with the well known Nadaraya-Watson estimator. The Bias, variance and Mean Integrated Square Error are computed explicitly. Using a selected bandwidth and a special stepsize, we showed that the proposed recursive estimators allowed us to obtain quite better results compared to the non-recursive regression estimator under α-mixing condition in terms of estimation error and much better in terms of computational costs.

Variable bandwidth kernel regression estimation

In this paper we propose a variable bandwidth kernel regression estimator fori.i.d. observations in ℝ2to improve the classical Nadaraya-Watson estimator. The bias is improved to the order ofO(hn4) under the condition that the fifth order derivative of the density function and the sixth order derivative of the regression function are bounded and continuous. We also establish the central limit theorems for the proposed ideal and true variable kernel regression estimators. The simulation study confirms our results and demonstrates the advantage of the variable bandwidth kernel method over the classical kernel method.

Potential trade vulnerability of Bangladesh: an exploratory analysis

Bangladesh's integration in the global economy has hitherto proven to be an important force behind its economic growth. However, exclusively trade dependent growth is also trade vulnerable growth. This paper explores the factors that may potentially affect the trade vulnerability of Bangladesh. Results of the bivariate non-parametric local linear kernel regression estimation reveal that the minimum wage of readymade garment workers, manufacturing cost, utilities cost, crude oil prices, and climate change are positively related to trade volatility, while improved political stability is negatively related to trade volatility. Multivariate kernel regressions with composite indicators show that macroeconomic factors, such as exchange rate, foreign exchange reserves and budget balance, have a statistically significant impact on the trade volatility of Bangladesh. In view of these preliminary results, a number of policy recommendations are suggested to deal with the short run and long run trade vulnerability of Bangladesh.

Uniform convergence rate of the kernel regression estimator adaptive to intrinsic dimension in presence of censored data

ABSTRACT The focus of the present paper is on the uniform in bandwidth consistency of kernel-type estimators of the regression function derived by modern empirical process theory, under weaker conditions on the kernel than previously used in the literature. Our theorems allow data-driven local bandwidths for these statistics. We extend existing uniform bounds on kernel regression estimator and making it adaptive to the intrinsic dimension of the underlying distribution of which will be characterising by the so-called intrinsic dimension. Moreover, we show, in the same context, the uniform in bandwidth consistency for nonparametric inverse probability of censoring weighted (I.P.C.W.) estimators of the regression function under random censorship. Statistical applications to the kernel-type estimators (density, regression, conditional distribution, derivative functions, entropy, mode and additive models) are given to motivate these results.

Global statistical inference for the difference between two regression mean curves with covariates possibly partially missing

In two sample problems it is of interest to examine the difference between the two regression curves or to detect whether certain functions are adequate to describe the overall trend of the difference. In this paper, we propose a simultaneous confidence band (SCB) as a global inference method with asymptotically correct coverage probabilities for the difference curve based on the weighted local linear kernel regression estimates in each sample. Our procedure allows for random designs, different sample sizes, heteroscedastic errors, and especially missing covariates. Simulation studies are conducted to investigate the finite sample properties of the new SCB which support our asymptotic theory. The proposed SCB is used to analyze two data sets, one of which is concerned with human event-related potentials data which are fully observed and the other is concerned with the Canada 2010/2011 youth student survey data with partially missing covariates, leading to a number of discoveries.

Some asymptotic properties of kernel regression estimators of the mode for stationary and ergodic continuous time processes.

In the present paper, we consider the nonparametric regression model with random design based on (mathbf{X}_mathrm{t},mathbf{Y}_mathrm{t})_{mathrm{t}ge 0} a mathbb {R}^{d}times mathbb {R}^{q}-valued strictly stationary and ergodic continuous time process, where the regression function is given by m(mathbf{x},psi ) = mathbb {E}(psi (mathbf{Y}) mid mathbf{X} = mathbf{x})), for a measurable function psi : mathbb {R}^{q} rightarrow mathbb {R}. We focus on the estimation of the location {varvec{Theta }} (mode) of a unique maximum of m(cdot , psi ) by the location widehat{{varvec{Theta }}}_mathrm{T} of a maximum of the Nadaraya–Watson kernel estimator widehat{m}_mathrm{T}(cdot , psi ) for the curve m(cdot , psi ). Within this context, we obtain the consistency with rate and the asymptotic normality results for widehat{{varvec{Theta }}}_mathrm{T} under mild local smoothness assumptions on m(cdot , psi ) and the design density f(cdot ) of mathbf{X}. Beyond ergodicity, any other assumption is imposed on the data. This paper extends the scope of some previous results established under the mixing condition. The usefulness of our results will be illustrated in the construction of confidence regions.

Transient stability assessment in large-scale power systems based on the sparse single index model

We apply a class of nonlinear semiparametric models to the problem of transient stability analysis of a large-scale power system. The mapping between the pre-contingency state and the transient stability boundary (TSB) is modelled by a block-oriented structure known as the single-index model (SIM). In this model one has a single dimensional projection that enters the unknown nonlinearity nonparametrically. Such models form a rich class of nonlinear mappings that includes classical models, e.g., linear and logistic regression, as special cases. The generalized case of the SIM is utilized that is taking into account the sparsity of the projection vector. This yields a low-dimensional nonlinear model suitable for high-dimensional data emerging in modern power systems. The parametric part of the model is obtained by the properly modified sparsity sensitive LASSO algorithm. The nonlinear nonparametric part of the model is estimated by the monotonically corrected kernel regression estimate. The precision of our modeling is verified for two fault cases of the 470-bus power system. It is shown that for the examined faults, the proposed modeling methodology exhibits a stronger prediction accuracy compared to the existing competing methods recently applied in the field, namely, kernel ridge regression, as well as LASSO linear modeling.

Kernel Density Estimation Based on the Distinct Units in Sampling with Replacement

This paper considers the problem of estimating density functions using the kernel method based on the set of distinct units in sampling with replacement. Using a combined design-model-based inference framework, which accounts for the underlying superpopulation model as well as the randomization distribution induced by the sampling design, we derive asymptotic expressions for the bias and integrated mean squared error (MISE) of a Parzen-Rosenblatt-type kernel density estimator (KDE) based on the distinct units from sampling with replacement. We also prove the asymptotic normality of the distinct units KDE under both design-based and combined inference frameworks. Additionally, we give the asymptotic MISE formulas of several alternative estimators including the estimator based on the full with-replacement sample and estimators based on without-replacement sampling of similar cost. Using the MISE expressions, we discuss how the various estimators compare asymptotically. Moreover, we use Mote Carlo simulations to investigate the finite sample properties of these estimators. Our simulation results show that the distinct units KDE and the without-replacement KDEs perform similarly but are all always superior to the full with-replacement sample KDE. Furthermore, we briefly discuss a Nadaraya-Watson-type kernel regression estimator based on the distinct units from sampling with replacement, derive its MSE under the combined inference framework, and demonstrate its finite sample properties using a small simulation study. Finally, we extend the distinct units density and regression estimators to the case of two-stage sampling with replacement.

Anomaly Detection Using Local Kernel Density Estimation and Context-Based Regression

Current local density-based anomaly detection methods are limited in that the local density estimation and the neighborhood density estimation are not accurate enough for complex and large databases, and the detection performance depends on the size parameter of the neighborhood. In this paper, we propose a new kernel function to estimate samples’ local densities and propose a weighted neighborhood density estimation to increase the robustness to changes in the neighborhood size. We further propose a local kernel regression estimator and a hierarchical strategy for combining information from the multiple scale neighborhoods to refine anomaly factors of samples. We apply our general anomaly detection method to image saliency detection by regarding salient pixels in objects as anomalies to the background regions. Local density estimation in the visual feature space and kernel-based saliency score propagation in the image enable the assignment of similar saliency values to homogenous object regions. Experimental results on several benchmark datasets demonstrate that our anomaly detection methods overall outperform several state-of-art anomaly detection methods. The effectiveness of our image saliency detection method is validated by comparison with several state-of-art saliency detection methods.

Nonparametric relative recursive regression

Abstract In this paper, we propose the problem of estimating a regression function recursively based on the minimization of the Mean Squared Relative Error (MSRE), where outlier data are present and the response variable of the model is positive. We construct an alternative estimation of the regression function using a stochastic approximation method. The Bias, variance, and Mean Integrated Squared Error (MISE) are computed explicitly. The asymptotic normality of the proposed estimator is also proved. Moreover, we conduct a simulation to compare the performance of our proposed estimators with that of the two classical kernel regression estimators and then through a real Malaria dataset.

Convergence rate of kernel regression estimation for time series data when both response and covariate are functional

We investigate kernel estimates in the functional nonparametric regression model when both the response and the explanatory variable (the covariate) are functional. The rates of almost complete and uniform almost complete convergence of the estimator are obtained under some mild $$\alpha $$-mixing functional sample. Finally, a simulation study is carried out to illustrate the finite sample performance of the estimator.

Random Noise Suppression of Magnetic Resonance Sounding Data with Intensive Sampling Sparse Reconstruction and Kernel Regression Estimation

The magnetic resonance sounding (MRS) method is a non-invasive, efficient and advanced geophysical method for groundwater detection. However, the MRS signal received by the coil sensor is extremely susceptible to electromagnetic noise interference. In MRS data processing, random noise suppression of noisy MRS data is an important research aspect. We propose an approach for intensive sampling sparse reconstruction (ISSR) and kernel regression estimation (KRE) to suppress random noise. The approach is based on variable frequency sampling, numerical integration and statistical signal processing combined with kernel regression estimation. In order to realize the approach, we proposed three specific sparse reconstructions, namely rectangular sparse reconstruction, trapezoidal sparse reconstruction and Simpson sparse reconstruction. To solve the distortion of peaks and valleys after sparse reconstruction, we introduced the KRE to deal with the processed data by the ISSR. Further, the simulation and field experiments demonstrate that the ISSR-KRE approach is a feasible and effective way to suppress random noise. Besides, we find that rectangular sparse reconstruction and trapezoidal sparse reconstruction are superior to Simpson sparse reconstruction in terms of noise suppression effect, and sampling frequency is positively correlated with signal-to-noise improvement ratio (SNIR). In one case of field experiment, the standard deviation of noisy MRS data was reduced from 1200.80 nV to 570.01 nV by the ISSR-KRE approach. The proposed approach provides theoretical support for random noise suppression and contributes to the development of MRS instrument with low power consumption and high efficiency. In the future, we will integrate the approach into MRS instrument and attempt to utilize them to eliminate harmonic noise from power line.

Optimal asymptotic MSE of kernel regression estimate for continuous time processes with missing at random response

Based on an incomplete observed sample (Xt,Yt,ζt)0≤t≤T, where ζt=1 if Yt is observed at time t and ζt=0 otherwise, the asymptotic mean square error of the regression estimator m̂T(x),x∈Rd, is showed to satisfy the following inequality: limT→∞T4∕(d+4)E(m̂T(x)−m(x))2<c where c is an explicit constant. A simulation study is performed to illustrate the effect of the sampling mesh as well as the missing at random mechanism on the consistency of the regression estimator for regular and irregular continuous time processes.

On the Nadaraya–Watson kernel regression estimator for irregularly spaced spatial data

We investigate the asymptotic normality of the Nadaraya–Watson kernel regression estimator for irregularly spaced data collected on a finite region of the lattice Zd where d is a positive integer. The results are stated for strongly mixing random fields in the sense of Rosenblatt (1956) and for weakly dependent random fields in the sense of Wu (2005). Only minimal conditions on the bandwidth parameter and simple conditions on the dependence structure of the data are assumed.

Wild bootstrap bandwidth selection of recursive nonparametric relative regression for independent functional data

We propose and investigate a new kernel regression estimator based on the minimization of the mean squared relative error. We study the properties of the proposed recursive estimator and compare it with the recursive estimator based on the minimization of the mean squared error proposed by Slaoui (2018). It turns out that, with an adequate choice of the parameters, the proposed estimator performs better than the recursive estimator based on the minimization of the mean squared error. We illustrate these theoretical results through a real chemometric dataset.

Skewing methods for variance-stabilizing local linear regression estimation

It is known that kernel regression estimators do not produce a constant estimator variance over a domain. To correct the problem, Nishida and Kanazawa proposed a variance-stabilizing (VS) local variable bandwidth for Local Linear (LL) regression estimator. In contrast, Choi and Hall proposed the skewing (SK) methods for a univariate LL estimator and constructed a convex combination of three SK estimators (the CC estimator) to eliminate bias terms. In this study, we show the CC estimator can also produce constant estimator variance by adjusting its weighting parameter and compare the performances of the two VS methods by simulations.

Bayesian Variance-Stabilizing Kernel Density Estimation Using Conjugate Prior

Kernel-type density or regression estimator does not produce a constant estimator variance over the domain. To correct this problem, K. Nishida and Y. Kanazawa (2011, 2015) proposed a variance-stabilizing (VS) local variable bandwidth for kernel regression estimators. K. Nishida (2017) proposed another strategy to construct VS local linear regression estimator using a convex combination of three skewing estimators proposed by Choi and Hall (1998). In this study, we show that variance stabilization can be accomplished by a Bayesian approach in the case of kernel density estimator using conjugate prior.

Semiparametric multiple kernel estimators and model diagnostics for count regression functions

This study concerns semiparametric approaches to estimate discrete multivariate count regression functions. The semiparametric approaches investigated consist of combining discrete multivariate nonparametric kernel and parametric estimations such that (i) a prior knowledge of the conditional distribution of model response may be incorporated and (ii) the bias of the traditional nonparametric kernel regression estimator of Nadaraya-Watson may be reduced. We are precisely interested in combination of the two estimations approaches with some asymptotic properties of the resulting estimators. Asymptotic normality results were showed for nonparametric correction terms of parametric start function of the estimators. The performance of discrete semiparametric multivariate kernel estimators studied is illustrated using simulations and real count data. In addition, diagnostic checks are performed to test the adequacy of the parametric start model to the true discrete regression model. Finally, using discrete semiparametric multivariate kernel estimators provides a bias reduction when the parametric multivariate regression model used as start regression function belongs to a neighborhood of the true regression model.