Kernel Regression Estimator Research Articles

Uniform Convergence Rates of Kernel Estimators with Heterogenous, Dependent Data

The main uniform convergence results of Hansen (2008) are generalized in two directions: Data is allowed to (i) be heterogenously dependent and (ii) depend on a (possibly unbounded) parameter. These results are useful in semiparametric estimation problems involving time-inhomogenous models and/or sampling of continuous-time processes. The usefulness of these results are demonstrated by two applications: Kernel regression estimation of a time-varying AR(1) model, and the kernel density estimation of a Markov chain that has not been intialized at its stationary distribution.

تقدير دالة الأنحدار اللامعلمي باستخدام بعض الطرائق اللامعلمية الرتيبة

This research was concerning to study monotone nonparametric methods for estimating the nonparametric regression function (i.e treatment outlier) to achieve a monotone function (increasing or decreasing).
 So we will use the monotone methods to treatment outlier but after estimate the regression function with use kernel estimator (Nadarya - Watson) these methods are:-
 1- Mukerjee method takes averages of maximums and minimum of subsets of the data was used to adjust the initial kernel regression estimates and use the researcher special case when .
 2- Algorithm least square isotonic regression.
 In the experimental aspect comparison was done of which is the best methods through the simulation procedure using Mote Carlo method using five models.
 While in the application aspect practical application was done on data represent the measurements for blood pressure patients.
 In both aspects we use two of the important statistical measures which are Mean square error (MSE) and efficiency. We find through the application that the best method is Mukerjee method for general case as it has minimum Mean square error and maximum efficiency.

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara–Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.

Kernel regression estimation for continuous spatial processes

We investigate here a kernel estimate of the spatial regression function r(x) = E(Yu | Xu = x), x ∈ ℝd, of a stationary multidimensional spatial process { Zu = (Xu, Yu), u ∈ ℝN}. The weak and strong consistency of the estimate is shown under sufficient conditions on the mixing coefficients and the bandwidth, when the process is observed over a rectangular domain of ℝN. Special attention is paid to achieve optimal and suroptimal strong rates of convergence. It is also shown that this suroptimal rate is preserved by using a suitable spatial sampling scheme.

Integral curves of noisy vector fields and statistical problems in diffusion tensor imaging: Nonparametric kernel estimation and hypotheses testing

Let $v$ be a vector field in a bounded open set $G\subset {\mathbb{R}}^d$. Suppose that $v$ is observed with a random noise at random points $X_i,\ i=1,\dots, n$, that are independent and uniformly distributed in $G$. The problem is to estimate the integral curve of the differential equation \[\frac{dx(t)}{dt}=v(x(t)),\qquad t\geq 0,x(0)=x_{0}\in G,\] starting at a given point $x(0)=x_0\in G$ and to develop statistical tests for the hypothesis that the integral curve reaches a specified set $\Gamma\subset G$. We develop an estimation procedure based on a Nadaraya–Watson type kernel regression estimator, show the asymptotic normality of the estimated integral curve and derive differential and integral equations for the mean and covariance function of the limit Gaussian process. This provides a method of tracking not only the integral curve, but also the covariance matrix of its estimate. We also study the asymptotic distribution of the squared minimal distance from the integral curve to a smooth enough surface $\Gamma\subset G$. Building upon this, we develop testing procedures for the hypothesis that the integral curve reaches $\Gamma$. The problems of this nature are of interest in diffusion tensor imaging, a brain imaging technique based on measuring the diffusion tensor at discrete locations in the cerebral white matter, where the diffusion of water molecules is typically anisotropic. The diffusion tensor data is used to estimate the dominant orientations of the diffusion and to track white matter fibers from the initial location following these orientations. Our approach brings more rigorous statistical tools to the analysis of this problem providing, in particular, hypothesis testing procedures that might be useful in the study of axonal connectivity of the white matter.

Generalized Kernel Regression Estimate for the Identification of Hammerstein Systems

Generalized Kernel Regression Estimate for the Identification of Hammerstein SystemsA modified version of the classical kernel nonparametric identification algorithm for nonlinearity recovering in a Hammerstein system under the existence of random noise is proposed. The assumptions imposed on the unknown characteristic are weak. The generalized kernel method proposed in the paper provides more accurate results in comparison with the classical kernel nonparametric estimate, regardless of the number of measurements. The convergence in probability of the proposed estimate to the unknown characteristic is proved and the question of the convergence rate is discussed. Illustrative simulation examples are included.

Rates of strong uniform consistency for local least squares kernel regression estimators

We establish exact rates of strong uniform consistency for the multivariate Nadaraya–Watson kernel estimator of the regression function and its derivatives. As a special case, we treat the local linear estimator of the regression and the local polynomial smoothers of derivatives of the regression in the more convenient univariate setting. Our methods of proofs are based upon modern empirical process theory in the spirit of the results of Einmahl and Mason [2000. An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab. 13.1, 1–37.] and Deheuvels and Mason [2004. General asymptotic confidence bands based on kernel-type function estimators. Statist. Infer. Stochastic Process. 7.3, pp. 225–277] relative to uniform deviations of nonparametric kernel estimators.

On Nonparametric Identification of Wiener Systems

In this paper, a new method for the identification of the Wiener nonlinear system is proposed. The system, being a cascade connection of a linear dynamic subsystem and a nonlinear memoryless element, is identified by a two-step semiparametric approach. The impulse response function of the linear part is identified via the nonlinear least-squares approach with the system nonlinearity estimated by a pilot nonparametric kernel regression estimate. The obtained estimate of the linear part is then used to form a nonparametric kernel estimate of the nonlinear element of the Wiener system. The proposed method permits recovery of a wide class of nonlinearities which need not be invertible. As a result, the proposed algorithm is computationally very efficient since it does not require a numerical procedure to calculate the inverse of the estimate. Furthermore, our approach allows non-Gaussian input signals and the presence of additive measurement noise. However, only linear systems with a finite memory are admissible. The conditions for the convergence of the proposed estimates are given. Computer simulations are included to verify the basic theory

Randomly censored partially linear single-index models

This paper proposes a method for estimation of a class of partially linear single-index models with randomly censored samples. The method provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure-time models for survival analysis. The estimation procedure involves three stages: first, transform the censored data into synthetic data or pseudo-responses unbiasedly; second, obtain quasi-likelihood estimates of the regression coefficients in both linear and single-index components by an iteratively algorithm; finally, estimate the unknown nonparametric regression function using techniques for univariate censored nonparametric regression. The estimators for the regression coefficients are shown to be jointly root- n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as all the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodology.

ON STUDY OF A MODIFIED LOCAL CONSTANT $M$-SMOOTHER

In the case of the equally spaced fixed design nonparametric regression, the local constant $M$-smoother (LCM) with local maximizing is proposed by Chu, Glad, Godtliebsen, and Marron (1998) to correct for the effect of discontinuity on the kernel regression estimator. It has the interesting property of jump-preserving. However, in the jump region, it is inconsistent when the magnitude of the noise is larger than the size of the jump in the regression. To adjust for this drawback to the ordinary LCM, we propose to construct the LCM with global maximizing instead of local maximizing as well as with binned data instead of original data. Our proposed estimator is analyzed by the asymptotic mean square error. Both binning and global maximizing have no effect on the asymptotic mean square error of the ordinary LCM in the smooth region, but have an effect on improving the inconsistency of the ordinary LCM in the jump region. Simulation studies demonstrate that the regression function estimate produced by our modified LCM is better than those by alternatives, in the sense of yielding smaller sample mean integrated square error, showing more accurately the location of jump point, and having smoother appearance.

Sequential Kernel Estimation of the Conditional Intensity of Nonstationary Point Processes

This paper develops adaptive nonparametric methods for analyzing seismic data. Kernel smoothing techniques are suitable for space–time point processes; however, they must be adapted to deal with the nonstationarity of earthquakes. By this we mean changes in the spatial and temporal pattern of point occurrences. A class of recursive kernel density and regression estimators are proposed to study the space–time evolution of earthquakes. Their smoothing coefficients can be designed in an optimal way with prediction error criteria. The entire method is naturally oriented to forecasting. An extensive application to the Northern California Earthquake Catalog (NCEC) data-set, and a simulation experiment illustrates and checks the approach.

Sharp large deviations in nonparametric estimation

Large deviation results for the kernel density estimator and the kernel regression estimator have been given by Louani [Louani, D., 1998, Large deviations limit theorems for the kernel density estimator. Scandinavian Journal of Statistics, 25, 243–253; Louani, D., 1999, Some large deviations limit theorems in conditional nonparametric statistics. Statistics, 33, 171–196]. We complete these works by establishing sharp large deviation results for the two estimators. This means that we study precisely the tail probabilities of the estimators. We distinguish two cases depending on the support of the kernel. To prove the results, we need an Edgeworth expansion obtained from a version of Cramer’s condition.

An approach to nonparametric smoothing techniques for regressions with discrete data

This paper proposes nonparametric regression estimation techniques for small samples in situations where the dependent variable involves count data. Often the form of a kernel will not matter asymptotically. However, in small samples the kernel structure may play a more important role in approximating the small sample distribution especially for discrete random variables. In particular for count data we introduce a Poisson kernel regression estimator and a binomial kernel regression estimator. These new regression methods are applied to coal mine wildcat strike data. We use cross validation to evaluate out-of-sample performance.

ON STUDY OF A MODIFIED LOCAL CONSTANT $M$-SMOOTHER

In the case of the equally spaced fixed design nonparametric regression, the local constant $M$-smoother (LCM) with local maximizing is proposed by Chu, Glad, Godtliebsen, and Marron (1998) to correct for the effect of discontinuity on the kernel regression estimator. It has the interesting property of jump-preserving. However, in the jump region, it is inconsistent when the magnitude of the noise is larger than the size of the jump in the regression. To adjust for this drawback to the ordinary LCM, we propose to construct the LCM with global maximizing instead of local maximizing as well as with binned data instead of original data. Our proposed estimator is analyzed by the asymptotic mean square error. Both binning and global maximizing have no effect on the asymptotic mean square error of the ordinary LCM in the smooth region, but have an effect on improving the inconsistency of the ordinary LCM in the jump region. Simulation studies demonstrate that the regression function estimate produced by our modified LCM is better than those by alternatives, in the sense of yielding smaller sample mean integrated square error, showing more accurately the location of jump point, and having smoother appearance.

Non-parametric regression estimation on closed Riemannian manifolds

The non-parametric estimation of the regression function of a real-valued random variable Y on a random object X valued in a closed Riemannian manifold M is considered. A regression estimator which generalizes kernel regression estimators on Euclidean sample spaces is introduced. Under classical assumptions on the kernel and the bandwidth sequence, the asymptotic bias and variance are obtained, and the estimator is shown to converge at the same L 2-rate as kernel regression estimators on Euclidean spaces.

A comparative study of monotone nonparametric kernel estimates

In this paper we present a detailed numerical comparison of three monotone nonparametric kernel regression estimates, which isotonize a nonparametric curve estimator. The first estimate is the classical smoothed isotone estimate of Brunk [Brunk, H.D., 1955, Maximum likelihood estimates of monotone parameters. The Annals of Mathematical Statistics, 26, 607–616.]. The second method has recently been proposed by Hall and Huang [Hall, P. and Huang, L.-S., 2001, Nonparametric kernel regression subject to monotonicity constraints. The Annals of Statistics, 29, 624–647.] and modifies the weights of a commonly used kernel estimate such that the resulting estimate is monotone. The third estimate was recently proposed by Dette et al. [Dette, H., Neumeyer, N. and Pilz, K.F., 2003, A simple non-parametric estimator of a monotone regression function. Technical report, Department of Mathematics. Available online at: http://www.ruhr-uni-bochum.de/mathematik3/preprint.htm] and combines density and regression estimation techniques to obtain a monotone curve estimate of the inverse of the isotone regression function. The three concepts are briefly reviewed and their finite sample properties are studied by means of a simulation study. Although all estimates are first-order asymptotically equivalent (provided that the unknown regression function is isotone) some differences for moderate sample sizes are observed.

An evaluation of the performance of kernel estimators for graduating mortality data

In the graduation of the age-specific mortality pattern, recent emphasis has been placed on the use of kernel regression estimators. Three such estimators are the Nadaraya-Watson, Gasser-Muller and kernel weighted local linear estimators. This paper discusses the theoretical background of each estimator and evaluates their accuracy in graduating age-specific mortality using data for France, Japan and Sweden. For comparison, we also fit the Heligman-Pollard model and its nine-parameter variant by Kostaki. The Gasser-Muller estimator is found to be superior to the two other kernel estimators in that it is both more stable and not influenced by boundary effects. Furthermore, compared with the two parametrric models, the Gasser-Muller estimator provides a more satisfactory graduation, especially at older adult ages, in terms both of smoothness and of fidelity between the observed and graduated rates.

Principal surfaces from unsupervised kernel regression

We propose a nonparametric approach to learning of principal surfaces based on an unsupervised formulation of the Nadaraya-Watson kernel regression estimator. As compared with previous approaches to principal curves and surfaces, the new method offers several advantages: First, it provides a practical solution to the model selection problem because all parameters can be estimated by leave-one-out cross-validation without additional computational cost. In addition, our approach allows for a convenient incorporation of nonlinear spectral methods for parameter initialization, beyond classical initializations based on linear PCA. Furthermore, it shows a simple way to fit principal surfaces in general feature spaces, beyond the usual data space setup. The experimental results illustrate these convenient features on simulated and real data.

A new model validation tool using kernel regression and density estimation

In physiological system modelling for control or decision support, model validation is a critical element. A nonparametric approach for assessing the validity of deterministic dynamic models against empirical data is developed, based on kernel regression and kernel density estimation, yielding visual graphical assessment tools as well as numerical metrics of compatibility between the model and the data. Nonparametric regression has been suggested for assessing a parametric statistical model by constructing a confidence band for the proposed model and then checking whether the nonparametric regression curve lies within the band. However, for deterministic models, there is no confidence band that can be constructed. A reversal of roles is therefore suggested--construct a probability band for the nonparametric regression curve and check whether the proposed model lies within the band. This approach extends the utility of nonparametric regression for model assessment to deterministic models. Weighted kernel density estimation is incorporated to derive a density profile for the regression curve, creating a local graphical validation tool. In addition, the density profile is used to define and compute two numerical measures--average normalized density (AND) and relative average normalized density (RAND), representing global statistical validity measures. These tools are demonstrated using a biomedical system model for agitation-sedation and sedation management control.

Nonparametric regression function estimation with surrogate data and validation sampling

This paper develops estimation approaches for nonparametric regression analysis with surrogate data and validation sampling when response variables are measured with errors. Without assuming any error model structure between the true responses and the surrogate variables, a regression calibration kernel regression estimate is defined with the help of validation data. The proposed estimator is proved to be asymptotically normal and the convergence rate is also derived. A simulation study is conducted to compare the proposed estimators with the standard Nadaraya–Watson estimators with the true observations in the validation data set and the complete observations, respectively. The Nadaraya–Watson estimator with the complete observations can serve as a gold standard, even though it is practically unachievable because of the measurement errors.