Bandwidth Selection Method Research Articles

On Bandwidth Selection in Local Polynomial Regression Analysis and Its Application to Multi-resolution Analysis of Non-uniform Data

This paper studies adaptive bandwidth selection method for local polynomial regression (LPR) and its application to multi-resolution analysis (MRA) of non-uniformly sampled data. In LPR, the observations are modeled locally by a polynomial using least-squares criterion with a kernel having a certain support or bandwidth so that a better bias-variance tradeoff can be achieved. In this paper, two bandwidth selection methods, namely the Fan and Gijbels's bandwidth selection (FGBS) method (Fan and Gijbels, Local Polynomial Modelling and Its Applications, Chapman and Hall, London, 1996; Fan and Gijbels, Stat Sin 57:371---394, 1995) in the statistical community and the intersection of confidence intervals (ICI) method commonly used in the signal and image processing communities, are reviewed and compared in terms of their performance and implementation complexity using standard testing data sets. Furthermore, using the result of Stankovi (IEEE Trans Signal Proc 52:1228---1234, 2004), a new refined ICI-based adaptive bandwidth selection method for LPR and its associated reliability analysis are proposed. In addition, recursive implementations of LPR with the two classes of bandwidth selection methods are considered for online applications. Simulation results show that the performances of the FGBS method and the refined ICI method are comparable for the data sets tested. Since LPR with adaptive bandwidths can be naturally applied to non-uniformly sampled noisy observations, we propose to use it as a pre-processing step to a conventional MRA so that a MRA of non-uniformly sampled data can be realized. Simulation results show that the proposed LPR-based MRA gives better results than conventional linear interpolation of the data.

Variable Selection in Logistic Discrimination Based on Local Likelihood

We consider the variable selection problem in the nonlinear discriminant procedure using local likelihood. The local likelihood method is an effective technique for analyzing data with complex structure,and various bandwidth selection methods have been suggested in recent years. Variable selection in a nonlinear model,however, is more complex than bandwidth selection,since the optimal bandwidth depends on the combination of the variables. We propose a technique for variable selection using generalized information criteria in logistic discrimination based on local likelihood. We derive the logistic discrimination method with a sample covariance matrix to account for the correlation of the variables. Real data examples are given to examine the effectiveness of our technique.

Nonparametric Econometrics: ThenpPackage

We describe the R <b>np</b> package via a series of applications that may be of interest to applied econometricians. The np package implements a variety of nonparametric and semiparametric kernel-based estimators that are popular among econometricians. There are also procedures for nonparametric tests of significance and consistent model specification tests for parametric mean regression models and parametric quantile regression models, among others. The <b>np</b> package focuses on kernel methods appropriate for the mix of continuous, discrete, and categorical data often found in applied settings. Data-driven methods of bandwidth selection are emphasized throughout, though we caution the user that data-driven bandwidth selection methods can be computationally demanding.

Relative error prediction via kernel regression smoothers

In this article, we introduce and study local constant and local linear nonparametric regression estimators when it is appropriate to assess performance in terms of mean squared relative error of prediction. We give asymptotic results for both boundary and non-boundary cases. These are special cases of more general asymptotic results that we provide concerning the estimation of the ratio of conditional expectations of two functions of the response variable. We also provide a good bandwidth selection method for the estimators. Examples of application, limited simulation results and discussion of related problems and approaches are also given.

Smooth Location-Dependent Bandwidth Selection for Local Polynomial Regression

A bandwidth function for local polynomial models is commonly obtained by optimizing a pointwise penalty criterion, such as an estimated mean squared error (MSE), over a grid of predictor locations. A resultant regression estimate may suffer from irregularities, such as discontinuities, and contextual information over nearby predictor locations is not used. To mediate these difficulties, ad hoc postprocessing is sometimes carried out in the form of smoothing of the penalty criterion and/or the bandwidth estimates. In this work a technique is developed for choosing a smooth bandwidth function that uses a smoothing spline selected based on new “fit” and “roughness” penalties. The fit penalty pushes the bandwidth estimate to adhere to the chosen pointwise criterion, whereas the roughness penalty is imposed on the fitted regression estimate as opposed to the bandwidth estimate, which usually is not of direct interest. The technique can be used in conjunction with various adaptive bandwidth selection methods and provides a systematic way of incorporating contextual information into bandwidth estimation. To justify a spline bandwidth function, we show that under mild regularity conditions, there exists a smooth, asymptotically optimal bandwidth function. We also demonstrate empirically that the technique outperforms the empirical-bias bandwidth selector (EBBS) of Ruppert when using an EBBS MSE pointwise penalty estimate.

A general and fast convergent bandwidth selection method of kernel estimator

A new method of bandwidth estimation is developed in this article. It has the attractive property of controlled rate of convergence, regardless of sample size. This approach is applied in different contexts of functional estimation leading to bandwidth selectors that enjoy minimal assumptions, are easy to compute and have fast orders of convergence. The method is presented to jointly apply to the density, its derivatives and the distribution functions, all in one setting, and is used then to deliver bandwidth selectors in estimating density derivative functionals.

Smooth semiparametric receiver operating characteristic curves for continuous diagnostic tests

We propose a semiparametric kernel distribution function estimator, based on which a new smooth semiparametric estimator of the receiver operating characteristic (ROC) curve is constructed. We derive the asymptotic bias and variance of the newly proposed distribution function estimator and show that it is more efficient than the traditional non-parametric kernel distribution estimator. We also derive the asymptotic bias and variance of our new ROC curve estimator and show that it is more efficient than the smooth non-parametric ROC curve estimator proposed by Zou et al. (Stat. Med. 1997; 16:2143-2156) and Lloyd (J. Am. Stat. Assoc. 1998; 93:1356-1364). For our proposed estimators, we derive data-based methods for bandwidth selection. In addition, we present some results on the analysis of two real data sets. Finally, a simulation study is presented to show that our estimators are better than the non-parametric counterparts in terms of bias, standard error, and mean-square error.

Optimal bandwidth selection for multivariate kernel deconvolution density estimation

Assume we have i.i.d. replications from the mismeasured random vector Y=X+e, where X and e are mutually independent. We consider a data-driven bandwidth, based on a cross-validation ideas, for multivariate kernel deconvolution estimator of the density of X. The proposed data-driven bandwidth selection method is shown to be asymptotically optimal. As a by-product of the proof of this result, we show that the average squared error, the integrated squared error, and the mean integrated squared error are asymptotically equivalent error measures.

Finite sample performance of kernel-based regression methods for non-parametric additive models under common bandwidth selection criterion

In this paper, we investigate the finite sample performance of four kernel-based estimators that are currently available for additive non-parametric regression models—the classic backfitting estimator (CBE), the smooth backfitting estimator, the marginal integration estimator, and two versions of a two-stage estimator of which the first is proposed by Kim, Linton and Hengartner (1999) and the second is proposed in this paper. The bandwidths are selected for each estimator by minimizing their respective asymptotic approximation of the mean average squared errors. In our simulations, we are particularly concerned with the performance of these estimators under this unified data-driven bandwidth selection method, since in this case both the asymptotic and the finite sample properties of all estimators are currently unavailable. The comparison is based on the estimators' average squared error. Our Monte Carlo results seem to suggest that the CBE is the best performing kernel-based procedure.

Bandwidth Selection for Fixed-Kernel Analysis of Animal Utilization Distributions

In studies of animal space use, researchers often use kernel-based techniques for estimating the size of an animal's home range and its utilization distribution from radiotracking data. However, the kernel estimator is highly sensitive to the bandwidth value used. Previous ecological studies recommended least-squares cross-validation (LSCV) as the default bandwidth selection method, but some statisticians consider this technique inferior to newer methods. We used simulations to compare the performance of the scaling LSCV and reference approaches to plug-in and solve-the-equation (STE) bandwidth methods. We generated samples of 20, 50, and 150 points from mixtures of 2, 4, and 16 bivariate normal distributions. We selected the ranges of potential variances for these distributions to create 4 distribution types with varied levels of clumping to simulate the diversity of location patterns expected from radiotracking data. For most distribution types, plug-in and STE methods performed as well or better than LSCV in % absolute error of home-range size estimates and overlap of estimated and true utilization distributions. Although the relative differences usually were small, the plug-in and STE approaches provide good alternatives to LSCV. However, LSCV performed better with distribution types composed entirely of tight clumps of points. The reference bandwidth performed poorly for most distributions. Surprisingly, it often had the lowest absolute error at outer contours for distributions consisting of a single very tight cluster surrounded by more dispersed points. Although our results demonstrate the utility of plug-in and STE approaches, no method was best across all distributions. Rather, choice of a bandwidth selection method may vary depending on the study goals, sample size, and patterns of space use by the study species. In general, we recommend plug-in and STE approaches for estimating relatively smooth outer contours. The LSCV approach is better at identifying tight clumps, including areas of peak use, although risk of LSCV failure also increases when a distribution has a very tight cluster of points. When planning to use kernel methods, researchers should consider these factors to make preliminary decisions about the bandwidth method expected to be most appropriate in their study.

A cross-validation method for data with ties in kernel density estimation

Limitation of the cross-validation method of bandwidth selection is well known when applied to data with ties. A method which resolves this problem and which is easy to understand and implement is proposed. We show that the proposed approach is viable in theory, by proving its asymptotic equivalence to the standard cross-validation method. The practical usefulness is shown in simulations and an application to a real data example.

Plug-in bandwidth selection in kernel hazard estimation from dependent data

The plug-in bandwidth selection method in nonparametric kernel hazard estimation is considered, and a weak dependence on the sample data is assumed. A general result of asymptotic optimality for the plug-in bandwidth is presented, that is valid for the hazard function, as well as for the density and distribution functions. In a simulation study, this method is compared with the “leave more than one out” cross-validation criterion under dependence. Simulations show that smaller errors and much less sample variability can be reached, and that a good selection of the pilot bandwidth can be done by means of “leave one out” cross-validation. Finally, an application to an earthquake data set is made.

Bandwidth selection for kernel binomial regression

In nonparametric binomial regression, the weighted kernel estimator of the regression function and its efficient bias-adjusted version have been proposed by Okumura and Naito (2004) with consideration to differences of variances of observed response proportions at covariates. The aim of this article is to propose an effective data-based method for bandwidth selection of the bias-adjusted estimator. The proposed method is developed through three steps: the plug-in method by Ruppert et al. (1995), a scale adjustment suggested by Yang and Tschernig (1999), and an effective use of the approach discussed by Grizzel et al. (1969) for the rule-of-thumb part. Theoretical considerations on the asymptotic performance of the selected bandwidth are given under the situation where the numbers of covariates and responses observed at each covariate increase.

RIGID OBJECT TRACKING USING MEAN-SHIFT WITH ADAPTIVE KERNEL BANDWIDTH SELECTION

The mean shift algorithm is an efficient way for object tracking. However, there is presently no clear mechanism for selecting kernel bandwidth while the object is changing in size. This paper presents a novel bandwidth selection method for mean shift based rigid object tracking. The kernel bandwidth is updated by discovering the scale parameters of the object's affine model that are estimated by using the correspondences between the corner object in two consecutive frames. The centroid of the object is registered by a special backward tracking method. Therefore, we can not only get translation parameter to simplify affine model but also improve the accuracy of finding corner correspondences. In addition, the M-estimate method is employed to reject mismatched pairs (outliers) so as to get better regression results. We have applied the proposed method to track vehicles changing in size with encouraging results.

Classification Using Kernel Density Estimates

The use of kernel density estimates in discriminant analysis is quite well known among scientists and engineers interested in statistical pattern recognition. Using a kernel density estimate involves properly selecting the scale of smoothing, namely the bandwidth parameter. The bandwidth that is optimum for the mean integrated square error of a class density estimator may not always be good for discriminant analysis, where the main emphasis is on the minimization of misclassification rates. On the other hand, cross-validation–based methods for bandwidth selection, which try to minimize estimated misclassification rates, may require huge computation when there are several competing populations. Besides, such methods usually allow only one bandwidth for each population density estimate, whereas in a classification problem, the optimum bandwidth for a class density estimate may vary significantly, depending on its competing class densities and their prior probabilities. Therefore, in a multiclass problem, it would be more meaningful to have different bandwidths for a class density when it is compared with different competing class densities. Moreover, good choice of bandwidths should also depend on the specific observation to be classified. Consequently, instead of concentrating on a single optimum bandwidth for each population density estimate, it is more useful in practice to look at the results for different scales of smoothing for the kernel density estimates. This article presents such a multiscale approach along with a graphical device leading to a more informative discriminant analysis than the usual approach based on a single optimum scale of smoothing for each class density estimate. When there are more than two competing classes, this method splits the problem into a number of two-class problems, which allows the flexibility of using different bandwidths for different pairs of competing classes and at the same time reduces the computational burden that one faces for usual cross-validation–based bandwidth selection in the presence of several competing populations. We present some benchmark examples to illustrate the usefulness of the proposed methodology.

Bandwidth Selection in Local Polynomial Regression Using Eigenvalues

SummaryLocal polynomial regression is commonly used for estimating regression functions. In practice, however, with rough functions or sparse data, a poor choice of bandwidth can lead to unstable estimates of the function or its derivatives. We derive a new expression for the leading term of the bias by using the eigenvalues of the weighted design matrix where the bias depends on the arrangement of the X-values in the bandwidth window. We then use this result to determine a local data-driven bandwidth selection method and to provide a diagnostic for poor bandwidths that are chosen by using other methods. We show that our data-driven bandwidth is asymptotically equivalent to the optimal local bandwidth and that it performs well for relatively small samples when compared with other methods. In addition, we provide simulation results for first-derivative estimation. We illustrate its performance with data from Mars Global Surveyor.

Bandwidth selection for the local polynomial estimator under dependence: A simulation study

Seven of the most popular methods for bandwidth selection in regression estimation are compared by means of a thorough simulation study, when the local polynomial estimator is used and the observations are dependent. The study is completed with two plug-in bandwidths for the generalized local polynomial estimator proposed by Vilar-Fernândez & Francisco-Fernandez (2002).

Bandwidth selection for smooth backfitting in additive models

The smooth backfitting introduced by Mammen, Linton and Nielsen [Ann. Statist. 27 (1999) 1443-1490] is a promising technique to fit additive regression models and is known to achieve the oracle efficiency bound. In this paper, we propose and discuss three fully automated bandwidth selection methods for smooth backfitting in additive models. The first one is a penalized least squares approach which is based on higher-order stochastic expansions for the residual sums of squares of the smooth backfitting estimates. The other two are plug-in bandwidth selectors which rely on approximations of the average squared errors and whose utility is restricted to local linear fitting. The large sample properties of these bandwidth selection methods are given. Their finite sample properties are also compared through simulation experiments.

Automatic Selection of Kernel-Bandwidth for Mean-Shift Object Tracking

Classic Mean-Shift based tracking algorithm uses fixed kernel-bandwidth, which limits the performance when the object scale exceeds the size of the tracking window. Based on the analysis of similarity of object kernel-histogram in different scales, i.e. the Bhattacharyya coefficient, a theorem is found and proved i.e. the changes of object scale and position within the kernel will not impact localization accuracy of Mean-Shift based tracking algorithm. Using this theorem an automatic bandwidth selection method is proposed based on backward tracking and object centroid registration. The proposed method is applied to track vehicle changing in size with encouraging results.

Estimation of semi-parametric additive coefficient model

In the multivariate regression setting, we propose a flexible varying coefficient model in which the regression coefficients of some predictors are additive functions of other predictors. Marginal integration estimators of the coefficients are developed and their asymptotic properties investigated. Under β -mixing, it is found that the estimators of the parameters in the regression coefficients have rate of convergence 1 / n , and the nonparametric additive components are estimated at the same rate of convergence as in univariate smoothing. A data-driven bandwidth selection method is developed based on asymptotic considerations. Its effectiveness is confirmed in a Monte-Carlo study. The procedure is applied to the real German GNP and Wolf's Sunspot data, where the semi-parametric additive coefficient model demonstrates superior performance in terms of out-of-sample forecasts.