Data-driven Bandwidth Selection Research Articles

A comprehensive exploration of complete cross-validation for circular data

Kernel density estimation of circular data has recently received considerable attention for its ability to model and analyse distributions on unit circles and other periodic domains. Our aim is to contribute to the literature on data-driven bandwidth selectors in circular kernel density estimation. We propose a novel circular-specific method that is based on a crossvalidation procedure with a von Mises density used as a kernel function. Using simulated data as well as real-world circular datasets, we evaluate and validate the proposed method and compare it with the existing methods.

Iterated Data Sharpening

Data sharpening in kernel regression has been shown to be an effective method of reducing bias while having minimal effects on variance. Earlier efforts to iterate the data sharpening procedure have been less effective, due to the employment of an inappropriate sharpening transformation. In this article, an iterated data sharpening algorithm is proposed which reduces the asymptotic bias at each iteration, while having modest effects on the variance. The efficacy of the iterative approach is demonstrated theoretically and via a simulation study. Boundary effects persist and the affected region successively grows when the iteration is applied to local constant regression. By contrast, boundary bias successively decreases for each iteration step when applied to local linear regression. This study also shows that after iteration, the resulting estimates are less sensitive to bandwidth choice, and a further simulation study demonstrates that iterated data sharpening with data-driven bandwidth selection via cross-validation can lead to more accurate regression function estimation. Examples with real data are used to illustrate the scope of change made possible by using iterated data sharpening and to also identify its limitations. Supplementary materials for this article are available online.

Data-driven resistant kernel regression

We investigate data-driven bandwidth selection within the confines of robust (resistant) kernel smoothing. While several approaches presently exist, they require user defined robustness parameters. We discuss identification issues within this setting and propose several tractable avenues to fully operationalise this approach. Simulations reveal that the proposed selection methods perform well relative to competing approaches and a small empirical example illustrates its usefulness.

Adaptive minimax density estimation on ℝd for Huber’s contamination model

Abstract We address the problem of adaptive minimax density estimation on $\mathbb{R}^{d}$ with $L_{p}$ loss functions under Huber’s contamination model. To investigate the contamination effect on the optimal estimation of the density, we first establish the minimax rate with the assumption that the density is in an anisotropic Nikol’skii class. We then develop a data-driven bandwidth selection procedure for kernel estimators, which can be viewed as a robust generalization of the Goldenshluger-Lepski method. We show that the proposed bandwidth selection rule can lead to the estimator being minimax adaptive to either the smoothness parameter or the contamination proportion. When both of them are unknown, we prove that finding any minimax-rate adaptive method is impossible. Extensions to smooth contamination cases are also discussed.

Boundary-adaptive kernel density estimation: the case of (near) uniform density

We consider nonparametric kernel estimation of density functions in the bounded-support setting having known support [ a , b ] using a boundary-adaptive kernel function and data-driven bandwidth selection, where a and b are finite and known prior to estimation. We observe, theoretically and in finite sample settings, that when bounds are known a priori this kernel approach is capable of outperforming even correctly specified parametric models, in the case of the uniform distribution. We demonstrate that this result has implications for modelling a range of densities other than the uniform case. Furthermore, when bounds [ a , b ] are unknown and the empirical support (i.e. [ min ( x i ) , max ( x i ) ] ) is used in their place, similar behaviour surfaces.

Mean group instrumental variable estimation of time-varying large heterogeneous panels with endogenous regressors

The large heterogeneous panel data models are extended to the setting where the heterogenous coefficients are changing over time and the regressors are endogenous. Kernel-based non-parametric time-varying parameter instrumental variable mean group (TVP-IV-MG) estimator is proposed for the time-varying cross-sectional mean coefficients. The uniform consistency is shown and the pointwise asymptotic normality of the proposed estimator is derived. A data-driven bandwidth selection procedure is also proposed. The finite sample performance of the proposed estimator is investigated through a Monte Carlo study and an empirical application on multi-country Phillips curve with time-varying parameters.

An exact bootstrap-based bandwidth selection rule for kernel quantile estimators

Bandwidth selection is key for kernel quantile estimators (KQEs), which estimate quantiles by averaging all of the order statistics with appropriate kernel-weighting functions. This paper provides a new data-driven bandwidth selection method for KQEs, named the exact bootstrap-based bandwidth selection (EBS) rule. By relying on the exact analytic expressions for the bootstrap mean and variance of KQEs, the error due to bootstrap resampling is eliminated, and thus the optimal bandwidth can be obtained by minimizing the mean squared error (MSE) estimate. The effectiveness of this EBS rule is confirmed by numerical experiments. First, the bandwidth selection performance of the EBS method is compared to that of a benchmark approach. Simulation studies show that the EBS method performs well, especially in selecting bandwidths for extreme quantiles and when applied to small sample sizes with skewed distributions and relatively large variances. Second, KQEs with a bandwidth determined by our EBS rule is compared with five other state-of-the-art quantile estimators over six typical distributions. The results further validate the efficiency of the EBS method. Third, the results of simulations of controlling actual type-I error rates that occur when two independent groups are compared through quantiles further demonstrate the precision of our EBS-based KQEs.

Stochastic smoothing of point processes for wildlife-vehicle collisions on road networks

Wildlife-vehicle collisions on road networks represent a natural problem between human populations and the environment, that affects wildlife management and raise a risk to the life and safety of car drivers. We propose a statistically principled method for kernel smoothing of point pattern data on a linear network when the first-order intensity depends on covariates. In particular, we present a consistent kernel estimator for the first-order intensity function that uses a convenient relationship between the intensity and the density of events location over the network, which also exploits the theoretical relationship between the original point process on the network and its transformed process through the covariate. We derive the asymptotic bias and variance of the estimator, and adapt some data-driven bandwidth selectors to estimate the optimal bandwidth. The performance of the estimator is analysed through a simulation study under inhomogeneous scenarios. We present a real data analysis on wildlife-vehicle collisions in a region of North-East of Spain.

Spatial Birth–Death–Move Processes: Basic Properties and Estimation of their Intensity Functions

AbstractMany spatiotemporal data record the time of birth and death of individuals, along with their spatial trajectories during their lifetime, whether through continuous-time observations or discrete-time observations. Natural applications include epidemiology, individual-based modelling in ecology, spatiotemporal dynamics observed in bioimaging and computer vision. The aim of this article is to estimate in this context the birth and death intensity functions that depend in full generality on the current spatial configuration of all alive individuals. While the temporal evolution of the population size is a simple birth–death process, observing the lifetime and trajectories of all individuals calls for a new paradigm. To formalise this framework, we introduce spatial birth–death–move processes, where the birth and death dynamics depends on the current spatial configuration of the population and where individuals can move during their lifetime according to a continuous Markov process with possible interactions. We consider non-parametric kernel estimators of their birth and death intensity functions. The setting is original because each observation in time belongs to a non-vectorial, infinite dimensional space and the dependence between observations is barely tractable. We prove the consistency of the estimators in the presence of continuous-time and discrete-time observations, under fairly simple conditions. We moreover discuss how we can take advantage in practice of structural assumptions made on the intensity functions and we explain how data-driven bandwidth selection can be conducted, despite the unknown (and sometimes undefined) second order moments of the estimators. We finally apply our statistical method to the analysis of the spatiotemporal dynamics of proteins involved in exocytosis in cells, providing new insights on this complex mechanism.

Adaptive nonparametric estimation of a component density in a two-class mixture model

A two-class mixture model, where the density of one of the components is known, is considered. We address the issue of the nonparametric adaptive estimation of the unknown probability density of the second component. We propose a randomly weighted kernel estimator with a fully data-driven bandwidth selection method, in the spirit of the Goldenshluger and Lepski method. An oracle-type inequality for the pointwise quadratic risk is derived as well as convergence rates over Hölder smoothness classes. The theoretical results are illustrated by numerical simulations.

Invariant density adaptive estimation for ergodic jump–diffusion processes over anisotropic classes

We consider the solution X=(Xt)t≥0 of a multivariate stochastic differential equation with Levy-type jumps and with unique invariant probability measure with density μ. We assume that a continuous record of observations XT=(Xt)0≤t≤T is available.In the case without jumps, Dalalyan and Reiss (2007) and Strauch (2018) have found convergence rates of invariant density estimators, under respectively isotropic and anisotropic Hölder smoothness constraints, which are considerably faster than those known from standard multivariate density estimation.We extend the previous works by obtaining, in presence of jumps, some estimators which have the same convergence rates they had in the case without jumps for d≥2 and a rate which depends on the degree of the jumps in the one-dimensional setting. We propose moreover a data driven bandwidth selection procedure based on the Goldenshluger and Lepski method (Goldenshluger and Lepski, 2011) which leads us to an adaptive non-parametric kernel estimator of the stationary density μ of the jump–diffusion X.

Nonparametric regression estimate with Berkson Laplace measurement error

In this paper, a nonparametric estimator for the regression function is constructed when the covariates are contaminated with the multivariate Laplace measurement error. The proposed estimator is based upon a simple relationship between the regression function and the conditional expectation of the regression function given the proxy data, as well as the second derivative of this expectation. Large sample properties of the proposed estimator, including the consistency and asymptotic normality, are established. The theoretical optimal bandwidth based on asymptotic integrated mean squared error is derived, and a data-driven bandwidth selector is recommended. Finite sample performance of the proposed estimator is evaluated by a simulation study.

Estimation of semi-varying coefficient error-in-variable models with surrogate data and validation sample

In this study, a semi-varying coefficient error-in-variable model with surrogate data and validation sample is proposed. Without specifying any error structure, we firstly use the local linear kernel smoothing technique to define the estimators and the proposed estimators are proved to be asymptotically normal. Then, we conduct generalized likelihood ratio (GLR) test on varying coefficient function. The data–driven bandwidth selection method is discussed. Finally, simulated studies are conducted to illustrate the finite sample properties of the proposed estimators and efficiency of the GLR methodology.

Bootstrap bandwidth selection in time-varying coefficient models with jumps

The time-varying coefficient models (TVCMs) are very important tools to explore the hidden structure between the time series response variable and its predictors. The assumption that the coefficient functions are smooth can be restrictive in practice. The TVCMs with jumps are needed to be considered. Selection of smoothing parameters plays a critical role in assessing the performance of estimations of coefficient functions. In this article, we first develop a nonparametric two-step estimation procedure for estimating a finite number of jumps of the coefficient functions. Then, based on the estimated jumps, we propose a bootstrapping bandwidth selection procedure in the TVCMs with jumps. Monte Carlo simulations are conducted to evaluate the finite sample performance of the proposed data-driven estimation and bootstrap bandwidth selection procedures. As an illustration, the proposed methodologies are further applied to a real data example.

Non-Parametric Decompounding of Pulse Pile-Up Under Gaussian Noise With Finite Data Sets

A novel estimator is proposed for estimating the energy distribution of photons incident upon a detector in X-ray spectroscopic systems. It is specifically designed for count-rate regimes where pulse pile-up is an issue. A key step in the derivation of the estimator is the novel reformulation of the problem as a decompounding problem of a compound Poisson process. A non-parametric decompounding algorithm is proposed for pile-up correction with finite-length data sets. Non-parametric estimation typically includes appropriately choosing a ‘kernel bandwidth’. Simulations demonstrate our data-driven bandwidth selection is close to optimal, and outperforms asymptotic-based selection in the typical regions of interest to spectroscopic applications. Non-parametric approaches are particularly useful when the shape of the detector response varies with each interaction. The method exhibits similar accuracy to other state-of-the-art non-parametric methods, while being much faster to compute.

Bootstrapping kernel intensity estimation for inhomogeneous point processes with spatial covariates

The bias–variance trade-off for inhomogeneous point processes with covariates is theoretically and empirically addressed. A consistent kernel estimator for the first-order intensity function based on covariates is constructed, which uses a convenient relationship between the intensity and the density of events location. The asymptotic bias and variance of the estimator are derived and hence the expression of its infeasible optimal bandwidth. Three data-driven bandwidth selectors are proposed to estimate the optimal bandwidth. One of them is based on a new smooth bootstrap proposal which is proved to be consistent under a Poisson assumption. The other two are a rule-of-thumb method based on assuming normality, and a simple non-model-based approach. An extensive simulation study is accomplished considering Poisson and non-Poisson scenarios, and including a comparison with other competitors. The practicality of the new proposals is shown through an application to real data about wildfires in Canada, using meteorological covariates.

Computation and application of robust data-driven bandwidth selection for gradient function estimation

The significance of gradient estimates in nonparametric regression cannot be neglected as it is a critical process for executing marginal effect in empirical application. However, the performance of resulting estimates is closely related to the selection of smoothing parameters. The existing methods of parameter choice are either too complicated or not robust enough. For improving the computational efficiency and robustness, a data-driven bandwidth selection procedure is proposed in this paper to compute the gradient of unknown function based on local linear composite quantile regression. Such bandwidth selection method can solve the difficulty of the infeasible selection program that requires the direct observation of true gradient. Moreover, the leading bias and variance of the estimated gradient are obtained under certain regular conditions. It is shown that the bandwidth selection method processes the oracle property in the sense that the selected bandwidth is asymptotically equivalent to the optimal bandwidth if the true gradient is known. Monte Carlo simulations and a real example are conducted to demonstrate the finite sample properties of the suggested method. Both simulation and application corroborate that our technique delivers more effective and robust derivative estimator than some existing approaches.

Higher‐Order Accurate Spectral Density Estimation of Functional Time Series

Under the frequency domain framework for weakly dependent functional time series, a key element is the spectral density kernel which encapsulates the second‐order dynamics of the process. We propose a class of spectral density kernel estimators based on the notion of a flat‐top kernel. The new class of estimators employs the inverse Fourier transform of a flat‐top function as the weight function employed to smooth the periodogram. It is shown that using a flat‐top kernel yields a bias reduction and results in a higher‐order accuracy in terms of optimizing the integrated mean square error (IMSE). Notably, the higher‐order accuracy of flat‐top estimation comes at the sacrifice of the positive semi‐definite property. Nevertheless, we show how a flat‐top estimator can be modified to become positive semi‐definite (even strictly positive definite) in finite samples while retaining its favorable asymptotic properties. In addition, we introduce a data‐driven bandwidth selection procedure realized by an automatic inspection of the estimated correlation structure. Our asymptotic results are complemented by a finite‐sample simulation where the higher‐order accuracy of flat‐top estimators is manifested in practice.

An evaluation of likelihood-based bandwidth selectors for spatial and spatiotemporal kernel estimates

ABSTRACTSpatial point pattern data sets are commonplace in a variety of different research disciplines. The use of kernel methods to smooth such data is a flexible way to explore spatial trends and make inference about underlying processes without, or perhaps prior to, the design and fitting of more intricate semiparametric or parametric models to quantify specific effects. The long-standing issue of ‘optimal’ data-driven bandwidth selection is complicated in these settings by issues such as high heterogeneity in observed patterns and the need to consider edge correction factors. We scrutinize bandwidth selectors built on leave-one-out cross-validation approximation to likelihood functions. A key outcome relates to previously unconsidered adaptive smoothing regimens for spatiotemporal density and multitype conditional probability surface estimation, whereby we propose a novel simultaneous pilot-global selection strategy. Motivated by applications in epidemiology, the results of both simulated and real-world analyses suggest this strategy to be largely preferable to classical fixed-bandwidth estimation for such data.

Cluster Robust Covariance Matrix Estimation in Panel Quantile Regression with Individual Fixed Effects

This paper develops cluster robust inference methods for panel quantile regression (QR) models with individual fixed effects, allowing temporal correlation within each individual. The conventional QR standard errors can seriously underestimate the uncertainty of estimators and therefore overestimate the significance of effects when outcomes are serially correlated. Thus, we propose a clustered covariance matrix estimator (CCM) that solves this problem. The CCM estimator is an extension of the heteroskedasticity and autocorrelation consistent covariance matrix estimator to QR models with fixed effects. The auto-covariance element in the CCM estimator can be substantially biased due to the incidental parameter problem. So we develop a bias correction method. We derive an optimal bandwidth formula that minimises the asymptotic mean squared errors and propose a data driven bandwidth selection rule. We also develop two cluster robust tests and establish their asymptotic properties. Simulation studies show that cluster robust standard errors and tests have excellent finite sample properties. We demonstrate the usefulness of the new methods using two empirical applications.