Kernel Bandwidth Research Articles

Variable Selection in Kernel Regression Using Measurement Error Selection Likelihoods

ABSTRACTThis article develops a nonparametric shrinkage and selection estimator via the measurement error selection likelihood approach recently proposed by Stefanski, Wu, and White. The measurement error kernel regression operator (MEKRO) has the same form as the Nadaraya–Watson kernel estimator, but optimizes a measurement error model selection likelihood to estimate the kernel bandwidths. Much like LASSO or COSSO solution paths, MEKRO results in solution paths depending on a tuning parameter that controls shrinkage and selection via a bound on the harmonic mean of the pseudo-measurement error standard deviations. We use small-sample-corrected AIC to select the tuning parameter. Large-sample properties of MEKRO are studied and small-sample properties are explored via Monte Carlo experiments and applications to data. Supplementary materials for this article are available online.

Effects of DNA Methylation on Progression to Interstitial Fibrosis and Tubular Atrophy in Renal Allograft Biopsies: A Multi-Omics Approach.

Progressive fibrosis of the interstitium is the dominant final pathway in renal destruction in native and transplanted kidneys. Over time, the continuum of molecular events following immunological and nonimmunological insults lead to interstitial fibrosis and tubular atrophy and culminate in kidney failure. We hypothesize that these insults trigger changes in DNA methylation (DNAm) patterns, which in turn could exacerbate injury and slow down the regeneration processes, leading to fibrosis development and graft dysfunction. Herein, we analyzed biopsy samples from kidney allografts collected 24 months posttransplantation and used an integrative multi-omics approach to understand the underlying molecular mechanisms. The role of DNAm and microRNAs on the graft gene expression was evaluated. Enrichment analyses of differentially methylated CpG sites were performed using GenomeRunner. CpGs were strongly enriched in regions that were variably methylated among tissues, implying high tissue specificity in their regulatory impact. Corresponding to this methylation pattern, gene expression data were related to immune response (activated state) and nephrogenesis (inhibited state). Preimplantation biopsies showed similar DNAm patterns to normal allograft biopsies at 2 years posttransplantation. Our findings demonstrate for the first time a relationship among epigenetic modifications and development of interstitial fibrosis, graft function, and inter-individual variation on long-term outcomes.

GPU-based volume reconstruction for freehand 3D ultrasound imaging.

Volume reconstruction plays an important role in improving image quality for freehand three-dimensional (3D) ultrasound imaging. The kernel regression provides an effective method for volume reconstruction in 3D ultrasound imaging, but it requires heavily computational time-cost. In this paper, a programmable graphic-processor-unit-(GPU) based fast kernel regression method is proposed for freehand 3D ultrasound volume reconstruction. The most significant aspect of our method is the adopting of powerful data-parallel computing capability of GPU to improve the overall efficiency. To produce higher image quality, the results of the kernel regression with various parameter settings is deeply investigated under the help of the fast implementation of the algorithm. Experimental results demonstrate that the computational performance of the proposed GPU-based method can be over 200 times faster than that on CPU. Better image quality for speckle reduction and details preservation can be obtained with the parameter setting of kernel window size of 5×5×5 and kernel bandwidth of 1.0.

A kernelized non-parametric classifier based on feature ranking in anisotropic Gaussian kernel

Non-parametric methods make no assumptions about the form of data distribution and estimate it directly from the data. Kernel density estimation is a non-parametric method which estimates the probability density function of an unknown distribution. To estimate the density using a kernel estimator, it is necessary to have a bandwidth selection procedure. This paper proposes a kernelized non-parametric classifier based on feature ranking in anisotropic Gaussian kernel (KNR-AGK) and focuses on the selection of different bandwidths in kernel density estimation. KNR-AGK uses the rank of features to learn the parameters of an anisotropic Gaussian kernel and considers these ranks as kernel bandwidths in different dimensions. In the proposed method, the rank of features is also used for feature selection based on filter methods to exclude low-ranked features that have a negative impact on the performance of KNR-AGK. To evaluate the performance of the proposed method, comprehensive experiments are conducted on several benchmark datasets. Experiment results show that the proposed classifier has better performance than Gaussian kernel density estimation based classifier.

The Application of a Geographically Weighted Principal Component Analysis for Exploring Twenty-three Years of Goat Population Change across Mongolia

The dzud are extreme weather events in Mongolia of deep snow, severe cold, or other conditions that render forage unavailable or inaccessible, which in turn results in extensive livestock deaths. Mongolia is economically vulnerable to extreme events due to an increase in nonprofessional herders and the livestock population, brought about by a deregularized industry. Thus, it is hugely informative to try to understand the spatial and temporal trends of livestock population change. To this end, annual livestock census data are exploited and a geographically weighted principal component analysis (GWPCA) is applied to goat data recorded from 1990 to 2012 in 341 regions. This application of GWPCA to temporal data is novel and is able to account for both temporal and spatial patterns in goat population change. Furthermore, the GWPCA methodology is extended to simultaneously optimize the number of components to retain and the kernel bandwidth. In doing so, this study not only advances the GWPCA method but provides a useful insight into the spatiotemporal variations of the Mongolian goat population.

A voting-based statistical cylinder detection framework applied to fallen tree mapping in terrestrial laser scanning point clouds

This paper introduces a statistical framework for detecting cylindrical shapes in dense point clouds. We target the application of mapping fallen trees in datasets obtained through terrestrial laser scanning. This is a challenging task due to the presence of ground vegetation, standing trees, DTM artifacts, as well as the fragmentation of dead trees into non-collinear segments. Our method shares the concept of voting in parameter space with the generalized Hough transform, however two of its significant drawbacks are improved upon. First, the need to generate samples on the shape’s surface is eliminated. Instead, pairs of nearby input points lying on the surface cast a vote for the cylinder’s parameters based on the intrinsic geometric properties of cylindrical shapes. Second, no discretization of the parameter space is required: the voting is carried out in continuous space by means of constructing a kernel density estimator and obtaining its local maxima, using automatic, data-driven kernel bandwidth selection. Furthermore, we show how the detected cylindrical primitives can be efficiently merged to obtain object-level (entire tree) semantic information using graph-cut segmentation and a tailored dynamic algorithm for eliminating cylinder redundancy. Experiments were performed on 3 plots from the Bavarian Forest National Park, with ground truth obtained through visual inspection of the point clouds. It was found that relative to sample consensus (SAC) cylinder fitting, the proposed voting framework can improve the detection completeness by up to 10 percentage points while maintaining the correctness rate.

GPU-accelerated Kernel Regression Reconstruction for Freehand 3D Ultrasound Imaging.

Volume reconstruction method plays an important role in improving reconstructed volumetric image quality for freehand three-dimensional (3D) ultrasound imaging. By utilizing the capability of programmable graphics processing unit (GPU), we can achieve a real-time incremental volume reconstruction at a speed of 25-50 frames per second (fps). After incremental reconstruction and visualization, hole-filling is performed on GPU to fill remaining empty voxels. However, traditional pixel nearest neighbor-based hole-filling fails to reconstruct volume with high image quality. On the contrary, the kernel regression provides an accurate volume reconstruction method for 3D ultrasound imaging but with the cost of heavy computational complexity. In this paper, a GPU-based fast kernel regression method is proposed for high-quality volume after the incremental reconstruction of freehand ultrasound. The experimental results show that improved image quality for speckle reduction and details preservation can be obtained with the parameter setting of kernel window size of [Formula: see text] and kernel bandwidth of 1.0. The computational performance of the proposed GPU-based method can be over 200 times faster than that on central processing unit (CPU), and the volume with size of 50 million voxels in our experiment can be reconstructed within 10 seconds.

Band Selection for Nonlinear Unmixing of Hyperspectral Images as a Maximal Clique Problem.

Kernel-based nonlinear mixing models have been applied to unmix spectral information of hyperspectral images when the type of mixing occurring in the scene is too complex or unknown. Such methods, however, usually require the inversion of matrices of sizes equal to the number of spectral bands. Reducing the computational load of these methods remains a challenge in large-scale applications. This paper proposes a centralized band selection (BS) method for supervised unmixing in the reproducing kernel Hilbert space. It is based upon the coherence criterion, which sets the largest value allowed for correlations between the basis kernel functions characterizing the selected bands in the unmixing model. We show that the proposed BS approach is equivalent to solving a maximum clique problem, i.e., searching for the biggest complete subgraph in a graph. Furthermore, we devise a strategy for selecting the coherence threshold and the Gaussian kernel bandwidth using coherence bounds for linearly independent bases. Simulation results illustrate the efficiency of the proposed method.

Variable Kernel Density Estimation in High-Dimensional Feature Spaces

Estimating the joint probability density function of a dataset is a central task in many machine learning applications. In this work we address the fundamental problem of kernel bandwidth estimation for variable kernel density estimation in high-dimensional feature spaces. We derive a variable kernel bandwidth estimator by minimizing the leave-one-out entropy objective function and show that this estimator is capable of performing estimation in high-dimensional feature spaces with great success. We compare the performance of this estimator to state-of-the art maximum-likelihood estimators on a number of representative high-dimensional machine learning tasks and show that the newly introduced minimum leave-one-out entropy estimator performs optimally on a number of high-dimensional datasets considered.

Adaptive Mean Shift-Based Identification of Individual Trees Using Airborne LiDAR Data

Identifying individual trees and delineating their canopy structures from the forest point clouddataacquiredbyanairborneLiDAR(LightDetectionAndRanging)hassignificantimplications in forestry inventory. Once accurately identified, tree structural attributes such as tree height, crown diameter, canopy based height and diameter at breast height can be derived. This paper focuses on a novel computationally efficient method to adaptively calibrate the kernel bandwidth of a computational scheme based on mean shift—a non-parametric probability density-based clustering technique—to segment the 3D (three-dimensional) forest point clouds and identify individual tree crowns. The basic concept of this method is to partition the 3D space over each test plot into small vertical units (irregular columns containing 3D spatial features from one or more trees) first, by using a fixed bandwidth mean shift procedure and a small square grouping technique, and then rough estimation of crown sizes for distinct trees within a unit, based on an original 2D (two-dimensional) incremental grid projection technique, is applied to provide a basis for dynamical calibration of the kernel bandwidth for an adaptive mean shift procedure performed in each partition. The adaptive mean shift-based scheme, which incorporates our proposed bandwidth calibration method, is validated on 10 test plots of a dense, multi-layered evergreen broad-leaved forest located in South China. Experimental results reveal that this approach can work effectively and when compared to the conventional point-based approaches (e.g., region growing, k-means clustering, fixed bandwidth or multi-scale mean shift), its accuracies are relatively high: it detects 86 percent of the trees (“recall”) and 92 percent of the identified trees are correct (“precision”), showing good potential for use in the area of forest inventory.

Birnbaum–Saunders power-exponential kernel density estimation and Bayes local bandwidth selection for nonnegative heavy tailed data

In this paper, we study the performance of the Birnbaum–Saunders-power-exponential (BS-PE) kernel and Bayesian local bandwidth selection in the context of kernel density estimation for nonnegative heavy tailed data. Our approach considers the BS-PE kernel estimator and treats locally the bandwidth h as a parameter with prior distribution. The posterior density of h at each point x (point where the density is estimated) is derived in closed form, and the Bayesian bandwidth selector is obtained by using popular loss functions. The performance evaluation of this new procedure is carried out by a simulation study and real data in web-traffic. The proposed method is very quick and very competitive in comparison with the existing global methods, namely biased cross-validation and unbiased cross-validation.

Tracking Algorithm with Adaptive Bandwidth of Kernel Function for UAV Autonomous Aerial Refueling

The use of machine vision to track the receiver aircraft’s receptacle is one of the key steps in automated aerial refueling. Traditional Meanshift tracking algorithm’s kernel bandwidth is fixed, when the target size changes, it will produce with the bias even with the loss of the situation. Aiming at this problem, this paper proposes a kernel bandwidth adaptive algorithm based on corner matching. First, use backward tracking method to amend the core window center. Then according to the Harris algorithm to detect the corner, use small-scale approach for the two points between frames to match and regression analysis, namely determine the update parameters of nuclear window width. Experiments show that the algorithm can track the target accurately when the size of tracking target change, and the kernel window width can be better suited to the target size.

Analysis of asynchronous longitudinal data with partially linear models

We study partially linear models for asynchronous longitudinal data to incorporate nonlinear time trend effects. Local and global estimating equations are developed for estimating the parametric and nonparametric effects. We show that with a proper choice of the kernel bandwidth parameter, one can obtain consistent and asymptotically normal parameter estimates for the linear effects. Asymptotic properties of the estimated nonlinear effects are established. Extensive simulation studies provide numerical support for the theoretical findings. Data from an HIV study are used to illustrate our methodology.

Bump hunting by topological data analysis

A topological data analysis approach is taken to the challenging problem of finding and validating the statistical significance of local modes in a data set. As with the SIgnificance of the ZERo (SiZer) approach to this problem, statistical inference is performed in a multi‐scale way, that is, across bandwidths. The key contribution is a two‐parameter approach to the persistent homology representation. For each kernel bandwidth, a sub‐level set filtration of the resulting kernel density estimate is computed. Inference based on the resulting persistence diagram indicates statistical significance of modes. It is seen through a simulated example, and by analysis of the famous Hidalgo stamps data, that the new method has more statistical power for finding bumps than SiZer. Copyright © 2017 John Wiley & Sons, Ltd.

Hazard Mapping of Landmines and ERW Using Geo-Spatial Techniques

Landmines and Explosive Remnants of War (ERW) continue to represent a significant nuisance for society in affected countries. Coping with humanitarian and development activities, mine action aims at both, reducing the impacts of the presence of landmines/ERW on the population, and ultimately returning cleared land to the communities. These are the main tasks of mine action decision makers. This study combines landmine/ERW contamination data with explanatory variables that contain information about underlying targets. They are integrated into a risk mapping framework using Geographic Information Systems with other information sources, such as remote sensing. The aim of this paper is to provide insights into the populations and/or locations at risk caused by landmine and ERW impacts on a broad and local scale. Thus, the concept of ‘hotspots’ is particularly useful because it provides a visual representation of exposure, aided by a geo-spatial representation of ‘priority areas for mine action planners to focus on. We apply the Kernel Density Estimator (KDE) to derive such ‘hotspots’. KDE is proposed as the basis to define landmine and ERW hazard, vulnerability, and element-at-risk maps, which enable producing a final output, the landmine/ERW risk map. This is accomplished by using an adaptive kernel bandwidth for datasets with highly heterogeneous spatial distributions, and a problem-specific method for generating point samples from polygon data, before using them as inputs for KDE. The geo-statistical model presented here is a time-and-cost-efficient method to construct a landmine risk map, that is as representative as those produced by mine action actors. It can be used as a complement to the risk area maps made by these actors because they are slightly different but show a large degree of overlap. Moreover, the method helps revealing the variables which are the most linked to landmine/ERW-related events in the study area.

Kernel-Based Sensor Fusion With Application to Audio-Visual Voice Activity Detection

In this paper, we address the problem of multiple view data fusion in the presence of noise and interferences. Recent studies have approached this problem using kernel methods, by relying particularly on a product of kernels constructed separately for each view. From a graph theory point of view, we analyze this fusion approach in a discrete setting. More specifically, based on a statistical model for the connectivity between data points, we propose an algorithm for the selection of the kernel bandwidth, a parameter that, as we show, has important implications on the robustness of this fusion approach to interferences. Then, we consider the fusion of audio-visual speech signals measured by a single microphone and by a video camera pointed to the face of the speaker. Specifically, we address the task of voice activity detection, i.e., the detection of speech and nonspeech segments, in the presence of structured interferences such as keyboard taps and office noise. We propose an algorithm for voice activity detection based on the audio-visual signal. Simulation results show that the proposed algorithm outperforms competing fusion and voice activity detection approaches. In addition, we demonstrate that a proper selection of the kernel bandwidth indeed leads to improved performance.

Multi-objective optimization of ethanol fuelled HCCI engine performance using hybrid GRNN–PSO

In this research study, the ethanol-fuelled homogeneous charge compression ignition (HCCI) engine operating conditions were optimized based on its performance and emission parameters. However, there is no proper relation among these performance and emission parameters. Therefore, it is a challenging task to optimize the HCCI engine. For this purpose, a hybrid generalised regression neural network (GRNN)–particle swarm optimisation (PSO) model was designed to optimize three input parameters, including the charge temperature, engine load, and EGR rate. The GRNN based prediction tool has been introduced to obtain the relation between input and output parameters. Initially, PSO generated a random particle and given as input to GRNN. The GRNN carries a nonlinear regression analysis among all the input and output parameters of the ethanol fuelled HCCI engine. The optimum kernel bandwidth in the GRNN model was determined by the grid search method for reducing the cross-validation error. The output of the GRNN regarding performance and emission parameters was given to the PSO algorithm to optimize the HCCI engine operating conditions. The developed algorithm has the flexibility to optimize the engine performance for any user-defined weights. The optimum HCCI engine operating conditions for the general criteria were found to be 170°C charge temperature, 72% engine load, and 4% EGR. The proposed model obtained the optimized HCCI engine input parameters within the short span of time i.e., 60–75ms. The new hybrid GRNN–PSO algorithm is anticipated to accomplish as a useful tool for a rapid engine optimization.

Predicting Melbourne ambulance demand using kernel warping

Predicting ambulance demand accurately in fine resolutions in space and time is critical for ambulance fleet management and dynamic deployment. Typical challenges include data sparsity at high resolutions and the need to respect complex urban spatial domains. To provide spatial density predictions for ambulance demand in Melbourne, Australia, as it varies over hourly intervals, we propose a predictive spatio-temporal kernel warping method. To predict for each hour, we build a kernel density estimator on a sparse set of the most similar data from relevant past time periods (labeled data), but warp these kernels to a larger set of past data irregardless of time periods (point cloud). The point cloud represents the spatial structure and geographical characteristics of Melbourne, including complex boundaries, road networks and neighborhoods. Borrowing from manifold learning, kernel warping is performed through a graph Laplacian of the point cloud and can be interpreted as a regularization toward, and a prior imposed for, spatial features. Kernel bandwidth and degree of warping are efficiently estimated via cross-validation, and can be made time- and/or location-specific. Our proposed model gives significantly more accurate predictions compared to a current industry practice, an unwarped kernel density estimation and a time-varying Gaussian mixture model.

A simple method to estimate the time-dependent receiver operating characteristic curve and the area under the curve with right censored data.

The time-dependent receiver operating characteristic curve is often used to study the diagnostic accuracy of a single continuous biomarker, measured at baseline, on the onset of a disease condition when the disease onset may occur at different times during the follow-up and hence may be right censored. Due to right censoring, the true disease onset status prior to the pre-specified time horizon may be unknown for some patients, which causes difficulty in calculating the time-dependent sensitivity and specificity. We propose to estimate the time-dependent sensitivity and specificity by weighting the censored data by the conditional probability of disease onset prior to the time horizon given the biomarker, the observed time to event, and the censoring indicator, with the weights calculated nonparametrically through a kernel regression on time to event. With this nonparametric weighting adjustment, we derive a novel, closed-form formula to calculate the area under the time-dependent receiver operating characteristic curve. We demonstrate through numerical study and theoretical arguments that the proposed method is insensitive to misspecification of the kernel bandwidth, produces unbiased and efficient estimators of time-dependent sensitivity and specificity, the area under the curve, and other estimands from the receiver operating characteristic curve, and outperforms several other published methods currently implemented in R packages.

Minimum local distance density estimation

ABSTRACTWe present a local density estimator based on first-order statistics. To estimate the density at a point, x, the original sample is divided into subsets and the average minimum sample distance to x over all such subsets is used to define the density estimate at x. The tuning parameter is thus the number of subsets instead of the typical bandwidth of kernel or histogram-based density estimators. The proposed method is similar to nearest-neighbor density estimators but it provides smoother estimates. We derive the asymptotic distribution of this minimum sample distance statistic to study globally optimal values for the number and size of the subsets. Simulations are used to illustrate and compare the convergence properties of the estimator. The results show that the method provides good estimates of a wide variety of densities without changes of the tuning parameter, and that it offers competitive convergence performance.