Gaussian Kernel Density Estimation Research Articles

Ridge-based method for finding curvilinear structures from noisy data

Extraction of curvilinear structures from noisy data is an essential task in many application fields such as data analysis, pattern recognition and machine vision. The proposed approach assumes a random process in which the samples are obtained from a generative model. The model specifies a set of generating functions describing curvilinear structures as well as sampling noise and background clutter. It is shown that ridge curves of the marginal density induced by the model can be used to estimate the generating functions. Given a Gaussian kernel density estimate for the marginal density, ridge curves of the density estimate are parametrized as the solution to a differential equation. Finally, a predictor–corrector algorithm for tracing the ridge curve set of such a density estimate is developed. Efficiency and robustness of the algorithm are demonstrated by numerical experiments on synthetic datasets as well as observational datasets from seismology and cosmology.

A statistical approach for sub-hourly solar radiation reconstruction

This paper proposes a computational-statistics based approach for solar radiation reconstruction at sub-hourly intervals. A dimensionless form of stochastic variable, V, which is defined as the difference between the theoretical global solar radiation in clear-sky conditions and the actual solar radiation, normalized by the clear-sky global solar radiation, is introduced and adopted in this work. The probability density function of V is calculated from historical data using a Gaussian kernel density estimator. With the developed model, the only input information required for the reconstruction procedure is the cloud condition of the sky (i.e., fair, partly cloudy, overcast, and rain/snow etc.). A case study in simulating solar radiation in Singapore is conducted to validate the accuracy of the model. The calculated results agree well with the measured data. The normalized root mean square error (NRMSE) is on average 23.4% and 7.2% for the one-minute temporal resolution and hourly integral values, respectively.

랜덤오더 심볼열과 상호 코렌트로피를 이용한 블라인드 알고리듬의 현실적 접근

The cross-correntropy concept can be expressed with inner products of two different probability density functions constructed by Gaussian-kernel density estimation methods. Blind algorithms based on the maximization of the cross-correntropy (MCC) and a symbol set of randomly generated N samples yield superior learning performance, but have a huge computational complexity in the update process at the aim of weight adjustment based on the MCC. In this paper, a method of reducing the computational complexity of the MCC algorithm that calculates recursively the gradient of the cross-correntropy is proposed. The proposed method has only O(N) operations per iteration while the conventional MCC algorithms that calculate its gradients by a block processing method has O(N 2 ). In the simulation results, the proposed method shows the same learning performance while

Adaptive kernels and transfer entropy for neural connectivity analysis in EEG signals

This paper aims at better understanding causal relationships between parts of the brain during epileptic seizures. Our objective is to detect effective connectivity, i.e. to discover whether neural activity in a given substructure influences activity in another part of the brain. Recent efforts have been devoted to develop nonlinear and nonparametric approaches, such as transfer entropy (TE), to overcome linear methods limitations. However, building efficient TE estimators still asks open questions. In this study, we propose a new strategy to improve TE estimation by introducing different nonparametric adaptive kernel density estimators. Among all techniques under study, the Gaussian adaptive kernel density estimator based approach presents the best behavior whatever the tested model (autoregressive or physiological model).

Curve Fitting of the Corporate Recovery Rates: The Comparison of Beta Distribution Estimation and Kernel Density Estimation

Recovery rate is essential to the estimation of the portfolio’s loss and economic capital. Neglecting the randomness of the distribution of recovery rate may underestimate the risk. The study introduces two kinds of models of distribution, Beta distribution estimation and kernel density distribution estimation, to simulate the distribution of recovery rates of corporate loans and bonds. As is known, models based on Beta distribution are common in daily usage, such as CreditMetrics by J.P. Morgan, Portfolio Manager by KMV and Losscalc by Moody’s. However, it has a fatal defect that it can’t fit the bimodal or multimodal distributions such as recovery rates of corporate loans and bonds as Moody’s new data show. In order to overcome this flaw, the kernel density estimation is introduced and we compare the simulation results by histogram, Beta distribution estimation and kernel density estimation to reach the conclusion that the Gaussian kernel density distribution really better imitates the distribution of the bimodal or multimodal data samples of corporate loans and bonds. Finally, a Chi-square test of the Gaussian kernel density estimation proves that it can fit the curve of recovery rates of loans and bonds. So using the kernel density distribution to precisely delineate the bimodal recovery rates of bonds is optimal in credit risk management.

A Predictive Maintenance System for Epitaxy Processes Based on Filtering and Prediction Techniques

Silicon epitaxial deposition is a process strongly influenced by wafer temperature behavior, which has to be constantly monitored to avoid the production of defective wafers. However, temperature measurements are not reliable, and the sensors have to be appropriately calibrated with some dedicated procedure. A predictive maintenance (PdM) system is proposed with the aim of predicting process behavior and scheduling control actions on the sensors in advance. Two different prediction techniques have been employed and compared: the Kalman predictor and the particle filter with Gaussian kernel density estimator. The accuracy of the PdM module has been tested on real industrial production datasets.

AUTOMATIC REGISTRATION OF MULTI-SOURCE DATA USING MUTUAL INFORMATION

Abstract. Automatic image registration is a basic step in multi-sensor data integration in remote sensing and photogrammetric applications such as data fusion. The effectiveness of Mutual Information (MI) as a technique for automated multi-sensor image registration has previously been demonstrated for medical and remote sensing applications. In this paper, a new General Weighted MI (GWMI) approach that improves the robustness of MI to local maxima, particularly in the case of registering optical imagery and 3D point clouds, is presented. Two different methods including a Gaussian Mixture Model (GMM) and Kernel Density Estimation have been used to define the weight function of joint probability, regardless of the modality of the data being registered. The Expectation Maximizing method is then used to estimate parameters of GMM, and in order to reduce the cost of computation, a multi-resolution strategy has been used. The performance of the proposed GWMI method for the registration of aerial orthotoimagery and LiDAR range and intensity information has been experimentally evaluated and the results obtained are presented.

Extreme quantile estimation with nonparametric adaptive importance sampling

In this article, we propose a nonparametric adaptive importance sampling (NAIS) algorithm to estimate rare event quantile. Indeed, Importance Sampling (IS) is a well-known adapted random simulation technique. It consists in generating random weighted samples from an auxiliary distribution rather than the distribution of interest. The optimization of this auxiliary distribution is often very difficult in practice. First, we review how to define the optimal auxiliary density of IS for quantile estimation in the general case and then propose a nonparametric method based on Gaussian kernel density estimator to approach the optimal auxiliary density that does not assume an initial PDF guess. This method is then finally applied to theoretical cases to demonstrate the efficiency of the proposed NAIS algorithm and on the quantile estimation of the temperature of a forest fire detection simulator.

From Gaussian kernel density estimation to kernel methods

This paper explores how a kind of probabilistic systems, namely, Gaussian kernel density estimation (GKDE), can be used to interpret several classical kernel methods, including the well-known support vector machine (SVM), support vector regression (SVR), one-class kernel classifier, i.e., support vector data description (SVDD) or equivalently minimal enclosing ball (MEB), and the fuzzy systems (FS). For the SVM, we reveal that the classical SVM with Gaussian density kernel attempts to find a noisy GKDE based Bayesian classifier with equal prior probabilities for each class. For the SVR, the classification based e-SVR attempts to obtain two noisy GKDEs for each class in the constructed binary classification dataset, and the decision boundary just corresponds to the mapping function of the original regression problem. For the MEB or SVDD, we reveal the equivalence between it and the integrated-squared-errors (ISE) criterion based GKDE and by using this equivalence a MEB based classifier with privacy-preserving function is proposed for one kind of classification tasks where the datasets contain privacy-preserving clouds. For the FS, we show that the GKDE for a regression dataset is equivalent to the construction of a zero-order Takagi–Sugeno–Kang (TSK) fuzzy system based on the same dataset. Our extensive experiments confirm the obtained conclusions and demonstrated the effectiveness of the proposed new machine learning and modeling methods.

What Do Kernel Density Estimators Optimize?

Some linkages between kernel and penalty methods of density estimation are explored. It is recalled that classical Gaussian kernel density estimation can be viewed as the solution of the heat equation with initial condition given by data. We then observe that there is a direct relationship between the kernel method and a particular penalty method of density estimation. For this penalty method, solutions can be characterized as a weighted average of Gaussian kernel density estimates, the average taken with respect to the bandwidth parameter. A Laplace transform argument shows that this weighted average of Gaussian kernel estimates is equivalent to a fixed bandwidth kernel estimate using a Laplace kernel. Extensions to higher order kernels are considered and some connections to penalized likelihood density estimators are made in the concluding sections.

A continuation approach to mode-finding of multivariate Gaussian mixtures and kernel density estimates

Gaussian mixtures (i.e. linear combinations of multivariate Gaussian probability densities) appear in numerous applications due to their universal ability to approximate multimodal probability distributions. Finding the modes (maxima) of a Gaussian mixture is a fundamental problem arising in many practical applications such as machine learning and digital image processing. In this paper, we propose a computationally efficient method for finding a significant mode of the Gaussian mixture. Such a mode represents an area of large probability, and it often coincides with the global mode of the mixture. The proposed method uses a Gaussian convolution in order to remove undesired local maxima of the Gaussian mixture and preserve its underlying structure. The transformation between the maximizers of the smoothed Gaussian mixture and the original one is formulated as a differential equation. A robust trust region method for tracing the solution curve of this differential equation is described. Our formulation also allows mixtures with negative weights or even negative values, which occur in some applications related to machine learning or quantum mechanics. The applicability of the method to mode-finding of Gaussian kernel density estimates obtained from experimental data is illustrated. Finally, some numerical results are given to demonstrate the ability of the method to find significant modes of Gaussian mixtures and kernel density estimates.

A Gaussian‐mixture ensemble transform filter

AbstractWe generalize the popular ensemble Kalman filter to an ensemble transform filter, in which the prior distribution can take the form of a Gaussian mixture or a Gaussian kernel density estimator. The design of the filter is based on a continuous formulation of the Bayesian filter analysis step. We call the new filter algorithm the ensemble Gaussian‐mixture filter (EGMF). The EGMF is implemented for three simple test problems (Brownian dynamics in one dimension, Langevin dynamics in two dimensions and the three‐dimensional Lorenz‐63 model). It is demonstrated that the EGMF is capable of tracking systems with non‐Gaussian uni‐ and multimodal ensemble distributions. Copyright © 2011 Royal Meteorological Society

Combining P values to improve classification of differential gene expression in the HTself software

HTself is a web-based bioinformatics tool designed to deal with the classification of differential gene expression in low replication microarray studies. It is based on a statistical test that uses self-self experiments to derive intensity-dependent cutoffs. We developed an extension of HTself, originally released in 2005, by calculating P values instead of using a fixed acceptance level α. As before, the statistic used to compute single-spot P values is obtained from the Gaussian kernel density estimator method applied to self-self data. Different spots corresponding to the same biological gene (replicas) give rise to a set of independent P values that can be combined by well-known statistical methods. The combined P value can be used to decide whether a gene can be considered differentially expressed or not. HTself2 is a new version of HTself that uses P values combination. It is implemented as a user-friendly desktop application to help laboratories without a bioinformatics infrastructure.

HTself2: Combining p-values to Improve Classification of Differential Gene Expression in HTself

AbstractHTself is a web-based bioinformatics tool designed to deal with the classification of differential gene expression for low replication microarray studies. It is based on a statistical test that uses self-self experiments to derive intensity-dependent cutoffs. The method was previously described in Vêncio et al, (DNA Res. 12: 211- e 214, 2005). In this work we consider an extension of HTself by calculating p-values instead of using a fixed credibility level α. As before, the statistic used to compute single spots p-values is obtained from the gaussian Kernel Density Estimator method applied to self-self data. Different spots corresponding to the same biological gene (replicas) give rise to a set of independent p-values which can be combined by well known statistical methods. The combined p-value can be used to decide whether a gene can be considered differentially expressed or not. HTself2 is a new version of HTself that uses the idea of p-values combination. It was implemented as a user-friendly desktop application to help laboratories without a bioinformatics infrastructure.

Topology modeling for Adaboost-cascade based object detection

Several important issues involved in Adaboost-cascade learning still remain open problems. In this work, several novel ideas are proposed for improved Adaboost-cascade object detection. The most important one is the novel topology oriented Adaboost (TOBoost) algorithm. TOBoost immediately minimizes the classification error of each selected feature, and thus enables the final detector to be more discriminative and to converge more quickly. Moreover, a simple cascading scheme is presented for tuning the cascade parameters of TOBoost; and Gaussian kernel density estimation is introduced to enhance the generalization ability of TOBoost. Another important contribution is the topology modeling of Haar-like (HL) features, which reveals an interesting property of negative HL features and significantly avoids unnecessary training computations. Non-adjacent Haar-like features are consequently configured for more effective object representation. The above enhancements result in a more efficient and stable detector with fewer features. Extensive experiments in the application of iris detection are conducted and encouraging performance is achieved.

Properties of series feature aggregation schemes

Feature aggregation is a critical technique in content-based image retrieval (CBIR) that combines multiple feature distances to obtain image dissimilarity. Conventional parallel feature aggregation (PFA) schemes failed to effectively filter out the irrelevant images using individual visual features before ranking images in collection. Series feature aggregation (SFA) is a new scheme that aims to address this problem. This paper investigates three important properties of SFA that are significant for design of systems. They reveal the irrelevance of feature order and the convertibility of SFA and PFA as well as the superior performance of SFA. Furthermore, based on Gaussian kernel density estimator, the authors propose a new method to estimate the visual threshold, which is the key parameter of SFA. Experiments, conducted with IAPR TC-12 benchmark image collection (ImageCLEF2006) that contains over 20,000 photographic images and defined queries, have shown that SFA can outperform conventional PFA schemes.

Nonparametric density estimation for positive time series

The Gaussian kernel density estimator is known to have substantial problems for bounded random variables with high density at the boundaries. For independent and identically distributed data, several solutions have been put forward to solve this boundary problem. In this paper, we propose the gamma kernel estimator as a density estimator for positive time series data from a stationary α -mixing process. We derive the mean (integrated) squared error and asymptotic normality. In a Monte Carlo simulation, we generate data from an autoregressive conditional duration model and a stochastic volatility model. We study the local and global behavior of the estimator and we find that the gamma kernel estimator outperforms the local linear density estimator and the Gaussian kernel estimator based on log-transformed data. We also illustrate the good performance of the h -block cross-validation method as a bandwidth selection procedure. An application to data from financial transaction durations and realized volatility is provided.

DSP-based bottleneck bandwidth estimation

This paper presents a new passive technique for estimating the bottleneck bandwidth based on transferring the Gaussian kernel density estimation of the packets inter-arrival times to the frequency domain. The resulting spectrum contains information about the transmission time of the bottleneck link and can reveal information about multiple bottlenecks if they exist along the end-to-end path. The advantage of the technique is that it provides a model that can be manipulated by the digital signal processing methods and, unlike prior work in the area, it relies less on statistical methods. The proposed technique was validated using the ns2 network simulator [1] on several topologies and traffic sources. Further experiments were conducted to test the strength of the patterns between flows that share a bottleneck by applying K-means algorithm to cluster the average packet inter-arrival times of these flows. The paper also presents a set of results from real traffic experiments conducted in order to infer both the bottleneck bandwidth and the capacity of the path using a passive approach.

Nonparametric Density Estimation for Positive Time Series

The Gaussian kernel density estimator is known to have substantial problems for bounded random variables with high density at the boundaries. For i.i.d. data several solutions have been put forward to solve this boundary problem. In this paper we propose the gamma kernel estimator as density estimator for positive data from a stationary alpha-mixing process. We derive the mean integrated squarred error, almost sure convergence and asymptotic normality. In a Monte Carlo study, where we generate data from an autoregressive conditional duration model and a stochastic volatility model, we find that the gamma kernel outperforms the local linear density estimator. An application to data from financial transaction durations, realized volatility and electricity price data is provided.

Using satellite radiotelemetry data to delineate and manage wildlife populations

The greatest promise of radiotelemetry always has been a better understanding of animal movements. Telemetry has helped us know when animals are active, how active they are, how far and how fast they move, the geographic areas they occupy, and whether individuals vary in these traits. Unfortunately, the inability to estimate the error in animals' utilization distributions (UDs), has prevented probabilistic linkage of movements data, which are always retrospective, with future management actions. We used the example of the harvested population of polar bears (Ursus maritimus) in the Southern Beaufort Sea to illustrate a method that provides that linkage. We employed a 2-dimensional Gaussian kernel density estimator to smooth and scale frequencies of polar bear radio locations within cells of a grid overlying our study area. True 2-dimensional smoothing allowed us to create accurate descriptions of the UDs of individuals and groups of bears. We used a new method of clustering, based upon the relative use collared bears made of each cell in our grid, to assign individual animals to populations. We applied the fast Fourier transform to make bootstrapped estimates of the error in UDs computationally feasible. Clustering and kernel smoothing identified 3 populations of polar bears in the region between Wrangel Island, Russia, and Banks Island, Canada. The relative probability of occurrence of animals from each population varied significantly among grid cells distributed across the study area. We displayed occurrence probabilities as contour maps wherein each contour line corresponded with a change in relative probability. Only at the edges of our study area and in some offshore regions were bootstrapped estimates of error in occurrence probabilities too high to allow prediction. Error estimates, which also were displayed as contours, allowed us to show that occurrence probabilities did not vary by season. Near Barrow, Alaska, 50% of bears observed are predicted to be from the Chukchi Sea population and 50% from the Southern Beaufort Sea population. At Tuktoyaktuk, Northwest Territories, Canada, 50% are from the Southern Beaufort Sea and 50% from the Northern Beaufort Sea population. The methods described here will aid managers of all wildlife that can be studied by telemetry to allocate harvests and other human perturbations to the appropriate populations, make risk assessments, and predict impacts of human activities. They will aid researchers by providing the refined descriptions of study populations that are necessary for population estimation and other investigative tasks.