Sparse Kernel Density Estimator Research Articles

Permissible Area Analyses of Measurement Errors with Required Fault Diagnosability Performance.

Fault diagnosability is the basis of fault diagnosis. Fault diagnosability evaluation refers to whether there is enough measurable information in the system to support the rapid and reliable detection of a fault. However, due to unavoidable measurement errors in a system, a quantitative evaluation index of system fault diagnosability is inadequate. In order to overcome the adverse effects of measurement errors, improve the accuracy of the quantitative evaluation of fault diagnosability, and improve the safety level of the system, a method for a permissible area analysis of measurement errors for a quantitative evaluation of fault diagnosability is proposed in this paper. Firstly, in order for the residuals obey normal distribution, a design method of the permissible area of measurement errors based on the Kullback–Leibler divergence (KLD) is given. Secondly, two key problems in calculating the KLD are solved by sparse kernel density estimation and the Monte Carlo method. Finally, the feasibility and validity of the method are analyzed through a case study.

Sparse density estimator with tunable kernels

A new sparse kernel density estimator with tunable kernels is introduced within a forward constrained regression framework whereby the nonnegative and summing-to-unity constraints of the mixing weights can easily be satisfied. Based on the minimum integrated square error criterion, a recursive algorithm is developed to select significant kernels one at time, and the kernel width of the selected kernel is then tuned using the gradient descent algorithm. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing very sparse kernel density estimators with competitive accuracy to existing kernel density estimators.

A novel sparse boosting method for crater detection in the high resolution planetary image

Impact craters distributed on planetary surface become one of the main barriers during the soft landing of planetary probes. In order to accelerate the crater detection, in this paper, we present a new sparse boosting (SparseBoost) method for automatic detection of sub-kilometer craters. The SparseBoost method integrates an improved sparse kernel density estimator (RSDE-WL1) into the Boost algorithm and the RSDE-WL1 estimator is achieved by introducing weighted l1 penalty term into the reduced set density estimator. An iterative algorithm is proposed to implement the RSDE-WL1. The SparseBoost algorithm has the advantage of fewer selected features and simpler representation of the weak classifiers compared with the Boost algorithm. Our SparseBoost based crater detection method is evaluated on a large and high resolution image of Martian surface. Experimental results demonstrate that the proposed method can achieve less computational complexity in comparison with other crater detection methods in terms of selected features.

Sparse Density Estimation on the Multinomial Manifold.

A new sparse kernel density estimator is introduced based on the minimum integrated square error criterion for the finite mixture model. Since the constraint on the mixing coefficients of the finite mixture model is on the multinomial manifold, we use the well-known Riemannian trust-region (RTR) algorithm for solving this problem. The first- and second-order Riemannian geometry of the multinomial manifold are derived and utilized in the RTR algorithm. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with an accuracy competitive with those of existing kernel density estimators.

Robust sparse kernel density estimation by inducing randomness

In this paper, a robust sparse kernel density estimation based on the reduced set density estimator is proposed. The key idea is to induce randomness to the plug-in estimation of weighting coefficients. The random fluctuations can inhibit these small nonzero weighting coefficients to cluster in regions of space with greater probability mass. By sequential minimal optimization, these coefficients are merged into a few larger weighting coefficients. Experimental studies show that the proposed model is superior to several related methods both in sparsity and accuracy of the estimation. Moreover, the proposed density estimation is extensively validated on novelty detection and binary classification.

Sparse probability density function estimation using the minimum integrated square error

We develop a new sparse kernel density estimator using a forward constrained regression framework, within which the nonnegative and summing-to-unity constraints of the mixing weights can easily be satisfied. Our main contribution is to derive a recursive algorithm to select significant kernels one at time based on the minimum integrated square error (MISE) criterion for both the selection of kernels and the estimation of mixing weights. The proposed approach is simple to implement and the associated computational cost is very low. Specifically, the complexity of our algorithm is in the order of the number of training data N, which is much lower than the order of N2 offered by the best existing sparse kernel density estimators. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with comparable accuracy to those of the classical Parzen window estimate and other existing sparse kernel density estimators.

Sparse kernel density estimations and its application in variable selection based on quadratic Renyi entropy

A novel sparse kernel density estimation method is proposed based on the sparse Bayesian learning with random iterative dictionary preprocessing. Using empirical cumulative distribution function as the response vectors, the sparse weights of density estimation are estimated by sparse Bayesian learning. The proposed iterative dictionary learning algorithm is used to reduce the number of kernel computations, which is an essential step of the sparse Bayesian learning. With the sparse kernel density estimation, the quadratic Renyi entropy based normalized mutual information feature selection method is proposed. The simulation of three examples demonstrates that the proposed method is comparable to the typical Parzen kernel density estimations. And compared with other state-of-art sparse kernel density estimations, our method also has a shown very good performance as to the number of kernels required in density estimation. For the last example, the Friedman data and Housing data are used to show the property of the proposed feature variables selection method.

Efficient Particle Filtering via Sparse Kernel Density Estimation

Particle filters (PFs) are Bayesian filters capable of modeling nonlinear, non-Gaussian, and nonstationary dynamical systems. Recent research in PFs has investigated ways to appropriately sample from the posterior distribution, maintain multiple hypotheses, and alleviate computational costs while preserving tracking accuracy. To address these issues, a novel utilization of the support vector data description (SVDD) density estimation method within the particle filtering framework is presented. The SVDD density estimate can be integrated into a wide range of PFs to realize several benefits. It yields a sparse representation of the posterior density that reduces the computational complexity of the PF. The proposed approach also provides an analytical expression for the posterior distribution that can be used to identify its modes for maintaining multiple hypotheses and computing the MAP estimate, and to directly sample from the posterior. We present several experiments that demonstrate the advantages of incorporating a sparse kernel density estimate in a particle filter.

Regression based D-optimality experimental design for sparse kernel density estimation

This paper derives an efficient algorithm for constructing sparse kernel density (SKD) estimates. The algorithm first selects a very small subset of significant kernels using an orthogonal forward regression (OFR) procedure based on the D-optimality experimental design criterion. The weights of the resulting sparse kernel model are then calculated using a modified multiplicative nonnegative quadratic programming algorithm. Unlike most of the SKD estimators, the proposed D-optimality regression approach is an unsupervised construction algorithm and it does not require an empirical desired response for the kernel selection task. The strength of the D-optimality OFR is owing to the fact that the algorithm automatically selects a small subset of the most significant kernels related to the largest eigenvalues of the kernel design matrix, which counts for the most energy of the kernel training data, and this also guarantees the most accurate kernel weight estimate. The proposed method is also computationally attractive, in comparison with many existing SKD construction algorithms. Extensive numerical investigation demonstrates the ability of this regression-based approach to efficiently construct a very sparse kernel density estimate with excellent test accuracy, and our results show that the proposed method compares favourably with other existing sparse methods, in terms of test accuracy, model sparsity and complexity, for constructing kernel density estimates.

A Forward-Constrained Regression Algorithm for Sparse Kernel Density Estimation

Using the classical Parzen window (PW) estimate as the target function, the sparse kernel density estimator is constructed in a forward-constrained regression (FCR) manner. The proposed algorithm selects significant kernels one at a time, while the leave-one-out (LOO) test score is minimized subject to a simple positivity constraint in each forward stage. The model parameter estimation in each forward stage is simply the solution of jackknife parameter estimator for a single parameter, subject to the same positivity constraint check. For each selected kernels, the associated kernel width is updated via the Gauss-Newton method with the model parameter estimate fixed. The proposed approach is simple to implement and the associated computational cost is very low. Numerical examples are employed to demonstrate the efficacy of the proposed approach.