Parzen Window Research Articles

Optimal convergence rates of high order Parzen windows with unbounded sampling

Abstract High order Parzen windows are considered with data drawn from unbounded sampling processes. Convergence analysis is established by imposing a moment hypothesis on the unbounded sampling outputs and some decay conditions on the marginal distributions. Our estimate of convergence rate is consistent with the previous result with bounded sampling.

Fuzzy Boolean Nets – a nature inspired model for learning and reasoning

This paper presents Fuzzy Boolean Nets (FBN), a nature inspired Boolean neural model, in the sense that it exhibits topologic similarities with natural systems, as well as learning dynamic activity at low level and high immunity to individual errors or deletions of neurons or synaptic connections. It is shown that one can interpret reasoning in FBN as a particular case of fuzzy qualitative reasoning. It is proven that the model is a Universal Approximator since it can be interpreted as a Parzen Window probability density estimator. Also, conditions for learning a discrete set of rules without cross influence are investigated, being proven that this goal can be achieved with an appropriate net parameters relationship. Moreover, adequate parameter choices for learning continuous functions with interpolation capabilities are deduced.

Resource-aware distributed particle filtering for cluster-based object tracking in wireless camera networks

This paper presents a novel resource-aware framework for the implementation of distributed particle filters in resource-constrained wireless camera networks WCNs. WCNs often suffer from communication failures caused by physical limitations of the communication channel as well as network congestion. Unreliable communication degrades the visual information shared by the cameras, such as visual feature data, and consequently leads to inaccurate vision processing at individual camera nodes. This paper focuses on the effects of communication failures on object tracking performance and presents a novel communication resource-aware tracking methodology, which adjusts the amount of data packets transmitted by the cameras according to the network conditions. We demonstrate the performance of the proposed framework using three different mechanisms to share the particle information among nodes: synchronised particles, Gaussian mixture models, and Parzen windows. The experimental results show that the proposed resource-aware method makes the distributed particle filters more tolerant to packet losses and also more energy efficient.

SVM Active Learning Approach for Image Classification Using Spatial Information

In the last few years, active learning has been gaining growing interest in the remote sensing community in optimizing the process of training sample collection for supervised image classification. Current strategies formulate the active learning problem in the spectral domain only. However, remote sensing images are intrinsically defined both in the spectral and spatial domains. In this paper, we explore this fact by proposing a new active learning approach for support vector machine classification. In particular, we suggest combining spectral and spatial information directly in the iterative process of sample selection. For this purpose, three criteria are proposed to favor the selection of samples distant from the samples already composing the current training set. In the first strategy, the Euclidean distances in the spatial domain from the training samples are explicitly computed, whereas the second one is based on the Parzen window method in the spatial domain. Finally, the last criterion involves the concept of spatial entropy. Experiments on two very high resolution images show the effectiveness of regularization in spatial domain for active learning purposes.

Speaker recognition using vocal source model

Speaker recognition is carried out in the space of the functional parameters of the area of the glottal cross-section, found by solving the inverse problem. This problem is solved in two stages: first, the signal obtained by inverse filtering is approximated using the vocal source model, and then the glottal area model parameters, which generate the calculated vocal source impulse, are computed. Speaker recognition is carried out on a database of Russian numerals from 0 to 9 separately for men (48 speakers) and women (37 speakers) at the segments of stressed vowels. Various methods of recognition are studied: the Gaussian mixture model (GMM), support vector machines (SVMs), discriminant analysis, naive Bayes classifier (NB), the method of classification trees (CTREE), and the Parzen window classifier. The best results were obtained using the method of SVMs and the Parzen method: the average total error of identification of men was 4.9% and 5.1%, and that of women--8.2% and 8.8%, respectively.

Comparative Analysis on Kernel Based Probability Density Estimation

In this paper, we compare the estimation performances of 7 different kernels (i.e., Uniform, Triangular, Epanechnikov, Biweight, Triweight, Cosine and Gaussian) when using them to conduct the probability density estimation with Parzen window method. We firstly analyze the efficiencies of these 7 kernels and then compare their estimation errors measured by mean squared error (MSE). The theoretical analysis and the experimental comparisons show that the mostly-used Gaussian kernel is not the best choice for the probability density estimation, of which the efficiency is low and estimation error is high. The derived conclusions give some guidelines for the selection of kernel in the practical application of probability density estimation.

Feature and instance reduction for PNN classifiers based on fuzzy rough sets

Instance reduction for K-nearest-neighbor classification rules (KNN) has attracted much attention these years, and most of the existing approaches lose the semantics of probability of original data. In this work, we propose a new reduced KNN rule, called FAIR-KNN, to perform feature and instance reduction based on fuzzy rough set theory. First, we use fuzzy rough sets to evaluate candidate features and select the most informative ones. The algorithm of feature selection returns the selected features and the membership values of samples to the lower approximations of their classes. These values reflect the distances of the samples to classification boundary and are used to compute probabilities of samples to be subsampled. Then we introduce a weighted Parzen window technique to estimate the probability from the weighted subsampled data. Thus we can not only reduce features and samples in original data, but also do not lose the semantics of probability. Finally, the memberships of samples to lower and upper approximations of decisions are interpreted as certainty and possibility degrees of samples belonging to the corresponding decisions, respectively. So the weighted averages with probability of the memberships of samples to lower and upper approximations are outputted as the certainty and possibility degrees of unseen samples belonging to some decisions, which enrich the semantics of KNN. Numerical experiments on artificial and real-world data validate the effectiveness of the proposed technique.

A New Nearest Neighbor Classification Algorithm Based on Local Probability Centers

The nearest neighbor is one of the most popular classifiers, and it has been successfully used in pattern recognition and machine learning. One drawback ofkNN is that it performs poorly when class distributions are overlapping. Recently, local probability center (LPC) algorithm is proposed to solve this problem; its main idea is giving weight to samples according to their posterior probability. However, LPC performs poorly when the value ofkis very small and the higher-dimensional datasets are used. To deal with this problem, this paper suggests that the gradient of the posterior probability function can be estimated under sufficient assumption. The theoretic property is beneficial to faithfully calculate the inner product of two vectors. To increase the performance in high-dimensional datasets, the multidimensional Parzen window and Euler-Richardson method are utilized, and a new classifier based on local probability centers is developed in this paper. Experimental results show that the proposed method yields stable performance with a wide range ofkfor usage, robust performance to overlapping issue, and good performance to dimensionality. The proposed theorem can be applied to mathematical problems and other applications. Furthermore, the proposed method is an attractive classifier because of its simplicity.

The effect of chronically occluded coronary artery recanalisation on baroreflex sensitivity and left ventricular systolic function

Recanalisation of chronic total occlusion (CTO) of a coronary artery can be reflected by improvements in various clinical parameters. Revealing increased parasympathetic activity would constitute an additional argument for performing this procedure. To assess the effect of CTO recanalisation on baroreflex sensitivity (BRS) and left ventricular ejection fraction (LVEF) in stable symptomatic patients with coronary artery disease. BRS (spectral analysis, transfer function, Blackman-Tukey algorithm, 0.03 Hz bandwidth Parzen window) and LVEF (echocardiography, Simpson's method) were analysed in 23 patients: one day (R1) before, one day (R2) after, and three months (R3) after CTO recanalisation. Patients were divided into two groups: those with depressed (≤ 3 ms/mm Hg) or preserved(> 3 ms/mm Hg) BRS. Significant BRS changes were observed in the study group compared to baseline values (p = 0.016). In the patients with a depressed reflex, BRS in R2 was similar to R1 and almost doubled in R3 (p = 0.018). In the patients with a preserved reflex, BRS significantly decreased in R2 (p = 0.024) and returned to the baseline value in R3. The behaviour of LVEF was homogenous in the groups, showing an increase from R1 to R3. The improvement in autonomic nervous system activity after successful CTO recanalisation is reflected by an increase in BRS, and the changes are dependent on the baseline value of the measurement: patients with a depressed BRS before recanalisation present a greater BRS improvement than patients with a preserved BRS. The CTO recanalisation leads to the improvement of LVEF in both groups.

Application of multidimensional data analysis to chromatography

Abstract This work presents analysis of chromatographic signal used to identify substances in samples. First part consists of chromatography overview and description of three classification methods (neural network with backpropagation, probabilistic neural network with Parzen window and support vector machines). Designed algorithm consists of several stages: signal filtering, peak detection and its approximation with sum of two Gaussian functions. The parameters of that two curves are the features vectors describing the peak of the substance. The last step is classification, for which two types of supervised machine learning were compared, based on the whole signal and on features vectors. Both types were tested for different classificators and their parameters. Verification was based on 55 chromatography signals. The best results for both methods of learning were achieved for probabilistic neural networks. The correct classification rate was 82% for the whole signal and 93% for feature vectors.

Nonrigid image registration using similarity measure based on spatially weighted global probability distribution function

In nonrigid image registration, similarity measures including spatial information have been shown to perform better than those measures without spatial information. In this work, we provide new insight to the relationships among regional mutual information, regional probability distribution functions (PDFs) and global PDFs, and propose a novel nonrigid registration scheme with spatially weighted global probability distribution function (SWGPDF). Similarity measures based on SWGPDF (SWGPDFSM) are constructed. Three different spatial sub-region division methods are compared: the equally spaced sub-region (ESSR), the local binary pattern sub-region (LBPSR) and the gradient sub-region (GSR). The registration scheme applies B-spline based free form deformations (FFDs) as the transformation model. A Parzen window and linear interpolation are used to construct histograms. The SWGPDFSM registration scheme with ESSR space division is compared with the traditional global mutual information (gMI), the traditional global normalized mutual information (gNMI), regional mutual information and the SWGPDFSM with LBPSR or GSR space division. The test results show that SWGPDFSM scheme with ESSR space division outperforms the other schemes for elastically aligning images in the presence of big geometrical transformations, bias fields and illumination changes.

An Analysis on Mammograms using KDA in Multi Transform Domain

a robust Computer Aided Diagnosis (CADx) system for mammograms analysis has been the challenging task for years. The slight difference in X-ray attenuation between normal and abnormal glandular tissues makes the mammogram diagnosis complicated. The proposed CAD system discriminates the abnormal severity of mammograms into normal-benign (non-cancerous, non-spreadable), normal- malign (cancerous) ans benign-,malign, using wavelet features combined with spectral domain. Classification is performed on 206 different mammogram images from Mias database using Kernel Discriminant Analysis (KDA). KDA affords a non-parametric statistical approach with parzen window density estimation to estimate density function from a given sample dataset. The study reveals that the optimal smoothing parameters are increasing functions of the sample size of the complementary classes, features used to classify and value of the bandwidth. KeywordsMammograms, KDA, Accuracy, MCC

SIMILARITY EVALUATION OF ONLINE SIGNATURES BASED ON MODIFIED DYNAMIC TIME WARPING

Many people are very accustomed to the process of signing their name and having it matched for authentication. In a signature verification system, the signatures are processed to extract features that are used for verification. These features should not be duplicable. A basic problem is intraclass variations that will greatly affect the matching scores produced. The problem of distinctiveness occurs when the expectation of signatures to vary significantly between individuals is not met. There may be a large number of similarities in the feature sets used to represent the signatures of two different individuals. The efficiency of any signature verification system depends mainly on the discrimination power and robustness of the features used in the system. This study evaluates 40 functional features of viewpoint classification error and consistency for extracting the best subset once a set of features provides maximal discrimination capability between genuine and forged signatures. A modified distance of the DTW algorithm is proposed to improve performance of the verification phase. The proposed system is evaluated on the public SVC2004 signature database. The experimental results show that first, the most discriminate and consistent features are velocity based. Second, the average EER for the proposed algorithm in comparison with the general DTW algorithm shows a 5.47% decrease. Moreover, a comparative study based on a different classifier with a skilled forgery shows that the best result has an EER of 1.73% using the Parzen window classifier.

An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

In this paper, an improved feature selection algorithm by conditional mutual information with Parzen window was proposed, which adopted conditional mutual information as an evaluation criterion of feature selection in order to overcome the deficiency of feature redundant and used Parzen window to estimate the probability density functions and calculate the conditional mutual information of continuous variables, in such a way as to achieve feature selection for continuous data.

Locally Orderless Registration

This paper presents a unifying approach for calculating a wide range of popular, but seemingly very different, similarity measures. Our domain is the registration of n-dimensional images sampled on a regular grid, and our approach is well suited for gradient-based optimization algorithms. Our approach is based on local intensity histograms and built upon the technique of Locally Orderless Images. Histograms by Locally Orderless Images are well posed and offer explicit control over the three inherent and unavoidable scales: the spatial resolution, intensity levels, and spatial extent of local histograms. Through Locally Orderless Images, we offer new insight into the relations between these scales. We demonstrate our unification by developing a Locally Orderless Registration algorithm for two quite different similarity measures, namely, Normalized Mutual Information and Sum of Squared Differences, and we compare these variations both theoretically and empirically. Finally, using our algorithm, we explain the empirically observed differences between two popular joint density estimation techniques used in registration: Parzen Windows and Generalized Partial Volume.

SU‐E‐J‐50: Maximum a Posteriori Proton CT Reconstruction Using Anatomic Prior

Purpose: It has been demonstrated that the information in one image modality can be used to significantly help the reconstruction of another image modality. Here we develop and evaluate a novel Maximum a Posteriori (MAP) approach to apply anatomical information in MRI or x‐ray CT image to facilitate the estimate of Proton CT (pCT) image. Method: To incorporate anatomical information of MRI or x‐ray CT into the reconstruction of pCT images through priors based on information theoretic similarity measures, we formulate a MAP optimization problem, which includes a least‐square data‐fitting term to account for the noise in the measurement and a penalty term to maximize the joint density of feature vectors extracted from the pCT image and anatomical image. We defined feature vectors that emphasize prominent boundaries in anatomical and functional images, and attach less importance to detail and noise that is less likely to be correlated in the two images. We use a non‐parametric, FFT‐based Parzen window approach to speed up the computation of the density function of the feature vector, the joint density of feature vector, and the gradients of joint entropy. The optimization problem is solved using conjugate gradient and Armijo‐backtracking line search. We performed realistic simulations using anthropomorphic phantom to validate our approach and compare its performance to MAP approach with a quadratic smoothness penalty function (without anatomic information from x‐ray). Results: The Preliminary results demonstrate the proposed MAP approach with MRI or x‐ray CT as prior can achieve better contrast recovery and biasvariance trade‐off curves compared with MAP approach without anatomical prior. Conclusions: The proposed MAP approach can achieve superior image quality by using anatomical information in the MRI/x‐ray CT image. In the future we will investigate how much data reduction we can achieve using the proposed method while maintain the same image quality.

Models and Methods for Automated Background Density Estimation in Hyperspectral Anomaly Detection

Anomaly detection (AD) in remotely sensed hyperspectral images has been proven to be valuable in many applications. In this paper, we propose a scheme for detecting global anomalies in which a likelihood ratio test-based decision rule is applied in conjunction with automated data-driven estimation of the background probability density function (PDF). Specifically, the use of both semiparametric (finite mixtures) and nonparametric (Parzen windows) models is investigated for background PDF estimation. Although such approaches are well known in multivariate data analysis, they have been very seldom applied to estimate the hyperspectral image background PDF, mostly due to the difficulty of reliably learning the model parameters without operator intervention. In this paper, semi and nonparametric estimators have been successfully employed to estimate the image background PDF with the aim of detecting global anomalies in a scene benefiting from the application of ad hoc Bayesian learning strategies. Two real hyperspectral images have been used to experimentally evaluate the ability of the proposed AD scheme resulting from the application of different global background PDF models and learning methods.

Quadratic Inverse Function Tsallis Entropy Multi-modulus Blind Equalization Algorithm

In underwater acoustic communication systems, inter-symbol interference (ISI) caused by communication channel distortion is the main factor affecting the quality of communication. Aiming at the shortcomings of computational complexity, slow convergence rate, and poor stability of Multi-Modulus Algorithm (MMA), a quadratic inverse function Tsallis entropy of Cascade Multi-Modulus blind equalization Algorithm (TCMMA) was proposed. In the proposed algorithm, quadratic inverse function Tsallis entropy is fused with Parzen window probability density estimation method, and directly used as the cost function in cascade blind equalizer to reduce the residual mean square error, and the normalized wavelet transform is used to speed up the convergence speed. The simulation results with underwater acoustic channel show the superiority of the proposed algorithm. DOI: http://dx.doi.org/10.11591/telkomnika.v11i4.2371

Sparse probability density function estimation using the minimum integrated square error

We develop a new sparse kernel density estimator using a forward constrained regression framework, within which the nonnegative and summing-to-unity constraints of the mixing weights can easily be satisfied. Our main contribution is to derive a recursive algorithm to select significant kernels one at time based on the minimum integrated square error (MISE) criterion for both the selection of kernels and the estimation of mixing weights. The proposed approach is simple to implement and the associated computational cost is very low. Specifically, the complexity of our algorithm is in the order of the number of training data N, which is much lower than the order of N2 offered by the best existing sparse kernel density estimators. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with comparable accuracy to those of the classical Parzen window estimate and other existing sparse kernel density estimators.

Authentication based on pole-zero models of signature velocity

With the increase of communication and financial transaction through internet, on-line signature verification is an accepted biometric technology for access control and plays a significant role in authenticity and authorization in modernized society. Therefore, fast and precise algorithms for the signature verification are very attractive. The goal of this paper is modeling of velocity signal that pattern and properties is stable for persons. With using pole-zero models based on discrete cosine transform, precise method is proposed for modeling and then features is founded from strokes. With using linear, parzen window and support vector machine classifiers, the signature verification technique was tested with a large number of authentic and forgery signatures and has demonstrated the good potential of this technique. The signatures are collected from three different database include a proprietary database, the SVC2004 and the Sabanci University signature database benchmark databases. Experimental results based on Persian, SVC2004 and SUSIG databases show that our method achieves an equal error rate of 5.91%, 5.62% and 3.91% in the skilled forgeries, respectively.