Parzen Window Density Estimation Research Articles

Manifold Alignment Aware Ants: A Markovian Process for Manifold Extraction.

The presence of manifolds is a common assumption in many applications, including astronomy and computer vision. For instance, in astronomy, low-dimensional stellar structures, such as streams, shells, and globular clusters, can be found in the neighborhood of big galaxies such as the Milky Way. Since these structures are often buried in very large data sets, an algorithm, which can not only recover the manifold but also remove the background noise (or outliers), is highly desirable. While other works try to recover manifolds either by pushing all points toward manifolds or by downsampling from dense regions, aiming to solve one of the problems, they generally fail to suppress the noise on manifolds and remove background noise simultaneously. Inspired by the collective behavior of biological ants in food-seeking process, we propose a new algorithm that employs several random walkers equipped with a local alignment measure to detect and denoise manifolds. During the walking process, the agents release pheromone on data points, which reinforces future movements. Over time the pheromone concentrates on the manifolds, while it fades in the background noise due to an evaporation procedure. We use the Markov chain (MC) framework to provide a theoretical analysis of the convergence of the algorithm and its performance. Moreover, an empirical analysis, based on synthetic and real-world data sets, is provided to demonstrate its applicability in different areas, such as improving the performance of t-distributed stochastic neighbor embedding (t-SNE) and spectral clustering using the underlying MC formulas, recovering astronomical low-dimensional structures, and improving the performance of the fast Parzen window density estimator.

Security risk assessment of wind integrated power system using Parzen window density estimation

Security risk assessment of wind integrated power system using Parzen window density estimation

Prediction of residual life of complex electromechanical systems based on small training samples

A model for predicting the residual life of complex electromechanical systems based on small training samples is presented. The developed algorithm involves the use of a probabilistic model of a complex electromechanical system. The statistical characteristics of the model parameters are determined by training samples, that is, by past experience of the functioning of this or similar system. The developed algorithm is based on recognition and forecasting algorithms based on a Wald’s sequential decision-making procedure, a modified V. S. Pugachev canonical decomposition, and an improved Parzen window density estimation algorithm.

Outlier Detection Using Improved Support Vector Data Description in Wireless Sensor Networks.

Wireless sensor networks (WSNs) are susceptible to faults in sensor data. Outlier detection is crucial for ensuring the quality of data analysis in WSNs. This paper proposes a novel improved support vector data description method (ID-SVDD) to effectively detect outliers of sensor data. ID-SVDD utilizes the density distribution of data to compensate SVDD. The Parzen-window algorithm is applied to calculate the relative density for each data point in a data set. Meanwhile, we use Mahalanobis distance (MD) to improve the Gaussian function in Parzen-window density estimation. Through combining new relative density weight with SVDD, this approach can efficiently map the data points from sparse space to high-density space. In order to assess the outlier detection performance, the ID-SVDD algorithm was implemented on several datasets. The experimental results demonstrated that ID-SVDD achieved high performance, and could be applied in real water quality monitoring.

Benefits of noise in M-estimators: Optimal noise level and probability density

For the robust estimation of a location parameter, we consider a parallel array of maximum likelihood type estimators (M-estimators). We investigate the possibility of added noise as a design variable of the M-estimators, and characterize a nonzero optimal amount of added noise maximizing the efficiency for estimation. The added noise shows its benefits to the asymptotic efficiency of the M-estimator when the noise level and the noise probability density are optimally tuned. The optimal noise level can be theoretically derived by maximizing the asymptotic efficiency as the probability density of added noise is given. Based on the Parzen-window density estimation technique, we approximate the infinite-dimensional non-convex optimization of the optimal probability density of added noise as a simpler optimization problem with respect to a finite-dimensional vector under certain constraints. This approximate solution for the optimal probability density of added noise shows its feasibility for various M-estimators with an arbitrary array size, which is also validated by simulation results.

BorderShift: toward optimal MeanShift vector for cluster boundary detection in high-dimensional data

We present a cluster boundary detection scheme that exploits MeanShift and Parzen window in high-dimensional space. To reduce the noises interference in Parzen window density estimation process, the kNN window is introduced to replace the sliding window with fixed size firstly. Then, we take the density of sample as the weight of its drift vector to further improve the stability of MeanShift vector which can be utilized to separate boundary points from core points, noise points, isolated points according to the vector models in multi-density data sets. Under such circumstance, our proposed BorderShift algorithm doesn’t need multi-iteration to get the optimal detection result. Instead, the developed Shift value of each data point helps to obtain it in a liner way. Experimental results on both synthetic and real data sets demonstrate that the F-measure evaluation of BorderShift is higher than that of other algorithms.

Improved linear local tangent space alignment and its application to pattern recognition

Considering the drawbacks of the linear local tangent space alignment, a Semi-Supervised Neighbourhood Adaptive Linear Local Tangent Space Alignment (SSNA-LLTSA) is proposed. The distance metric combining the Cosine similarity and the Euclidean distance is used in the algorithm instead of the Euclidean distance, and the algorithm realises the semi-supervised learning and neighbourhood adaptive adjustment by integrating some of the known category information and the method of Parzen window density estimation into the dimensionality reduction process. The simulation experiment of UCI standard data sets and the pattern recognition example of hydraulic pump show that the redefined distance metric has better performance than the Euclidean distance, and SSNA-LLTSA can overcome the defect that LLTSA is unsupervised. Meanwhile, the capability of neighbourhood adaptive adjustment makes the algorithm find the low-dimensional manifold of the data sets more effectively, which can further improve the accuracy of pattern recognition.

Mahalanobis Distance Based Approach for Anomaly Detection of Analog Filters Using Frequency Features and Parzen Window Density Estimation

Analog filters play a very important role in insuring the availability of electronic systems. Early detection of anomalies of analog filters can prevent the impending failures and enhance reliability. The complex architecture and the tolerances of multiple components make it very difficult to detect anomalies in analog filters. To address this concern, A Mahalanobis distance (MD) based anomaly detection method for analog filters is proposed in this paper. The conventional frequency features and the moment of frequency response are selected as the feature vector. Mahalanobis distance is used to transform the frequency feature vector to one dimensional MD data. The anomaly detection threshold is obtained based on probability density of the health MD data sets which is estimated by Parzen window density estimation method. The efficiency of the proposed method has been verified by two case studies. In the case studies, a comprehensive indicator constructed by miss alarm and false alarm is used to obtain an optimal anomaly detection threshold. One class SVM (OCSVM) based anomaly detection method is used as a comparison with our approach. The results illustrate that: (1) the proposed frequency features can effectively clarify the degradation of analog filters; (2) the proposed MD based approach can detect anomalies in analog filters effectively at an early time stage. (3) the proposed MD based approach can detect anomalies in analog filters more accurately than OCSVM based method.

Parzen Window Density Estimator-Based Probabilistic Power Flow With Correlated Uncertainties

This paper presents a numerical-based algorithm to solve the probabilistic power flow problem. Parzen window density estimator is used to efficiently estimate probabilistic characteristics of power flow outputs. Correlations between wind generation, load, and plug-in hybrid electric vehicle charging stations are taken into account. The proposed algorithm works properly for random variables with various probability distribution functions and is very useful when limited information is available for each random variable. The algorithm is tested on the IEEE 14-bus and IEEE 118-bus systems considering correlated and uncorrelated conditions. Comparison between the proposed algorithm with 2n, $\text{2n} + 1$ point estimation methods as well as Monte Carlo simulation and linear diffusion method are provided. In addition, probability density and cumulative distribution functions are determined using the proposed algorithm, diffusion method, and the combined Cumulants and Gram-Charlier for $\text{2n} + 1$ point estimation method. Error indices are introduced to evaluate all random variables in a single benchmark. Simulation results show the effectiveness of the proposed algorithm to provide complete statistical information for probabilistic power flow outputs.

Keystroke dynamics-based user authentication using long and free text strings from various input devices

Keystroke dynamics, which refers to the typing pattern of an individual, has been highlighted as a practical behavioral biometric feature that does not require any additional recognition device for strengthening user authentication or identification. However, research in the area of keystroke dynamics-based user authentication (KDA) has been primarily focused only on the short predefined text, such as identification (ID) and password, typed on a traditional personal computer (PC) keyboard. In this paper, we aim to explore the extendability of KDA by considering long and free text strings from various input devices. Three fundamental questions are raised about the dependence of authentication performance on (1) the type of input device, (2) the length of text strings, and (3) the type of authentication algorithm. Based on the experimental tests, we observe that (1) the usage of a PC keyboard reported the highest authentication accuracy, followed by a soft keyboard and a touch keyboard; (2) the authentication accuracy could be strengthened by increasing the length of either reference or test keystrokes; (3) the R+A and RA measures report the best performance with a PC keyboard, while the Cramér–von Mises criterion reports the best performance with the other input devices for most cases, followed by the Parzen window density estimator.

An Analysis on Mammograms using KDA in Multi Transform Domain

a robust Computer Aided Diagnosis (CADx) system for mammograms analysis has been the challenging task for years. The slight difference in X-ray attenuation between normal and abnormal glandular tissues makes the mammogram diagnosis complicated. The proposed CAD system discriminates the abnormal severity of mammograms into normal-benign (non-cancerous, non-spreadable), normal- malign (cancerous) ans benign-,malign, using wavelet features combined with spectral domain. Classification is performed on 206 different mammogram images from Mias database using Kernel Discriminant Analysis (KDA). KDA affords a non-parametric statistical approach with parzen window density estimation to estimate density function from a given sample dataset. The study reveals that the optimal smoothing parameters are increasing functions of the sample size of the complementary classes, features used to classify and value of the bandwidth. KeywordsMammograms, KDA, Accuracy, MCC

Rényi entropy and cauchy-schwartz mutual information applied to mifs-u variable selection algorithm: a comparative study

This paper approaches the algorithm of selection of variables named MIFS-U and presents an alternative method for estimating entropy and mutual information, "measures" that constitute the base of this selection algorithm. This method has, for foundation, the Cauchy-Schwartz quadratic mutual information and the Rényi quadratic entropy, combined, in the case of continuous variables, with Parzen Window density estimation. Experiments were accomplished with public domain data, being such method compared with the original MIFS-U algorithm, broadly used, that adopts the Shannon entropy definition and makes use, in the case of continuous variables, of the histogram density estimator. The results show small variations between the two methods, what suggest a future investigation using a classifier, such as Neural Networks, to qualitatively evaluate these results, in the light of the final objective which is greater accuracy of classification.

Optimal control strategies for multi-chiller system based on probability density distribution of cooling load ratio

Probability density distribution profile of cooling load ratio is obtained by Parzen window density estimation method, which is used to describe the building cooling load variation. Based on the analysis of this profile, the optimal sequence control strategies (AI, AII and AIII) are presented. In order to optimize the COP of total chillers, the optimal load allocation control strategy B is presented. The optimal load allocation control strategy BI is presented to improve the strategy B for simplifying the process of the optimal calculation and maintaining stability of the system operating. Simulation results show that these optimal control strategies presented can save more power consumption of chillers operating than the original sequence control strategy. Comparing the strategy B, the strategy BI can avoid unnecessary switch on/off actions of chillers and decrease computational costs.

An efficient gene selection algorithm based on mutual information

Gene selection, a significant preprocessing of the discriminant analysis of microarray data, is to select the most informative genes from the whole gene set. In this paper, an efficient mutual information-based gene selection algorithm (MIGS) is proposed, in which genes are sequentially forward selected according to an approximate measure of the mutual information between the class and the selected genes. In order to improve the efficiency of the MIGS, an effective pruning strategy is introduced in the selection procedure as well as the employment of Parzen window density estimation technique. Extensive experiments are conducted on three public gene expression datasets and the experimental results confirm the efficiency and effectiveness of the algorithm. Though the computational cost of MIGS-Pruning increases with the number of selected genes, it still has good performance applied in the microarray problems.

ParCrys: a Parzen window density estimation approach to protein crystallization propensity prediction

The ability to rank proteins by their likely success in crystallization is useful in current Structural Biology efforts and in particular in high-throughput Structural Genomics initiatives. We present ParCrys, a Parzen Window approach to estimate a protein's propensity to produce diffraction-quality crystals. The Protein Data Bank (PDB) provided training data whilst the databases TargetDB and PepcDB were used to define feature selection data as well as test data independent of feature selection and training. ParCrys outperforms the OB-Score, SECRET and CRYSTALP on the data examined, with accuracy and Matthews correlation coefficient values of 79.1% and 0.582, respectively (74.0% and 0.227, respectively, on data with a 'real-world' ratio of positive:negative examples). ParCrys predictions and associated data are available from www.compbio.dundee.ac.uk/parcrys.

A New Adaptive Filter Algorithm for System Identification Using Independent Component Analysis

This paper proposes a new adaptive filter algorithm for system identification by using an independent component analysis (ICA) technique, which separates the signal from noisy observation under the assumption that the signal and noise are independent. We first introduce an augmented state-space expression of the observed signal, representing the problem in terms of ICA. By using a nonparametric Parzen window density estimator and the stochastic information gradient, we derive an adaptive algorithm to separate the noise from the signal. The proposed ICA-based algorithm does not suppress the noise in the least mean square sense but to maximize the independence between the signal part and the noise. The computational complexity of the proposed algorithm is compared with that of the standard NLMS algorithm. The stationary point of the proposed algorithm is analyzed by using an averaging method. We can directly use the new ICA-based algorithm in an acoustic echo canceller without double-talk detector. Some simulation results are carried out to show the superiority of our ICA method to the conventional NLMS algorithm.

Some Equivalences between Kernel Methods and Information Theoretic Methods

In this paper, we discuss some equivalences between two recently introduced statistical learning schemes, namely Mercer kernel methods and information theoretic methods. We show that Parzen window-based estimators for some information theoretic cost functions are also cost functions in a corresponding Mercer kernel space. The Mercer kernel is directly related to the Parzen window. Furthermore, we analyze a classification rule based on an information theoretic criterion, and show that this corresponds to a linear classifier in the kernel space. By introducing a weighted Parzen window density estimator, we also formulate the support vector machine in this information theoretic perspective.

Adaptive linear filtering design with minimum symbol error probability criterion

Adaptive digital filtering has traditionally been developed based on the minimum mean square error (MMSE) criterion and has found ever-increasing applications in communications. This paper presents an alternative adaptive filtering design based on the minimum symbol error rate (MSER) criterion for communication applications. It is shown that the MSER filtering is smarter, as it exploits the non-Gaussian distribution of filter output effectively. Consequently, it provides significant performance gain in terms of smaller symbol error over the MMSE approach. Adopting Parzen window or kernel density estimation for a probability density function, a block-data gradient adaptive MSER algorithm is derived. A stochastic gradient adaptive MSER algorithm, referred to as the least symbol error rate, is further developed for sample-by-sample adaptive implementation of the MSER filtering. Two applications, involving single-user channel equalization and beamforming assisted receiver, are included to demonstrate the effectiveness and generality of the proposed adaptive MSER filtering approach.

A Face and Speech Biometric Verification System Using A Simple Bayesian Structure

Identity verification systems that use a mono modal biometric always have to contend with sensor noise and limitations of the feature extractor and matcher, while combining information from different biometrics modalities may well provide higher and more consistent performance levels. However, an intelligent scheme is required to fuse the decisions produced by the individual sensors. This paper presents a decision fusion technique for a bimodal biometric verification system that makes use of facial and speech biometrics. The decision fusion schemes considered have simple Bayesian structures (SBS) that particularize the univariat Gaussian density function, Beta density function or Parzen window density estimation. SBS has advantages in terms of computation speed, storage space and its open framework. The performances of SBS is evaluated and compared with that of other classical classification approaches, such as sum rule and Multilayer Perceptron, on a bimodal database.

Rényi continuous entropy of DNA sequences

Entropy measures of DNA sequences estimate their randomness or, inversely, their repeatability. L-block Shannon discrete entropy accounts for the empirical distribution of all length-L words and has convergence problems for finite sequences. A new entropy measure that extends Shannon's formalism is proposed. Rényi's quadratic entropy calculated with Parzen window density estimation method applied to CGR/USM continuous maps of DNA sequences constitute a novel technique to evaluate sequence global randomness without some of the former method drawbacks. The asymptotic behaviour of this new measure was analytically deduced and the calculation of entropies for several synthetic and experimental biological sequences was performed. The results obtained were compared with the distributions of the null model of randomness obtained by simulation. The biological sequences have shown a different p-value according to the kernel resolution of Parzen's method, which might indicate an unknown level of organization of their patterns. This new technique can be very useful in the study of DNA sequence complexity and provide additional tools for DNA entropy estimation. The main MATLAB applications developed and additional material are available at the webpage http://bioinformatics.musc.edu/renyi. Specialized functions can be obtained from the authors.