Parzen Window Method Research Articles

Synthesis of stochastic algorithms for image registration by the criterion of maximum mutual information

We discuss a synthesis of stochastic algorithms, obtaining expressions for gradients of Shannon, Renyi and Tsallis mutual information on the basis of the mathematical apparatus of stochastic gradient adaptation of algorithms for estimating image registration parameters. To obtain the expressions, derivatives of the image entropy with respect to the estimated parameters are used. The entropies are calculated using a Parzen window method. A comparative study of the synthesized algorithms in terms of stability and accuracy of the registration parameter estimates, including in conditions of additive noise, is carried out.

Online Monitoring of Automotive Engine Lubricating Oil Based on Internet of Things Technology

In order to realize the online monitoring of automotive engine lubricating oil, a method based on Internet of Things technology is proposed. This method uses the self-organizing neural network of the Internet of Things to fuse the original multidimensional feature data to obtain the fusion value. The Parzen window method is used to formulate the limit value of fusion value, and the samples are divided into three states: normal, warning, and abnormal. Weka software is used to extract rules from oil data. This method can identify different wear state information from oil spectral data, extract knowledge rules, and use them to build the knowledge base of automobile engine wear diagnosis system, so as to realize the automation and intelligence of automobile engine fault diagnosis based on lubricating oil spectral wear data. The measurement method of the lubricating oil sensor is mainly to comprehensively reflect the relationship between oil quality and electrical signal, so as to effectively provide users with reliable information. After many studies, it is concluded that the conductivity of lubricating oil has a good linear relationship with its acid value, metal particles, moisture content, and the change of additive content. Measuring the change of conductivity is an effective means to detect the change of lubricating oil quality. The experimental results show that using the extracted knowledge rules to verify the state of samples, the recognition rate is 97.47%. In order to more fully explain the difference between important element fusion and all feature fusion, all features are extracted. At the same time, the fault diagnosis and recognition rate of all features is not high, only 62.39%. It is proved that the Internet of Things technology can effectively realize online monitoring of the automotive engine lubricating oil.

Margin-based approach for outlier detection of industrial design data using a modified general regression neural network

AbstractThe choice of components in industrial design involves setting design parameters that typically must reside inside permissible ranges called “design margins”. This paper proposes a novel automated method called the Margin-Based General Regression Neural Network (MB-GRNN) that classifies design errors for design parameters that are outside of permissible ranges as outliers, directly from industrial design data, using an unsupervised machine learning approach. The method is based on a modified GRNN that estimates extremal margin boundaries of design parameters by self-learning the features from datasets. These extremal permissible margin boundaries are determined by “stretching out” the upper and lower GRNN surfaces using an iterative application of stretch factors (a second kernel weighting factor). The method creates a variable insensitive band surrounding the data cloud, interlinked with the normal regression function, providing upper and lower margin boundaries. These boundaries can then be used to determine outliers and to predict a range of permissible values of design parameters during design. Pushing out extremal margin boundaries reduce the false identification of outliers. This classification technique could be used by industrial engineers to detect likely outliers and to predict a range of permissible output limits for chosen design parameters. The efficacy of this method has been validated against the widespread Parzen window method by comparing experimental results from three multivariate datasets. It was found that the two methods have different but complementary capabilities. The MB-GRNN also uses a modified algorithm for estimating the smoothing parameter using a combination of clustering, k-nearest neighbor, and localized covariance matrix.

Investigation of the effectiveness of metric classification methods in identifying attacks in VANET

This paper discusses the problem of improving the efficiency of metric machine learning methods of identification attacks in vehicular adhoc networks (VANETs). The main idea of this research is to select the type of nonlinear functions for calculating the distances between the objects of the sample, describing the traffic of VANET using metric methods, such as the method of k-nearest neighbour with linearly decreasing weights and the Parzen window method. The analysis of the effectiveness of the methods considered was carried out on a synthetically generated sample with three different types of attacks on the network. Computational experiments have shown that the k-nearest neighbour method with decreasing weights based on an exponential function with base a < 1 is more efficient than the Parzen window method by about 0.3% and has an accuracy of 84.15%.

Brain Image Segmentation Based on Multi-Weight Probability Map

Due to the complexity of the brain image itself, the brain image segmentation technology has become a bottle for further application and development of the system. Considering the inconsistency of intensity, partial volume effect, and noise in medical images, this paper studies the brain image segmentation technology based on the multi-weight probability. The multi-weight probability method mainly models the data set with outliers and non-Gaussian noise. First, the probabilistic local ELM model is established. Based on this, the Parzen window method is used to establish the probability distribution of the local model, and then, the probability distribution is used as the weight to fuse. All local models are used to build a global robustness model. The method successfully applied the brain and UCI examples and compared with traditional ELM, regularized ELM, and robust ELM. The results show that the probability weight ELM shows better modeling performance.

Artificial bee colony algorithm based on Parzen window method

Artificial Bee Colony (ABC) algorithm, based on the metaphor in foraging behavior of honey fee swarm, has been repeatedly criticized for its poor convergence, due to its known exploration bias. In order to enhance the performance of ABC, the paper develops a novel approach (named ABCPW). First, three popular search strategies with different characteristics are employed to construct a strategy candidate pool for obtaining high quality candidate individuals. Next, to cut down on computational cost, the Parzen window method is applied to estimate these candidate individuals and then select one as the offspring. In addition, two different neighborhood mechanisms are adopted to balance the convergence and the population diversity. Finally, the performance of ABCPW is tested on a series of benchmark functions. The experimental results not only demonstrate the stability and convergence of ABCPW, but also show ABCPW outperforms several popular algorithms.

Star cluster detection and characterization using generalized Parzen density estimation

Star cluster studies hold the key to understanding star formation, stellar evolution, and origin of galaxies. The detection and characterization of clusters depend on the underlying background density and the cluster richness. We examine the ability of the Parzen Density Estimation (a.k.a. Parzen Windows) method, which is a generalization of the well-known Star Count method, to detect clusters and measure their properties. We apply it on a range of simulated and real star fields, considering square and circular windows, with and without Gaussian kernel smoothing. Our method successfully identifies clusters and we suggest an optimal standard deviation of the Gaussian Parzen window for obtaining the best estimates of these parameters. Finally, we demonstrate that the Parzen Windows with Gaussian kernels are able to detect small clusters in regions of relatively high background density where the Star Count method fails.

Density estimation of grey-level co-occurrence matrices for image texture analysis

The Haralick texture features are common in the image analysis literature, partly because of their simplicity and because their values can be interpreted. It was recently observed that the Haralick texture features are very sensitive to the size of the GLCM that was used to compute them, which led to a new formulation that is invariant to the GLCM size. However, these new features still depend on the sample size used to compute the GLCM, i.e. the size of the input image region-of-interest (ROI).The purpose of this work was to investigate the performance of density estimation methods for approximating the GLCM and subsequently the corresponding invariant features.Three density estimation methods were evaluated, namely a piece-wise constant distribution, the Parzen-windows method, and the Gaussian mixture model. The methods were evaluated on 29 different image textures and 20 invariant Haralick texture features as well as a wide range of different ROI sizes.The results indicate that there are two types of features: those that have a clear minimum error for a particular GLCM size for each ROI size, and those whose error decreases monotonically with increased GLCM size. For the first type of features, the Gaussian mixture model gave the smallest errors, and in particular for small ROI sizes (less than about ).In conclusion, the Gaussian mixture model is the preferred method for the first type of features (in particular for small ROIs). For the second type of features, simply using a large GLCM size is preferred.

State estimation pre-filtering with overlapping tiling of autoencoders

This paper presents a new concept for an approach to deal with measurements contaminated with gross errors, prior to power system state estimation. Instead of a simple filtering operation, the new procedure develops a screen-and-repair process, going through the phases of detection, identification and correction of multiple gross errors.The method is based on the definition of the coverage of the measurement set by a tiling scheme of 3-overlapping autoencoders, trained with denoising techniques and correntropy, that produce an ensemble-like set of three proposals for each measurement. These proposals are then subject to a process of fusion to produce a vector of proposed/corrected measurements, and two fusion methods are compared, with advantage to the Parzen Windows method. The original measurement vector can then be recognized as clean or diagnosed with possible gross errors, together with corrections that remove these errors. The repaired vectors can then serve as input to classical state estimation procedures, as only a small noise remains. A test case illustrates the effectiveness of the technique, which could deal with four simultaneous gross errors and achieve a result close to full recognition and correction of the errors.

Information Entropy and Fuzzy Logic Based Equalizer for PolMux QAM Coherent Optical Communication Systems

In polarization-multiplexed (PolMux) coherent optical communication systems, adaptive blind equalizer is efficient in demultiplexing and mitigating intersymbol interference (ISI). A novel blind algorithm based on Information Entropy and fuzzy logic is proposed, in which the Renyi's $\alpha$ entropy is adopted to measure the uncertainty of error between the desired and estimated probability density function (PDF). The nonparametric PDF estimator of Parzen window method is employed to estimate the PDF of symbols. Meantime, a fuzzy-logic tuning unit is designed to adjust the kernel size of Parzen window, which leads to fast convergence rate and small steady mean-square error. By simulation in PolMux-16 quadrature amplitude modulation (QAM) coherent systems, the correctness and effectiveness of the proposed algorithm are verified.

Peaks Over Threshold–based detector design for structural health monitoring: Application to aerospace structures

Structural health monitoring offers new approaches to interrogate the integrity of complex structures. The structural health monitoring process classically relies on four sequential steps: damage detection, localization, classification, and quantification. The most critical step of such process is the damage detection step since it is the first one and because performances of the following steps depend on it. A common method to design such a detector consists of relying on a statistical characterization of the damage indexes available in the healthy behavior of the structure. On the basis of this information, a decision threshold can then be computed in order to achieve a desired probability of false alarm. To determine the decision threshold corresponding to such desired probability of false alarm, the approach considered here is based on a model of the tail of the damage indexes distribution built using the Peaks Over Threshold method extracted from the extreme value theory. This approach of tail distribution estimation is interesting since it is not necessary to know the whole distribution of the damage indexes to develop a detector, but only its tail. This methodology is applied here in the context of a composite aircraft nacelle (where desired probability of false alarm is typically between 10−4 and 10−9) for different configurations of learning sample size and probability of false alarm and is compared to a more classical one which consists of modeling the entire damage indexes distribution by means of Parzen windows. Results show that given a set of data in the healthy state, the effective probability of false alarm obtained using the Peaks Over Threshold method is closer to the desired probability of false alarm than the one obtained using the Parzen-window method, which appears to be more conservative.

Multi-Objective Optimization of University Bus Based on Passenger Probability Density Estimation

In this paper, a multi-objective optimization model is presented, based on Parzen Window method which can make a theoretical analysis of the probability density function of passengers and determine the demand over a corresponding period by integration. This model can minimize the total of waiting time of passengers and the running cost of school buses with several constrains including the passengers’ numbers and the longest endurance time. Moreover, the data from Wuhan University of Technology, used as an example, verify the feasibility of the model. At last, the whole results are calculated by the Particle Swarm Optimization.

The Parzen Window method: In terms of two vectors and one matrix

Pattern classification methods assign an object to one of several predefined classes/categories based on features extracted from observed attributes of the object (pattern). When L discriminatory features for the pattern can be accurately determined, the pattern classification problem presents no difficulty. However, precise identification of the relevant features for a classification algorithm (classifier) to be able to categorize real world patterns without errors is generally infeasible. In this case, the pattern classification problem is often cast as devising a classifier that minimizes the misclassification rate. One way of doing this is to consider both the pattern attributes and its class label as random variables, estimate the posterior class probabilities for a given pattern and then assign the pattern to the class/category for which the posterior class probability value estimated is maximum. More often than not, the form of the posterior class probabilities is unknown.The so-called Parzen Window approach is widely employed to estimate class-conditional probability (class-specific probability) densities for a given pattern. These probability densities can then be utilized to estimate the appropriate posterior class probabilities for that pattern. However, the Parzen Window scheme can become computationally impractical when the size of the training dataset is in the tens of thousands and L is also large (a few hundred or more). Over the years, various schemes have been suggested to ameliorate the computational drawback of the Parzen Window approach, but the problem still remains outstanding and unresolved.In this paper, we revisit the Parzen Window technique and introduce a novel approach that may circumvent the aforementioned computational bottleneck. The current paper presents the mathematical aspect of our idea. Practical realizations of the proposed scheme will be given elsewhere.

The Existence and Application of Facial Spectral Information

Current face representation methods focus on the feature extraction in the appearance of face image, ignoring the influence of spectrum information of the imaging waveband. By analyzing the characteristics of texture patterns of face image in different wavebands, this article presents that the face image implicitly contains spectrum information which reflects the essential nature of imaging waveband, and the spectrum information can be extracted by learning the texture patterns with the Parzen window method, and can be used to enhance the representation capability of current LBP-based face descriptor by embedding it. Extensive experiments clearly show the effectiveness of the proposed method on the FERET face image sets.

SVM Active Learning Approach for Image Classification Using Spatial Information

In the last few years, active learning has been gaining growing interest in the remote sensing community in optimizing the process of training sample collection for supervised image classification. Current strategies formulate the active learning problem in the spectral domain only. However, remote sensing images are intrinsically defined both in the spectral and spatial domains. In this paper, we explore this fact by proposing a new active learning approach for support vector machine classification. In particular, we suggest combining spectral and spatial information directly in the iterative process of sample selection. For this purpose, three criteria are proposed to favor the selection of samples distant from the samples already composing the current training set. In the first strategy, the Euclidean distances in the spatial domain from the training samples are explicitly computed, whereas the second one is based on the Parzen window method in the spatial domain. Finally, the last criterion involves the concept of spatial entropy. Experiments on two very high resolution images show the effectiveness of regularization in spatial domain for active learning purposes.

Comparative Analysis on Kernel Based Probability Density Estimation

In this paper, we compare the estimation performances of 7 different kernels (i.e., Uniform, Triangular, Epanechnikov, Biweight, Triweight, Cosine and Gaussian) when using them to conduct the probability density estimation with Parzen window method. We firstly analyze the efficiencies of these 7 kernels and then compare their estimation errors measured by mean squared error (MSE). The theoretical analysis and the experimental comparisons show that the mostly-used Gaussian kernel is not the best choice for the probability density estimation, of which the efficiency is low and estimation error is high. The derived conclusions give some guidelines for the selection of kernel in the practical application of probability density estimation.

A New Nearest Neighbor Classification Algorithm Based on Local Probability Centers

The nearest neighbor is one of the most popular classifiers, and it has been successfully used in pattern recognition and machine learning. One drawback ofkNN is that it performs poorly when class distributions are overlapping. Recently, local probability center (LPC) algorithm is proposed to solve this problem; its main idea is giving weight to samples according to their posterior probability. However, LPC performs poorly when the value ofkis very small and the higher-dimensional datasets are used. To deal with this problem, this paper suggests that the gradient of the posterior probability function can be estimated under sufficient assumption. The theoretic property is beneficial to faithfully calculate the inner product of two vectors. To increase the performance in high-dimensional datasets, the multidimensional Parzen window and Euler-Richardson method are utilized, and a new classifier based on local probability centers is developed in this paper. Experimental results show that the proposed method yields stable performance with a wide range ofkfor usage, robust performance to overlapping issue, and good performance to dimensionality. The proposed theorem can be applied to mathematical problems and other applications. Furthermore, the proposed method is an attractive classifier because of its simplicity.

Fractal and Statistical Features for the Discrimination Between Patients With Amyotrophic Lateral Sclerosis and Healthy Adults

Back to table of contents Previous article Next article LettersFull AccessFractal and Statistical Features for the Discrimination Between Patients With Amyotrophic Lateral Sclerosis and Healthy AdultsAli Falaki, Ph.D. Student, Farshad Almasganj, Ph.D., and Farzad Towhidkhah, Ph.D.Ali FalakiSearch for more papers by this author, Ph.D. Student, Farshad AlmasganjSearch for more papers by this author, Ph.D., and Farzad TowhidkhahSearch for more papers by this author, Ph.D.Published Online:1 Jan 2013https://doi.org/10.1176/appi.neuropsych.12010020AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InEmail To the Editor: Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder, by which 2 in 100,000 individuals are affected each year,1 that affects primary motor neurons in cerebral cortex, brain stem, and spinal cord.2 Degeneration of motor neurons will hinder sending cerebrum impulses to the muscle fibers, impairing voluntary movements. Muscle twitching and spasticity, difficulties in speech, swallowing, and breathing are most commonly known symptoms of ALS. One of the most salient features in determining disease severity and appropriate treatment is gait disorder in form of increase in mean stride interval and decrease in mean speed of walking.3 Stride-to-stride fluctuations in healthy adults show alteration in a chaotic manner; these have a fractal (self-similar) structure more than a random one,4,5 which would approach a stochastic state.5Hausdorff et al.3 are one of the first investigators who have worked on the neurological disorders, namely, Parkinson disease, Huntington disease, and ALS, whose recorded data have been utilized by many researches so far. According to their quantitative analysis, gait dynamic would be altered in statistical characteristics like stride interval fluctuation magnitude as well as fractal dimension characteristics.3 Wu and Shi6 used a nonparametric method based on the parzen-window method to analyze altered gait cycle duration in ALS and to classify patients with ALS and healthy adults based on a least-square support vector machine. They obtained classification accuracy of 92.3%, which is a bit higher than the classification rate of 91.7% obtained by Dutta et al.7 using Elman’s recurrent neural network.In recent years, chaotic theories have suggested novel approaches in analyzing biological systems; resulting in a better understanding of these complex nonlinear systems. Although chaotic characteristics might be altered according to progression of the disease, little has been done to classify healthy and patient groups based on the fractal dimension method. Fractal dimension of a signal can be used as a valuable tool to analyze transient changes and data-points’ correlation. Moreover, gait symmetry would be affected by neurodegenerative disorders.8 Hence, we suggested classifying patients and healthy subjects using fractal dimension features besides statistical ones. Since gait asymmetry is a feature of a healthy motor control system, we can use both left and right feet stride intervals to gain a more accurate classifier.Based on mentioned informative feature vector, fractal and statistical features, discrimination between patients and healthy subjects could be well achieved. Using this feature vector and considering asymmetry between left and right feet, different neurological disorders are distinguished from each other, their severity estimated, and appropriate treatment can be accordingly determined more accurately.Amirkabir University of Technology Biomedical Engineering Tehran Islamic Republic of IranCorrespondence: [email protected]comReferences1 Rothstein JD: Current hypotheses for the underlying biology of amyotrophic lateral sclerosis. Ann Neurol 2009; 65(Suppl 1):S3–S9Crossref, Medline, Google Scholar2 Brown RH: Amyotrophic lateral sclerosis. Insights from genetics. Arch Neurol 1997; 54:1246–1250Crossref, Medline, Google Scholar3 Hausdorff JM, Lertratanakul A, Cudkowicz ME, et al.: Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis. J Appl Physiol 2000; 88:2045–2053Crossref, Medline, Google Scholar4 Scafetta N, Marchi D, West BJ: Understanding the complexity of human gait dynamics. Chaos 2009; 19:026108Crossref, Medline, Google Scholar5 Hausdorff JM: Gait dynamics, fractals and falls: finding meaning in the stride-to-stride fluctuations of human walking. Hum Mov Sci 2007; 26:555–589Crossref, Medline, Google Scholar6 Wu Y, Shi L: Analysis of altered gait cycle duration in amyotrophic lateral sclerosis based on nonparametric probability density function estimation. Med Eng Phys 2011; 33:347–355Crossref, Medline, Google Scholar7 Dutta S, Chatterjee A, Munshi S: An automated hierarchical gait pattern identification tool employing cross-correlation-based feature extraction and recurrent neural network based classification. Expert Syst 2009; 26:202–217Crossref, Google Scholar8 Liao F, Wang J, He P: Multi-resolution entropy analysis of gait symmetry in neurological degenerative diseases and amyotrophic lateral sclerosis. Med Eng Phys 2008; 30:299–310Crossref, Medline, Google Scholar FiguresReferencesCited byDetailsCited byNone Volume 25Issue 1 Winter 2013Pages E22-E22 Metrics PDF download History Published online 1 January 2013 Published in print 1 January 2013

Statistical Analysis of Gait Rhythm in Patients With Parkinson's Disease

To assess the gait variability in patients with Parkinson's disease (PD), we first used the nonparametric Parzen-window method to estimate the probability density functions (PDFs) of stride interval and its two subphases (i.e., swing interval and stance interval). The gait rhythm standard deviation (sigma) parameters computed with the PDFs indicated that the gait variability is significantly increased in PD. Signal turns count (STC) was also derived from each outlier-processed gait rhythm time series to serve as a dominant feature, which could be used to characterize the gait variability in PD. Since it was observed that the statistical parameters of swing interval or stance interval were highly correlated with those of stride interval, this article only used the stride interval parameters, i.e., sigma(r) and STC(r) , to form the feature vector in the pattern classification experiments. The results evaluated with the leave-one-out cross-validation method demonstrated that the least squares support vector machine with polynomial kernels was able to provide a classification accurate rate of 90.32% and an area (Az) of 0.952 under the receiver operating characteristic curve, both of which were better than the results obtained with the linear discriminant analysis (accuracy: 67.74%, Az: 0.917). The features and the classifiers used in the present study could be useful for monitoring of the gait in PD.

Analysis and Adaptive Estimation of the Registration Noise Distribution in Multitemporal VHR Images

This paper analyzes the problem of change detection in very high resolution (VHR) multitemporal images by studying the effects of residual misregistration [registration noise (RN)] between images acquired on the same geographical area at different times. In particular, according to an experimental analysis driven from a theoretical study, the main effects of RN on VHR images are identified and some important properties are derived and described in a polar framework for change vector analysis. In addition, a technique for an adaptive and unsupervised explicit estimation of the RN distribution in the polar domain is proposed. This technique derives the RN distribution according to both a multiscale analysis of the distribution of spectral change vectors and the Parzen windows method. Experimental results obtained on simulated and real multitemporal data sets confirm the validity of the proposed analysis, the reliability of the derived properties on RN, and the effectiveness of the proposed estimation technique. This technique represents a very promising tool for the definition of change-detection methods for VHR multitemporal images robust to RN.