Parzen Method Research Articles

Assessment of computing systems reliability by non-parametric method by small samples of operational data

A new method for assessing the reliability of small computing systems that allows the generation of only small samples of operational data is proposed in the article. The result of applying the technique is the posterior failure distribution density, on the basis of which various reliability indicators can be calculated. The methodology consists of two stages: the first is the preparation of operational data, including detection of failures using machine learning methods, and the second is the construction of the failure distribution density using the adapted Rosenblatt–Parzen method. Increasing the efficiency of estimates using the proposed method is achieved by taking into account censored data, compensating for the shift of failure distribution densities and finding the optimal smoothing parameter.

Speaker recognition using vocal source model

Speaker recognition is carried out in the space of the functional parameters of the area of the glottal cross-section, found by solving the inverse problem. This problem is solved in two stages: first, the signal obtained by inverse filtering is approximated using the vocal source model, and then the glottal area model parameters, which generate the calculated vocal source impulse, are computed. Speaker recognition is carried out on a database of Russian numerals from 0 to 9 separately for men (48 speakers) and women (37 speakers) at the segments of stressed vowels. Various methods of recognition are studied: the Gaussian mixture model (GMM), support vector machines (SVMs), discriminant analysis, naive Bayes classifier (NB), the method of classification trees (CTREE), and the Parzen window classifier. The best results were obtained using the method of SVMs and the Parzen method: the average total error of identification of men was 4.9% and 5.1%, and that of women--8.2% and 8.8%, respectively.

On the Smoothed Minimum Error Entropy Criterion

Recent studies suggest that the minimum error entropy (MEE) criterion can outperform the traditional mean square error criterion in supervised machine learning, especially in nonlinear and non-Gaussian situations. In practice, however, one has to estimate the error entropy from the samples since in general the analytical evaluation of error entropy is not possible. By the Parzen windowing approach, the estimated error entropy converges asymptotically to the entropy of the error plus an independent random variable whose probability density function (PDF) corresponds to the kernel function in the Parzen method. This quantity of entropy is called the smoothed error entropy, and the corresponding optimality criterion is named the smoothed MEE (SMEE) criterion. In this paper, we study theoretically the SMEE criterion in supervised machine learning where the learning machine is assumed to be nonparametric and universal. Some basic properties are presented. In particular, we show that when the smoothing factor is very small, the smoothed error entropy equals approximately the true error entropy plus a scaled version of the Fisher information of error. We also investigate how the smoothing factor affects the optimal solution. In some special situations, the optimal solution under the SMEE criterion does not change with increasing smoothing factor. In general cases, when the smoothing factor tends to infinity, minimizing the smoothed error entropy will be approximately equivalent to minimizing error variance, regardless of the conditional PDF and the kernel.

GENERALIZED CELLULAR AUTOMATA FOR STUDYING THE BEHAVIOR OF CELL POPULATIONS

Cellular automata are generalized to handle non-local rules, i.e. rules taking into account the states of the whole set of objects present in the array. Considering a binary state (presence/absence of objects), evolution rules are defined as a function of the local density of objects, estimated according to the Parzen method. This allows tackling problems found in the field of cellular sociology, such as cell clustering or cell sorting.

Rényi continuous entropy of DNA sequences

Entropy measures of DNA sequences estimate their randomness or, inversely, their repeatability. L-block Shannon discrete entropy accounts for the empirical distribution of all length-L words and has convergence problems for finite sequences. A new entropy measure that extends Shannon's formalism is proposed. Rényi's quadratic entropy calculated with Parzen window density estimation method applied to CGR/USM continuous maps of DNA sequences constitute a novel technique to evaluate sequence global randomness without some of the former method drawbacks. The asymptotic behaviour of this new measure was analytically deduced and the calculation of entropies for several synthetic and experimental biological sequences was performed. The results obtained were compared with the distributions of the null model of randomness obtained by simulation. The biological sequences have shown a different p-value according to the kernel resolution of Parzen's method, which might indicate an unknown level of organization of their patterns. This new technique can be very useful in the study of DNA sequence complexity and provide additional tools for DNA entropy estimation. The main MATLAB applications developed and additional material are available at the webpage http://bioinformatics.musc.edu/renyi. Specialized functions can be obtained from the authors.

A probabilistic neural network approach for modeling and classification of bacterial growth/no-growth data

In this paper, we propose to use probabilistic neural networks (PNNs) for classification of bacterial growth/no-growth data and modeling the probability of growth. The PNN approach combines both Bayes theorem of conditional probability and Parzen's method for estimating the probability density functions of the random variables. Unlike other neural network training paradigms, PNNs are characterized by high training speed and their ability to produce confidence levels for their classification decision. As a practical application of the proposed approach, PNNs were investigated for their ability in classification of growth/no-growth state of a pathogenic Escherichia coli R31 in response to temperature and water activity. A comparison with the most frequently used traditional statistical method based on logistic regression and multilayer feedforward artificial neural network (MFANN) trained by error backpropagation was also carried out. The PNN-based models were found to outperform linear and nonlinear logistic regression and MFANN in both the classification accuracy and ease by which PNN-based models are developed.

A CMOS binary pattern classifier based on Parzen's method

Biological circuitry in the brain that has been associated with the Parzen method of classification inspired an analog CMOS binary pattern classifier. The circuitry resides on three separate chips. The first chip computes the closeness of a test vector to each training vector stored on the chip where "vector closeness" is defined as the number of bits two vectors have in common above some thresholds. The second chip computes the closeness of the test vector to each possible category where "category closeness" is defined as the sum of the closenesses of the test vector to each training vector in a particular category. Category closenesses are coded by currents which feed into an "early bird" winner-take-all circuit on the third chip that selects the category closest to the test vector. Parzen classifiers offer superior classification accuracy than the common nearest neighbor Hamming networks. A high degree of parallelism allows for O(1) time complexity and the chips are tillable for increased training vector storage capacity. Proof-of-concept chips were fabricated through the MOSIS chip prototyping service and successfully tested.

Sparse random networks with LTP learning rules approximate Bayes classifiers via Parzen's method

Analysis is performed of a neuronal cell layer consisting of sparse random anatomical innervation, incorporating anatomically accurate distribution of excitatory and inhibitory cells, appropriate physiological time courses of excitatory-inhibitory interactions, and synaptic modification via long-term potentiation (LTP). One of two input pathways selects cells that learn and the other uses LTP-based learning rules to modify synapses of selected cells. It is found that the resulting network implements Parzen's method for approximating a Bayes classifier. The simple one-layer network can be generalized to include a class of networks including one that adds an input layer of neurons resulting in changes to the kernel used in the Parzen method. The LTP learning rule is nondecremental and limited to a maximum ceiling, which will result in essential saturation of synaptic weights; nonetheless, calculations using reasonable estimates of network parameters indicate a large network capacity.