Class-conditional Probability Density Functions Research Articles

Optimization of Probability Density Functions Applicable for Hourly Rainfall

In order to improve the calculation accuracy of the rainfall probability distribution in related applications, this study aimed to select a theoretical function from applicable functions for three classes of the class-conditional probability density function (CCPD) of hourly rainfall series. The three applicable functions are generalized gamma distribution (GΓD), generalized normal distribution (GND), and Weibull distribution. For the reason that it is hard to distinguish the advantages and disadvantages of the three functions by the probability plot and error analysis of fitted values, optimization criteria are proposed, which are the Bayesian information criterion (BIC) and the estimated accuracy of both the annual average rainfall (AAR) and the annual average continuous rainfall (AACR). The results show that by using three applicable functions in 15 regions, the relative fitting deviations for CCPD1 were less than 2.3% and less than 3.3% for ln(CCPD1). The goodness-of-fit values were all above 0.98 for CCPD1 and greater than 0.94 for ln(CCPD1). The fitting effect of the Weibull distribution was relatively poor from the perspective of the probability plot and error analysis of the fitted values, while the three applicable functions could be used to fit CCPD. GΓD had the highest fitting accuracy for the three classes of CCPDs, but there is concern about overfitting due to its broad spectrum. GND, with fewer parameters, had comparable performance to GΓD, and when fitting CCPD1 using GND, the mean relative fitting deviation was 0.6%, the coefficient of determination was 0.999, and for ln(CCPD1), they were 1.45% and 0.989. At the same time, GND performed well in estimating the AARs, with an 8.6% relative error and a 0.92 correlation coefficient in the fifteen regions, indicating that GND can well reflect the spatial variation characteristics of the AAR. Moreover, the function form of GND is simple. GND follows the parsimonious principle, and it is suitable for the whole domain. Therefore, GND is recommended as the theoretical density function based on the optimization criteria. The genetic algorithm was adopted to obtain the approximate solution of the parameters through optimization, which can simplify the derivation and calculation steps. The multiplicative and additive fitting errors were both used in the objective functions, which gave comprehensive consideration to both ends of the fitting curve.

New multivariate kernel density estimator for uncertain data classification

Uncertainty in data occurs in diverse applications due to measurement errors, data incompleteness, and multiple repeated measurements. Several classifiers for uncertain data have been developed to tackle this uncertainty. However, the existing classifiers do not consider the dependencies among uncertain features, even though this dependency has a critical effect on classification accuracy. Therefore, we propose a new Bayesian classification model that considers the correlation among uncertain features. To handle the uncertainty of data, new multivariate kernel density estimators are developed to estimate the class conditional probability density function of categorical, continuous, and mixed uncertain data. Experimental results with simulated data and real-life data sets show that the proposed approach is better than the existing approaches for classification of uncertain data in terms of classification accuracy.

Simultaneously Spatiospectral Pattern Learning and Contaminated Trial Pruning for Electroencephalography-Based Brain Computer Interface

Electroencephalography (EEG)-based brain computer interfaces (BCIs) translate motor imagery commands into the movements of an external device (e.g., a robotic arm). The automatic design of spectral and spatial filters is a challenging task, as the frequency bands of the spectral filters must be predefined by previously published studies and given that they may be affected during trials by artifacts and improper motor imagery (MI). This study aimed to eliminate the contaminated trials automatically during classifier training, and to simultaneously learn the spectral and spatial patterns without the need for predefined frequency bands. Compared with previous studies that measured the discriminative power of a frequency band based on mutual information, this study determined the difference of the class conditional probability density function between two MI classes. This information was further shared to measure the contamination level of the trial that simplified the computation. A particle-based approximation technique iteratively constructed a filter bank that extracted discriminative features, and simultaneously removed potentially contaminated trials. The particle weight was estimated by an analysis of variance F-test instead of mutual information as commonly used in previous studies. The experimental results of a publicly available dataset revealed that the proposed method outperformed the other BCI in terms of the classification accuracy. Asymmetrical spatial patterns were found on left- versus right-hand MI classifications. The learnt spectral and spatial patterns were consistent with prior neurophysiological knowledge.

Learning from Positive and Unlabeled Data without Explicit Estimation of Class Prior

Learning a classifier from positive and unlabeled data may occur in various applications. It differs from the standard classification problems by the absence of labeled negative examples in the training set. So far, two main strategies have typically been used for this issue: the likely negative examplesbased strategy and the class prior-based strategy, in which the likely negative examples or the class prior is required to be obtained in a preprocessing step. In this paper, a new strategy based on the Bhattacharyya coefficient is put forward, which formalizes this learning problem as an optimization problem and does not need a preprocessing step. We first show that with the given positive class conditional probability density function (PDF) and the mixture PDF of both the positive class and the negative class, the class prior can be estimated by minimizing the Bhattacharyya coefficient of the positive class with respect to the negative class. We then show how to use this result in an implicit mixture model of restricted Boltzmann machines to estimate the positive class conditional PDF and the negative class conditional PDF directly to obtain a classifier without the explicit estimation of the class prior. Many experiments on real and synthetic datasets illustrated the superiority of the proposed approach.

Anomaly Detection for Controller Area Network in Braking Control System With Dynamic Ensemble Selection

The controller area networks (CAN) in the braking control system of metro trains are used to transmit the important control instruction and condition information, whose anomaly will endanger the security of trains running seriously. Due to the harsh work environment, there are various known and previously unknown fault types, current scheduled maintenance cannot detect early anomaly in time, and constructing an accurate and stable anomaly detector is a challenging task. In this paper, an anomaly detection approach is proposed to detect anomaly based on a dynamic ensemble selection system (DESS) without the expert knowledge, which involves two-class and one-class classifiers, and the base classifiers are trained with the network features extracted from the physical-layer information. To conduct the fusion, the support function of “distance-based” classifier is redefined as a class-conditional probability density function, and the source competence of base classifier is estimated by the entropy-based method in validate space and extended to entire decision space using the normalized Gaussian potential function. For different fault types, the competence classifiers are selected and the anomaly detection result is finally achieved by weighted majority voting. The comparative experiments are included in this paper to demonstrate the effectiveness and robustness in anomaly detection, including varying fault types.

Variational inference based bayes online classifiers with concept drift adaptation

We present VIGO, a novel online Bayesian classifier for both binary and multiclass problems. In our model, variational inference for multivariate distribution technique is exploited to approximate the class conditional probability density functions of data in an online manner. To handle concept drift that could arise in streaming data, we develop 2 new adaptive methods based on VIGO, which we called VIGOw and VIGOd. While VIGOw naturally adapts to any kind of changing environments, VIGOd maximises the benefit of a static environment as long as it does not detect any change. Extensive experiments on big/medium real-world/synthetic datasets demonstrate the superior performance of our algorithms over many state-of-the-art methods in the literature.

Drowsiness Detection by Bayesian-Copula Discriminant Classifier Based on EEG Signals During Daytime Short Nap.

Daytime short nap involves individual physiological states including alertness and drowsiness. In order to have a better understanding of the periodical rhymes of physiological states and then promote a good interpretability of alertness, the aim of this study is to detect drowsiness during daytime short nap. A method of Bayesian-copula discriminant classifier (BCDC) was introduced to detect individual drowsiness based on the physiological features extracted from electroencephalogram (EEG) signals. As an extension of traditional Bayesian decision theory, the BCDC method tries to construct the class-conditional probability density functions by exploiting the theory of copula and kernel density estimation. The proposed BCDC method was validated with experimental dataset and compared with other traditional methods for drowsiness detection. The obtained results showed that our method outperformed other methods in terms of three evaluation criteria. Our proposed method is effective to detect drowsiness with superior performance. Additionally, the BCDC method is relatively robust to different parameter settings on the group-level dataset. The proposed method is likely to be a useful tool to improve the correctness of the estimated class-conditional probability density functions. Since features are extracted from spontaneous EEG recordings, the results of this study can be further generalized to other experimental environment to detect vigilance level or driver drowsiness.

Non-stationary feature fusion of face and palmprint multimodal biometrics

Multimodal biometrics provides high performance in biometric recognition systems with respect to unibiometric systems as they offer a more universal approach, added security and better recognition accuracy. Moreover, data acquisition at the feature level brings out rich information from the traits, thus fusion of modalities at this level is desirable. In this paper we propose a novel fusion technique called non-stationary feature fusion where a new structure of interleaved matrix is constructed using local features extracted from two modalities i.e. face and palmprint images. A block based Discrete Cosine Transform (DCT) algorithm is used to construct a fused feature vector by extracting independent feature vectors from each spatial image. This fused feature vector contains nonlinear information that is used to train a Gaussian Mixture Models (GMM) based statistical model. The model provides accurate estimation of the class conditional probability density function of the fused feature vector. The proposed method produces recognition rates as high as 99.7% and 97% when tested on benchmark databases-ORL-PolyU and FERET-PolyU respectively. These rates are achieved using 23% low frequency DCT coefficients. The new technique is shown to outperform existing feature level fusion methods including methods based on matching and decision level fusion.

Neural network based class-conditional probability density function using kernel trick for supervised classifier

The practical limitation of the Bayes classifier used in pattern recognition is computing the class-conditional probability density function (pdf) of the vectors belonging to the corresponding classes. In this paper, a neural network based approach is proposed to model the class-conditional pdf, which can be further used in supervised classifier (e.g. Bayes classifier). It is also suggested to use kernel version (using kernel trick) of the proposed approach to obtain the class-conditional pdf of the corresponding training vectors in the higher dimensional space. This is used for better class separation and hence better classification rate is achieved. The performance of the proposed technique is validated by using the synthetic data and the real data. The simulation results show that the proposed technique on synthetic data and the real data performs well (in terms of classification accuracy) when compared with the classical Fisher׳s Linear Discriminant Analysis (LDA) and Gaussian based Kernel-LDA.

Mapping model validation metrics to subject matter expert scores for model adequacy assessment

This paper develops a novel approach to incorporate the contributions of both quantitative validation metrics and qualitative subject matter expert (SME) evaluation criteria in model validation assessment. The relationship between validation metrics (input) and SME scores (output) is formulated as a classification problem, and a probabilistic neural network (PNN) is constructed to execute this mapping. Establishing PNN classifiers for a wide variety of combinations of validation metrics allows for a quantitative comparison of validation metric performance in representing SME judgment. An advantage to this approach is that it semi-automates the model validation process and subsequently is capable of incorporating the contributions of large data sets of disparate response quantities of interest in model validation assessment. The effectiveness of this approach is demonstrated on a complex real-world problem involving the shock qualification testing of a floating shock platform. A data set of experimental and simulated pairs of time history comparisons along with associated SME scores and computed validation metrics is obtained and utilized to construct the PNN classifiers through K-fold cross validation. A wide range of validation metrics for time history comparisons is considered including feature-specific metrics (phase and magnitude error), a frequency metric (shock response spectra), a time-frequency metric (wavelet decomposition), and a global metric (index of agreement). The PNN classifiers constructed using a Parzen kernel for the class conditional probability density function whose smoothing parameter is optimized using a genetic algorithm performs well in representing SME judgment.

A speaker invariant speech recognition technique using HFCC features in isolated Hindi words

A speaker invariant speech recognition system is proposed by analysing the characteristics of speech signal. The distinctive features are derived from the speech data using discrete wavelet transforms (DWT) and human factor cepestral coefficient (HFCC) technique. This HFCC technique provides an immense impact on signal decoupling process for adjusting parameters in noise smoothing and spectral resolution. We have created a speech repository of 12 isolated Hindi words. The principal component analysis (PCA) is applied on speech features obtained from HFCC analysis in order to reduce the dimension of feature space. We have applied Bayes' decision rule for classification with multivariate normal distribution which follows the class conditional probability density function for each training classes. The performance of the classifier has been evaluated by calculating the misclassification error probability. Experimental results of proposed method are analysed and compared with the existing methods like MFCC with DWT, MFCC with PCA, DWT with PCA, etc. We have achieved promising classification results using HFCC-based speech features for speaker invariant speech identification system.

A Bayesian Model for Crowd Escape Behavior Detection

People naturally escape from a place when unexpected events happen. Based on this observation, efficient detection of crowd escape behavior in surveillance videos is a promising way to perform timely detection of anomalous situations. In this paper, we propose a Bayesian framework for escape detection by directly modeling crowd motion in both the presence and absence of escape events. Specifically, we introduce the concepts of potential destinations and divergent centers to characterize crowd motion in the above two cases respectively, and construct the corresponding class-conditional probability density functions of optical flow. Escape detection is finally performed based on the proposed Bayesian framework. Although only data associated with nonescape behavior are included in the training set, the density functions associated with the case of escape can also be adaptively updated using observed data. In addition, the identified divergent centers indicate possible locations at which the unexpected events occur. The performance of our proposed method is validated in a number of experiments on crowd escape detection in various scenarios.

SUPERVISED CLASSIFICATION OF POLARIMETRIC SAR IMAGERY USING TEMPORAL AND CONTEXTUAL INFORMATION

Abstract. Using the context as a source of ancillary information in classification process provides a powerful tool to obtain better class discrimination. Modelling context using Markov Random Fields (MRFs) and combining with Bayesian approach, a context-based supervised classification method is proposed. In this framework, to have a full use of the statistical a priori knowledge of the data, the spatial relation of the neighbouring pixels was used. The proposed context-based algorithm combines a Gaussian-based wishart distribution of PolSAR images with temporal and contextual information. This combination was done through the Bayes decision theory: the class-conditional probability density function and the prior probability are modelled by the wishart distribution and the MRF model. Given the complexity and similarity of classes, in order to enhance the class separation, simultaneously two PolSAR images from two different seasons (leaf-on and leaf-off) were used. According to the achieved results, the maximum improvement in the overall accuracy of classification using WMRF (Combining Wishart and MRF) compared to the wishart classifier when the leaf-on image was used. The highest accuracy obtained was when using the combined datasets. In this case, the overall accuracy of the wishart and WMRF methods were 72.66% and 78.95% respectively.

Bayesian classifiers based on probability density estimation and their applications to simultaneous fault diagnosis

A key characteristic of simultaneous fault diagnosis is that the features extracted from the original patterns are strongly dependent. This paper proposes a new model of Bayesian classifier, which removes the fundamental assumption of naive Bayesian, i.e., the independence among features. In our model, the optimal bandwidth selection is applied to estimate the class-conditional probability density function (p.d.f.), which is the essential part of joint p.d.f. estimation. Three well-known indices, i.e., classification accuracy, area under ROC curve, and probability mean square error, are used to measure the performance of our model in simultaneous fault diagnosis. Simulations show that our model is significantly superior to the traditional ones when the dependence exists among features.

Joint segmentation of collectively moving objects using a bag-of-words model and level set evolution

In scenes with collectively moving objects, to disregard the individual objects and take the entire group into consideration for motion characterization is a promising approach with wide application prospects. In contrast to studies on the segmentation of independently moving objects, our purpose is to construct a segmentation of these objects to characterize their motions at a macroscopic level. In general, the collectively moving objects in a group have very similar motion behavior with their neighbors and appear as a kind of global collective motion. This paper presents a joint segmentation approach for these collectively moving objects. In our model, we extract these macroscopic movement patterns based on optical flow field sequences. Specifically, a group of collectively moving objects correspond to a region where the optical flow field has high magnitude and high local direction coherence. As a result, our problem can be addressed by identifying these coherent optical flow field regions. The segmentation is performed through the minimization of a variational energy functional derived from the Bayes classification rule. Specifically, we use a bag-of-words model to generate a codebook as a collection of prototypical optical flow patterns, and the class-conditional probability density functions for different regions are determined based on these patterns. Finally, the minimization of our proposed energy functional results in the gradient descent evolution of segmentation boundaries which are implicitly represented through level sets. The application of our proposed approach is to segment and track multiple groups of collectively moving objects in a large variety of real-world scenes.

Bayes classifier based on tree-structured Gaussian mixtures

A novel approach is proposed to constructing a Bayes classifier in a multidimensional space of features by using tree-structured Gaussian mixtures as estimates of class-conditional probability density functions. A training procedure is developed for the classifier that is reduced to finding numbers of mixture components and their thresholds in order to realize rejections for the given classes. The mixture parameters are optimized by a cross-validation method. Classification error rate is estimated on a set of 3D vectors of textual features of a monochrome image. Comparative error rates are obtained for classifiers that use histograms, individual Gaussian densities, and Gaussian mixtures constructed using the EM (expectation-maximization) algorithm. The practical application of the developed classifier is illustrated by results of image segmentation for a satellite picture. The image represents a fragment of the Earth surface and it is obtained using the Google Earth program.

Fully automatic identification and discrimination of sperm’s parts in microscopic images of stained human semen smear

In the last years, digital image processing and analysis are used for computer assisted evaluation of semen quality with therapeutic goals or to estimate its fertility by means of spermatozoid motility and morphology. Sperm morphology is assessed routinely as part of standard laboratory analysis in the diagnosis of human male infertility. Nowadays assessments of sperm morphology are mostly done based on subjective criteria. In order to avoid subjectivity, numerous studies that incorporate image analysis techniques in the assessment of sperm morphology have been proposed. The primary step of all these methods is segmentation of sperm’s parts. In this paper, we have proposed a new method for segmentation of sperm’s Acrosome, Nucleus, Mid-piece and identification of sperm’s tail through some points which are placed on the sperm’s tail, accurately. These estimated points could be used to verify the morphological characteristics of sperm’s tail such as length, shape and etc. At first, sperm’s Acrosome, Nucleus and Mid-piece are segmented through a method based on a Bayesian classifier which utilizes the entropy based expectation–maximization (EM) algorithm and Markov random field (MRF) model to obtain and upgrade the class conditional probability density function (CCPDF) and the apriori probability of each class. Then, a pixel at the end of sperm’s Mid-piece, is selected as an initial point. To find other pixels which are placed on the sperm’s tail, structural similarity index (SSIM) is used in an iterative scheme. In order to stop the algorithm automatically at the end of sperm’s tail, local entropy is estimated and used as a feature to determine if a point is located on the sperm’s tail or not. To compare the performance of the proposed approach with those of previous approaches including manual segmentation, the Accuracy, Sensitivity and Specificity were calculated.

Retinal Blood Vessel Segmentation Using Gabor Wavelet and Line Operator

In this paper, we propose a method for segmenting blood vessels from retinal images. We extract two sets of features for image classification: features based on Gabor wavelet and line operator. At each pixel of retinal image we construct a feature vector consisting of the pixel intensity, four features from Gabor wavelet transform in different scales and two features from orthogonal line operators. We compare the result of classification using two classifiers: Bayesian and SVM. First we estimate class-conditional probability density functions for vessel and non-vessel using Gaussian mixture model. Then using a Bayesian classifier we implement a fast classification. The result of experiments show the combination of Gabor features and line features provides a good performance for vessel segmentation. We tested the proposed algorithm on DRIVE database which is publicly available. As the second classifier we employ Support Vector Machine. The results shows SVM classifier in some cases performs better than Bayesian classifier.

Segmentation of MS lesions using entropy-based EM algorithm and Markov random fields

This paper presents an approach for fully automatic segmentation of MS lesions in fluid attenuated inversion recovery (FLAIR) Magnetic Resonance (MR) images. The proposed method estimates a gaussian mixture model with three kernels as cerebrospinal fluid (CSF), normal tissue and Multiple Sclerosis lesions. To estimate this model, an automatic Entropy based EM algorithm is used to find the best estimated Model. Then, Markov random field (MRF) model and EM algorithm are utilized to obtain and upgrade the class conditional probability density function and the apriori probability of each class. After estimation of Model parameters and apriori probability, brain tissues are classified using bayesian classification. To evaluate the result of the proposed method, similarity criteria of different slices related to 20 MS patients are calculated and compared with other methods which include manual segmentation. Also, volume of segmented lesions are computed and compared with gold standard using correlation coefficient. The proposed method has better performance in comparison with previous works which are reported here.

Bayesian texture classification and retrieval based on multiscale feature vector

This paper proposes a supervised multiscale Bayesian texture classifier. The classifier exploits the dual-tree complex wavelet transform (DT-CWT) to obtain complex-valued multiscale representations of training texture samples for each texture class. The high-pass subbands of DT-CWT decomposition of a texture image are used to form a multiscale feature vector representing magnitude and phase features. For computational efficiency, the dimensionality of feature vectors is reduced using principal component analysis (PCA). The class conditional probability density function of low-dimensional feature vectors for each texture class is then estimated by using Parzen-window estimate with identical Gaussian kernels and is used to represent the texture class. A query texture image is classified as the corresponding texture class with the highest a posteriori probability according to a Bayesian inferencing. The superior performance and robustness of the proposed classifier is demonstrated for classifying texture images from image databases. The proposed multiscale texture feature vector extracted from both magnitude and phase of DT-CWT subbands of a query image is also shown to be effective for texture retrieval.