Posterior Class Probabilities Research Articles

A new formant feature and its application in Mandarin vowel pronunciation quality assessment

In order to improve the Mandarin vowel pronunciation quality assessment, a novel formant feature was proposed and applied to formant classification for Chinese Mandarin vowel pronunciation quality evaluation. Formant candidates of each frame were plotted on the time-frequency plane to form a bitmap, and its Gabor feature was extracted to represent the formant trajectory. The feature was then classified by using GMM model and the classification posterior probability was mapped to pronunciation quality grade. The experiments of comparing the Gabor transformation based formant trajectory feature with several other kinds of traditionally used features show that with this method, a human-machine scoring correlation coefficient (CC) of 0.842 can be achieved, which is better than the result of 0.832 by traditional speech recognition techniques. At the same time, considering that the long-term information of formant classification and the short-term information of speech recognition technique are complementary to each other, it is investigated to combine their results with linear or nonlinear methods to further improve the evaluation performance. As a result, experiments on PSK show that the best CC of 0.913, which is very close to the correlation of inter-human rating of 0.94, is gotten by using neural network.

Combining Multi-class SVMs with Linear Ensemble Methods that Estimate the Class Posterior Probabilities

Roughly speaking, there is one main model of pattern recognition support vector machine, with several variants of lower popularity. On the contrary, among the different multi-class support vector machines which can be found in the literature, none is clearly favoured. On the one hand, they exhibit distinct statistical properties. On the other hand, multiple comparative studies between multi-class support vector machines and decomposition methods have highlighted the fact that each model has its advantages and drawbacks. These observations call for the evaluation of combinations of multi-class support vector machines. In this article, we study the combination of multi-class support vector machines with linear ensemble methods. Their sample complexity is low, which should prevent them from overfitting, and the outputs of two of them are estimates of the class posterior probabilities.

Multiclass relevance units machine: benchmark evaluation and application to small ncRNA discovery.

BackgroundClassification is the problem of assigning each input object to one of a finite number of classes. This problem has been extensively studied in machine learning and statistics, and there are numerous applications to bioinformatics as well as many other fields. Building a multiclass classifier has been a challenge, where the direct approach of altering the binary classification algorithm to accommodate more than two classes can be computationally too expensive. Hence the indirect approach of using binary decomposition has been commonly used, in which retrieving the class posterior probabilities from the set of binary posterior probabilities given by the individual binary classifiers has been a major issue.MethodsIn this work, we present an extension of a recently introduced probabilistic kernel-based learning algorithm called the Classification Relevance Units Machine (CRUM) to the multiclass setting to increase its applicability. The extension is achieved under the error correcting output codes framework. The probabilistic outputs of the binary CRUMs are preserved using a proposed linear-time decoding algorithm, an alternative to the generalized Bradley-Terry (GBT) algorithm whose application to large-scale prediction settings is prohibited by its computational complexity. The resulting classifier is called the Multiclass Relevance Units Machine (McRUM).ResultsThe evaluation of McRUM on a variety of real small-scale benchmark datasets shows that our proposed Naïve decoding algorithm is computationally more efficient than the GBT algorithm while maintaining a similar level of predictive accuracy. Then a set of experiments on a larger scale dataset for small ncRNA classification have been conducted with Naïve McRUM and compared with the Gaussian and linear SVM. Although McRUM's predictive performance is slightly lower than the Gaussian SVM, the results show that the similar level of true positive rate can be achieved by sacrificing false positive rate slightly. Furthermore, McRUM is computationally more efficient than the SVM, which is an important factor for large-scale analysis.ConclusionsWe have proposed McRUM, a multiclass extension of binary CRUM. McRUM with Naïve decoding algorithm is computationally efficient in run-time and its predictive performance is comparable to the well-known SVM, showing its potential in solving large-scale multiclass problems in bioinformatics and other fields of study.

Intelligence-based condition monitoring model

Condition-based maintenance (CBM) is a maintenance strategy that reduces equipment downtime, production loss, and maintenance cost based on changes in equipment condition (e.g., changes in vibration, changes in power usage, changes in operating performance, changes in temperatures, changes in noise levels, changes in chemical composition, increase in debris content and changes in volume of material). In this study, we present the newly developed condition monitoring model (CMM) based on Bayesian decision theory, which takes vibration signals from the equipment, and classifies them to either normal or abnormal condition. Using conditional risk function, the equipment condition can be classified to either normal or abnormal condition. The conditional risk function is calculated based on loss table and the class posterior probabilities. The developed model can efficiently avoid unnecessary maintenance and make timely actions through analysing the received vibration signals from the equipment. An illustrative example is demonstrated to present the application of the model. Also, the results derived from CMM programme coded in Visual Basic are discussed.

CONSTRUCTION OF A CALIBRATED PROBABILISTIC CLASSIFICATION CATALOG: APPLICATION TO 50k VARIABLE SOURCES IN THE ALL-SKY AUTOMATED SURVEY

With growing data volumes from synoptic surveys, astronomers must become more abstracted from the discovery and introspection processes. Given the scarcity of follow-up resources, there is a particularly sharp onus on the frameworks that replace these human roles to provide accurate and well-calibrated probabilistic classification catalogs. Such catalogs inform the subsequent follow-up, allowing consumers to optimize the selection of specific sources for further study and permitting rigorous treatment of purities and efficiencies for population studies. Here, we describe a process to produce a probabilistic classification catalog of variability with machine learning from a multi-epoch photometric survey. In addition to producing accurate classifications, we show how to estimate calibrated class probabilities, and motivate the importance of probability calibration. We also introduce a methodology for feature-based anomaly detection, which allows discovery of objects in the survey that do not fit within the predefined class taxonomy. Finally, we apply these methods to sources observed by the All Sky Automated Survey (ASAS), and unveil the Machine-learned ASAS Classification Catalog (MACC), which is a 28-class probabilistic classification catalog of 50,124 ASAS sources. We estimate that MACC achieves a sub-20% classification error rate, and demonstrate that the class posterior probabilities are reasonably calibrated. MACC classifications compare favorably to the classifications of several previous domain-specific ASAS papers and to the ASAS Catalog of Variable Stars, which had classified only 24% of those sources into one of 12 science classes. The MACC is publicly available at http://www.bigmacc.info.

Cost-Sensitive Sequences of Bregman Divergences

The minimization of the empirical risk based on an arbitrary Bregman divergence is known to provide posterior class probability estimates in classification problems, but the accuracy of the estimate for a given value of the true posterior depends on the specific choice of the divergence. Ad hoc Bregman divergences can be designed to get a higher estimation accuracy for the posterior probability values that are most critical for a particular cost-sensitive classification scenario. Moreover, some sequences of Bregman loss functions can be constructed in such a way that their minimization guarantees, asymptotically, minimum number of errors in nonseparable cases, and maximum margin classifiers in separable problems. In this paper, we analyze general conditions on the Bregman generator to satisfy this property, and generalize the result for cost-sensitive classification.

POST-CLASSIFICATION APPROACH BASED ON GEOSTATISTICS TO REMOTE SENSING IMAGES : SPECTRAL AND SPATIAL INFORMATION FUSION

Abstract. Classification of remote sensing imagery provides an inexpensive yet efficient approach to land cover mapping. In supervised image classification, training samples are collected through certain sampling schemes, which are used to derive classification rules, aiming for adequate accuracy for the applications at hand. However, in conventional classification methods, the potential of training samples in terms of locational information is not tapped further, confounding the classification accuracy to the limited separability inherent to the given input feature vector. This paper explores two methods pertaining to geostatistics, i.e., simple kriging with local mean and cokriging, to predict class occurrences based on training samples' indicator transforms (location and classes) and spectrally derived class probabilities, thus calibrating the a posterior class probability vectors derived from initial spectral classification. The results showed that classification accuracy is significantly increased by these two methods for utilizing spatial information contained in training samples and initial spectral classification, compared with those obtainable with spectral classification. Moreover, the proposed methods constitute a valuable strategy for making fuller use of information residing in training data for improving spectrally derived classification, which is independent of the specific classifiers initially adopted for image classification.

Statistical approaches for farm and parasitic risk profiling in geographical veterinary epidemiology

We address the problem of farm and parasitic risk profiling in the context of Veterinary Epidemiology. We take advantage of a cross-sectional study carried out in the Campania Region in order to study the spatial distribution of 16 parasites in 121 ovine farms. We propose a tri-level hierarchical Bayesian model, which account for multivariate spatially structured overdispersion, to obtain estimate of posterior classification probabilities, that is for each parasite and farm the probability to belong to the set of the null hypothesis. We explore four decision rules based on either posterior probabilities or posterior means and compare the results in terms of the number of false discoveries/non-discoveries or the rate of false discovery/non-discovery. Our approach proved useful for parasitological risk profiling and we show that decision rules can be easily handled.

Application of Evidence Reasoning in Multi-Class Image Steganalysis

In steganalysis, it is important to ascertain the stego algorithm. In this paper, a blind steganalysis scheme based on evidence reasoning, which is capable of assinging stego images to known steganagraphic programs, is put forward. The evidences are extracted from the dct domain, pixel domain and dwt domain. The BPA function is obtained by transforming the svm based a posterior class probability. Three different scheme are tested and compared. Experimental results reveal the fact that our scheme is more effective than others and applying the evidence reasoning into steganalysis can integrate the advantages of different features, and improve the detection rate accordingly

Application of Evidence Reasoning in Multi-Class Image Steganalysis

In steganalysis, it is important to ascertain the stego algorithm. In this paper, a blind steganalysis scheme based on evidence reasoning, which is capable of assinging stego images to known steganagraphic programs, is put forward. The evidences are extracted from the dct domain, pixel domain and dwt domain. The BPA function is obtained by transforming the svm based a posterior class probability. Three different scheme are tested and compared. Experimental results reveal the fact that our scheme is more effective than others and applying the evidence reasoning into steganalysis can integrate the advantages of different features, and improve the detection rate accordingly

Hyperspectral Image Segmentation Using a New Bayesian Approach With Active Learning

This paper introduces a new supervised Bayesian approach to hyperspectral image segmentation with active learning, which consists of two main steps. First, we use a multinomial logistic regression (MLR) model to learn the class posterior probability distributions. This is done by using a recently introduced logistic regression via splitting and augmented Lagrangian algorithm. Second, we use the information acquired in the previous step to segment the hyperspectral image using a multilevel logistic prior that encodes the spatial information. In order to reduce the cost of acquiring large training sets, active learning is performed based on the MLR posterior probabilities. Another contribution of this paper is the introduction of a new active sampling approach, called modified breaking ties, which is able to provide an unbiased sampling. Furthermore, we have implemented our proposed method in an efficient way. For instance, in order to obtain the time-consuming maximum a posteriori segmentation, we use the α-expansion min-cut-based integer optimization algorithm. The state-of-the-art performance of the proposed approach is illustrated using both simulated and real hyperspectral data sets in a number of experimental comparisons with recently introduced hyperspectral image analysis methods.

Bayesian Hyperspectral Image Segmentation With Discriminative Class Learning

This paper introduces a new supervised technique to segment hyperspectral images: the Bayesian segmentation based on discriminative classification and on multilevel logistic (MLL) spatial prior. The approach is Bayesian and exploits both spectral and spatial information. Given a spectral vector, the posterior class probability distribution is modeled using multinomial logistic regression (MLR) which, being a discriminative model, allows to learn directly the boundaries between the decision regions and, thus, to successfully deal with high-dimensionality data. To control the machine complexity and, thus, its generalization capacity, the prior on the multinomial logistic vector is assumed to follow a componentwise independent Laplacian density. The vector of weights is computed via the fast sparse multinomial logistic regression (FSMLR), a variation of the sparse multinomial logistic regression (SMLR), conceived to deal with large data sets beyond the reach of the SMLR. To avoid the high computational complexity involved in estimating the Laplacian regularization parameter, we have also considered the Jeffreys prior, as it does not depend on any hyperparameter. The prior probability distribution on the class-label image is an MLL Markov–Gibbs distribution, which promotes segmentation results with equal neighboring class labels. The <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\alpha$</tex></formula> -expansion optimization algorithm, a powerful graph-cut-based integer optimization tool, is used to compute the maximum a posteriori segmentation. The effectiveness of the proposed methodology is illustrated by comparing its performance with the state-of-the-art methods on synthetic and real hyperspectral image data sets. The reported results give clear evidence of the relevance of using both spatial and spectral information in hyperspectral image segmentation.

Gaussian process classification: singly versus doubly stochastic models, and new computational schemes

The aim of this paper is to compare four different methods for binary classification with an underlying Gaussian process with respect to theoretical consistency and practical performance. Two of the inference schemes, namely classical indicator kriging and simplicial indicator kriging, are analytically tractable and fast. However, these methods rely on simplifying assumptions which are inappropriate for categorical class labels. A consistent and previously described model extension involves a doubly stochastic process. There, the unknown posterior class probability f(·) is considered a realization of a spatially correlated Gaussian process that has been squashed to the unit interval, and a label at position x is considered an independent Bernoulli realization with success parameter f(x). Unfortunately, inference for this model is not known to be analytically tractable. In this paper, we propose two new computational schemes for the inference in this doubly stochastic model, namely the “Aitchison Maximum Posterior” and the “Doubly Stochastic Gaussian Quadrature”. Both methods are analytical up to a final step where optimization or integration must be carried out numerically. For the comparison of practical performance, the methods are applied to storm forecasts for the Spanish coast based on wave heights in the Mediterranean Sea. While the error rate of the doubly stochastic models is slightly lower, their computational cost is much higher.

On the use of double cross-validation for the combination of proteomic mass spectral data for enhanced diagnosis and prediction

We consider a proteomic mass spectrometry case–control study for the calibration of a diagnostic rule for the detection of early-stage breast cancer. For each patient, a pair of two distinct mass spectra is recorded, each of which is derived from a different prior fractionation procedure on the available patient serum. We propose a procedure for combining the distinct spectral expressions from patients for the calibration of a diagnostic discriminant rule. This is achieved by first calibrating two distinct prediction rules separately, each on only one of the two available spectral data sources. A double cross-validatory approach is used to summarize the available spectral data using the two classifiers to posterior class probabilities, on which a combined predictor can be calibrated.

Accounting for uncertainty in extremal dependence modeling using Bayesian model averaging techniques

Modeling the joint tail of an unknown multivariate distribution can be characterized as modeling the tail of each marginal distribution and modeling the dependence structure between the margins. Classical methods for modeling multivariate extremes are based on the class of multivariate extreme value distributions. However, such distributions do not allow for the possibility of dependence at finite levels that vanishes in the limit. Alternative models have been developed that account for this asymptotic independence, but inferential statistical procedures seeking to combine the classes of asymptotically dependent and asymptotically independent models have been of limited use. We overcome these difficulties by employing Bayesian model averaging to account for both types of asymptotic behavior, and for subclasses within the asymptotically independent framework. Our approach also allows for the calculation of posterior probabilities of different classes of models, allowing for direct comparison between them. We demonstrate the use of joint tail models based on our broader methodology using two oceanographic datasets and a brief simulation study.

MCP2009 – 6th International Conference on Multiple Comparison Procedures

This Special Issue is for the 6th International Conference on Multiple Comparison Procedures (MCP 2009), which was held on 24–27 March 2009 at the Tokyo University of Science, Tokyo, Japan. At the conference approximately 250 research scientists from different subject matter areas, regulatory authorities and industrial practitioners from 20 different countries met to exchange new ideas on the theory and application of the multiple comparison procedures. The conference covered indeed a wide range of topics of multiple testing, estimation, monitoring and decision making. It was opened with keynote lectures by Yoav Benjamini and Kei Takeuchi and we are happy that both of them contributed to this Special Issue. These two keynote lectures were followed by 8 invited speakers, 75 regular oral presentations and 8 poster contributions. Benjamini gave a short review of historical trends in multiple comparisons research calling the previous decade a second golden era of this field. Various new methods for multiple comparisons have been developed in these years, motivated by the needs of applications as genome analysis, brain research as well as clinical trials. Undoubtedly, as Benjamini stated, innovative applications have been and will continue to be the driving force for the further development of multiple comparisons methodology. Takeuchi outlined fundamental ideas and concepts of multiple comparisons. As elaborated by Takeuchi, many of these ideas have been proposed already in the 1970s and had a great impact on the Japanese statistical community. Takeuchi proposed a general principle for multiple decision processes that he illustrated with a procedure to determine the sign and ordering of normal means. Takeuchi also discussed the bias arising in estimation after selection. Unfortunately, however, his work published in his seminal book (Takeuchi, 1973) did not spread beyond Japan since it was published in Japanese. We are therefore very pleased that Professor Takeuchi accepted our invitation to contribute a review of these results together with new methodology on sequential approaches. In addition to the contributions by the two keynote speakers, this Special Issue contains nine peer-reviewed papers out of 18 submissions. Seven of the accepted papers are from the field of clinical trials with four of them related to adaptive designs. Wang, Hung and O'Neil investigated the inflation of the family-wise error rate in adaptive designs with treatment selection at interim, where the final test strategy does not account for the number of candidate treatments at the first stage. In the papers by Luo, Wu and Xiong and Bebu, Luta and Dragalin new methods to address the problem of estimation after selection are proposed. Padmanabhan and Dragalin propose a combined D- and c-optimality criterion for adaptive dose ranging studies and improve the estimation of dose–response and the identification of the minimum effective dose. Hung and Wang discuss more generally the challenges in regulatory practices to multiple testing problems such as multiple doses and endpoints, non-inferiority and superiority, time of onset of a treatment effect and so on. Glimm and Läuter discuss a directional test for multiple endpoints in the two sample problem and propose two methods improving the classical approach. Imada and Yamamoto consider the same problem but in Dunnett's setting and propose the approximate likelihood ratio test approach. On the other hand Zhao, Wang and Cui discuss the basic consistency problems in multiple testing procedures (MTP). They propose a general framework consisting of several adjustment methods to construct strongly consonant and coherent MTP. Catelan, Lagazio and Biggeri applied the hierarchical Bayesian modeling to the problem of mapping the standard mortality ratio for lung cancer among males in Tuscany, Italy, and describe how to estimate the posterior classification probabilities. Through this editorial process, we were impressed by the high number and quality of submissions. Emphasis was put on the reproducible research aspect (Hothorn et al., 2009), and for many of the papers computer code is available as Supporting Information. We wish to thank all reviewers for completing the difficult task in a limited time. This Special Issue continues the tradition of the past MCP conferences (Tel Aviv, 1996; Berlin, 2000; Bethesda, 2002; Shanghai, 2005; Vienna, 2007), which resulted in special issues with selected papers from the meetings (Benjamini et al., 1999; Bauer et al., 2001; Westfall and Tamhane, 2003; Bretz et al., 2007; Posch et al., 2008). Apart from the scientific program of the MCP 2009 in Tokyo we cannot forget the Japanese traditional atmosphere around the conference venue as well as the very nice timing of flowering of the cherry blossoms. Finally, we look forward to the next MCP that will take place in Washington, DC, in September 2011.

A hierarchical Bayesian approach to multiple testing in disease mapping

We propose a Bayesian approach to multiple testing in disease mapping. This study was motivated by a real example regarding the mortality rate for lung cancer, males, in the Tuscan region (Italy). The data are relative to the period 1995–1999 for 287 municipalities. We develop a tri-level hierarchical Bayesian model to estimate for each area the posterior classification probability that is the posterior probability that the municipality belongs to the set of non-divergent areas. We show also the connections of our model with the false discovery rate approach. Posterior classification probabilities are used to explore areas at divergent risk from the reference while controlling for multiple testing. We consider both the Poisson-Gamma and the Besag, York and Mollié model to account for extra Poisson variability in our Bayesian formulation. Posterior inference on classification probabilities is highly dependent on the choice of the prior. We perform a sensitivity analysis and suggest how to rely on subject-specific information to derive informative a priori distributions. Hierarchical Bayesian models provide a sensible way to model classification probabilities in the context of disease mapping.

Fast Keypoint Recognition Using Random Ferns

While feature point recognition is a key component of modern approaches to object detection, existing approaches require computationally expensive patch preprocessing to handle perspective distortion. In this paper, we show that formulating the problem in a naive Bayesian classification framework makes such preprocessing unnecessary and produces an algorithm that is simple, efficient, and robust. Furthermore, it scales well as the number of classes grows. To recognize the patches surrounding keypoints, our classifier uses hundreds of simple binary features and models class posterior probabilities. We make the problem computationally tractable by assuming independence between arbitrary sets of features. Even though this is not strictly true, we demonstrate that our classifier nevertheless performs remarkably well on image data sets containing very significant perspective changes.

Bayesian adaptive nearest neighbor

AbstractThe k nearest neighbor classification (k‐NN) is a very simple and popular method for classification. However, it suffers from a major drawback, it assumes constant local class posterior probability. It is also highly dependent on and sensitive to the choice of the number of neighbors k. In addition, it severely lacks the desired probabilistic formulation. In this article, we propose a Bayesian adaptive nearest neighbor method (BANN) that can adaptively select the shape of the neighborhood and the number of neighbors k. The shape of the neighborhood is automatically selected according to the concentration of the data around each query point with the help of discriminants. The neighborhood size is not predetermined and is kept free using a prior distribution. Thus, we are able to make the model to select the appropriate neighborhood size. The model is fitted using Markov Chain Monte Carlo (MCMC), so we are not using exactly one neighborhood size but a mixture of k. Our BANN model is highly flexible, determining any local pattern in the data‐generating process, and adapting it to give an improved prediction. We have applied our model on four simulated data sets with special structures and five real‐life benchmark data sets. Our proposed BANN method demonstrates substantial improvement over k‐NN and discriminant adaptive nearest neighbor (DANN) in all nine case studies. It also outperforms the probabilistic nearest neighbor (PNN) in most of the data analyses. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 92‐105, 2010

LEARNING DECISION TREES WITH LOG CONDITIONAL LIKELIHOOD

In machine learning and data mining, traditional learning models aim for high classification accuracy. However, accurate class probability prediction is more desirable than classification accuracy in many practical applications, such as medical diagnosis. Although it is known that decision trees can be adapted to be class probability estimators in a variety of approaches, and the resulting models are uniformly called Probability Estimation Trees (PETs), the performances of these PETs in class probability estimation, have not yet been investigated. We begin our research by empirically studying PETs in terms of class probability estimation, measured by Log Conditional Likelihood (LCL). We also compare a PET called C4.4 with other representative models, including Naïve Bayes, Naïve Bayes Tree, Bayesian Network, KNN and SVM, in LCL. From our experiments, we draw several valuable conclusions. First, among various tree-based models, C4.4 is the best in yielding precise class probability prediction measured by LCL. We provide an explanation for this and reveal the nature of LCL. Second, compared with non tree-based models, C4.4 also performs best. Finally, LCL does not dominate another well-established relevant metric — AUC, which suggests that different decision-tree learning models should be used for different objectives. Our experiments are conducted on the basis of 36 UCI sample sets. We run all the models within a machine learning platform — Weka. We also explore an approach to improve the class probability estimation of Naïve Bayes Tree. We propose a greedy and recursive learning algorithm, where at each step, LCL is used as the scoring function to expand the decision tree. The algorithm uses Naïve Bayes created at leaves to estimate class probabilities of test samples. The whole tree encodes the posterior class probability in its structure. One benefit of improving class probability estimation is that both classification accuracy and AUC can be possibly scaled up. We call the new model LCL Tree (LCLT). Our experiments on 33 UCI sample sets show that LCLT outperforms all state-of-the-art learning models, such as Naïve Bayes Tree, significantly in accurate class probability prediction measured by LCL, as well as in classification accuracy and AUC.