Standard Gaussian Mixture Model Research Articles

Effective fuzzy clustering algorithm with Bayesian model and mean template for image segmentation

Fuzzy c-means (FCMs) with spatial constraints have been considered as an effective algorithm for image segmentation. The well-known Gaussian mixture model (GMM) has also been regarded as a useful tool in several image segmentation applications. In this study, the authors propose a new algorithm to incorporate the merits of these two approaches and reveal some intrinsic relationships between them. In the authors model, the new objective function pays more attention on spatial constraints and adopts Gaussian distribution as the distance function. Thus, their model can degrade to the standard GMM as a special case. Our algorithm is fully free of the empirically pre-defined parameters that are used in traditional FCM methods to balance between robustness to noise and effectiveness of preserving the image sharpness and details. Furthermore, in their algorithm, the prior probability of an image pixel is influenced by the fuzzy memberships of pixels in its immediate neighbourhood to incorporate the local spatial information and intensity information. Finally, they utilise the mean template instead of the traditional hidden Markov random field (HMRF) model for estimation of prior probability. The mean template is considered as a spatial constraint for collecting more image spatial information. Compared with HMRF, their method is simple, easy and fast to implement. The performance of their proposed algorithm, compared with state-of-the-art technologies including extensions of possibilistic fuzzy c-means (PFCM), GMM, FCM, HMRF and their hybrid models, demonstrates its improved robustness and effectiveness.

An extension Gaussian mixture model for brain MRI segmentation.

The segmentation of brain magnetic resonance (MR) images into gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) has been an intensive studied area in the medical image analysis community. The Gaussian mixture model (GMM) is one of the most commonly used model to represent the intensity of different tissue types. However, as a histogram-based model, the spatial relationship between pixels is discarded in the GMM, making it sensitive to noise. Herein we present a new framework which aims to incorporate spatial information into the standard GMM, where each pixel is assigned its individual prior by leveraging its neighborhood information. Expectation maximization (EM) is modified to estimate the parameters of the proposed method. The method is validated on both synthetic and real brain MR images, showing its effectiveness in the segmentation task.

Modified Expectation Maximization Method for Automatic Segmentation of MR Brain Images

An automated method of MR Brain image segmentation is presented. A block based Expectation Maximization method is presented for the tissue classification of MR Brain images. The standard Gaussian Mixture Model is the most widely used method for MR Brain Image Segmentation and Expectation Maximization algorithm is used to estimate the model parameters. The Gaussian Mixture Model considers each pixel as independent and does not take into account the spatial correlation between the neighbouring pixels. Hence the segmentation result obtained using standard GMM is highly sensitive to Inensity Non-Uniformity and noise. The image is divided into blocks before applying EM since the GMM is preserved in the local image blocks. Also, Nonsubsampled Contourlet Transform is employed to incorporate the spatial correlation among the neighbouring pixels. The method is applied to the 12 MR Brain volumes of MRBRAINS13 test data and the White Matter, Gray Matter and CSF structures were segmented.

Text Segmentation from Images with Various Light Conditions Based on Gaussian Mixture Model

Standard Gaussian Mixture Model (GMM) is a well-known method for image segmentation. However, one of its problems is that we consider the pixel as independent to each other, which can cause the segmentation results sensitive to noise. It explains why some of existing algorithms still cannot segment texts from the background clearly. Therefore, we present a new method in which we incorporate the spatial relationship between a pixel and its neighbors inside 3x3 windows to segment the text. Our approach works well with images containing texts, which has different sizes, shapes or colors in case of light changes or complex background. Experimental results demonstrate the robustness, accuracy and effectiveness of the proposed model in image segmentation compared to other methods.

Sound texture recognition through dynamical systems modeling of empirical mode decomposition

This paper describes a system for modeling, recognizing, and classifying sound textures. The described system translates contemporary approaches from video texture analysis, creating a unique approach in the realm of audio and music. The signal is first represented as a set of mode functions by way of the Empirical Mode Decomposition technique for time/frequency analysis, before expressing the dynamics of these modes as a linear dynamical system (LDS). Both linear and nonlinear techniques are utilized in order to learn the system dynamics, which leads to a successful distinction between unique classes of textures. Five classes of sounds comprised a data set, consisting of crackling fire, typewriter action, rainstorms, carbonated beverages, and crowd applause, drawing on a variety of source recordings. Based on this data set the system achieved a classification accuracy of 90%, which outperformed both a Mel-Frequency Cepstral Coefficient based LDS-modeling approach from the literature, as well as one based on a standard Gaussian Mixture Model classifier.

Voice Conversion Using Dynamic Frequency Warping With Amplitude Scaling, for Parallel or Nonparallel Corpora

In Voice Conversion (VC), the speech of a source speaker is modified to resemble that of a particular target speaker. Currently, standard VC approaches use Gaussian mixture model (GMM)-based transformations that do not generate high-quality converted speech due to “over-smoothing” resulting from weak links between individual source and target frame parameters. Dynamic Frequency Warping (DFW) offers an appealing alternative to GMM-based methods, as more spectral details are maintained in transformation; however, the speaker timbre is less successfully converted because spectral power is not adjusted explicitly. Previous work combines separate GMM- and DFW-transformed spectral envelopes for each frame. This paper proposes a more effective DFW-based approach that (1) does not rely on the baseline GMM methods, and (2) functions on the acoustic class level. To adjust spectral power, an amplitude scaling function is used that compares the average target and warped source log spectra for each acoustic class. The proposed DFW with Amplitude scaling (DFWA) outperforms standard GMM and hybrid GMM-DFW methods for VC in terms of both speech quality and timbre conversion, as is confirmed in extensive objective and subjective testing. Furthermore, by not requiring time-alignment of source and target speech, DFWA is able to perform equally well using parallel or nonparallel corpora, as is demonstrated explicitly.

Gaussian-Mixture-Model-Based Spatial Neighborhood Relationships for Pixel Labeling Problem

In this paper, we present a new algorithm for pixel labeling and image segmentation based on the standard Gaussian mixture model (GMM). Unlike the standard GMM where pixels themselves are considered independent of each other and the spatial relationship between neighboring pixels is not taken into account, the proposed method incorporates this spatial relationship into the standard GMM. Moreover, the proposed model requires fewer parameters compared with the models based on Markov random fields. In order to estimate model parameters from observations, instead of utilizing an expectation-maximization algorithm, we employ gradient method to minimize a higher bound on the data negative log-likelihood. The performance of the proposed model is compared with methods based on both standard GMM and Markov random fields, demonstrating the robustness, accuracy, and effectiveness of our method.

Joint Parsimonious Modeling and Model Order Selection for Multivariate Gaussian Mixtures

Multivariate Gaussian mixture models (GMMs) are widely for density estimation, model-based data clustering, and statistical classification. A difficult problem is estimating the model order, i.e., the number of mixture components, and model structure. Use of full covariance matrices, with number of parameters quadratic in the feature dimension, entails high model complexity, and thus may underestimate order, while naive Bayes mixtures may introduce model bias and lead to order overestimates. We develop a parsimonious modeling and model order and structure selection method for GMMs which allows for and optimizes over parameter tying configurations across mixture components applied to each individual parameter, including the covariates. We derive a generalized Expectation-Maximization algorithm for [(Bayesian information criterion (BIC)-based] penalized likelihood minimization. This, coupled with sequential model order reduction, forms our joint learning and model selection. Our method searches over a rich space of models and, consistent with minimizing BIC, achieves fine-grained matching of model complexity to the available data. We have found our method to be effective and largely robust in learning accurate model orders and parameter-tying structures for simulated ground-truth mixtures. We compared against naive Bayes and standard full-covariance GMMs for several criteria: 1) model order and structure accuracy (for synthetic data sets); 2) test set log-likelihood; 3) unsupervised classification accuracy; and 4) accuracy when class-conditional mixtures are used in a plug-in Bayes classifier. Our method, which chooses model orders intermediate between standard and naive Bayes GMMs, gives improved accuracy with respect to each of these performance measures.

Including Probe-Level Measurement Error in Robust Mixture Clustering of Replicated Microarray Gene Expression

Probabilistic mixture models provide a popular approach to cluster noisy gene expression data for exploring gene function. Since gene expression data obtained from microarray experiments are often associated with significant sources of technical and biological noise, replicated experiments are typically used to deal with data variability, and internal replication (e.g. from multiple probes per gene in an experiment) provides valuable information about technical sources of noise. However, current implementations of mixture models either do not consider the correlation between the replicated measurements for the same experimental condition, or ignore the probe-level measurement error, and thus overlook the rich information about technical noise. Moreover, most current methods use non-robust Gaussian components to describe the data, and these methods are therefore sensitive to non-Gaussian clusters and outliers. In many cases, this will lead to over-estimation of the number of model components as multiple Gaussian components are used to fit a non-Gaussian cluster. We propose a robust Student's t-mixture model, which explicitly handles replicated gene expression data, includes the consideration of probe-level measurement error when available and automatically selects the appropriate number of model components using a minimum message length criterion. We apply the model to gene expression data using probe-level measurements from an Affymetrix probe-level model, multi-mgMOS, which provides uncertainty estimates. The proposed Student's t-mixture model shows robust performance on synthetic data sets with realistic noise characteristics in comparison to a standard Gaussian mixture model and two other previously published methods. We also compare performance with these methods on two yeast time-course data sets and show that the new method obtains more biologically meaningful clusters in terms of enrichment statistics for GO categories and interactions between transcription factors and genes. Automatically selecting the number of components is more computationally efficient than using a model selection approach and allows the methods to be applied to larger data sets.

Modeling Prosodic Features With Joint Factor Analysis for Speaker Verification

In this paper, we introduce the use of continuous prosodic features for speaker recognition, and we show how they can be modeled using joint factor analysis. Similar features have been successfully used in language identification. These prosodic features are pitch and energy contours spanning a syllable-like unit. They are extracted using a basis consisting of Legendre polynomials. Since the feature vectors are continuous (rather than discrete), they can be modeled using a standard Gaussian mixture model (GMM). Furthermore, speaker and session variability effects can be modeled in the same way as in conventional joint factor analysis. We find that the best results are obtained when we use the information about the pitch, energy, and the duration of the unit all together. Testing on the core condition of NIST 2006 speaker recognition evaluation data gives an equal error rate of 16.6% and 14.6%, with prosodic features alone, for all trials and English-only trials, respectively. When the prosodic system is fused with a state-of-the-art cepstral joint factor analysis system, we obtain a relative improvement of 8% (all trials) and 12% (English only) compared to the cepstral system alone.

Including probe-level uncertainty in model-based gene expression clustering.

BackgroundClustering is an important analysis performed on microarray gene expression data since it groups genes which have similar expression patterns and enables the exploration of unknown gene functions. Microarray experiments are associated with many sources of experimental and biological variation and the resulting gene expression data are therefore very noisy. Many heuristic and model-based clustering approaches have been developed to cluster this noisy data. However, few of them include consideration of probe-level measurement error which provides rich information about technical variability.ResultsWe augment a standard model-based clustering method to incorporate probe-level measurement error. Using probe-level measurements from a recently developed Affymetrix probe-level model, multi-mgMOS, we include the probe-level measurement error directly into the standard Gaussian mixture model. Our augmented model is shown to provide improved clustering performance on simulated datasets and a real mouse time-course dataset.ConclusionThe performance of model-based clustering of gene expression data is improved by including probe-level measurement error and more biologically meaningful clustering results are obtained.

Unsupervised Texture Classification and Segmentation

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non-Gaussian densities. The algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non-Gaussian structure of the data. This improves classification accuracy compared with standard Gaussian mixture models. When applied to textures, the algorithm can learn basis functions for images that capture the statistically significant structure intrinsic in the images. We apply this technique to the problem of unsupervised texture classification and segmentation. intrinsic structure of its image texture. In a mixture model, the observed data can be categorized into several mutually exclusive classes (3). When the data in each class are modeled as multivariate Gaussian, it is called a Gaussian mixture model. We generalize this by assuming that the data in each class are generated by a linear combination of independent, non-Gaussian sources, called as ICA mixture model. The algorithm for learning the parameters of the model uses gradient descent algorithm to maximize log likelihood function. We apply this learning algorithm to the problem of unsupervised classification and segmentation of textures. A large number of approaches for texture classification and segmentation have been suggested. Commonly, two types of approaches are distinguished, adapted respectively to macro- and microtextures, namely, the structural and statistical approaches. As far as the latter is concerned, we can site probabilistic methods based on texture modelling, statistical methods which characterize an image in terms of numerical attributes or features and new tools like neural networks, wavelets, multiresolution and multiscale approaches, and fuzzy modelling. A few methods also come from signal processing and seem to be promising: bidimensional autoregressive modelling and, time-frequency and time-scale representations. Claude.I, smolarz. A (17) focus on stochastic approaches and, specifically, on texture modelling by bidimensional autoregressive models (2D-AR models). They describe the AR model and propose a method for choosing an adapted neighbourhood and evaluation. Then, the segmentation algorithm is presented with the classification criterion and the contextual information.

Unsupervised image classification, segmentation, and enhancement using ICA mixture models

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non-Gaussian structure of the data. This improves classification accuracy compared with standard Gaussian mixture models. When applied to images, the algorithm can learn efficient codes (basis functions) for images that capture the statistically significant structure intrinsic in the images. We apply this technique to the problem of unsupervised classification, segmentation, and denoising of images. We demonstrate that this method was effective in classifying complex image textures such as natural scenes and text. It was also useful for denoising and filling in missing pixels in images with complex structures. The advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either Gaussian mixture models or standard independent component analysis (ICA) algorithms.

ICA mixture models for unsupervised classification of non-Gaussian classes and automatic context switching in blind signal separation

An unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the density of each class and is able to model class distributions with non-Gaussian structure. The new algorithm can improve classification accuracy compared with standard Gaussian mixture models. When applied to blind source separation in nonstationary environments, the method can switch automatically between classes, which correspond to contexts with different mixing properties. The algorithm can learn efficient codes for images containing both natural scenes and text. This method shows promise for modeling non-Gaussian structure in high-dimensional data and has many potential applications.