Multivariate Gaussian Mixture Model Research Articles

Gaussian process assisted coevolutionary estimation of distribution algorithm for computationally expensive problems

In order to reduce the computation of complex problems, a new surrogate-assisted estimation of distribution algorithm with Gaussian process was proposed. Coevolution was used in dual populations which evolved in parallel. The search space was projected into multiple subspaces and searched by sub-populations. Also, the whole space was exploited by the other population which exchanges information with the sub-populations. In order to make the evolutionary course efficient, multivariate Gaussian model and Gaussian mixture model were used in both populations separately to estimate the distribution of individuals and reproduce new generations. For the surrogate model, Gaussian process was combined with the algorithm which predicted variance of the predictions. The results on six benchmark functions show that the new algorithm performs better than other surrogate-model based algorithms and the computation complexity is only 10% of the original estimation of distribution algorithm.

Genotype Copy Number Variations using Gaussian Mixture Models: Theory and Algorithms

Copy number variations (CNVs) are important in the disease association studies and are usually targeted by most recent microarray platforms developed for GWAS studies. However, the probes targeting the same CNV regions could vary greatly in performance, with some of the probes carrying little information more than pure noise. In this paper, we investigate how to best combine measurements of multiple probes to estimate copy numbers of individuals under the framework of Gaussian mixture model (GMM). First we show that under two regularity conditions and assume all the parameters except the mixing proportions are known, optimal weights can be obtained so that the univariate GMM based on the weighted average gives the exactly the same classification as the multivariate GMM does. We then developed an algorithm that iteratively estimates the parameters and obtains the optimal weights, and uses them for classification. The algorithm performs well on simulation data and two sets of real data, which shows clear advantage over classification based on the equal weighted average.

A latent variables approach for clustering mixed binary and continuous variables within a Gaussian mixture model

For clustering objects, we often collect not only continuous variables, but binary attributes as well. This paper proposes a model-based clustering approach with mixed binary and continuous variables where each binary attribute is generated by a latent continuous variable that is dichotomized with a suitable threshold value, and where the scores of the latent variables are estimated from the binary data. In economics, such variables are called utility functions and the assumption is that the binary attributes (the presence or the absence of a public service or utility) are determined by low and high values of these functions. In genetics, the latent response is interpreted as the `liability' to develop a qualitative trait or phenotype. The estimated scores of the latent variables, together with the observed continuous ones, allow to use a multivariate Gaussian mixture model for clustering, instead of using a mixture of discrete and continuous distributions. After describing the method, this paper presents the results of both simulated and real-case data and compares the performances of the multivariate Gaussian mixture model and of a mixture of joint multivariate and multinomial distributions. Results show that the former model outperforms the mixture model for variables with different scales, both in terms of classification error rate and reproduction of the clusters means.

Initializing the EM algorithm in Gaussian mixture models with an unknown number of components

An approach is proposed for initializing the expectation–maximization (EM) algorithm in multivariate Gaussian mixture models with an unknown number of components. As the EM algorithm is often sensitive to the choice of the initial parameter vector, efficient initialization is an important preliminary process for the future convergence of the algorithm to the best local maximum of the likelihood function. We propose a strategy initializing mean vectors by choosing points with higher concentrations of neighbors and using a truncated normal distribution for the preliminary estimation of dispersion matrices. The suggested approach is illustrated on examples and compared with several other initialization methods.

The North Atlantic jet stream: a look at preferred positions, paths and transitions

AbstractPreferred jet stream positions and their link to regional circulation patterns over the winter North Atlantic/European sector are investigated to corroborate findings of multimodal behaviour of the jet positions and to analyse patterns of preferred paths and transition probabilities between jet regimes using ERA‐40 data. Besides the multivariate Gaussian mixture model, hierarchical clustering and data image techniques are used for this purpose. The different approaches all yield circulation patterns that correspond to the preferred jet regimes, namely the southern, central and the northern positions associated respectively with the Greenland anticyclone or blocking, and two opposite phases of an East Atlantic‐like flow pattern. Growth and decay patterns as well as preferred paths of the system trajectory are studied using the mixture model within the delay space. The analysis shows that the most preferred paths are associated with central to north and north to south jet stream transitions with a typical time‐scale of about 5 days, and with life cycles of 1–2 weeks. The transition paths are found to be consistent with transition probabilities. The analysis also shows that wave breaking seems to be the dominant mechanism behind Greenland blocking. Copyright © 2011 Royal Meteorological Society

Segmentation d’Images Texturées Couleur à l’aide de modèles paramétriques pour approche la distribution des erreurs de prédiction linéaires

We propose novel a priori parametric models to approximate the distribution of the two dimensional multichannel linear prediction error in order to improve the performance of color texture segmentation algorithms. Two dimensional linear prediction models are used to characterize the spatial structures in color images. The multivariate linear prediction error of these texture models is approximated with Wishart distribution and multivariate Gaussian mixture models. A novel color texture segmentation framework based on these models and a spatial regularization model of initial class label fields is presented. For the proposed method and with different color spaces, experimental results show better performances in terms of percentage segmentation error, in comparison with the use of a multivariate Gaussian law.

Parametric models of linear prediction error distribution for color texture and satellite image segmentation

In this article we present a Bayesian color texture segmentation framework based on the multichannel linear prediction error. Two-dimensional causal and non-causal real (in RGB color space) and complex (in IHLS and L ∗ a ∗ b ∗ color spaces) multichannel linear prediction models are used to characterize the spatial structures in color images. The main contribution of this segmentation methodology resides in the robust parametric approximations proposed for the multichannel linear prediction error distribution. These are composed of a unimodal approximation based on the Wishart distribution and a multimodal approximation based on the multivariate Gaussian mixture models. For the spatial regularization of the initial class label estimates, computed through the proposed parametric priors, we compare the conventional Potts model to a Potts model fusioned with a region size energy term. We provide performances of the method when using Iterated Conditional Modes algorithm and simulated annealing. Experimental results for the segmentation of synthetic color textures as well as high resolution QuickBird and IKONOS satellite images validate the application of this approach for highly textured images. Advantages of using these priors instead of classical Gaussian approximation and improved label field model are shown by these results. They also verify that the L ∗ a ∗ b ∗ color space exhibits better performance among the used color spaces, indicating its significance for the characterization of color textures through this approach.

Feature selection using factor analysis for Alzheimer's diagnosis using PET images

This article presents a computer-aided diagnosis technique for improving the accuracy of the early diagnosis of Alzheimer's disease (AD). Two hundred and ten 18F-FDG PET images from the ADNI initiative [52 normal controls (NC), 114 mild cognitive impairment (MCI), and 53 AD subjects] are studied. The proposed methodology is based on the selection of voxels of interest using the t-test and a posterior reduction of the feature dimension using factor analysis. Factor loadings are used as features for three different classifiers: Two multivariate Gaussian mixture model, with linear and quadratic discriminant function, and a support vector machine with linear kernel. An accuracy rate up to 95% when NC and AD are considered and an accuracy rate up to 88% and 86% for NC-MCI and NC-MCI,AD, respectively, are obtained using SVM with linear kernel. Results are compared to the voxel-as-features and a PCA- based approach and the proposed methodology achieves better classification performance.

A comparison of procedures for the calculation of forensic likelihood ratios from acoustic–phonetic data: Multivariate kernel density (MVKD) versus Gaussian mixture model–universal background model (GMM–UBM)

Two procedures for the calculation of forensic likelihood ratios were tested on the same set of acoustic–phonetic data. One procedure was a multivariate kernel density procedure (MVKD) which is common in acoustic–phonetic forensic voice comparison, and the other was a Gaussian mixture model–universal background model (GMM–UBM) which is common in automatic forensic voice comparison. The data were coefficient values from discrete cosine transforms fitted to second-formant trajectories of /aɪ/, /eɪ/, /oʊ/, /aʊ/, and /ɔɪ/ tokens produced by 27 male speakers of Australian English. Scores were calculated separately for each phoneme and then fused using logistic regression. The performance of the fused GMM–UBM system was much better than that of the fused MVKD system, both in terms of accuracy (as measured using the log-likelihood-ratio cost, C llr ) and precision (as measured using an empirical estimate of the 95% credible interval for the likelihood ratios from the different-speaker comparisons).

Joint Parsimonious Modeling and Model Order Selection for Multivariate Gaussian Mixtures

Multivariate Gaussian mixture models (GMMs) are widely for density estimation, model-based data clustering, and statistical classification. A difficult problem is estimating the model order, i.e., the number of mixture components, and model structure. Use of full covariance matrices, with number of parameters quadratic in the feature dimension, entails high model complexity, and thus may underestimate order, while naive Bayes mixtures may introduce model bias and lead to order overestimates. We develop a parsimonious modeling and model order and structure selection method for GMMs which allows for and optimizes over parameter tying configurations across mixture components applied to each individual parameter, including the covariates. We derive a generalized Expectation-Maximization algorithm for [(Bayesian information criterion (BIC)-based] penalized likelihood minimization. This, coupled with sequential model order reduction, forms our joint learning and model selection. Our method searches over a rich space of models and, consistent with minimizing BIC, achieves fine-grained matching of model complexity to the available data. We have found our method to be effective and largely robust in learning accurate model orders and parameter-tying structures for simulated ground-truth mixtures. We compared against naive Bayes and standard full-covariance GMMs for several criteria: 1) model order and structure accuracy (for synthetic data sets); 2) test set log-likelihood; 3) unsupervised classification accuracy; and 4) accuracy when class-conditional mixtures are used in a plug-in Bayes classifier. Our method, which chooses model orders intermediate between standard and naive Bayes GMMs, gives improved accuracy with respect to each of these performance measures.

On the Origin of Planetary-Scale Extratropical Winter Circulation Regimes

Abstract Sectorial and planetary-scale winter circulation regimes are studied and the relationship between them is investigated in order to find how much the simultaneous occurrence of sectorial regimes contributes to the occurrence of hemispheric regimes. The strategy is based on the multivariate Gaussian mixture model. The number of components in the model is estimated using two approaches. The first one is based on arguments from order statistics of the mixture proportions and the second uses a more severe test based on reproducibility. The procedure is applied next to the 500-hPa height field over the North Pacific, the North Atlantic, and the Northern Hemisphere using the empirical orthogonal function state space. Two highly significant regimes are found in each case, namely, the Pacific–North America (pattern) (±PNA)–North Atlantic Oscillation (±NAO) for the hemisphere—±PNA for the Pacific sector and ±NAO for the Atlantic sector. The sectorial regimes reflect mainly blocking and no-blocking flows. The results are tested further by applying a spatial clustering algorithm and are found to be consistent, particularly along the regime axes in the system state space. The relationship between hemispheric and sectorial circulation regimes is investigated. The data in each sector are first classified and then the times of simultaneous occurrence of sectorial regimes are identified. A new hemispheric dataset is then obtained by discarding maps corresponding to those co-occurrence times, and a new regime analysis is conducted. The results show that the hemispheric regime behavior has significantly decreased, suggesting that synchronization between sectorial circulation regimes could play an important role in the occurrence of planetary circulation regimes. The interannual variability of regime events is also discussed.

Is there a best color space for color image characterization or representation based on Multivariate Gaussian Mixture Model?

The aim of this work is the evaluation of Multivariate Gaussian Mixture Model (MGMM) in different color spaces (RGB, L∗ a∗ b∗ and YC1C2 issued from IHLS). We have analysed the approach for the description of the color information in an image on the one hand and for the representation of a color image on the other hand. To achieve this goal we need an accurate estimation method of MGMM parameter set firstly and distances for evaluating MGMMs secondly. A SEMmean-EM procedure and two distances between color images and synthesized images are then proposed. First distance is based on normalized histograms and second distance is based on a psychovisual distance between two colors. To compute these distances, two methods for synthezising images are proposed from a direct procedure of site classification: for each site, sample a color from its associated multivariate Gaussian law or attribute the mean color. The conclusion of our study is that estimation accuracy is better using L∗ a∗ b∗ color space than using RGB or YC1C2 and discrimination performances based on the two distances are also better in L∗ a∗ b∗ color space. A comparison between different criteria for choosing the number of components in the mixture is also done. Integrated Completed Likelihood criterion may be used if one only needs to characterize color information and Bayesian Information Criterion or ϕ β min criterion may be used if one needs to characterize color information along with color spatial localization.

Annotating novel genes by integrating synthetic lethals and genomic information

BackgroundLarge scale screening for synthetic lethality serves as a common tool in yeast genetics to systematically search for genes that play a role in specific biological processes. Often the amounts of data resulting from a single large scale screen far exceed the capacities of experimental characterization of every identified target. Thus, there is need for computational tools that select promising candidate genes in order to reduce the number of follow-up experiments to a manageable size.ResultsWe analyze synthetic lethality data for arp1 and jnm1, two spindle migration genes, in order to identify novel members in this process. To this end, we use an unsupervised statistical method that integrates additional information from biological data sources, such as gene expression, phenotypic profiling, RNA degradation and sequence similarity. Different from existing methods that require large amounts of synthetic lethal data, our method merely relies on synthetic lethality information from two single screens. Using a Multivariate Gaussian Mixture Model, we determine the best subset of features that assign the target genes to two groups. The approach identifies a small group of genes as candidates involved in spindle migration. Experimental testing confirms the majority of our candidates and we present she1 (YBL031W) as a novel gene involved in spindle migration. We applied the statistical methodology also to TOR2 signaling as another example.ConclusionWe demonstrate the general use of Multivariate Gaussian Mixture Modeling for selecting candidate genes for experimental characterization from synthetic lethality data sets. For the given example, integration of different data sources contributes to the identification of genetic interaction partners of arp1 and jnm1 that play a role in the same biological process.

Tropospheric Planetary Wave Dynamics and Mixture Modeling: Two Preferred Regimes and a Regime Shift

AbstractInvestigation of preferred structures of planetary wave dynamics is addressed using multivariate Gaussian mixture models. The number of components in the mixture is obtained using order statistics of the mixing proportions, hence avoiding previous difficulties related to sample sizes and independence issues. The method is first applied to a few low-order stochastic dynamical systems and data from a general circulation model. The method is next applied to winter daily 500-hPa heights from 1949 to 2003 over the Northern Hemisphere. A spatial clustering algorithm is first applied to the leading two principal components (PCs) and shows significant clustering. The clustering is particularly robust for the first half of the record and less for the second half. The mixture model is then used to identify the clusters. Two highly significant extratropical planetary-scale preferred structures are obtained within the first two to four EOF state space. The first pattern shows a Pacific–North American (PNA) pattern and a negative North Atlantic Oscillation (NAO), and the second pattern is nearly opposite to the first one. It is also observed that some subspaces show multivariate Gaussianity, compatible with linearity, whereas others show multivariate non-Gaussianity. The same analysis is also applied to two subperiods, before and after 1978, and shows a similar regime behavior, with a slight stronger support for the first subperiod. In addition a significant regime shift is also observed between the two periods as well as a change in the shape of the distribution. The patterns associated with the regime shifts reflect essentially a PNA pattern and an NAO pattern consistent with the observed global warming effect on climate and the observed shift in sea surface temperature around the mid-1970s.

Multivariate segregation analysis for quantitative traits in line crosses

Segregation analysis is a method of detecting major genes for quantitative traits without using marker information. It serves as an important tool in helping investigators to plan further studies such as quantitative trait loci mapping or more sophisticated genomic analyses. However, current methods of segregation analysis for a single trait typically have low statistical power. We propose a multivariate segregation analysis (MSA) that takes advantage of the correlation structure of multiple quantitative traits to detect major genes. This method not only increases the statistical power, but allows dissection of the genetic architecture underlying the trait complex. In MSA the observed phenotypes of multiple correlated traits are fitted to a multivariate Gaussian mixture model. Model parameters are estimated under the maximum likelihood framework via the expectation-maximization algorithm. The presence of major genes is tested using likelihood ratio test statistics. Pleiotropy is distinguished from close linkage by comparing three possible models using the Bayesian information criterion. Two simulation experiments were performed based on the F(2) mating design. In the first, the statistical properties of MSA under varying heritabilities and sample sizes were investigated and the results compared with those obtained from single-trait analysis. In the second simulation the efficacy of MSA in separating pleiotropy from close linkage was demonstrated. Finally, the new method was applied to real data and detected a major gene responsible for both plant height and tiller number in rice.

Quantitative Trait Associated Microarray Gene Expression Data Analysis

Selection on phenotypes may cause genetic change. To understand the relationship between phenotype and gene expression from an evolutionary viewpoint, it is important to study the concordance between gene expression and profiles of phenotypes. In this study, we use a novel method of clustering to identify genes whose expression profiles are related to a quantitative phenotype. Cluster analysis of gene expression data aims at classifying genes into several different groups based on the similarity of their expression profiles across multiple conditions. The hope is that genes that are classified into the same clusters may share underlying regulatory elements or may be a part of the same metabolic pathways. Current methods for examining the association between phenotype and gene expression are limited to linear association measured by the correlation between individual gene expression values and phenotype. Genes may be associated with the phenotype in a nonlinear fashion. In addition, groups of genes that share a particular pattern in their relationship to phenotype may be of evolutionary interest. In this study, we develop a method to group genes based on orthogonal polynomials under a multivariate Gaussian mixture model. The effect of each expressed gene on the phenotype is partitioned into a cluster mean and a random deviation from the mean. Genes can also be clustered based on a time series. Parameters are estimated using the expectation-maximization algorithm and implemented in SAS. The method is verified with simulated data and demonstrated with experimental data from 2 studies, one clusters with respect to severity of disease in Alzheimer's patients and another clusters data for a rat fracture healing study over time. We find significant evidence of nonlinear associations in both studies and successfully describe these patterns with our method. We give detailed instructions and provide a working program that allows others to directly implement this method in their own analyses.

Minimum phone error training of precision matrix models

Gaussian mixture models (GMMs) are commonly used as the output density function for large-vocabulary continuous speech recognition (LVCSR) systems. A standard problem when using multivariate GMMs to classify data is how to accurately represent the correlations in the feature vector. Full covariance matrices yield a good model, but dramatically increase the number of model parameters. Hence, diagonal covariance matrices are commonly used. Structured precision matrix approximations provide an alternative, flexible, and compact representation. Schemes in this category include the extended maximum likelihood linear transform and subspace for precision and mean models. This paper examines how these precision matrix models can be discriminatively trained and used on state-of-the-art speech recognition tasks. In particular, the use of the minimum phone error criterion is investigated. Implementation issues associated with building LVCSR systems are also addressed. These models are evaluated and compared using large vocabulary continuous telephone speech and broadcast news English tasks.

Multivariate Gaussian 함수를 이용한 센서 네트워크의 수화 인식에의 적용

Sensor networks are the results of convergence of very important technologies such as wireless communication and micro electromechanical systems. In recent years, sensor networks found a wide applicability in various fields such as health, environment and habitat monitoring, military, etc. A very important step for these many applications is pattern classification and recognition of data collected by sensors installed or deployed in different ways. But, pattern classification and recognition are sometimes difficult to perform. Systematic approach to pattern classification based on modern learning techniques like Multivariate Gaussian mixture models, can greatly simplify the process of developing and implementing real-time classification models. This paper proposes a new recognition system which is hierarchically composed of many sensor nodes having the capability of simple processing and wireless communication. The proposed system is able to perform classification of sensed data using the Multivariate Gaussian function. In order to verify the usefulness of the proposed system, it was applied to hand gesture recognition system.

Color texture analysis using the wavelet-based hidden Markov model

The wavelet-domain hidden Markov model (WD HMM), in particular the hidden Markov tree (HMT), has recently been proposed and applied to gray texture analysis with encouraging results. For color texture analysis, the previous WD HMM can only be used to model the different color planes individually, assuming they are independent of each other. However, this assumption in general is unrealistic. We show in this paper that the wavelet coefficients have certain inter-dependences between color planes. This paper presents a novel approach to modeling the dependences between color planes as well as the interactions across scales. In our approach, the wavelet coefficients at the same location, scale and sub-band, but different color planes are grouped into one vector. We then propose a multivariate Gaussian mixture model (MGMM) for approximating the marginal distribution of the wavelet coefficient vectors in one scale and capturing the interactions of different color planes. In addition, the statistical dependence between different scales is captured by the transition matrix of the hidden Markov tree. Using this approach, we can improve the performance of the WD HMM on the color texture classification. The experiments show that our WD HMM approach provides 85% of correct classifications (PCC) on 68 color textures from Oulu texture database and outperforms the other wavelet-based methods we study. In this paper, we also investigated the classification performance of the WD HMM methods on different color spaces.

Robust Reduced Rank Mixture Discriminant Analysis

ABSTRACT In the case of a large number of feature vector variables, using multivariate Gaussian mixture models, discrimination in a reduced subspace is studied, generalizing Hastie and Tibshirani's (1996) work, to a situation in which the outliers are present in the data. In the case of the Gaussian Mixture models, the reduced rank discriminant analysis is equivalent to the weighted rank k linear discriminant analysis (LDA). The reduced rank solution in the mixtures of multivariate Gaussian models was obtained from the full rank robust mixture solution. The classification in the new dimensions was compared with the discriminant analysis approach based on the original coordinates, using robust S-estimators. In most of the cases, the robust reduced rank mixture discriminant analysis (mda) performed better for the test data. However, for the case of common component covariance being diagonal, the robust reduced rank mixture discriminant analysis performed better than the robust full rank mixture discriminant analysis producing smaller errors in classification.