Gaussian Mixture Model Estimation Research Articles

Gaussian Mixture Estimation from Lower-Dimensional Data with Application to PET Imaging

In positron emission tomography (PET), the original points of emission are unknown, and the scanners record pairs of photons emitting from those origins and creating lines of response (LORs) in random directions. This presents a latent variable problem, since at least one dimension of relevant information is lost. This can be solved by a statistical approach to image reconstruction—modeling the image as a Gaussian mixture model (GMM). This allows us to obtain a high-quality continuous model that is not computationally demanding and does not require postprocessing. In this paper, we propose a novel method of GMM estimation in the PET setting, directly from lines of response. This approach utilizes some well-known and convenient properties of the Gaussian distribution and the fact that the random slopes of the lines are independent from the points of origin. The expectation–maximization (EM) algorithm that is most commonly used to estimate GMMs in the traditional setting here is adapted to lower-dimensional data. The proposed estimation method is unbiased, and simulations and experiments show that accurate reconstruction on synthetic data is possible from relatively small samples.

Unsupervised incremental estimation of Gaussian mixture models with 1D split moves

In this paper, we propose a new type of split rule for incremental estimation of Gaussian Mixture Models with model selection. Split-based methods typically start with a mixture composed of a single component representing all data, and successively split and optimize components for a given model selection criterion. These algorithms are typically faster than alternatives, but depend critically on the component splitting method, since a good split rule promotes a faster convergence of the mixture optimization phase. We propose a new efficient and robust split rule that projects mixture components onto a 1D subspace and fits a two-component model to the projected data with the Expectation Maximization algorithm. The proposed approach is fast and robust to parameter tuning, being the ideal choice for applications that favor speed while still maintaining an acceptable accuracy. We illustrate the validity of the method through a series of experiments on synthetic and real datasets comparing the proposed method to alternatives of the state-of-the-art in terms of efficiency, accuracy, and sensitivity to parameter tuning.

BMI-CNV: a Bayesian framework for multiple genotyping platforms detection of copy number variants.

Whole-exome sequencing (WES) enables the detection of copy number variants (CNVs) with high resolution in protein-coding regions. However, variants in the intergenic or intragenic regions are excluded from studies. Fortunately, many of these samples have been previously sequenced by other genotyping platforms which are sparse but cover a wide range of genomic regions, such as SNP array. Moreover, conventional single sample-based methods suffer from a high false discovery rate due to prominent data noise. Therefore, methods for integrating multiple genotyping platforms and multiple samples are highly demanded for improved copy number variant detection. We developed BMI-CNV, a Bayesian Multisample and Integrative CNV (BMI-CNV) profiling method with data sequenced by both whole-exome sequencing and microarray. For the multisample integration, we identify the shared copy number variants regions across samples using a Bayesian probit stick-breaking process model coupled with a Gaussian Mixture model estimation. With extensive simulations, BMI-copy number variant outperformed existing methods with improved accuracy. In the matched data from the 1000 Genomes Project and HapMap project data, BMI-CNV also accurately detected common variants and significantly enlarged the detection spectrum of whole-exome sequencing. Further application to the data from The Research of International Cancer of Lung consortium (TRICL) identified lung cancer risk variant candidates in 17q11.2, 1p36.12, 8q23.1, and 5q22.2 regions.

Novel optimal time selection method for synthetic aperture radar data processing via expectation‐maximization algorithm

A novel optimal time selection method for synthetic aperture radar (SAR) data processing via the expectation-maximization (EM) algorithm is proposed, which can reduce the computational complexity and improve the image quality simultaneously. First, the Doppler frequency feature after the SAR processing is analyzed from the point of probability. Then, the Doppler frequency is modelled as the Gaussian mixture model (GMM), and the issue of optimal time selection can be implemented by the parameter estimation of GMM. The results of simulated and real measured data are given to demonstrate the effectiveness of the proposed method.

Expectation‐Maximization Algorithm of Gaussian Mixture Model for Vehicle‐Commodity Matching in Logistics Supply Chain

A vehicle‐commodity matching problem (VCMP) is presented for service providers to reduce the cost of the logistics system. The vehicle classification model is built as a Gaussian mixture model (GMM), and the expectation‐maximization (EM) algorithm is designed to solve the parameter estimation of GMM. A nonlinear mixed‐integer programming model is constructed to minimize the total cost of VCMP. The matching process between vehicle and commodity is realized by GMM‐EM, as a preprocessing of the solution. The design of the vehicle‐commodity matching platform for VCMP is designed to reduce and eliminate the information asymmetry between supply and demand so that the order allocation can work at the right time and the right place and use the optimal solution of vehicle‐commodity matching. Furthermore, the numerical experiment of an e‐commerce supply chain proves that a hybrid evolutionary algorithm (HEA) is superior to the traditional method, which provides a decision‐making reference for e‐commerce VCMP.

Personalized Power Consumption Control Method for Intelligent Buildings That Perceiving Speech Recognition Identity

With the development of IoT technology, the meaning of the concept of energy conservation and environmental protection in people’s minds is also constantly changing, and it is moving towards the direction of intelligence and personalization. At present, China’s economy has entered the era of artificial intelligence. Electricity control such as building lighting has gradually changed from a single manual control to a multi-modal control. This paper designs an embedded control method based on speech recognition of personnel to achieve personalized power consumption, and improves the existing defects of intelligent power consumption in buildings such as unconfirmed operators and single decision-making factors.The method is to extract the mel frequency cepstrum coefficient feature vector after preprocessing the speech signal; Aiming at the problem of singularity matrix in the parameter estimation of Gaussian mixture model, the modified EM algorithm was used to optimize the parameter estimation of the Gaussian mixture model to complete the identification of personnel. The experimental test results show that the system can control the electricity facilities in combination with the environmental information and realize the personalized lighting control function while confirming the identity of the personnel.

Adaptive parameter estimation of GMM and its application in clustering

Parameter estimation of Gaussian mixture model (GMM) has begun to gain attention in the field of science and engineering, and it has gradually become one of the most popular research topics. In particular, the rapid development of theoretical progress on globally optimal convergence promotes the widespread application of Gaussian mixtures in data clustering. So this paper introduces a novel parameter estimation algorithm called TDAVBEM, which combines the Tsallis entropy and a deterministic annealing (DA) algorithm on the basis of the variational bayesian expected maximum (VBEM) to simultaneously implement the parameter estimation and select the optimal components of GMM. We experimentally certified the effectiveness and robustness of our proposed algorithm passes through comparing it with several parameter evaluation methods and its application in data clustering.

Estimation of Gaussian mixture models via tensor moments with application to online learning

In this paper, we present an alternating gradient descent algorithm for estimating parameters of a spherical Gaussian mixture model by the method of moments (AGD-MoM). We formulate the problem as a constrained optimisation problem which simultaneously matches the third order moments from the data, represented as a tensor, and the second order moment, which is the empirical covariance matrix. We derive the necessary gradients (and second derivatives), and use them to implement alternating gradient search to estimate the parameters of the model. We show that the proposed method is applicable in both a batch as well as in a streaming (online) setting. Using synthetic and benchmark datasets, we demonstrate empirically that the proposed algorithm outperforms the more classical algorithms like Expectation Maximisation and variational Bayes.

Adaptive Sparse Group Variable Selection for a Robust Mixture Regression Model Based on Laplace Distribution

The traditional estimation of Gaussian mixture model is sensitive to heavy-tailed errors; thus we propose a robust mixture regression model by assuming that the error terms follow a Laplace distribution in this article. And for the variable selection problem in our new robust mixture regression model, we introduce the adaptive sparse group Lasso penalty to achieve sparsity at both the group-level and within-group-level. As numerical experiments show, compared with other alternative methods, our method has better performances in variable selection and parameter estimation. Finally, we apply our proposed method to analyze NBA salary data during the period from 2018 to 2019.

Improved EXIT Algorithm Based on Gaussian Mixture Model and its Application to LDPC Construction in Coding Cooperative Systems With Hybrid Fading

In this study, an EXIT algorithm based on Gaussian mixture coding and multi-dimensional node information calculation is proposed. The objective is to solve the problem of performance analysis of cooperative communication and LDPC code construction under hybrid fading and impulse noise. First, Gaussian mixture model estimation of the decoding initialization information is performed by the expectation maximized algorithm. In addition, key parameters, such as weight, mean, and variance of the Gaussian mixture model are obtained. Then, the multi-dimensional information processing is performed on the variable node and the check node information in the traditional EXIT algorithm based on the multi-dimensional Gaussian mixture model and system symmetry analysis. Finally, the objective function of LDPC code optimization is constructed, the constraints are given, and the optimal LDPC code degree distribution is obtained by the differential evolution algorithm. The simulation results show that compared with the traditional regular and other irregular LDPC codes, the newly constructed degree distribution has better performance in waterfall characteristics and error floor characteristics. Such distribution provides a necessary method to analyze and optimize the performance of cooperative coding system under hybrid media or hybrid fading.

Stochastic Approximation Algorithm with Randomization at the Input for Unsupervised Parameters Estimation of Gaussian Mixture Model with Sparse Parameters

We consider the possibilities of using stochastic approximation algorithms with randomization on the input under unknown but bounded interference in studying the clustering of data generated by a mixture of Gaussian distributions. The proposed algorithm, which is robust to external disturbances, allows us to process the data “on the fly” and has a high convergence rate. The operation of the algorithm is illustrated by examples of its use for clustering in various difficult conditions.

Bridging Mixture Model Estimation and Information Bounds Using I-MMSE

We derive bounds on mutual information for arbitrary estimation problems in additive noise, modeled using Gaussian mixtures. Previous work exploiting the I-minimum-mean-squared-error (MMSE) formula to formulate a bridge between bounds on the MMSE for Gaussian mixture model estimation problems and bounds on the mutual information are generalized to allow arbitrary noise modeling. A novel upper bound on estimation information is also developed for the general estimation case. In addition, limits are analyzed to develop bounds on arbitrary entropy, asymptotic behavior of all bounds, and bound errors with some results bridged back to the MMSE domain.

A data driven equivariant approach to constrained Gaussian mixture modeling

Maximum likelihood estimation of Gaussian mixture models with different class-specific covariance matrices is known to be problematic. This is due to the unboundedness of the likelihood, together with the presence of spurious maximizers. Existing methods to bypass this obstacle are based on the fact that unboundedness is avoided if the eigenvalues of the covariance matrices are bounded away from zero. This can be done imposing some constraints on the covariance matrices, i.e. by incorporating a priori information on the covariance structure of the mixture components. The present work introduces a constrained approach, where the class conditional covariance matrices are shrunk towards a pre-specified target matrix $$\varvec{\varPsi }.$$ Data-driven choices of the matrix $$\varvec{\varPsi },$$ when a priori information is not available, and the optimal amount of shrinkage are investigated. Then, constraints based on a data-driven $$\varvec{\varPsi }$$ are shown to be equivariant with respect to linear affine transformations, provided that the method used to select the target matrix be also equivariant. The effectiveness of the proposal is evaluated on the basis of a simulation study and an empirical example.

Robust image reconstruction enhancement based on Gaussian mixture model estimation

The low quality of an image is often characterized by low contrast and blurred edge details. Gradients have a direct relationship with image edge details. More specifically, the larger the gradients, the clearer the image details become. Robust image reconstruction enhancement based on Gaussian mixture model estimation is proposed here. First, image is transformed to its gradient domain, obtaining the gradient histogram. Second, the gradient histogram is estimated and extended using a Gaussian mixture model, and the predetermined function is constructed. Then, using histogram specification technology, the gradient field is enhanced with the constraint of the predetermined function. Finally, a matrix sine transform-based method is applied to reconstruct the enhanced image from the enhanced gradient field. Experimental results show that the proposed algorithm can effectively enhance different types of images such as medical image, aerial image, and visible image, providing high-quality image information for high-level processing.

Generative part-based Gabor object detector

Discriminative part-based models have become the approach for visual object detection. The models learn from a large number of positive and negative examples with annotated class labels and location (bounding box). In contrast, we propose a part-based generative model that learns from a small number of positive examples. This is achieved by utilizing “privileged information”, sparse class-specific landmarks with semantic meaning. Our method uses bio-inspired complex-valued Gabor features to describe local parts. Gabor features are transformed to part probabilities by unsupervised Gaussian Mixture Model (GMM). GMM estimation is robustified for a small amount of data by a randomization procedure inspired by random forests. The GMM framework is also used to construct a probabilistic spatial model of part configurations. Our detector is invariant to translation, rotation and scaling. On part level invariance is achieved by pose quantization which is more efficient than previously proposed feature transformations. In the spatial model, invariance is achieved by mapping parts to an “aligned object space”. Using a small number of positive examples our generative method performs comparably to the state-of-the-art discriminative method.

Compact and information loss‐bounded estimation of Gaussian mixture model for 3D spatial representation

A new Gaussian mixture model (GMM) estimation technique is presented for three-dimensional (3D) spatial representation. The GMM generated by the proposed technique is compact with bounded information loss as a result of using robust estimators and the Kullback-Leibler divergence-based Gaussian mixture reduction method. In addition, the proposed technique is not only robust to outliers, but quite close to invariant under similarity transformation. Experiments have demonstrate that the compactness and the consistency of the GMM are improved compared with existing 3D spatial representation models.

Impact of Quality-Based Fusion Techniques for Video-Based Iris Recognition at a Distance

In this paper, we consider the problem of iris recognition in the context of video-based distant acquisition. We propose several systems aiming at improving the poor performance resulting from image degradations (low resolution, blur, and lack of texture) obtained from such acquisitions. Our approach is based on simple super-resolution techniques applied at the pixel level on the different frames of a video, improved by considering some quality criteria. Our main novelty is the introduction of a local quality measure in the fusion scheme. This measure relies on a gaussian mixture model estimation of clean iris texture distribution. It can also be used to compute a global quality measure of the normalized iris image which can be used either for the selection of the best images in a sequence or in the fusion scheme. Extensive experiments on the QFIRE database at different acquisition distances (5, 7, and 11 ft) show the big improvement brought by the use of the global quality for both scenarios. Moreover, the local quality-based fusion scheme further increases the performance due to its ability to consider locally the different parts of the image, and therefore, to discard poorly segmented pixels in the fusion.

Maximum likelihood estimation of Gaussian mixture models without matrix operations

The Gaussian mixture model (GMM) is a popular tool for multivariate analysis, in particular, cluster analysis. The expectation---maximization (EM) algorithm is generally used to perform maximum likelihood (ML) estimation for GMMs due to the M-step existing in closed form and its desirable numerical properties, such as monotonicity. However, the EM algorithm has been criticized as being slow to converge and thus computationally expensive in some situations. In this article, we introduce the linear regression characterization (LRC) of the GMM. We show that the parameters of an LRC of the GMM can be mapped back to the natural parameters, and that a minorization---maximization (MM) algorithm can be constructed, which retains the desirable numerical properties of the EM algorithm, without the use of matrix operations. We prove that the ML estimators of the LRC parameters are consistent and asymptotically normal, like their natural counterparts. Furthermore, we show that the LRC allows for simple handling of singularities in the ML estimation of GMMs. Using numerical simulations in the R programming environment, we then demonstrate that the MM algorithm can be faster than the EM algorithm in various large data situations, where sample sizes range in the tens to hundreds of thousands and for estimating models with up to 16 mixture components on multivariate data with up to 16 variables.

GroupTracker: Video tracking system for multiple animals under severe occlusion

Quantitative analysis of behaviors shown by interacting multiple animals can provide a key for revealing high-order functions of their nervous systems. To resolve these complex behaviors, a video tracking system that preserves individual identity even under severe overlap in positions, i.e., occlusion, is needed. We developed GroupTracker, a multiple animal tracking system that accurately tracks individuals even under severe occlusion. As maximum likelihood estimation of Gaussian mixture model whose components can severely overlap is theoretically an ill-posed problem, we devised an expectation–maximization scheme with additional constraints on the eigenvalues of the covariance matrix of the mixture components. Our system was shown to accurately track multiple medaka (Oryzias latipes) which freely swim around in three dimensions and frequently overlap each other. As an accurate multiple animal tracking system, GroupTracker will contribute to revealing unexplored structures and patterns behind animal interactions. The Java source code of GroupTracker is available at https://sites.google.com/site/fukunagatsu/software/group-tracker.

Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems

A new expectation maximization (EM) algorithm for time-critical supervised classification tasks in remote sensing is proposed. Compared to standard EM and other approaches, it has the following advantages: (1) No knowledge about the class distributions is needed. (2) The number of components is estimated. (3) It does not require careful initialization. (4) Singular estimates are avoided due to the ability of pruning components. (5) The best discriminating features are identified simultaneously. (6) The features are identified by incorporating Mahalanobis distances.Three experiments are carried out in order to demonstrate the relevance of the method. The main findings are the following: (1) Feature selection is very important in terms of prediction quality of models. (2) In the experiments the proposed method estimates better models than other state-of-the-art methods. (3) The incorporation of Mahalanobis distances is very valuable for the identification of relevant features. (4) The proposed method is more robust than the compared methods. (5) In case of complex data distributions the new approach is able to provide better results.