Parameters Of Gaussian Mixture Model Research Articles

Parameter Estimation of a Gaussian Mixture Model for Wind Power Forecast Error by Riemann L-BFGS Optimization

With the increasing penetration of wind power into the electricity grid, wind power forecast error analysis plays an important role in operations scheduling. To better describe the characteristics of power forecast error, a probability density function should be established. Compared with the Kernel density estimation method, this paper adopts the Gaussian mixture model (GMM), which is flexible enough to capture different error distribution characteristics, such as bias, heavy tail, multi-peak, and so on. In addition, for GMM parameter estimation, when dealing with a large number of multi-dimensional data sets or unbalanced overlapping mixtures, the expectation maximization (EM) algorithm shows a slower convergence speed and requires a high number of iterations. In this paper, a new L-BFGS optimization method, based on the Riemannian manifold, is used for GMM parameter estimation. Based on actual wind power forecast error data, the suitability of the model and the new optimization algorithm was verified in large, multi-dimensional data sets. The new optimization algorithm has fewer iterations than the EM algorithm, with an improved convergence speed.

Condition assessment of cables by pattern recognition of vehicle-induced cable tension ratio

The stay cables are one of most critical elements for cable-stayed bridges. This paper proposes a machine-learning based condition assessment method for stay cables by using the monitored cable tension force. First, based on the correlation of cable tension response between cable pairs (defined as the two cables at the upriver side and the opposite downriver side in the double cable planes), cable tension ratio is extracted as the feature variable, and the cable tension ratio is defined as the ratio of vehicle-induced cable tension between a cable pair. It is found that cable tension ratio is only related with cable properties and the transverse position of a vehicle over the deck. Vehicles on the bridge naturally cluster themselves into a few clusters that correspond to the traffic lanes, i.e. the vehicles in one lane form a cluster. Consequently, the vehicle-induced cable tension ratio forms the corresponding clusters or patterns. Gaussian Mixture Model (GMM) is employed for modelling the patterns of cable tension ratio, and each pattern (corresponds to a certain traffic lane) is modelled by a mono-Gaussian distribution. The Gaussian distribution parameters of tension ratio are used as condition indicator of stay cables because they are only related to cable conditions (the information of vehicle transverse location is presented in the number of tension ration patterns). The number of patterns which represents the model complexity are determined by Bayesian Information Criteria (BIC), while other parameters of GMM are estimated by using Expectation-Maximization algorithm under the Maximum Likelihood criteria, based on the monitored cable tension force. The cable condition is then evaluated according to the variation in estimated parameters of GMM. It is noted that pre-process of source separation is conducted to make the cable tension ratio independent from vehicle weight, environmental variant, and possible sensor errors. An FE model analysis is carried out to qualitatively illustrate the principle of the proposed method and physical sense of the cable tension ratio.

Hierarchical Bayesian combination of plug-in maximum a posteriori decoders in deep neural networks-based speech recognition and speaker adaptation

We propose a novel decoding framework by dynamically combining K multiple plug-in maximum a posteriori (MAP) decoders, with each solving for a sequence of symbols in a state-by-state manner in time and according to a set of constraints on the symbol sequences in space. The score combination occurs at the state level with the set of K combination weights either chosen to be equal (i.e., equal weighting scheme) or learned from a collection of data through a hierarchical Bayesian setting. When applied to automatic speech recognition (ASR), leveraging upon some characteristic differences in computing acoustic probabilities with both feed-forward deep neural networks (DNNs) and Gaussian mixture models (GMMs) at the hidden Markov phone state level, these scores can be discriminatively combined in plug-in MAP decoding. The DNN and GMM parameters can be trained from a large collection of speaker-independent (SI) speech data and further refined with a small set of speaker adaptation (SA) utterances. The per-speaker, per-state combination weights can be learned from SA data through the proposed hierarchical Bayesian approach. Experimental results on the Switchboard ASR task show that an ad hoc fixed-weight combination already reduces the word error rate (WER) to 16.9% from a SI WER of 17.4%. Model adaptation with 20 utterances can reduce the WER to 16.7%, which is further reduced to 16.1% using the SA models and fixed-weight combination decoding. The best WER of 15.3% is attained by using the proposed hierarchical Bayesian learned weights combining the two SA and two SI systems. Finally, we contrast the proposed technique with a state-of-the-art static system combination approach based on multiple word lattices generated by different ASR systems, and minimum Bayes risk. The experimental results demonstrate that static system combination cannot boost system performance of the individual systems, and the proposed dynamic combination scheme is needed.

Using cystoscopy to segment bladder tumors with a multivariate approach in different color spaces.

Nowadays the diagnosis of bladder lesions relies upon cystoscopy examination and depends on the interpreter's experience. State of the art of bladder tumor identification are based on 3D reconstruction, using CT images (Virtual Cystoscopy) or images where the structures are exalted with the use of pigmentation, but none uses white light cystoscopy images. An initial attempt to automatically identify tumoral tissue was already developed by the authors and this paper will develop this idea. Traditional cystoscopy images processing has a huge potential to improve early tumor detection and allows a more effective treatment. In this paper is described a multivariate approach to do segmentation of bladder cystoscopy images, that will be used to automatically detect and improve physician diagnose. Each region can be assumed as a normal distribution with specific parameters, leading to the assumption that the distribution of intensities is a Gaussian Mixture Model (GMM). Region of high grade and low grade tumors, usually appears with higher intensity than normal regions. This paper proposes a Maximum a Posteriori (MAP) approach based on pixel intensities read simultaneously in different color channels from RGB, HSV and CIELab color spaces. The Expectation-Maximization (EM) algorithm is used to estimate the best multivariate GMM parameters. Experimental results show that the proposed method does bladder tumor segmentation into two classes in a more efficient way in RGB even in cases where the tumor shape is not well defined. Results also show that the elimination of component L from CIELab color space does not allow definition of the tumor shape.

A real-time CFAR thresholding method for target detection in hyperspectral images

In order to support immediate decision-making in critical circumstances such as military reconnaissance and disaster rescue, real-time onboard implementation of target detection is greatly desired. In this paper, a real-time thresholding method (RT-THRES) is proposed to obtain the constant false alarm rate (CFAR) thresholds for target detection in real-time circumstances. RT-THRES utilizes Gaussian mixture model (GMM) to track and fit the distribution of the target detector's outputs. GMM is an extension to Gaussian probability density function, which could approximate any distribution smoothly. In this method, GMM is utilized to model the detector's output, and then the detection threshold is calculated to achieve a CFAR detection. The conventional GMM's parameter estimation by Expectation-Maximization (EM) requires all data samples in the dataset to be involved during the procedure and the the parameters would be re-estimated when new data samples available. Thus, GMM is difficult to be applied in real-time processing when newly observed data samples coming progressively. To improve GMM's application availability in time-critical circumstance, an optimization strategy is proposed by introducing the Incremental GMM (IGMM) which allows GMM's parameter to be estimated online incrementally. Experiments on real hyperspectral image and synthetic dataset suggest that RT-THRES can track and model the detection outputs' distribution accurately which ensures the accuracy of the calculation of CFAR thresholds. Moreover, by applying the optimization strategy the computational consumption of RT-THRES maintains relatively low.

EXIT Chart-Based IRA Code Design for TDMR Turbo-Equalization System

We present an extrinsic information transfer (EXIT) chart-based design technique for irregular repeat-accumulate (IRA) codes used in 2-D magnetic recording (TDMR) turbo-equalization systems. The channel model includes Voronoi magnetic grains, 2-D intersymbol interference (2D-ISI) and additive white Gaussian noise (AWGN). The receiver uses a 2D-ISI BCJR equalizer and an IRA decoder. For one outer equalizer-decoder iteration, we propose theory and simulation-based methods for computing EXIT curves. The simulation method calculates experimental EXIT curves for the check node decoder (CND) and the combination of the variable node decoder (VND) and an equalizer. The theoretical approach recursively calculates CND and VND Gaussian mixture model parameters in order to calculate EXIT curves. We then fit the VND and CND EXIT curves to find optimized variable node degree distributions. Simulation results show that the TDMR-optimized IRA codes achieve up to a 6.2% density increase in user bits/grain (U/G) compared with IRA codes designed for AWGN channels. The theory-based code designs achieve the same or better U/G as the simulation-based designs, but require 98% less design computation time. We also derive optimized IRA codes for iterative turbo-equalization; these codes can achieve simultaneous U/G gains and SNR savings compared with AWGN-optimized codes.

A variable step learning algorithm for Gaussian mixture models based on the Bhattacharyya coefficient and correlation coefficient criterion

The expectation maximization (EM) algorithm is one of the most enduring ways to estimate the parameters of Gaussian mixture models. However, the EM algorithm needs to know in advance the true number of mixing components, and its performance highly depends on the initial parameters, which leads to use it difficultly in practice. To overcome the above two drawbacks, a variable step learning algorithm is proposed for learning a Gaussian mixture models. The novelty of the algorithm is that a variable step search strategy is proposed to find the actual number of Gaussian mixture components, or slightly above it. This number is obtained by maximizing the Bhattacharyya coefficient criterion, which in turn quantifies how close the Gaussian mixtures models fit the observed data. Furthermore, the initialization parameters are determined by the correlation coefficient criterion representing the shape characteristics between the new GMM and the histograms of the data. Based on these two criteria, the proposed algorithm can make both parameter learning and model selection more efficient, as shown by the conducted experiments.

Marine Engine-Centered Data Analytics for Ship Performance Monitoring

This study proposes marine engine centered data analytics as a part of the ship energy efficiency management plan (SEEMP). The SEEMP enforces various emission control measures to improve ship energy efficiency by considering vessel performance and navigation data. The proposed data analytics is developed in the engine-propeller combinator diagram (i.e., one propeller shaft with a direct drive main engine). Three operating regions from the initial data analysis are under the combinator diagram noted to capture the shape of these regions by the proposed data analytics. The data analytics consists of implementing Gaussian mixture models (GMMs) to classify the most frequent operating regions of the main engine. Furthermore, the expectation maximization (EM) algorithm calculates the parameters of GMMs. This approach, also named data clustering algorithm, facilitates an iterative process for capturing the operating regions of the main engine (i.e., in the combinatory diagram) with the respective mean and covariance matrices. Hence, these data analytics can monitor ship performance and navigation conditions with respect to engine operating regions as a part of the SEEMP. Furthermore, development of advanced mathematical models for ship performance monitoring within the operational regions (i.e., data clusters) of marine engines is expected.

An Interactive Image Segmentation Method in Hand Gesture Recognition.

In order to improve the recognition rate of hand gestures a new interactive image segmentation method for hand gesture recognition is presented, and popular methods, e.g., Graph cut, Random walker, Interactive image segmentation using geodesic star convexity, are studied in this article. The Gaussian Mixture Model was employed for image modelling and the iteration of Expectation Maximum algorithm learns the parameters of Gaussian Mixture Model. We apply a Gibbs random field to the image segmentation and minimize the Gibbs Energy using Min-cut theorem to find the optimal segmentation. The segmentation result of our method is tested on an image dataset and compared with other methods by estimating the region accuracy and boundary accuracy. Finally five kinds of hand gestures in different backgrounds are tested on our experimental platform, and the sparse representation algorithm is used, proving that the segmentation of hand gesture images helps to improve the recognition accuracy.

Classification and Detection of Adulteration in Olive Oil Using Improved Gaussian Mixture Model and Regression by Artificial Bee Colony Algorithm

Gaussian mixture model (GMM) and Gaussian mixture regression (GMR) can be used to detect adulteration in extra virgin olive oil. The estimate of the GMM parameters is commonly obtained from the expectation- maximization (EM) algorithm. EM algorithm has some limitations such as local optimum problems and sensitivity to the initial values. In this paper, artificial bee colony (ABC) algorithm is used to determine the optimal parameters in GMM and GMR. To improve the optimized performance and reduce computational effort of ABC algorithm, the information sharing mechanism among the global best food sources is introduced in ABC. The improved GMM and GMR by artificial bee colony algorithm (GMMRABC) were used to discriminate and quantify the adulteration of extra virgin olive oil with rapeseed oil using FT-IR spectroscopy. It has been demonstrated that the proposed method is an accurate, rapid, stable strategy for identifying and quantifying the extra virgin olive oil.

Denoising model for parallel magnetic resonance imaging images using higher‐order Markov random fields

This study presents a novel robust method for Bayesian denoising of parallel magnetic resonance imaging (pMRI) images. For the first time, the authors’ proposal applies fields of experts (FoE), a filter-based higher-order Markov random field (MRF), to model the prior of the pMRI image statistics. The noise in pMRI data behaves to be non-central Chi (nc-χ) distributed. In practice, correlation between coils exists, resulting in that nc-χ distribution does not hold anymore and the spatially varying noise problem. Thus, preservation of fine textures requires to adapt locally the estimation. Therefore, more precisely, the noise is reduced by using a sliding window scheme. In each window, the likelihood probability function is accurately modelled from corrupted data by using an innovative Gaussian mixture model (GMM). The parameters of GMM are calculated by applying an iterative expectation maximisation approach. With the priors via the learned FoE model and the likelihood function via GMM, a maximum a posteriori (MAP) estimator is formulated. Then, the noise in the each window is filtered by applying an efficient non-linear quasi-Newton method to explore an optimal solution for the MAP estimator. Finally, experiments have been conducted on both the simulated and real data to compare the proposed model with some state-of-the-art denoising methods. The experimental results demonstrate the robustness and effectiveness of the proposed denoising model.

Model selection for Gaussian mixture model based on desirability level criterion

The expectation maximization (EM) algorithm is the most enduring way to estimate the parameters of Gaussian mixture models. However, use the EM algorithm needs to know in advance the true number of mixing components. Therefore, unless this key information is available, it is usually not straightforward to perform this algorithm. On the other hand, its performance highly depends on the initial parameters. To alleviate these problems, a new model selection criterion, i.e., the desirability level criterion, is proposed to choose the number of components. In particular, we proposed a variable step until find either coincides with the actual number or slightly exceeds it, which maximize the value of the desirability level criterion that provides an efficient index to quantify the distance between the Gaussian mixture model fits the observation data. Furthermore, unwanted components can be suppressed by setting the threshold of the desirability level criterion. Numerical examples are provided to illustrate the effectiveness of our desirability level criterion.

Uncertainty quantification for functional dependent random variables

This paper proposes a new methodology to model uncertainties associated with functional random variables. This methodology allows to deal simultaneously with several dependent functional variables and to address the specific case where these variables are linked to a vectorial variable, called covariate. In this case, the proposed uncertainty modelling methodology has two objectives: to retain both the most important features of the functional variables and their features which are the most correlated to the covariate. This methodology is composed of two steps. First, the functional variables are decomposed on a functional basis. To deal simultaneously with several dependent functional variables, a Simultaneous Partial Least Squares algorithm is proposed to estimate this basis. Second, the joint probability density function of the coefficients selected in the decomposition is modelled by a Gaussian mixture model. A new sparse method based on a Lasso penalization algorithm is proposed to estimate the Gaussian mixture model parameters and reduce their number. Several criteria are introduced to assess the methodology performance: its ability to approximate the functional variables probability distribution, their dependence structure and their features which explain the covariate. Finally, the whole methodology is applied on a simulated example and on a nuclear reliability test case.

Joint Learning of Unsupervised Dimensionality Reduction and Gaussian Mixture Model

Dimensionality reduction (DR) has been one central research topic in information theory, pattern recognition, and machine learning. Apparently, the performance of many learning models significantly rely on dimensionality reduction: successful DR can largely improve various approaches in clustering and classification, while inappropriate DR may deteriorate the systems. When applied on high-dimensional data, some existing research approaches often try to reduce the dimensionality first, and then input the reduced features to other available models, e.g., Gaussian mixture model (GMM). Such independent learning could however significantly limit the performance, since the optimal subspace given by a particular DR approach may not be appropriate for the following model. In this paper, we focus on investigating how unsupervised dimensionality reduction could be performed together with GMM and if such joint learning could lead to improvement in comparison with the traditional unsupervised method. In particular, we engage the mixture of factor analyzers with the assumption that a common factor loading exists for all the components. Based on that, we then present EM-algorithm that converges to a local optimal solution. Such setting exactly optimizes a dimensionality reduction together with the parameters of GMM. We describe the framework, detail the algorithm, and conduct a series of experiments to validate the effectiveness of our proposed approach. Specifically, we compare the proposed joint learning approach with two competitive algorithms on one synthetic and six real data sets. Experimental results show that the joint learning significantly outperforms the comparison methods in terms of three criteria.

Object Tracking with Multiple Instance Learning and Gaussian Mixture Model

Recently, Multiple Instance Learning (MIL) technique has been introduced for object tracking\linebreak applications, which has shown its good performance to handle drifting problem. While some instances in positive bags not only contain objects, but also contain the background, it is not reliable to simply assume that each feature of instances in positive bags obeys a single Gaussian distribution. In this paper, a tracker based on online multiple instance boosting has been developed, which employs Gaussian Mixture Model (GMM) and single Gaussian distribution respectively to model features of instances in positive and negative bags. The differences between samples and the model are integrated into the process of updating the parameters for GMM. With the Haar-like features extracted from the bags, a set of weak classifiers are trained to construct a strong classifier, which is used to track the object location at a new frame. And the classifier can be updated online frame by frame. Experimental results have shown that our tracker is more stable and efficient when dealing with the illumination, rotation, pose and appearance changes.

Feature Selection for GUMI Kernel-Based SVM in Speech Emotion Recognition

Speech emotion recognition is the indispensable requirement for efficient human machine interaction. Most modern automatic speech emotion recognition systems use Gaussian mixture models (GMM) and Support Vector Machines (SVM). GMM are known for their performance and scalability in the spectral modeling while SVM are known for their discriminatory power. A GMM-supervector characterizes an emotional style by the GMM parameters (mean vectors, covariance matrices, and mixture weights). GMM-supervector SVM benefits from both GMM and SVM frameworks. In this paper, the GMM-UBM mean interval (GUMI) kernel based on the Bhattacharyya distance is successfully used. CFSSubsetEval combined with Best first algorithm and Greedy stepwise were also utilized on the supervectors space in order to select the most important features. This framework is illustrated using Mel-frequency cepstral (MFCC) coefficients and Perceptual Linear Prediction (PLP) features on two different emotional databases namely the Surrey Audio-Expressed Emotion and the Berlin Emotional speech Database.

Gaussian Mixture Model Based Classification of Stuttering Dysfluencies

AbstractThe classification of dysfluencies is one of the important steps in objective measurement of stuttering disorder. In this work, the focus is on investigating the applicability of automatic speaker recognition (ASR) method for stuttering dysfluency recognition. The system designed for this particular task relies on the Gaussian mixture model (GMM), which is the most widely used probabilistic modeling technique in ASR. The GMM parameters are estimated from Mel frequency cepstral coefficients (MFCCs). This statistical speaker-modeling technique represents the fundamental characteristic sounds of speech signal. Using this model, we build a dysfluency recognizer that is capable of recognizing dysfluencies irrespective of a person as well as what is being said. The performance of the system is evaluated for different types of dysfluencies such as syllable repetition, word repetition, prolongation, and interjection using speech samples from the University College London Archive of Stuttered Speech (UCLASS).

SU‐E‐J‐130: Automating Liver Segmentation Via Combined Global and Local Optimization

Purpose:The aim of this work is to develop a robust algorithm for accurate segmentation of liver with special attention paid to the problems with fuzzy edges and tumor.Methods:200 CT images were collected from radiotherapy treatment planning system. 150 datasets are selected as the panel data for shape dictionary and parameters estimation. The remaining 50 datasets were used as test images. In our study liver segmentation was formulated as optimization process of implicit function. The liver region was optimized via local and global optimization during iterations. Our method consists five steps: 1)The livers from the panel data were segmented manually by physicians, and then We estimated the parameters of GMM (Gaussian mixture model) and MRF (Markov random field). Shape dictionary was built by utilizing the 3D liver shapes. 2)The outlines of chest and abdomen were located according to rib structure in the input images, and the liver region was initialized based on GMM. 3)The liver shape for each 2D slice was adjusted using MRF within the neighborhood of liver edge for local optimization. 4)The 3D liver shape was corrected by employing SSR (sparse shape representation) based on liver shape dictionary for global optimization. Furthermore, H‐PSO(Hybrid Particle Swarm Optimization) was employed to solve the SSR equation. 5)The corrected 3D liver was divided into 2D slices as input data of the third step. The iteration was repeated within the local optimization and global optimization until it satisfied the suspension conditions (maximum iterations and changing rate).Results:The experiments indicated that our method performed well even for the CT images with fuzzy edge and tumors. Comparing with physician delineated results, the segmentation accuracy with the 50 test datasets (VOE, volume overlap percentage) was on average 91%–95%.Conclusion:The proposed automatic segmentation method provides a sensible technique for segmentation of CT images.This work is supported by NIH/NIBIB (1R01‐EB016777), National Natural Science Foundation of China (No.61471226 and No.61201441), Research funding from Shandong Province (No.BS2012DX038 and No.J12LN23), and Research funding from Jinan City (No.201401221 and No.20120109).

Cooperative Localization in WSNs Using Gaussian Mixture Modeling: Distributed ECM Algorithms

We study cooperative sensor network localization in a realistic scenario where the underlying measurement errors more probably follow a non-Gaussian distribution; the measurement error distribution is unknown without conducting massive offline calibrations; and non-line-of-sight identification is not performed due to the complexity constraint and/or storage limitation. The underlying measurement error distribution is approximated parametrically by a Gaussian mixture with finite number of components, and the expectation-conditional maximization (ECM) criterion is adopted to approximate the maximum-likelihood estimator of the unknown sensor positions and an extra set of Gaussian mixture model parameters. The resulting centralized ECM algorithms lead to easier inference tasks and meanwhile retain several convergence properties with a proof of the “space filling” condition. To meet the scalability requirement, we further develop two distributed ECM algorithms where an average consensus algorithm plays an important role for updating the Gaussian mixture model parameters locally. The proposed algorithms are analyzed systematically in terms of computational complexity and communication overhead. Various computer based tests are also conducted with both simulation and experimental data. The results pin down that the proposed distributed algorithms can provide overall good performance for the assumed scenario even under model mismatch, while the existing competing algorithms either cannot work without the prior knowledge of the measurement error statistics or merely provide degraded localization performance when the measurement error is clearly non-Gaussian.

Modeling the Affective Content of Music with a Gaussian Mixture Model

Modeling the association between music and emotion has been considered important for music information retrieval and affective human computer interaction. This paper presents a novel generative model called acoustic emotion Gaussians (AEG) for computational modeling of emotion. Instead of assigning a music excerpt with a deterministic (hard) emotion label, AEG treats the affective content of music as a (soft) probability distribution in the valence-arousal space and parameterizes it with a Gaussian mixture model (GMM). In this way, the subjective nature of emotion perception is explicitly modeled. Specifically, AEG employs two GMMs to characterize the audio and emotion data. The fitting algorithm of the GMM parameters makes the model learning process transparent and interpretable. Based on AEG, a probabilistic graphical structure for predicting the emotion distribution from music audio data is also developed. A comprehensive performance study over two emotion-labeled datasets demonstrates that AEG offers new insights into the relationship between music and emotion (e.g., to assess the “affective diversity” of a corpus) and represents an effective means of emotion modeling. Readers can easily implement AEG via the publicly available codes. As the AEG model is generic, it holds the promise of analyzing any signal that carries affective or other highly subjective information.