Components Of Gaussian Mixture Model Research Articles

The cluster analysis of traditional Chinese medicine authenticity identification technique assisted by chemometrics

This study explore the authenticity identification technique of traditional Chinese medicine (TCM) using chemometrics in conjunction with cluster analysis. A clustering Gaussian mixture model was constructed and applied for the data clustering analysis of four types of TCM. Chemical measurements combined with discrete wavelet transform (DWT), Fourier transform infrared spectroscopy (FTIR), and Fourier self-deconvolution (FSD) were utilized for the detailed differentiation of Bupleurum scorzonerifolium, Bupleurum yinchowense, Bupleurum marginatum, and Bupleurum smithii Wolff var. parvifolium. Differences in the attenuated total reflection-FTIR (ATR-FTIR) spectra among the four TCMs were observed. Utilizing clustering algorithms, the one-dimensional DWT of the infrared spectra of samples was employed for the authentication of Chinese herbal medicines. The model demonstrates optimal performance throughout 2000 rounds of network training. The accuracy (88.6 %), sensitivity (86.5 %), and specificity (82.7 %) of the model constructed in this study significantly surpassed those of the CNN model: accuracy (67.7 %), sensitivity (70.4 %), and specificity (68.5 %) (P < 0.05). By setting the cluster size K = 5 and the number of Gaussian mixture model components to 5, the model effectively fits the actual number of categories within the dataset. Infrared spectroscopy analysis revealed distinct carbon-oxygen stretching vibration absorption peaks between 1025 and 1200 cm−1 for Bupleurum scorzonerifolium, Bupleurum yinchowense, Bupleurum marginatum, and Bupleurum smithii Wolff var. parvifolium, indicating strong absorption peaks of carbohydrates. A comprehensive structural information analysis revealed a similarity of above 0.982 among the four types of TCM. Combined with chemometrics and intelligent algorithm-based cluster analysis, successful and accurate authentication of TCM authenticity was achieved, providing an effective methodology for quality control in TCM.

Unsupervised classification of multi-contrast magnetic resonance histology of peripheral arterial disease lesions using a convolutional variational autoencoder with a Gaussian mixture model in latent space: A technical feasibility study

PurposeTo investigate the feasibility of a deep learning algorithm combining variational autoencoder (VAE) and two-dimensional (2D) convolutional neural networks (CNN) for automatically quantifying hard tissue presence and morphology in multi-contrast magnetic resonance (MR) images of peripheral arterial disease (PAD) occlusive lesions. MethodsMulti-contrast MR images (T2-weighted and ultrashort echo time) were acquired from lesions harvested from six amputated legs with high isotropic spatial resolution (0.078 mm and 0.156 mm, respectively) at 9.4 T. A total of 4014 pseudo-color combined images were generated, with 75% used to train a VAE employing custom 2D CNN layers. A Gaussian mixture model (GMM) was employed to classify the latent space data into four tissue classes: I) concentric calcified (c), II) eccentric calcified (e), III) occluded with hard tissue (h) and IV) occluded with soft tissue (s). Test image probabilities, encoded by the trained VAE were used to evaluate model performance. ResultsGMM component classification probabilities ranged from 0.92 to 0.97 for class (c), 1.00 for class (e), 0.82–0.95 for class (h) and 0.56–0.93 for the remaining class (s). Due to the complexity of soft-tissue lesions reflected in the heterogeneity of the pseudo-color images, more GMM components (n=17) were attributed to class (s), compared to the other three (c, e and h) (n=6). ConclusionCombination of 2D CNN VAE and GMM achieves high classification probabilities for hard tissue-containing lesions. Automatic recognition of these classes may aid therapeutic decision-making and identifying uncrossable lesions prior to endovascular intervention.

Semiparametric constant false alarm rate method for radar and sonar images

AbstractThis work proposed a novel constant false alarm rate detection method based on Gaussian mixture model (GMM). The main contribution contains two aspects. First, the Gabor wavelet is used to determine the GMM component number. Second, the expression of probability of false alarm is derived based on the GMM parameter.

Orbital Uncertainty Propagation Based on Adaptive Gaussian Mixture Model under Generalized Equinoctial Orbital Elements

The number of resident space objects (RSOs) has been steadily increasing over time, posing significant risks to the safe operation of on-orbit assets. The accurate prediction of potential collision events and implementation of effective and nonredundant avoidance maneuvers require the precise estimation of the orbit positions of objects of interest and propagation of their associated uncertainties. Previous research mainly focuses on striking a balance between accurate propagation and efficient computation. A recently proposed approach that integrates uncertainty propagation with different coordinate representations has the potential to achieve such a balance. This paper proposes combining the generalized equinoctial orbital elements (GEqOE) representation with an adaptive Gaussian mixture model (GMM) for uncertainty propagation. Specifically, we implement a reformulation for the orbital dynamics so that the underlying state and the moment feature of the GMM are propagated under the GEqOE coordinates. Starting from an initial Gaussian probability distribution function (PDF), the algorithm iteratively propagates the uncertainty distribution using a detection-splitting module. A differential entropy-based nonlinear detector and a splitting library are utilized to adjust the number of GMM components dynamically. Component splitting is triggered when a predefined threshold of differential entropy is violated, generating several GMM components. The final probability density function (PDF) is obtained by a weighted summation of the component distributions at the target time. Benefiting from the nonlinearity reduction caused by the GEqOE representation, the number of triggered events largely decreases, causing the necessary number of components to maintain uncertainty realism also to decrease, which enables the proposed approach to achieve good performance with much more efficiency. As demonstrated by the results of propagation in three scenarios with different degrees of complexity, compared with the Cartesian-based approach, the proposed approach achieves comparable accuracy to the Monte Carlo method while largely reducing the number of components generated during propagation. Our results confirm that a judicious choice of coordinate representation can significantly improve the performance of uncertainty propagation methods in terms of accuracy and computational efficiency.

Representing Multimodal Behaviors With Mean Location for Pedestrian Trajectory Prediction.

Representing multimodal behaviors is a critical challenge for pedestrian trajectory prediction. Previous methods commonly represent this multimodality with multiple latent variables repeatedly sampled from a latent space, encountering difficulties in interpretable trajectory prediction. Moreover, the latent space is usually built by encoding global interaction into future trajectory, which inevitably introduces superfluous interactions and thus leads to performance reduction. To tackle these issues, we propose a novel Interpretable Multimodality Predictor(IMP) for pedestrian trajectory prediction, whose core is to represent a specific mode by its mean location. We model the distribution of mean location as a Gaussian Mixture Model (GMM) conditioned on sparse spatio-temporal features, and sample multiple mean locations from the decoupled components of GMM to encourage multimodality. Our IMP brings four-fold benefits: 1) Interpretable prediction to provide semantics about the motion behavior of a specific mode; 2) Friendly visualization to present multimodal behaviors; 3) Well theoretical feasibility to estimate the distribution of mean locations supported by the central-limit theorem; 4) Effective sparse spatio-temporal features to reduce superfluous interactions and model temporal continuity of interaction. Extensive experiments validate that our IMP not only outperforms state-of-the-art methods but also can achieve a controllable prediction by customizing the corresponding mean location.

Measure of Similarity between GMMs Based on Autoencoder-Generated Gaussian Component Representations

A novel similarity measure between Gaussian mixture models (GMMs), based on similarities between the low-dimensional representations of individual GMM components and obtained using deep autoencoder architectures, is proposed in this paper. Two different approaches built upon these architectures are explored and utilized to obtain low-dimensional representations of Gaussian components in GMMs. The first approach relies on a classical autoencoder, utilizing the Euclidean norm cost function. Vectorized upper-diagonal symmetric positive definite (SPD) matrices corresponding to Gaussian components in particular GMMs are used as inputs to the autoencoder. Low-dimensional Euclidean vectors obtained from the autoencoder’s middle layer are then used to calculate distances among the original GMMs. The second approach relies on a deep convolutional neural network (CNN) autoencoder, using SPD representatives to generate embeddings corresponding to multivariate GMM components given as inputs. As the autoencoder training cost function, the Frobenious norm between the input and output layers of such network is used and combined with regularizer terms in the form of various pieces of information, as well as the Riemannian manifold-based distances between SPD representatives corresponding to the computed autoencoder feature maps. This is performed assuming that the underlying probability density functions (PDFs) of feature-map observations are multivariate Gaussians. By employing the proposed method, a significantly better trade-off between the recognition accuracy and the computational complexity is achieved when compared with other measures calculating distances among the SPD representatives of the original Gaussian components. The proposed method is much more efficient in machine learning tasks employing GMMs and operating on large datasets that require a large overall number of Gaussian components.

Fitting Imbalanced Uncertainties in Multi-output Time Series Forecasting

We focus on multi-step ahead time series forecasting with the multi-output strategy. From the perspective of multi-task learning (MTL), we recognize imbalanced uncertainties between prediction tasks of different future time steps. Unexpectedly, trained by the standard summed Mean Squared Error (MSE) loss, existing multi-output forecasting models may suffer from performance drops due to the inconsistency between the loss function and the imbalance structure. To address this problem, we reformulate each prediction task as a distinct Gaussian Mixture Model (GMM) and derive a multi-level Gaussian mixture loss function to better fit imbalanced uncertainties in multi-output time series forecasting. Instead of using the two-step Expectation-Maximization (EM) algorithm, we apply the self-attention mechanism on the task-specific parameters to learn the correlations between different prediction tasks and generate the weight distribution for each GMM component. In this way, our method jointly optimizes the parameters of the forecasting model and the mixture model simultaneously in an end-to-end fashion, avoiding the need of two-step optimization. Experiments on three real-world datasets demonstrate the effectiveness of our multi-level Gaussian mixture loss compared to models trained with the standard summed MSE loss function. All the experimental data and source code are available at https://github.com/smallGum/GMM-FNN .

Deep Generative Mixture Model for Robust Imbalance Classification.

Discovering hidden pattern from imbalanced data is a critical issue in various real-world applications. Existing classification methods usually suffer from the limitation of data especially for minority classes, and result in unstable prediction and low performance. In this paper, a deep generative classifier is proposed to mitigate this issue via both model perturbation and data perturbation. Specially, the proposed generative classifier is derived from a deep latent variable model where two variables are involved. One variable is to capture the essential information of the original data, denoted as latent codes, which are represented by a probability distribution rather than a single fixed value. The learnt distribution aims to enforce the uncertainty of model and implement model perturbation, thus, lead to stable predictions. The other variable is a prior to latent codes so that the codes are restricted to lie on components in Gaussian Mixture Model. As a confounder affecting generative processes of data (feature/label), the latent variables are supposed to capture the discriminative latent distribution and implement data perturbation. Extensive experiments have been conducted on widely-used real imbalanced image datasets. Experimental results demonstrate the superiority of our proposed model by comparing with popular imbalanced classification baselines on imbalance classification task.

Measure of Similarity between GMMs Based on Geometry-Aware Dimensionality Reduction

Gaussian Mixture Models (GMMs) are used in many traditional expert systems and modern artificial intelligence tasks such as automatic speech recognition, image recognition and retrieval, pattern recognition, speaker recognition and verification, financial forecasting applications and others, as simple statistical representations of underlying data. Those representations typically require many high-dimensional GMM components that consume large computing resources and increase computation time. On the other hand, real-time applications require computationally efficient algorithms and for that reason, various GMM similarity measures and dimensionality reduction techniques have been examined to reduce the computational complexity. In this paper, a novel GMM similarity measure is proposed. The measure is based on a recently presented nonlinear geometry-aware dimensionality reduction algorithm for the manifold of Symmetric Positive Definite (SPD) matrices. The algorithm is applied over SPD representations of the original data. The local neighborhood information from the original high-dimensional parameter space is preserved by preserving distance to the local mean. Instead of dealing with high-dimensional parameter space, the method operates on much lower-dimensional space of transformed parameters. Resolving the distance between such representations is reduced to calculating the distance among lower-dimensional matrices. The method was tested within a texture recognition task where superior state-of-the-art performance in terms of the trade-off between recognition accuracy and computational complexity has been achieved in comparison with all baseline GMM similarity measures.

Automatic Evaluation of Motor Rehabilitation Exercises Based on Deep Mixture Density Neural Networks

An automatic assessment system for physical telerehabilitation could reduce the time and cost of treatments. But such assessment involves stochastic uncertainties, nonlinearities, and complexities of human movement. Probabilistic models and deep structures are two categories that could, respectively, address the stochastic uncertainty and complexity of motion data. In the proposed Deep Mixture Density Network (DMDN), probabilistic models were concurrently processed along with deep neural networks. More specifically, a multi-branch convolutional layer extracted the deep features of motion data, a Long Short Term Memory (LSTM) learned its temporal dependencies, and a Gaussian Mixture Model (GMM) handled the stochastic interaction of its preceding layers in reaching a more valid assessment and improved generalization to new movements. Finally, the weighted mean of the GMM components was used as the performance score for exercises. Input data were the time series related to the joints' position and orientation extracted from the Kinect v2 sensor video. A clinical reference score for each movement was also included for training the DMDN. In addition to comparisons with the state-of-the-art algorithms, an ablation study of the various elements comprising the DMDN was performed. Three configurations of convolutional filter window transitions across input data were also investigated. Results indicate that the proposed DMDN with one-dimensional parallel window transitions outperforms the other competing strategies in the ablation study. It also achieves higher reliability in terms of a lower RMSE standard deviation against a DMDN without GMM strategy while ranking competitively according to the Spearman correlation coefficient and Root Mean Square Error.

Clustering Methods for Particle Filters With Gaussian Mixture Models

New clustering methods are proposed to develop novel particle filters with Gaussian mixture models (PFGMM). In the PFGMM, the propagated samples are clustered to recover a Gaussian mixture model (GMM) using a clustering algorithm, which plays a fundamental role in the filter's performance. Two clustering methods are introduced that simultaneously minimize the covariance of each of the GMM components and maximize the likelihood function. Under the scenarios considered in this paper, it is shown through numerical simulation that the PFGMMs with the proposed clustering algorithms lead to better performance than the PFGMM employing the K-means or the expectation-maximization (EM) algorithms as well as the regularized particle filter (RPF) and the Gaussian sum particle filter (GSPF).

An Asymptotically MSE-Optimal Estimator Based on Gaussian Mixture Models

This paper investigates a channel estimator based on Gaussian mixture models (GMMs) in the context of linear inverse problems with additive Gaussian noise. We fit a GMM to given channel samples to obtain an analytic probability density function (PDF) which approximates the true channel PDF. Then, a conditional mean estimator (CME) corresponding to this approximating PDF is computed in closed form and used as an approximation of the optimal CME based on the true channel PDF. This optimal CME cannot be calculated analytically because the true channel PDF is generally unknown. We present mild conditions which allow us to prove the convergence of the GMM-based CME to the optimal CME as the number of GMM components is increased. Additionally, we investigate the estimator's computational complexity and present simplifications based on common model-based insights. Further, we study the estimator's behavior in numerical experiments including multiple-input multiple-output (MIMO) and wideband systems.

Measure of Similarity between GMMs by Embedding of the Parameter Space That Preserves KL Divergence

In this work, we deliver a novel measure of similarity between Gaussian mixture models (GMMs) by neighborhood preserving embedding (NPE) of the parameter space, that projects components of GMMs, which by our assumption lie close to lower dimensional manifold. By doing so, we obtain a transformation from the original high-dimensional parameter space, into a much lower-dimensional resulting parameter space. Therefore, resolving the distance between two GMMs is reduced to (taking the account of the corresponding weights) calculating the distance between sets of lower-dimensional Euclidean vectors. Much better trade-off between the recognition accuracy and the computational complexity is achieved in comparison to measures utilizing distances between Gaussian components evaluated in the original parameter space. The proposed measure is much more efficient in machine learning tasks that operate on large data sets, as in such tasks, the required number of overall Gaussian components is always large. Artificial, as well as real-world experiments are conducted, showing much better trade-off between recognition accuracy and computational complexity of the proposed measure, in comparison to all baseline measures of similarity between GMMs tested in this paper.

Stable Autonomous Robotic Wheelchair Navigation in the Environment With Slope Way

In this article, we present a path planning approach that is capable of generating a feasible trajectory for stable robotic wheelchair navigation in the environment with slope way. Firstly, the environment is modeled by a lightweight navigation map, with which the proposed sampling-based path planning scheme with a modified extension function can generate a feasible path. Then, the path is further optimized by the proposed utility function involving the human comfort and the path cost. To improve the searching efficiency of an optimal trajectory, we present an adaptive weighting Gaussian Mixture Model (GMM) based sampling strategy. Particularly, the weights of the components in GMM are adjusted adaptively in the planning process. It is also worth noting that the proposed sampling-based planning paradigm can indicate the unsafe regions in the navigation map, which forms a traversable map and further guarantees the safety of the wheelchair robot navigation. Furthermore, the effectiveness and the efficiency of the proposed path planning method are verified in both simulation and real-world experiments.

Speaker age and gender classification using GMM supervector and NAP channel compensation method

One of the most important factors affecting the performance of speech-based recognition systems is the differences between training and test conditions. The Nuisance attribute projection (NAP) is an effective method for eliminating these differences, called channel effects. In this study, the effects of the NAP approach in determining age and gender groups are investigated. Mel-frequency cepstral coefficients and delta coefficients are used as a feature and Gaussian mixture models (GMM) adapted from the universal background model by maximum-a-posteriori method are used for the modeling of age and gender classes. After the GMMs corresponding to each speech are converted into mean supervectors, they are applied to a Support Vector Machine (SVM), and speeches are classified according to the age and gender group of the speakers. While linear GMM kernel based on Kullback–Leibler divergence is used instead of standard SVM kernels, the NAP channel subspace size is changed between 20 and 200 and the number of GMM components is changed between 32 and 512 to determine the optimum values for these parameters. In the tests on the aGender database, the optimum number of components is determined as 128, and the optimum NAP channel subspace size is determined as 45. The age and gender classification accuracy of the system, which is developed using these optimum parameters, is increased from 60.52 to 62.03% with the use of NAP. In addition, age classification accuracy is increased from 60.23 to 61.82% and gender classification accuracy is increased from 91.71 to 92.30%.

Adaptive parameter estimation of GMM and its application in clustering

Parameter estimation of Gaussian mixture model (GMM) has begun to gain attention in the field of science and engineering, and it has gradually become one of the most popular research topics. In particular, the rapid development of theoretical progress on globally optimal convergence promotes the widespread application of Gaussian mixtures in data clustering. So this paper introduces a novel parameter estimation algorithm called TDAVBEM, which combines the Tsallis entropy and a deterministic annealing (DA) algorithm on the basis of the variational bayesian expected maximum (VBEM) to simultaneously implement the parameter estimation and select the optimal components of GMM. We experimentally certified the effectiveness and robustness of our proposed algorithm passes through comparing it with several parameter evaluation methods and its application in data clustering.

GMM with parameters initialization based on SVD for network threat detection

In the field of security, the data labels are unknown or the labels are too expensive to label, so that clustering methods are used to detect the threat behavior contained in the big data. The most widely used probabilistic clustering model is Gaussian Mixture Models(GMM), which is flexible and powerful to apply prior knowledge for modelling the uncertainty of the data. Therefore, in this paper, we use GMM to build the threat behavior detection model. Commonly, Expectation Maximization (EM) and Variational Inference (VI) are used to estimate the optimal parameters of GMM. However, both EM and VI are quite sensitive to the initial values of the parameters. Therefore, we propose to use Singular Value Decomposition (SVD) to initialize the parameters. Firstly, SVD is used to factorize the data set matrix to get the singular value matrix and singular matrices. Then we calculate the number of the components of GMM by the first two singular values in the singular value matrix and the dimension of the data. Next, other parameters of GMM, such as the mixing coefficients, the mean and the covariance, are calculated based on the number of the components. After that, the initialization values of the parameters are input into EM and VI to estimate the optimal parameters of GMM. The experiment results indicate that our proposed method performs well on the parameters initialization of GMM clustering using EM and VI for estimating parameters.

Compressed Image Restoration via Artifacts-Free PCA Basis Learning and Adaptive Sparse Modeling

Visually unpleasant compression artifacts frequently appear in block-based transform coding, especially at low bit rates. This paper presents a new artifact reduction scheme based on Bayesian sparse modeling and artifacts-free PCA basis learning. To avoid the effect of blocking artifacts, we propose to learn artifacts-free PCA basis from clean images. We concatenate the clean patches and their compressed counterparts to learn paired distribution prior via the Gaussian Mixture Model (GMM). By this way, the GMM characterizes the mapping between the clean image and its compressed version. To restore a compressed patch, the best matched GMM component is assigned using the patch in the compressed image subspace. The artifacts-free PCA basis is obtained according to the mapping learned by the paired GMM. In practice, the statistical distributions of different sparse coefficients in different patches may dramatically vary with image contents. Instead of using a global zero-mean distribution for all coefficients, we propose to adaptively model the prior of each band in a Bayesian framework. The expectation and variance of each band are adaptively learned from the similar patches within the image. Thus, different transform bands are regularized unequally according to the learned priors. Experimental results show that the proposed scheme outperforms most of the compared schemes in terms of both objective quality and perceptual quality.

Registration of multi-view point sets under the perspective of expectation-maximization.

Multi-view registration plays a critical role in 3D model reconstruction. To solve this problem, most previous methods align point sets by either partially exploring available information or blindly utilizing unnecessary information, which may lead to undesired results or extra computation complexity. Accordingly, we propose a novel solution for the multi-view registration under the perspective of Expectation-Maximization (EM). The proposed method assumes that each data point is generated from one unique Gaussian Mixture Model (GMM), where its corresponding points in other point sets are regarded as Gaussian centroids with equal covariance and membership probabilities. As it is difficult to obtain real corresponding points in the registration problem, they are approximated by the nearest neighbor in each other aligned point sets. Based on this assumption, it is reasonable to define the likelihood function including all rigid transformations, which require to be estimated for multi-view registration. Subsequently, the EM algorithm is derived to estimate rigid transformations with one Gaussian covariance by maximizing the likelihood function. Since the GMM component number is automatically determined by the number of point sets, there is no trade-off between registration accuracy and efficiency in the proposed method. Finally, the proposed method is tested on several benchmark data sets and compared with state-of-the-art algorithms. Experimental results demonstrate its superior performance on the accuracy, efficiency, and robustness for multi-view registration.

PESA: Probabilistic Efficient Storage Algorithm for Time-Domain Spectrum Measurements.

Wireless communication is an essential part of daily life for users globally with applications in medical devices, cellular phones, Internet of Things nodes, and others. Accordingly, there is a need to understand the patterns and properties of radio frequency spectrum use by acquiring accurate spectrum utilization measurements. However, the massive storage volume needed to execute spectrum surveys-especially when a fast sampling rate is used-is an impeding factor in terms of cost and ease-of-access. In this article, a probabilistic efficient storage algorithm (PESA) is proposed to facilitate high-accuracy, time-domain spectrum surveys conducted at a fast sample acquisition rate to detect sporadic spectrum occupancy patterns that could be on the order of microseconds. PESA divides the dynamic range of a monitoring equipment into bins-each represented by one component of a Gaussian mixture model (GMM). Windows of activity and inactivity in the measurements are established by comparing with a threshold and then indicators to the GMM component that best describes a window are recorded. Hence, reducing required storage volume. Results demonstrate that ≈ 99% reduction in storage volume is achievable while maintaining an accurate estimation of channel utilization and activity/inactivity periods. Furthermore, a Lab-VIEW implementation of PESA on a hardware platform was executed and used to survey Wi-Fi channel 1 in a healthcare environment for seven consecutive hours. Although more than 25 billion samples were observed, resulting data only occupied 96.28 megabytes.