Mixture Of Normal Distributions Research Articles

Comparative Analysis of Improved Dirichlet Process Mixture Model

Due to the development of information technology, large amounts of data are generated every day in various industries such as engineering, healthcare, finance, anomaly detection, image recognition, and artificial intelligence. This massive data poses the challenge of analyzing accurately and appropriate classifications. The traditional clustering methods require specifying the number of clusters and are mostly based on distance, which cannot effectively consider the correlations between different indicators of high-dimensional and multi-source data. Moreover, the number of clusters cannot automatically adjust when new data is generated. In order to improve the clustering analysis of high-dimensional and multi-source data in a big data environment, this study utilizes non-parametric mixture models based on distribution clustering, which does not require specifying the number of clusters and can auto update with the data. By combining Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and the non-parametric Bayesian method called Dirichlet Process Mixture Model (DPMM), the Bayesian non-parametric PCA model (PCA-DPMM) and Bayesian non-parametric t-SNE model (TSNE-DPMM) are proposed. The Chinese restaurant process of DPMM is used for sampling by introducing a finite normal mixture distribution. The clustering results on the iris dataset are compared and analyzed. The accuracy of DPMM and TSNE-DPMM reaches 0.97, while PCA-DPMM achieves a maximum accuracy of only 0.94. When different numbers of iterations are set, TSNE-DPMM maintains an accuracy ranging from 0.92 to 0.97, DPMM ranges from 0.66 to 0.97, and PCA-DPMM ranges from 0.73 to 0.94. Therefore, the proposed TSNE-DPMM ensures accuracy and exhibits better model stability in clustering results. Future research can explore the improvement of the model by incorporating deep learning algorithms, among others, to further enhance its performance. Additionally, applying the TSNE-DPMM model to data analysis in other fields is also a future research direction. Through these efforts, we can better tackle the challenges of analyzing high-dimensional and multi-source data in a big data environment and extract valuable information from it.

Bootstrapping ARMA time series models after model selection

Inference after model selection is a very important problem. This article derives the asymptotic distribution of some model selection estimators for autoregressive moving average time series models. Under strong regularity conditions, the model selection estimators are asymptotically normal, but generally the asymptotic distribution is a non normal mixture distribution. Hence bootstrap confidence regions that can handle this complicated distribution were used for hypothesis testing. A bootstrap technique to eliminate selection bias is to fit the model selection estimator β ̂ M S ∗ to a bootstrap sample to find a submodel, then draw another bootstrap sample, and fit the same submodel to get the bootstrap estimator β ̂ M I X ∗ .

Multi-modal hybrid modeling strategy based on Gaussian Mixture Variational Autoencoder and spatial–temporal attention: Application to industrial process prediction

The industrial process is characterized by its multi-modal nature and complex spatial and temporal correlations. Despite the fact that several multi-modal methods have been proposed, few of them can effectively extract deep multi-modal representations and the highly intricate spatial and temporal relationships. In this paper, a novel multi-modal hybrid modeling strategy (GMVAE-STA) is proposed for industrial process prediction. This strategy combines the Gaussian Mixture Variational Autoencoder (GMVAE) and the spatial–temporal attention based Gated Recurrent Unit (STA-GRU). First, the GMVAE maps the raw data to the latent space, which follows a Gaussian Mixture distribution, and the data with the highest probability in each Gaussian are identified as a mode. Then, the STA-GRU captures the complex spatial and temporal relationships within each mode and makes predictions. Experimental results on the Tennessee Eastman process and a real-world fluid catalytic cracking process demonstrate the effectiveness of mode classification and prediction of the proposed method.

Maximum Likelihood Estimation for Non-Stationary Location Models with Mixture of Normal Distributions

We consider an observation-driven location model where the unobserved location variable is modeled as a random walk process and where the error variable is from a mixture of normal distributions. The time-varying location can be extended with a stationary process to account for cyclical and/or higher order autocorrelation. The mixed normal distribution can accurately approximate many continuous error distributions. We obtain a flexible modeling framework for the robust filtering and forecasting based on time-series models with non-stationary and nonlinear features. We provide sufficient conditions for strong consistency and asymptotic normality of the maximum likelihood estimator of the parameter vector in the specified model. The asymptotic properties are valid under correct model specification and can be generalized to allow for potential misspecification of the model. A simulation study is carried out to monitor the forecast accuracy improvements when extra mixture components are added to the model. In an empirical study we show that our approach is able to outperform alternative observation-driven location models in forecast accuracy for a time-series of electricity spot prices.

Bayesian boundary trend filtering

Estimating boundary curves has many applications such as economics, climate science, and medicine. Bayesian trend filtering has been developed as one of locally adaptive smoothing methods to estimate the non-stationary trend of data. This paper develops a Bayesian trend filtering for estimating the boundary trend. To this end, the truncated multivariate normal working likelihood and global-local shrinkage priors based on the scale mixtures of normal distribution are introduced. In particular, well-known horseshoe prior for difference leads to locally adaptive shrinkage estimation for boundary trend. However, the full conditional distributions of the Gibbs sampler involve high-dimensional truncated multivariate normal distribution. To overcome the difficulty of sampling, an approximation of truncated multivariate normal distribution is employed. Using the approximation, the proposed models lead to an efficient Gibbs sampling algorithm via the Pólya-Gamma data augmentation. The proposed method is also extended by considering a nearly isotonic constraint. The performance of the proposed method is illustrated through some numerical experiments and real data examples.

Graph signal interpolation and extrapolation over manifold of Gaussian mixture

Signals with an underlying irregular geometric structure are prevalent in modern applications and are often well-represented using graphs. The field of graph signal processing has emerged to accommodate such signals’ analysis, processing, and interpolation. In this paper, we address the latter, where signal samples are given only on a subset of graph nodes, and the goal is to estimate the graph signal on the remaining nodes. In addition, we consider a related extrapolation task in which new graph nodes that were not part of the original graph are added, and the goal is to estimate the graph signal on those nodes as well. We present a new approach for both the interpolation and extrapolation tasks, which is based on modeling the graph nodes as samples from a continuous manifold with a Gaussian mixture distribution and the graph signal as samples of a continuous function in a reproducing kernel Hilbert space. This model allows us to propose an interpolation and extrapolation algorithm that utilizes the closed-form expressions of the Gaussian kernel eigenfunctions. We test our algorithm on synthetic and real-world signals and compare it to existing methods. We demonstrate superior or on-par accuracy results achieved in significantly shorter run times.

Analysis of the characteristics of skill-based street dance movements based on the improved K-means algorithm

Abstract This paper first analyzes the K-mean algorithm from the core idea, algorithm process and advantages and disadvantages, then further improves the K-mean algorithm by using Gaussian mixture distribution and constructs the skill-based street dance movement recognition model based on the improved algorithm. Finally, the street dance teaching video is used as an example for dance movement acquisition and data pre-processing, and the recognition accuracy analysis of the street dance movement dataset is conducted based on the improved K-mean algorithm. The average recognition rates of the recognition model in the four data sets of the data set were 72.34%, 74.65%, 73.15% and 86.70%, respectively. This shows that analyzing the characteristics of street dance movements using the improved K-mean algorithm is beneficial for optimizing and improving existing street dance movements.

ScaffoldGVAE: scaffold generation and hopping of drug molecules via a variational autoencoder based on multi-view graph neural networks

In recent years, drug design has been revolutionized by the application of deep learning techniques, and molecule generation is a crucial aspect of this transformation. However, most of the current deep learning approaches do not explicitly consider and apply scaffold hopping strategy when performing molecular generation. In this work, we propose ScaffoldGVAE, a variational autoencoder based on multi-view graph neural networks, for scaffold generation and scaffold hopping of drug molecules. The model integrates several important components, such as node-central and edge-central message passing, side-chain embedding, and Gaussian mixture distribution of scaffolds. To assess the efficacy of our model, we conduct a comprehensive evaluation and comparison with baseline models based on seven general generative model evaluation metrics and four scaffold hopping generative model evaluation metrics. The results demonstrate that ScaffoldGVAE can explore the unseen chemical space and generate novel molecules distinct from known compounds. Especially, the scaffold hopped molecules generated by our model are validated by the evaluation of GraphDTA, LeDock, and MM/GBSA. The case study of generating inhibitors of LRRK2 for the treatment of PD further demonstrates the effectiveness of ScaffoldGVAE in generating novel compounds through scaffold hopping. This novel approach can also be applied to other protein targets of various diseases, thereby contributing to the future development of new drugs. Source codes and data are available at https://github.com/ecust-hc/ScaffoldGVAE.

The skewness of mean–variance normal mixtures

Mean–variance mixtures of normal distributions are very flexible: they model many nonnormal features, such as skewness, kurtosis and multimodality. Special cases include generalized asymmetric Laplace distributions, mixtures of two normal distributions with proportional covariance matrices, scale mixtures of normal distributions and normal distributions. This paper investigates the skewness of multivariate mean–variance normal mixtures. The special case of mixtures of two normal distributions with proportional covariance matrices is treated in greater detail. The paper derives the analytical forms of prominent measures of multivariate skewness and applies them to model-based clustering, normalizing linear transformations, projection pursuit and normality testing. The practical relevance of the theoretical results is assessed with both real and simulated data.

GMM-Based Expanded Feature Space as a Way to Extract Useful Information for Rare Cell Subtypes Identification in Single-Cell Mass Cytometry.

Cell subtype identification from mass cytometry data presents a persisting challenge, particularly when dealing with millions of cells. Current solutions are consistently under development, however, their accuracy and sensitivity remain limited, particularly in rare cell-type detection due to frequent downsampling. Additionally, they often lack the capability to analyze large data sets. To overcome these limitations, a new method was suggested to define an extended feature space. When combined with the robust clustering algorithm for big data, it results in more efficient cell clustering. Each marker's intensity distribution is presented as a mixture of normal distributions (Gaussian Mixture Model, GMM), and the expanded space is created by spanning over all obtained GMM components. The projection of the initial flow cytometry marker domain into the expanded space employs GMM-based membership functions. An evaluation conducted on three established cellular identification algorithms (FlowSOM, ClusterX, and PARC) utilizing the most substantial publicly available annotated dataset by Samusik et al. demonstrated the superior performance of the suggested approach in comparison to the standard. Although our approach identified 20 cell clusters instead of the expected 24, their intra-cluster homogeneity and inter-cluster differences were superior to the 24-cluster FlowSOM-based solution.

Information criteria for matrix exponential spatial specifications

In this study, we suggest using information criteria for nested and non-nested model selection problems for the matrix exponential spatial specifications (MESS) under both homoskedasticity and heteroskedasticity. To this end, we consider the deviance information criterion, the Akaike information criterion and the Bayesian information criterion in a Bayesian setting. In the heteroskedastic case, we assume that the error terms have a scale mixture of normal distributions, where the scale mixture variables are latent variables that lead to different distributions. We demonstrate how the integrated likelihood function can be obtained analytically by integrating out the scale mixture variables from the complete-data likelihood function, and how this integrated likelihood function can be used to formulate the information criteria. We investigate the finite sample performance of these criteria in selecting the true model in a simulation study. The results show that these criteria perform satisfactorily and can be useful for selecting the correct model in specification search exercises. Finally, we apply the proposed information criteria to a spatially augmented growth model and a carbon emission model to show their usefulness for both nested and non-nested model selection problems.

Robust joint modelling of sparsely observed paired functional data

AbstractA reduced‐rank mixed‐effects model is developed for robust modelling of sparsely observed paired functional data. In this model, the curves for each functional variable are summarized using a few functional principal components, and the association of the two functional variables is modelled through the association of the principal component scores. A multivariate‐scale mixture of normal distributions is used to model the principal component scores and the measurement errors in order to handle outlying observations and achieve robust inference. The mean functions and principal component functions are modelled using splines, and roughness penalties are applied to avoid overfitting. An EM algorithm is developed for computation of model fitting and prediction. A simulation study shows that the proposed method outperforms an existing method, which is not designed for robust estimation. The effectiveness of the proposed method is illustrated through an application of fitting multiband light curves of Type Ia supernovae.

A Novel Remaining Useful Life Probability Prediction Approach for Aero-Engine with Improved Bayesian Uncertainty Estimation Based on Degradation Data

As the heart of aircraft, the aero-engine is not only the main power source for aircraft flight but also an essential guarantee for the safe flight of aircraft. Therefore, it is of great significance to find effective methods for remaining useful life (RUL) prediction for aero-engines in order to avoid accidents and reduce maintenance costs. With the development of deep learning, data-driven approaches show great potential in dealing with the above problem. Although many attempts have been made, few works consider the error of the point prediction result caused by uncertainties. In this paper, we propose a novel RUL probability prediction approach for aero-engines with prediction uncertainties fully considered. Before forecasting, a principal component analysis (PCA) is first utilized to cut down the dimension of sensor data and extract the correlation between multivariate data to reduce the network computation. Then, a multi-layer bidirectional gate recurrent unit (BiGRU) is constructed to predict the RUL of the aero-engine, while prediction uncertainties are quantized by the improved variational Bayesian inference (IVBI) with a Gaussian mixture distribution. The proposed method can give not only the point prediction of RUL but also the confidence interval of the prediction result, which is very helpful for real-world applications. Finally, the experimental study illustrates that the proposed method is feasible and superior to several other comparative models.

Robust information filtering for distributed networks using mixed Gaussian distribution

This paper proposes a robust distributed state estimation algorithm for sensor networks. In a distributed sensor network, sensor nodes may be affected by outliers and thus influence the information fusion results. In this respect, the measurement likelihood and one-step state prediction are modeled by a mixture of Gaussian distributions. In this Gaussian mixture, one Gaussian uses nominal covariance, while the other uses adaptive covariance with larger values and lower probabilities. We build a centralized multi-sensor fusion model in the variational Bayesian framework based on the aforementioned mixture of Gaussian distributions. Then, a consensus strategy is employed to convert the centralized method into a distributed one. Each node completes estimation tasks separately based on the information from itself and nearby nodes. In addition, the weight of sensor information will adaptively update according to the Gaussian distribution that mainly affects the current sensor measurement noise. Finally, simulation experiments demonstrate the effectiveness of the proposed method in handling outlier effects.

Three-Part Composite Pareto Modelling for Income Distribution in Malaysia

Income distribution models can be useful for describing the economic properties of a population. In this study, three-part composite Pareto models are fitted to the income distribution in Malaysia for the years 2007, 2009, 2012, 2014, and 2016. The three-part composite Pareto models divide the population into three parts, each following a different distribution model. The lower part follows the inverse Pareto distribution, the upper part follows the Pareto distribution, and the middle part follows another unspecified distribution model. For application in income data, the use of Gaussian mixture distribution is proposed for the middle part, making the inverse Pareto–Gaussian mixture-Pareto distribution model semi-parametric. From the model, it is found that the levels of income inequality in the lower and upper income groups decrease over the period of study. Additionally, the proportion of data following the inverse Pareto distribution in the model is highly correlated with the official absolute poverty incidence.

A generalized framework for quantifying and monitoring the severity of meteorological drought

The current study proposes a new framework for quantifying and monitoring the severity of meteorological drought. The proposed framework consists of three phases. The first phase of the framework uses K-component Gaussian Mixture Distribution (GMD) in the computation. The second phase is mainly based on the dissimilarity matrix-based clustering using C-index and Monte Carlo Feature-based Selection (MCFS) method. The third phase uses the Markov chain, transition probabilities and a non-homogeneous Poisson process under the Bayesian estimation. The Relative Importance (RI) values are used to choose appropriate stations. The Deviance Information Criteria (DIC) is used to check model suitability, and Root Mean Square Error (RMSE) is utilized for determining model performance. The proposed framework is validated to the 52 meteorological stations in Pakistan for 49 years from 1968 to 2016. Moreover, the outcomes of the current analysis provide insight to quantify and monitor meteorological drought comprehensively and accurately.

A robust sparse Bayesian learning method for the structural damage identification by a mixture of Gaussians

Sparse Bayesian learning methods have been successfully applied to the community of structural damage identification, which commonly assumes that the uncertainties from modeling errors and measurement noises follow Gaussian distribution. However, this assumption typically compromises the accuracy of damage identification because these real uncertainties may not follow a Gaussian distribution. A mixture of Gaussian distributions can more accurately quantify the real uncertainties of damage identification than a Gaussian distribution because it can theoretically approximate any continuous distribution. Motivated by this, we utilized a robust sparse Bayesian learning method to investigate the problem of structural damage identification and further improve the accuracy of damage identification. This method includes a sparse Bayesian learning model with a mixture of Gaussians and an expectation–maximization algorithm based on the Laplace approximation technique. The numerical and experimental results illustrate that our method improves the damage identification accuracy by 6.78% and 11.92% than the existing two sparse Bayesian learning ones, respectively.

Regression modeling of censored data based on compound scale mixtures of normal distributions

Regression modeling of censored data based on compound scale mixtures of normal distributions

Robust quantitative susceptibility mapping via approximate message passing with parameter estimation.

For quantitative susceptibility mapping (QSM), the lack of ground-truth in clinical settings makes it challenging to determine suitable parameters for the dipole inversion. We propose a probabilistic Bayesian approach for QSM with built-in parameter estimation, and incorporate the nonlinear formulation of the dipole inversion to achieve a robust recovery of the susceptibility maps. From a Bayesian perspective, the image wavelet coefficients are approximately sparse and modeled by the Laplace distribution. The measurement noise is modeled by a Gaussian-mixture distribution with two components, where the second component is used to model the noise outliers. Through probabilistic inference, the susceptibility map and distribution parameters can be jointly recovered using approximate message passing (AMP). We compare our proposed AMP with built-in parameter estimation (AMP-PE) to the state-of-the-art L1-QSM, FANSI, and MEDI approaches on the simulated and in vivo datasets, and perform experiments to explore the optimal settings of AMP-PE. Reproducible code is available at: https://github.com/EmoryCN2L/QSM_AMP_PE. On the simulated Sim2Snr1 dataset, AMP-PE achieved the lowest NRMSE, deviation from calcification moment and the highest SSIM, while MEDI achieved the lowest high-frequency error norm. On the in vivo datasets, AMP-PE is robust and successfully recovers the susceptibility maps using the estimated parameters, whereas L1-QSM, FANSI and MEDI typically require additional visual fine-tuning to select or double-check working parameters. AMP-PE provides automatic and adaptive parameter estimation for QSM and avoids the subjectivity from the visual fine-tuning step, making it an excellent choice for the clinical setting.

Exploring the reform of the environmental design education model under the OBE theory model of professional and creative depth integration

Abstract To be able to better improve the professional ability of environmental design students and cultivate applied technical talents, this paper proposes an analysis of the OBE theory for the reform of specialized and deeply integrated environmental design education. By establishing a mathematical model, the database is divided into different classes based on the similarity between data. The error value of the objective function is gradually reduced by repeated iterative operations. The data points are randomly selected to calculate the center distance, and the dense area of the input domain is obtained in the data set obeying Gaussian mixture distribution. Through iteration to find the optimal solution, adjust its speed and position so that the fitness function takes the maximum value. The speed is constantly revised according to its own experience and group information to enhance the global seeking ability, improve the local seeking ability of particles and avoid the oscillation phenomenon, and select the particle with the highest adaptability to calculate its average fluctuation value, to analyze the environmental design education reform under OBE theory. The result analysis shows that the correct rate of the algorithm analysis performance of this paper reaches 90.1%, which ensures more complete and accurate environmental design education data.