Joint Gaussian Mixture Model Research Articles

Joint Gaussian mixture model for versatile deep visual model explanation

Explainable AI (XAI) promotes understandability and credibility in the complex decision-making process of machine learning models. Post-hoc explanations for deep neural networks are crucial because they focus on faithful explanation of the learned representations, decision mechanisms, and uncertainty of the model. Explaining deep convolutional neural networks (DCNN) is particularly challenging because of the high dimensionality of deep features and the complexity of model inference. Most post-hoc explanation methods follow one single paradigm of explanation, confining the diversity and consistency of the explanation. This study proposes a joint Gaussian mixture model (JGMM), which is a probabilistic model that jointly models deep inter-layer features and produces faithful and consistent post-hoc explanations. The JGMM explains deep features by Gaussian mixture model and model inference by the posterior distribution of the latent component variables. JGMM enables a versatile explanation framework that unifies the interpretable proxy model and mining global and local explanatory examples. Experiments are performed on various DCNN image classifiers in comparison with other explaining methods. The results show that the JGMM can efficiently produce versatile, consistent, faithful, and understandable explanations.

Maximizing Nonlocal Self-Similarity Prior for Single Image Super-Resolution

Prior knowledge plays an important role in the process of image super-resolution reconstruction, which can constrain the solution space efficiently. In this paper, we utilized the fact that clear image exhibits stronger self-similarity property than other degradated version to present a new prior called maximizing nonlocal self-similarity for single image super-resolution. For describing the prior with mathematical language, a joint Gaussian mixture model was trained with LR and HR patch pairs extracted from the input LR image and its lower scale, and the prior can be described as a specific Gaussian distribution by derivation. In our algorithm, a large scale of sophisticated training and time-consuming nearest neighbor searching is not necessary, and the cost function of this algorithm shows closed form solution. The experiments conducted on BSD500 and other popular images demonstrate that the proposed method outperforms traditional methods and is competitive with the current state-of-the-art algorithms in terms of both quantitative metrics and visual quality.

Compressive Sensing With Side Information: How to Optimally Capture This Extra Information for GMM Signals?

This paper studies how to optimally capture side information to aid in the reconstruction of high-dimensional signals from low-dimensional random linear and noisy measurements, by assuming that both the signal of interest and the side information signal are drawn from a joint Gaussian mixture model. In particular, we derive sufficient and (occasionally) necessary conditions on the number of linear measurements for the signal reconstruction minimum mean squared error (MMSE) to approach zero in the low-noise regime; moreover, we also derive closed-form linear side information measurement designs for the reconstruction MMSE to approach zero in the low-noise regime. Our designs suggest that a linear projection kernel that optimally captures side information is such that it measures the attributes of side information that are maximally correlated with the signal of interest. A number of experiments both with synthetic and real data confirm that our theoretical results are well aligned with numerical ones. Finally, we offer a case study associated with a panchromatic sharpening (pan sharpening) application in the presence of compressive hyperspectral data that demonstrates that our proposed linear side information measurement designs can lead to reconstruction peak signal-to-noise ratio (PSNR) gains in excess of 2 dB over other approaches in this practical application.

Ensemble image registration by a spatially constrained clustering approach

In this article, a novel spatially constrained clustering approach is proposed for ensemble image registration. We use a spatially constrained Gaussian mixture model, which is based on a joint Gaussian mixture model and Markov random field, to model the joint intensity scatter plot of the unregistered images. The spatially constrained Gaussian mixture model has the capability of performing the correlation among neighboring observations. A cost function of reducing the dispersion in the joint intensity scatter plot is proposed using the spatially constrained Gaussian mixture model to simultaneously register a group of images. We derive an expectation maximization algorithm for the proposed model. Computer simulations demonstrate the effectiveness of the proposed method.

Ensemble Registration of Multisensor Images by a Variational Bayesian Approach

In this paper, a novel image registration method, called ensemble image registration, is proposed. We use an infinite Gaussian mixture model (IGMM), which is based on a joint Gaussian mixture model and a Dirichlet process (DP), to model the joint intensity scatter plot (JISP) of the unregistered images. The DP is the cornerstone of nonparametric Bayesian statistics, and has capability of determining a proper number of mixing components. To simultaneously register a group of images, the cost function of reducing the dispersion in the JISP is preformed by a Bayesian method using IGMM. A variational Bayesian framework is developed to inference the posterior distribution of the parameters in the IGMM. To evaluate the performance of the proposed method, experiments of ensemble image registration are presented. The results show that the proposed method has improved performance compared with conventional methods.

Voice Conversion Based on Hybrid SVR and GMM

A novel VC (voice conversion) method based on hybrid SVR (support vector regression) and GMM (Gaussian mixture model) is presented in the paper, the mapping abilities of SVR and GMM are exploited to map the spectral features of the source speaker to those of target ones. A new strategy of F0 transfor- mation is also presented, the F0s are modeled with spectral features in a joint GMM and predicted from the converted spectral features using the SVR method. Subjective and objective tests are carried out to evaluate the VC performance; experimental results show that the converted speech using the proposed method can obtain a better quality than that using the state-of-the-art GMM method. Meanwhile, a VC method based on non-parallel data is also proposed, the speaker-specific information is investigated us- ing the SVR method and preliminary subjective experiments demonstrate that the proposed method is feasible when a parallel corpus is not available.

Polyphonic Pitch Estimation and Instrument Identification by Joint Modeling of Sustained and Attack Sounds

Polyphonic pitch estimation and musical instrument identification are some of the most challenging tasks in the field of music information retrieval (MIR). While existing approaches have focused on the modeling of harmonic partials, we design a joint Gaussian mixture model of the harmonic partials and the inharmonic attack of each note. This model encodes the power of each partial over time as well as the spectral envelope of the attack part. We derive an expectation-maximization (EM) algorithm to estimate the pitch and the parameters of the notes. We then extract timbre features both from the harmonic and the attack part via principal component analysis (PCA) over the estimated model parameters. Musical instrument recognition for each estimated note is finally carried out with a support vector machine (SVM) classifier. Experiments conducted on mixtures of isolated notes as well as real-world polyphonic music show higher accuracy over state-of-the-art approaches based on the modeling of harmonic partials only.

Radar HRRP Statistical Recognition: Parametric Model and Model Selection

Statistical modeling for radar high-resolution range profile (HRRP) is a challenging task in radar HRRP statistical recognition. Theoretical analysis and experimental results show that elements in an HRRP sample are statistically correlated and non-Gaussian distributed. First, this paper introduces three joint-Gaussian models, i.e., subspace approximation model, probability principal components analysis (PPCA) model and factor analysis (FA) model, into radar HRRP statistical recognition. Due to the experimental results, we can have the conclusion that the jointly non-Gaussian distributed HRRP samples approximately follow the joint-Gaussian distribution described by FA model. Therefore, we can apply FA model to radar HRRP statistical recognition rather than a joint-Gaussian mixture model, e.g., PPCA mixture model or FA mixture model, which is a more accurate choice for modeling non-Gaussian distributed correlations in multidimensional data but with high learning complexity and large computation burden, and the difficulty in the statistical modeling for HRRP samples is largely reduced. Second, this paper concerns model selection of FA model in radar HRRP statistical recognition, in which there are two issues, i.e., the partition of target-aspect frames and the determination of the number of factors in each frame. Based on the Akaike information criterion (AIC) and the Bayes' information criterion (BIC), an iterated algorithm for model selection is proposed in this paper, which can automatically give the optimal aspect-frame boundaries and determine the optimal number of factors in each aspect-frame. The recognition experiments based on measured data show that the proposed adaptive partition approach can further improve the recognition performance with higher recognition efficiency.