Process Gaussian Mixture Model Research Articles

Credit rating algorithm of corporate bonds based on Gaussian process mixture model and improved K-means

The primary challenge in credit analysis revolves around uncovering the correlation between repayment terms and yield to maturity, constituting the interest rate term structure-an essential model for corporate credit term evaluation. Presently, interest rate term structures are predominantly examined through economic theoretical models and quantitative models. However, predicting treasury bond yields remains a challenging task for both approaches. Leveraging the clustering analysis algorithm theory and the attributes of an insurance company’s customer database, this paper enhances the K-means clustering algorithm, specifically addressing the selection of initial cluster centers in extensive sample environments. Utilizing the robust data fitting and analytical capabilities of the Gaussian process mixture model, the study applies this methodology to model and forecast Treasury yields. Additionally, the research incorporates customer credit data from a property insurance company to investigate the application of clustering algorithms in the analysis of insurance customer credit.

Gaussian process mixture model for prediction based on maximum posterior distribution

In this paper, a numerical experiment was proposed, which is based on the Bayesian procedure and Gaussian Process. Gaussian Process prior distribution was supposed as a before evaluating the hyperparameters of the Gaussian Process model via GyPytorch. Finally, the Gaussian process posterior was achieved and then confidence regions were visualized in two-dimensional planes. The Gaussian Process model could perform under common or specified kernel functions, while it depends on the prior information to a great extent.

Bayesian Pure-Tone Audiometry Through Active Learning Under Informed Priors.

Pure-tone audiometry—the process of estimating a person's hearing threshold from “audible” and “inaudible” responses to tones of varying frequency and intensity—is the basis for diagnosing and quantifying hearing loss. By taking a probabilistic modeling approach, both optimal tone selection (in terms of expected information gain) and hearing threshold estimation can be derived through Bayesian inference methods. The performance of probabilistic model-based audiometry methods is directly linked to the quality of the underlying model. In recent years, Gaussian process (GP) models have been shown to provide good results in this context. We present methods to improve the efficiency of GP-based audiometry procedures by improving the underlying model. Instead of a single GP, we propose to use a GP mixture model that can be conditioned on side-information about the subject. The underlying idea is that one can typically distinguish between different types of hearing thresholds, enabling a mixture model to better capture the statistical properties of hearing thresholds among a population. Instead of modeling all hearing thresholds by a single GP, a mixture model allows specific types of hearing thresholds to be modeled by independent GP models. Moreover, the mixing coefficients can be conditioned on side-information such as age and gender, capturing the correlations between age, gender, and hearing threshold. We show how a GP mixture model can be optimized for a specific target population by learning the parameters from a data set containing annotated audiograms. We also derive an optimal tone selection method based on greedy information gain maximization, as well as hearing threshold estimation through Bayesian inference. The proposed models are fitted to a data set containing roughly 176 thousand annotated audiograms collected in the Nordic countries. We compare the predictive accuracies of optimized mixture models of varying sizes with that of an optimized single-GP model. The usefulness of the optimized models is tested in audiometry simulations. Simulation results indicate that an optimized GP mixture model can significantly outperform an optimized single-GP model in terms of predictive accuracy, and leads to significant increases the efficiency of the resulting Bayesian audiometry procedure.

Discovering key transcriptomic regulators in pancreatic ductal adenocarcinoma using Dirichlet process Gaussian mixture model

Pancreatic Ductal Adenocarcinoma (PDAC) is the most lethal type of pancreatic cancer, late detection leading to its therapeutic failure. This study aims to determine the key regulatory genes and their impacts on the disease’s progression, helping the disease’s etiology, which is still mostly unknown. We leverage the landmark advantages of time-series gene expression data of this disease and thereby identified the key regulators that capture the characteristics of gene activity patterns in the cancer progression. We have identified the key gene modules and predicted the functions of top genes from a reconstructed gene association network (GAN). A variation of the partial correlation method is utilized to analyze the GAN, followed by a gene function prediction task. Moreover, we have identified regulators for each target gene by gene regulatory network inference using the dynamical GENIE3 (dynGENIE3) algorithm. The Dirichlet process Gaussian process mixture model and cubic spline regression model (splineTimeR) are employed to identify the key gene modules and differentially expressed genes, respectively. Our analysis demonstrates a panel of key regulators and gene modules that are crucial for PDAC disease progression.

Multiview learning with variational mixtures of Gaussian processes

Gaussian processes (GPs) are powerful Bayesian nonparametric tools widely used in probabilistic modeling, and the mixture of GPs (MGPs) were introduced afterwards to make data modeling more flexible. However, MGPs are not directly applicable to multiview learning. In order to improve the modeling ability of MGPs, in this paper, we propose a new framework of multiview learning for the MGPs and instantiate it for classification. We make the divergence between views as small as possible while ensuring that the posterior probability of each view is as large as possible. Specifically, we regularize the posterior distribution of latent variables with the consistency of posterior distributions of the latent functions between different views. Since it is intractable to solve the model analytically, the variational inference and optimization algorithms of the classification model are also presented in this paper. Experimental results on multiple real-world datasets have shown that the proposed method has outperformed the original MGP model and several state-of-the-art multiview learning methods, which indicate the effectiveness of the proposed multiview learning framework for MGPs.

A Gaussian process mixture model-based hard-cut iterative learning algorithm for air quality prediction

Air quality is closely related to concentrations of gaseous pollutants, and the prediction of gaseous pollutant concentration plays a decisive role in regulating plant and vehicle emissions. Due to the non-linear and chaotic characteristics of the gas concentration series, traditional models may not easily capture the complex time series pattern. In this study, the Gaussian Process Mixture (GPM) model, which adopts hidden variables posterior hard-cut (HC) iterative learning algorithm, is first applied to the prediction of gaseous pollutant concentration in order to improve prediction performance. This algorithm adopts iterative learning and uses the maximizing a posteriori (MAP) estimation to achieve the optimal grouping of samples which effectively improves the expectation–maximization (EM) learning in GPM. The empirical results of the GPM model reveals improved prediction accuracy in gaseous pollutant concentration prediction, as compared with the kernel regression (K-R), minimax probability machine regression (MPMR), linear regression (L-R) and Gaussian Processes (GP) models. Furthermore, GPM with various learning algorithms, namely the HC algorithm, Leave-one-out Cross Validation (LOOCV), and variational algorithms, respectively, are also examined in this study. The results also show that the GPM with HC learning achieves superior performance compared with other learning algorithms.

Enhanced Gaussian process mixture model for short-term electric load forecasting

This research used a hard-cut iterative training algorithm to improve a Gaussian process mixture (GPM) model. Our enhanced GPM (EGPM) concisely estimates distribution parameters to the greatest extent possible. GPM models are powerful tools for data presentation and forecasting owing to their linear mix of multiple Gaussian process (GP) models. The hidden posterior probability distribution variables in the GPM model, which are based on the hard-cut algorithm, are 0 and 1, respectively, which can simplify the training process and reduce calculation requirements by training each GP via a maximum likelihood estimation method. The EGPM model is then used for a short-term electric load forecasting problem and compared with various forecasting models. First, the EGPM results are compared with those of two previous GPM training algorithms: the variational and leave-one-out cross validation (LOOCV) algorithms. The experimental results indicate that the EGPM model can accurately and more reliably forecast electric loads. The GP, support vector machine, and radial basis function network are also assessed for their ability to solve the short-term electric load forecasting problem. The empirical results indicate that the performance of the proposed EGPM is superior to that of the other methods in terms of forecasting accuracy.

Clustering gene expression time series data using an infinite Gaussian process mixture model.

Transcriptome-wide time series expression profiling is used to characterize the cellular response to environmental perturbations. The first step to analyzing transcriptional response data is often to cluster genes with similar responses. Here, we present a nonparametric model-based method, Dirichlet process Gaussian process mixture model (DPGP), which jointly models data clusters with a Dirichlet process and temporal dependencies with Gaussian processes. We demonstrate the accuracy of DPGP in comparison to state-of-the-art approaches using hundreds of simulated data sets. To further test our method, we apply DPGP to published microarray data from a microbial model organism exposed to stress and to novel RNA-seq data from a human cell line exposed to the glucocorticoid dexamethasone. We validate our clusters by examining local transcription factor binding and histone modifications. Our results demonstrate that jointly modeling cluster number and temporal dependencies can reveal shared regulatory mechanisms. DPGP software is freely available online at https://github.com/PrincetonUniversity/DP_GP_cluster.

Multimodality Prediction of Chaotic Time Series with Sparse Hard-Cut EM Learning of the Gaussian Process Mixture Model**Supported by the National Natural Science Foundation of China under Grant No 60972106, the China Postdoctoral Science Foundation under Grant No 2014M561053, the Humanity and Social Science Foundation of Ministry of Education of China under Grant No 15YJA630108, and the Hebei Province Natural Science

The contribution of this work is twofold: (1) a multimodality prediction method of chaotic time series with the Gaussian process mixture (GPM) model is proposed, which employs a divide and conquer strategy. It automatically divides the chaotic time series into multiple modalities with different extrinsic patterns and intrinsic characteristics, and thus can more precisely fit the chaotic time series. (2) An effective sparse hard-cut expectation maximization (SHC-EM) learning algorithm for the GPM model is proposed to improve the prediction performance. SHC-EM replaces a large learning sample set with fewer pseudo inputs, accelerating model learning based on these pseudo inputs. Experiments on Lorenz and Chua time series demonstrate that the proposed method yields not only accurate multimodality prediction, but also the prediction confidence interval. SHC-EM outperforms the traditional variational learning in terms of both prediction accuracy and speed. In addition, SHC-EM is more robust and insusceptible to noise than variational learning.

Bayesian Nonparametric Modeling for Rapid Design of Metamaterial Microstructures

We consider the problem of rapid design of massive metamaterial (MTM) microstructures from a statistical point of view. A Bayesian nonparametric model, namely, Gaussian Process (GP) mixture, is developed to generate the mapping relationship from the microstructure’s geometric dimension to the electromagnetic response, which is approximately expressed in a closed form of Drude-Lorentz type model. This GP mixture model is neatly able to tackle nonstationarity, discontinuities in the mapping function. The inference is performed using a Markov chain relying on Gibbs sampling. Experimental results demonstrate that the proposed approach is highly efficient in facilitating rapid design of MTM with accuracy.

A latent variable Gaussian process model with Pitman–Yor process priors for multiclass classification

Gaussian processes (GPs) constitute one of the most important Bayesian machine learning approaches. Several researchers have considered postulating mixtures of Gaussian processes as a means of dealing with non-stationary covariance functions, discontinuities, multi-modality, and overlapping output signals. In existing works, mixtures of Gaussian processes are based on the introduction of a gating function defined over the space of model input variables. This way, each postulated mixture component Gaussian process is effectively restricted in a limited subset of the input space. Additionally, the applicability of these models is limited to regression tasks. In this paper, for the first time in the literature, we devise a Gaussian process mixture model especially suitable for multiclass classification applications: We consider a GP classification scheme the prior distribution of which is a fully generative nonparametric Bayesian model with power-law behavior, generating Gaussian processes over the whole input space of the learned task. We provide an efficient algorithm for model inference, based on the variational Bayesian framework, and exhibit its efficacy using benchmark and real-world classification datasets.

Learning gas distribution models using sparse Gaussian process mixtures

In this paper, we consider the problem of learning two-dimensional spatial models of gas distributions. To build models of gas distributions that can be used to accurately predict the gas concentration at query locations is a challenging task due to the chaotic nature of gas dispersal. We formulate this task as a regression problem. To deal with the specific properties of gas distributions, we propose a sparse Gaussian process mixture model, which allows us to accurately represent the smooth background signal and the areas with patches of high concentrations. We furthermore integrate the sparsification of the training data into an EM procedure that we apply for learning the mixture components and the gating function. Our approach has been implemented and tested using datasets recorded with a real mobile robot equipped with an electronic nose. The experiments demonstrate that our technique is well-suited for predicting gas concentrations at new query locations and that it outperforms alternative and previously proposed methods in robotics.

Hierarchical Gaussian process mixtures for regression

As a result of their good performance in practice and their desirable analytical properties, Gaussian process regression models are becoming increasingly of interest in statistics, engineering and other fields. However, two major problems arise when the model is applied to a large data-set with repeated measurements. One stems from the systematic heterogeneity among the different replications, and the other is the requirement to invert a covariance matrix which is involved in the implementation of the model. The dimension of this matrix equals the sample size of the training data-set. In this paper, a Gaussian process mixture model for regression is proposed for dealing with the above two problems, and a hybrid Markov chain Monte Carlo (MCMC) algorithm is used for its implementation. Application to a real data-set is reported.