MCMC Algorithm Research Articles

AddiVortes: (Bayesian) Additive Voronoi Tessellations

The Additive Voronoi Tessellations (AddiVortes) model is a multivariate regression model that uses Voronoi tessellations to partition the covariate space in an additive ensemble model. Unlike other partition methods, such as decision trees, this has the benefit of allowing the boundaries of the partitions to be non-orthogonal and non-parallel to the covariate axes. The AddiVortes model uses a similar sum-of-tessellations approach and a Bayesian backfitting MCMC algorithm to the BART model. We utilize regularization priors to limit the strength of individual tessellations and accepts new models based on a likelihood. The performance of the AddiVortes model is illustrated through testing on several data sets and comparing the performance to other models along with a simulation study to verify some of the properties of the model. In many cases, the AddiVortes model outperforms random forests, BART and other leading black-box regression models when compared using a range of metrics. Supplementary materials for this article are available online.

Jump around: Selecting Markov Chain Monte Carlo parameters and diagnostics for improved food web model quality and ecosystem representation

Capturing ecological data variability in food web models is an important step for improving model representation of empirical systems. One approach is to use linear inverse modelling and Markov Chain Monte Carlo (LIM-MCMC) techniques to set up an inverse LIM problem using empirical data constraints, and then sample multiple plausible food webs from the inverse problem using an MCMC algorithm. We describe the set of plausible food webs as an ‘ensemble’ of solutions to the inverse problem sampled with the LIM-MCMC algorithm. The extent of data variability eventually integrated into an ensemble depends on how well the LIM-MCMC algorithm samples the solution space. Algorithm quality can be adjusted via user-defined parameters describing starting points, jump sizes, and number of iterations or food webs produced. However, little information exists on how each LIM-MCMC algorithm parameter affects the degree of empirical data variability introduced into the ensemble. Further, post hoc algorithm quality diagnostics with commonly used trace plots and the coefficient of variation (CoV) rarely address critical aspects of algorithm quality, such as (1) if the returned ensemble successfully targeted the solution space distribution (stationarity), (2) correlation between ensemble solutions (mixing), and (3) if the ensemble contains enough solutions to adequately capture input data variability (sampling efficiency). Therefore, we used several established MCMC convergence diagnostics to (1) quantify how algorithm parameters affect ensemble flow values and if these differences propagate to ecological indicators and (2) evaluate algorithm quality and compare to current evaluation and ecosystem modelling methods. We applied 30 LIM-MCMC algorithm combinations of varying starting points, jump sizes, and number of iterations to solve food web ensembles from a single food web model. We analysed ensembles with Ecological Network Analysis (ENA) to calculate indicators describing system function. Results show that LIM-MCMC algorithm parameters, in particular the jump size, affect ensemble flow values, which propagate to ecological indicators describing different ecosystem function of the same model. Thereafter, comparisons of post hoc diagnostics show that MCMC convergence diagnostics provided more robust estimates of algorithm quality than trace plots and CoV. Together, these findings underpin several novel recommendations to enhance LIM-MCMC algorithm parameter selection and quality assessments applicable to any ecological ensemble network study.

Enhancing subset simulation through Bayesian inference

Analyzing and reducing uncertainty in estimating failure probability has always been a crucial part of reliability analysis using Monte Carlo simulation. This paper employs Bayesian inference to capture uncertainty within Subset Simulation (SuS), with a specific emphasis on constructing the posterior distribution of failure probability. Two types of Bayesian models are discussed. The first type is based on the distribution form of the failure probability estimator from the SuS, where the Log-normal and Beta distributions are compared and analyzed. The second type is based on the distribution of statistics in Bernoulli trials in SuS, involving models such as the Beta-Binomial and Uniform-Bernoulli trials with Sequential Dependence. After evaluating all models among two types, the Log-normal distribution-based model was selected as the optimal solution due to its consideration of sample correlation and closed form of Maximum A Posteriori (MAP). Further exploration of this model reveals that its MAP estimation can provide less bias and deviation compared to mean estimation when SuS is run multiple times. Additionally, when compared to the point estimation of failure probability in conventional SuS, the posterior distribution of failure probability provides a more comprehensive overview, capturing the effects of sample size and MCMC algorithms.

Using Early Rejection Markov Chain Monte Carlo and Gaussian Processes to Accelerate ABC Methods

Approximate Bayesian computation (ABC) is a class of Bayesian inference algorithms that targets problems with intractable or unavailable likelihood functions. It uses synthetic data drawn from the simulation model to approximate the posterior distribution. However, ABC is computationally intensive for complex models in which simulating synthetic data is very expensive. In this article, we propose an early rejection Markov chain Monte Carlo (ejMCMC) sampler based on Gaussian processes to accelerate inference speed. We early reject samples in the first stage of the kernel using a discrepancy model, in which the discrepancy between the simulated and observed data is modeled by Gaussian process (GP). Hence, synthetic data is generated only if the parameter space is worth exploring. We demonstrate through theory, simulation experiments, and real data analysis that the new algorithm significantly improves inference efficiency compared to existing early-rejection MCMC algorithms. In addition, we employ our proposed method within an ABC sequential Monte Carlo (SMC) sampler. In our numerical experiments, we use examples of ordinary differential equations, stochastic differential equations, and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We develop an R package that is available at https://github.com/caofff/ejMCMC.

ASASSN-21js: A multi-year transit of a ringed disc

Aims. The early-type star ASASSN-21js started to fade in 2021, as was detected by the All Sky Automated Survey for Supernovae, undergoing a multi-year eclipse that is still underway. We interpret this event as being due to a structured disc of material transiting in front of the star. The disc is in orbit around a substellar object with the mass and luminosity of a brown dwarf or smaller. We want to determine the expected duration and ending date of the eclipse. Methods. We modelled a tilted and inclined azimuthally symmetric ring system around an unseen companion and calculated the resulting time-varying light curve as the object transited in front of the star. We made an initial estimate of the ring parameters and used these as inputs to an MCMC algorithm to determine the geometric properties of the rings with associated uncertainties. Results. The model most consistent with the light curve to date is a two-ring system at high inclination with respect to the line of sight that has a semi-major axis of 71.6 stellar radii. With an estimate of the stellar radius, the transverse velocity is around 0.7 km s−1, which if bound to the star is an orbit with a semi-major axis of around 13 000 au, placing it in the Oort cloud of the parent star. The transit is ongoing and will finish around MJD 61526 (May 1 2027). We encourage the community to continue observing this object in order to understand its properties.

Research on Molten Iron Quality Prediction Based on Machine Learning

The quality of molten iron not only has a significant impact on the strength, toughness, smelting cost and service life of cast iron but also directly affects the satisfaction of users. The establishment of timely and accurate blast furnace molten iron quality prediction models is of great significance for the improvement of the production efficiency of blast furnace. In this paper, Si, S and P content in molten iron is taken as the important index to measure the quality of molten iron, and the 989 sets of production data from a No.1 blast furnace from August to October 2020 are selected as the experimental data source, predicting the quality of molten iron by the I-GWO-CNN-BiLSTM model. First of all, on the basis of the traditional data processing method, the missing data values are classified into correlation data, temporal data, periodic data and manual input data, and random forest, the Lagrangian interpolation method, the KNN algorithm and the SVD algorithm are used to complete them, so as to obtain a more practical data set. Secondly, CNN and BiLSTM models are integrated and I-GWO optimized hyperparameters are used to form the I-GWO-CNN-BiLSTM model, which is used to predict Si, S and P content in molten iron. Then, it is concluded that using the I-GWO-CNN-BiLSTM model to predict the molten iron quality can obtain high prediction accuracy, which can provide data support for the regulation of blast furnace parameters. Finally, the MCMC algorithm is used to analyze the influence of the input variables on the Si, S and P content in molten iron, which helps the steel staff control the quality of molten iron in a timely manner, which is conducive to the smooth running of blast furnace production.

PyIRoGlass: An open-source, Bayesian MCMC algorithm for fitting baselines to FTIR spectra of basaltic-andesitic glasses

PyIRoGlass: An open-source, Bayesian MCMC algorithm for fitting baselines to FTIR spectra of basaltic-andesitic glasses

Identification of an Unknown Stationary Emission Source in Urban Geometry Using Bayesian Inference

Estimating the parameters of an unidentified toxic pollutant source is crucial for public safety, especially in densely populated urban areas. Implementing source term estimation methods in real-world urban environments is challenging due to complex phenomena and the absence of concentration observational data. This work combines a computational fluid dynamics numerical simulation with the Metropolis–Hastings MCMC algorithm to identify the location and quantify the release rate of an unknown source within the geometry of Augsburg city center. To address the lack of concentration measurements, synthetic observations are generated by a forward dispersion model. The methodology is tested using these datasets, both as directly calculated by the forward model and with added Gaussian noise under different source release and wind flow scenarios. The results indicate that in most cases, both the source location and the release rate are estimated accurately. Although a higher performance is achieved using synthetic datasets without additional noise, high accuracy predictions are also obtained in many applications of noisy measurement datasets. In general, the outcomes demonstrate that the presented methodology can be a useful tool for estimating unknown source parameters in real-world urban applications.

Copula-based Bayesian estimation of probabilities for two dependent non homogenous compound Poisson processes

Bivariate copulas for two dependent non homogeneous compound Poisson processes (NCPP) are considered. In this scenario, the arrival times of events are assumed to follow two independent non homogeneous Poisson processes (NHPP), but the corresponding renewal variables are dependent through a bivariate copula. The article aims to provide a computational tool for the estimation of the probability that the random sum associated with the first NCPP is less than the other random sum by a truncated time. The proposed ML and Bayesian estimation methods in a general setting can be applied using bivariate copulas such as Gumbel (for upper tail dependency), Clayton (for lower tail dependency), and Frank (for symmetric dependency). Power Law (PL) intensity function for the NHPPs and Inverse-Gaussian as renewal distribution are considered as examples. Bayes estimates of NHPPs parameters use gamma priors, and for Bayes estimates of Inverse-Gaussian distribution, an MCMC algorithm is employed. Our method has a broad range of practical applications, especially in the insurance sector, where the renewal variable may represent the dollar amount associated with a claim. To illustrate our approach, we analyze injury claims in North Carolina and South Carolina over a specific period, using a dataset sourced from docs.databricks.com.

Using traffic assignment models to assist Bayesian inference for origin–destination matrices

Estimation of traffic volumes between each origin and destination of travel is a standard practice in transport engineering. Commonly the available data constitute traffic counts at various locations on the network, supplemented by a survey-based prior estimate of mean origin–destination traffic volumes. Statistical inference in this type of network tomography problem is known to be challenging. Moreover, the difficulties are increased in practice by the presence of a large number of nuisance parameters corresponding to route choice probabilities, for which we have no direct prior information. Working in a Bayesian framework, we determine these parameters using a stochastic user equilibrium route choice model. We develop an MCMC algorithm for model fitting. This requires repeated computation of stochastic user equilibrium flows, and so we develop a computationally cheap emulator. Our methods are tested on numerical examples based on a section of the road network in the English city of Leicester.

Parallel MCMC algorithms: theoretical foundations, algorithm design, case studies

Abstract Parallel Markov Chain Monte Carlo (pMCMC) algorithms generate clouds of proposals at each step to efficiently resolve a target probability distribution $\mu $. We build a rigorous foundational framework for pMCMC algorithms that situates these methods within a unified ‘extended phase space’ measure-theoretic formalism. Drawing on our recent work that provides a comprehensive theory for reversible single-proposal methods, we herein derive general criteria for multiproposal acceptance mechanisms that yield ergodic chains on general state spaces. Our formulation encompasses a variety of methodologies, including proposal cloud resampling and Hamiltonian methods, while providing a basis for the derivation of novel algorithms. In particular, we obtain a top-down picture for a class of methods arising from ‘conditionally independent’ proposal structures. As an immediate application of this formalism, we identify several new algorithms including a multiproposal version of the popular preconditioned Crank–Nicolson (pCN) sampler suitable for high- and infinite-dimensional target measures that are absolutely continuous with respect to a Gaussian base measure. To supplement the aforementioned theoretical results, we carry out a selection of numerical case studies that evaluate the efficacy of these novel algorithms. First, noting that the true potential of pMCMC algorithms arises from their natural parallelizability and the ease with which they map to modern high-performance computing architectures, we provide a limited parallelization study using TensorFlow and a graphics processing unit to scale pMCMC algorithms that leverage as many as 100k proposals at each step. Second, we use our multiproposal pCN algorithm (mpCN) to resolve a selection of problems in Bayesian statistical inversion for partial differential equations motivated by fluid measurement. These examples provide preliminary evidence of the efficacy of mpCN for high-dimensional target distributions featuring complex geometries and multimodal structures.

Bayesian Statistical Inference for Factor Analysis Models with Clustered Data

Clustered data are a complex and frequently used type of data. Traditional factor analysis methods are effective for non-clustered data, but they do not adequately capture correlations between multiple observed individuals or variables in clustered data. This paper proposes a Bayesian approach utilizing MCMC and Gibbs sampling algorithms to accurately estimate parameters of interest within the clustered factor analysis model. The mean traversal graph of parameters ensures that the Markov chain converges, and the Bayesian case-deletion model is used to analyze the model’s impact and identify outliers in clustered data using Cook’s posterior mean distance. The applicability and validity of the principal-component-method-based factor analysis model for clustered data are demonstrated by comparing the parameter estimation of this method with the principal component method, the clustered data with and without internal relationships are compared by example analysis, and the anomalous groups are identified by the Cook’s posterior mean distance.

A Bayesian multilevel model for populations of networks using exponential-family random graphs

The collection of data on populations of networks is becoming increasingly common, where each data point can be seen as a realisation of a network-valued random variable. Moreover, each data point may be accompanied by some additional covariate information and one may be interested in assessing the effect of these covariates on network structure within the population. A canonical example is that of brain networks: a typical neuroimaging study collects one or more brain scans across multiple individuals, each of which can be modelled as a network with nodes corresponding to distinct brain regions and edges corresponding to structural or functional connections between these regions. Most statistical network models, however, were originally proposed to describe a single underlying relational structure, although recent years have seen a drive to extend these models to populations of networks. Here, we describe a model for when the outcome of interest is a network-valued random variable whose distribution is given by an exponential random graph model. To perform inference, we implement an exchange-within-Gibbs MCMC algorithm that generates samples from the doubly-intractable posterior. To illustrate this approach, we use it to assess population-level variations in networks derived from fMRI scans, enabling the inference of age- and intelligence-related differences in the topological structure of the brain’s functional connectivity.

A Projection Approach to Local Regression with Variable-Dimension Covariates

Incomplete covariate vectors are known to be problematic for estimation and inferences on model parameters, but their impact on prediction performance is less understood. We develop an imputation-free method that builds on a random partition model admitting variable-dimension covariates. Cluster-specific response models further incorporate covariates via linear predictors, facilitating estimation of smooth prediction surfaces with relatively few clusters. We exploit marginalization techniques of Gaussian kernels to analytically project response distributions according to any pattern of missing covariates, yielding a local regression with internally consistent uncertainty propagation that uses only one set of coefficients per cluster. Aggressive shrinkage of these coefficients regulates uncertainty due to missing covariates. The method allows in- and out-of-sample prediction for any missingness pattern, even if the pattern in a new subject’s incomplete covariate vector was not seen in the training data. We develop an MCMC algorithm for posterior sampling that improves a computationally expensive update for latent cluster allocation. Finally, we demonstrate the model’s effectiveness for nonlinear point and density prediction under various circumstances by comparing with other recent methods for regression of variable dimensions on synthetic and real data. Supplemental materials for this article are available online.

Graph-Aligned Random Partition Model (GARP)

Bayesian nonparametric mixtures and random partition models are powerful tools for probabilistic clustering. However, standard independent mixture models can be restrictive in some applications such as inference on cell lineage due to the biological relations of the clusters. The increasing availability of large genomic data requires new statistical tools to perform model-based clustering and infer the relationship between homogeneous subgroups of units. Motivated by single-cell RNA data we develop a novel dependent mixture model to jointly perform cluster analysis and align the clusters on a graph. Our flexible graph-aligned random partition model (GARP) exploits Gibbs-type priors as building blocks, allowing us to derive analytical results for the probability mass function (pmf) on the graph-aligned random partition. We derive a generalization of the Chinese restaurant process from the pmf and a related efficient and neat MCMC algorithm to implement Bayesian inference. We illustrate posterior inference under the GARP using single-cell RNA-seq data from mice stem cells. We further investigate the performance of the model in recovering the underlying clustering structure as well as the underlying graph by means of simulation studies. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Spatio-temporal characterization of earthquake sequence parameters and forecasting of strong aftershocks in Xinjiang based on the ETAS model.

In this paper, the Integrated Nested Laplace Algorithm (INLA) is applied to the Epidemic Type Aftershock Sequence (ETAS) model, and the parameters of the ETAS model are obtained for the earthquake sequences active in different regions of Xinjiang. By analyzing the characteristics of the model parameters over time, the changes in each earthquake sequence are studied in more detail. The estimated values of the ETAS model parameters are used as inputs to forecast strong aftershocks in the next period. We find that there are significant differences in the aftershock triggering capacity and aftershock attenuation capacity of earthquake sequences in different seismic regions of Xinjiang. With different cutoff dates set, we observe the characteristics of the earthquake sequence parameters changing with time after the mainshock occurs, and the model parameters of the Ms7.3 earthquake sequence in Hotan region change significantly with time within 15 days after the earthquake. Compared with the MCMC algorithm, the ETAS model fitted with the INLA algorithm can forecast the number of earthquakes in the early period after the occurrence of strong aftershocks more effectively and can forecast the sudden occurrence time of earthquakes more accurately.

Bayes goes big: Distributed MCMC and the drivers of E-commerce conversion

This work researches the drivers of e-commerce conversion of one of the largest e-commerce companies in the Netherlands. We focus on product page conversion, i.e., the probability that a customer who visits a specific product page also buys the product. This probability differs between products, which we explain by a variety of factors like pricing, delivery times, reviews, seller type, and the quality of the content. Understanding the drivers of conversion is the first step in increasing it, and therefore an important step in increasing overall sales. We describe the process of transforming Big Data into valuable insights using Bayesian statistics and apply a Bayesian Binomial model to a dataset of 15 million records using a distributed MCMC algorithm. In addition, we apply a simple Binomial GLM on a small sample of these 15 million records for comparison. We find that using the full 15 million records results in a significant reduction in the variance of the estimated parameters. In terms of the actual drivers of e-commerce conversion, we observe that products with competitive pricing, short delivery times, and good content all achieve a higher conversion on average. Furthermore, reviews, average review score, and the number of reviews for a product have a large positive effect on conversion compared to other characteristics. This provides a unique insight into what makes people decide whether or not to purchase a product.

Bayesian hidden mark interaction model for detecting spatially variable genes in imaging-based spatially resolved transcriptomics data.

Recent technology breakthroughs in spatially resolved transcriptomics (SRT) have enabled the comprehensive molecular characterization of cells whilst preserving their spatial and gene expression contexts. One of the fundamental questions in analyzing SRT data is the identification of spatially variable genes whose expressions display spatially correlated patterns. Existing approaches are built upon either the Gaussian process-based model, which relies on ad hoc kernels, or the energy-based Ising model, which requires gene expression to be measured on a lattice grid. To overcome these potential limitations, we developed a generalized energy-based framework to model gene expression measured from imaging-based SRT platforms, accommodating the irregular spatial distribution of measured cells. Our Bayesian model applies a zero-inflated negative binomial mixture model to dichotomize the raw count data, reducing noise. Additionally, we incorporate a geostatistical mark interaction model with a generalized energy function, where the interaction parameter is used to identify the spatial pattern. Auxiliary variable MCMC algorithms were employed to sample from the posterior distribution with an intractable normalizing constant. We demonstrated the strength of our method on both simulated and real data. Our simulation study showed that our method captured various spatial patterns with high accuracy; moreover, analysis of a seqFISH dataset and a STARmap dataset established that our proposed method is able to identify genes with novel and strong spatial patterns.

Automatic Regenerative Simulation via Non-Reversible Simulated Tempering

Simulated Tempering (ST) is an MCMC algorithm for complex target distributions that operates on a path between the target and a more amenable reference distribution. Crucially, if the reference enables iid sampling, ST is regenerative and can be parallelized across independent tours. However, the difficulty of tuning ST has hindered its widespread adoption. In this work, we develop a simple nonreversible ST (NRST) algorithm, a general theoretical analysis of ST, and an automated tuning procedure for ST. A core contribution that arises from the analysis is a novel performance metric—Tour Effectiveness (TE)—that controls the asymptotic variance of estimates from ST for bounded test functions. We use the TE to show that NRST dominates its reversible counterpart. We then develop an automated tuning procedure for NRST algorithms that targets the TE while minimizing computational cost. This procedure enables straightforward integration of NRST into existing probabilistic programming languages. We provide extensive experimental evidence that our tuning scheme improves the performance and robustness of NRST algorithms on a diverse set of probabilistic models. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Topical hidden genome: discovering latent cancer mutational topics using a Bayesian multilevel context-learning approach.

Inferring the cancer-type specificities of ultra-rare, genome-wide somatic mutations is an open problem. Traditional statistical methods cannot handle such data due to their ultra-high dimensionality and extreme data sparsity. To harness information in rare mutations, we have recently proposed a formal multilevel multilogistic "hidden genome" model. Through its hierarchical layers, the model condenses information in ultra-rare mutations through meta-features embodying mutation contexts to characterize cancer types. Consistent, scalable point estimation of the model can incorporate 10s of millions of variants across thousands of tumors and permit impressive prediction and attribution. However, principled statistical inference is infeasible due to the volume, correlation, and noninterpretability of mutation contexts. In this paper, we propose a novel framework that leverages topic models from computational linguistics to effectuate dimension reduction of mutation contexts producing interpretable, decorrelated meta-feature topics. We propose an efficient MCMC algorithm for implementation that permits rigorous full Bayesian inference at a scale that is orders of magnitude beyond the capability of existing out-of-the-box inferential high-dimensional multi-class regression methods and software. Applying our model to the Pan Cancer Analysis of Whole Genomes dataset reveals interesting biological insights including somatic mutational topics associated with UV exposure in skin cancer, aging in colorectal cancer, and strong influence of epigenome organization in liver cancer. Under cross-validation, our model demonstrates highly competitive predictive performance against blackbox methods of random forest and deep learning.