Efficient Bayesian Method Research Articles

Pop-cosmos: Scaleable Inference of Galaxy Properties and Redshifts with a Data-driven Population Model

We present an efficient Bayesian method for estimating individual photometric redshifts and galaxy properties under a pretrained population model (pop-cosmos) that was calibrated using purely photometric data. This model specifies a prior distribution over 16 stellar population synthesis (SPS) parameters using a score-based diffusion model, and includes a data model with detailed treatment of nebular emission. We use a GPU-accelerated affine-invariant ensemble sampler to achieve fast posterior sampling under this model for 292,300 individual galaxies in the COSMOS2020 catalog, leveraging a neural network emulator (Speculator) to speed up the SPS calculations. We apply both the pop-cosmos population model and a baseline prior inspired by Prospector-α, and compare these results to published COSMOS2020 redshift estimates from the widely used EAZY and LePhare codes. For the ∼12,000 galaxies with spectroscopic redshifts, we find that pop-cosmos yields redshift estimates that have minimal bias (∼10−4), high accuracy (σ MAD = 7 × 10−3), and a low outlier rate (1.6%). We show that the pop-cosmos population model generalizes well to galaxies fainter than its r < 25 mag training set. The sample we have analyzed is ≳3× larger than has previously been possible via posterior sampling with a full SPS model, with average throughput of 15 GPU-sec per galaxy under the pop-cosmos prior, and 0.6 GPU-sec per galaxy under the Prospector prior. This paves the way for principled modeling of the huge catalogs expected from upcoming Stage IV galaxy surveys.

Fast Bayesian Functional Regression for Non-Gaussian Spatial Data

Functional generalized linear models (FGLM) have been widely used to study the relations between non-Gaussian response and functional covariates. However, most existing works for FGLM assume independence among observations and therefore they are of limited applicability for correlated data. A particularly important example is spatial functional data, where we observe functions over spatial domains, such as the age population curve or temperature curve at each areal unit. In this paper, we extend FGLM by incorporating spatial random effects. Especially, we study the relationship between the non-Gaussian response variable and functional covariates that are spatially observed. However, such model has computational and inferential challenges. The high-dimensional spatial random effects cause the slow mixing of Markov chain Monte Carlo (MCMC) algorithms. Furthermore, spatial confounding can lead to bias in parameter estimates and inflate their variances. To address these issues, we propose an efficient Bayesian method using a sparse reparameterization of high-dimensional random effects. We also study the average coverage probabilities of the credible intervals of functional parameters. We apply our methods to simulated and real data examples, including malaria incidence data and US COVID-19 data. The proposed method is fast while providing accurate functional estimates.

An efficient Bayesian method with intrusive homotopy surrogate model for stochastic model updating

AbstractThis paper proposes a new stochastic model updating method based on the homotopy surrogate model (HSM) and Bayesian sampling. As a novel intrusive surrogate model, the HSM is established by the homotopy stochastic finite element (FE) method. Then combining the advanced delayed‐rejection adaptive Metropolis–Hastings sampling technology with HSM, the structural FE model can be updated by uncertain measurement modal data. The numerical results show that the updating effectiveness of the proposed method is better than that of the Bayesian methods with the non‐intrusive surrogate models, such as stochastic response surface model and Kriging model. Compared to the Bayesian method with the intrusive second‐order perturbation model, the updating results of the proposed method are more accurate, especially when the fluctuation of the uncertain measured data is large and the stiffness of the structure significantly changes. The model updating results of a cable‐stayed bridge show that the statistic modal properties of the updated bridge model have a very good agreement with the uncertain measurement modal data.

Large-scale sandwich structures optimization using Bayesian method

Benefiting from advanced features like high stiffness-to-weight ratios, sandwich structures are widely used in aerospace for primary and secondary structures. As tasks grow more complex and structures increase in scale, high-dimensional design spaces inevitably arise. Optimizing large-scale sandwich structures efficiently and intelligently presents certain challenges. Additionally, functional requirements and constraints, such as thermal deformation, should be fully considered in the design of practical structures like solar arrays, which involve a large expensive analysis and make the problem more complicated. This paper proposes to use efficient Bayesian optimization with the active subspace (AC) method to address this type of problem. The active subspace method, combined with the adaptive kriging and global sensitivity analysis (GSA), is employed for dimension reduction and reconstructing the design space. Then the structure is optimized within the reconstructed design space using the efficient Bayesian method. The proposed optimization strategy is applied to a case study of solar array with sandwich panels, demonstrating that the developed framework is feasible and effective for structural optimization of large-scale sandwich structures.

Bayesian uncertainty quantification for anaerobic digestion models

Uncertainty quantification is critical for ensuring adequate predictive power of computational models used in biology. Focusing on two anaerobic digestion models, this article introduces a novel generalized Bayesian procedure, called VarBUQ, ensuring a correct tradeoff between flexibility and computational cost. A benchmark against three existing methods (Fisher’s information, bootstrapping and Beale’s criteria) was conducted using synthetic data. This Bayesian procedure offered a good compromise between fitting ability and confidence estimation, while the other methods proved to be repeatedly overconfident. The method’s performances notably benefitted from inductive bias brought by the prior distribution, although it requires careful construction. This article advocates for more systematic consideration of uncertainty for anaerobic digestion models and showcases a new, computationally efficient Bayesian method. To facilitate future implementations, a Python package called ‘aduq’ is made available.

Multi-population mortality modelling: a Bayesian hierarchical approach

AbstractModelling mortality co-movements for multiple populations has significant implications for mortality/longevity risk management. This paper assumes that multiple populations are heterogeneous sub-populations randomly drawn from a hypothetical super-population. Those heterogeneous sub-populations may exhibit various patterns of mortality dynamics across different age groups. We propose a hierarchical structure of these age patterns to ensure the model stability and use a Vector Error Correction Model (VECM) to fit the co-movements over time. Especially, a structural analysis based on the VECM is implemented to investigate potential interdependence among mortality dynamics of the examined populations. An efficient Bayesian Markov Chain Monte-Carlo method is also developed to estimate the unknown parameters to address the computational complexity. Our empirical application to the mortality data collected for the Group of Seven nations demonstrates the efficacy of our approach.

An Efficient Bayesian Method for Estimating the Degree of the Skewness of X Chromosome Inactivation Based on the Mixture of General Pedigrees and Unrelated Females

Skewed X chromosome inactivation (XCI-S) has been reported to be associated with some X-linked diseases. Several methods have been proposed to estimate the degree of XCI-S (denoted as γ) for quantitative and qualitative traits based on unrelated females. However, there is no method available for estimating γ based on general pedigrees. Therefore, in this paper, we propose a Bayesian method to obtain the point estimate and the credible interval of γ based on the mixture of general pedigrees and unrelated females (called mixed data for brevity), which is also suitable for only general pedigrees. We consider the truncated normal prior and the uniform prior for γ. Further, we apply the eigenvalue decomposition and Cholesky decomposition to our proposed methods to accelerate the computation speed. We conduct extensive simulation studies to compare the performances of our proposed methods and two existing Bayesian methods which are only applicable to unrelated females. The simulation results show that the incorporation of general pedigrees can improve the efficiency of the point estimation and the precision and the accuracy of the interval estimation of γ. Finally, we apply the proposed methods to the Minnesota Center for Twin and Family Research data for their practical use.

Efficient Bayesian method for characterizing multiple soil parameters using parametric bootstrap

This study proposes an efficient Bayesian method using parametric bootstrap in place of the inefficient conventional Bayesian method for characterizing the multivariate PDF of multiple soil parameters. The improvement of the efficiency is achieved by replacing the complex multi-dimensional integral in the conventional Bayesian method with a simple arithmetic average in the proposed method. The good efficiency of the proposed method makes it possible to characterize the multivariate PDF of multiple soil parameters using their limited multivariate site-specific test data and prior knowledge on the site. Moreover, the proposed method has no limitation on the multivariate distribution model and prior distribution of multiple soil parameters. It does not require that the multivariate distribution model is a multivariate normal distribution and the prior distribution is conjugate to the multivariate normal distribution. Two practical examples using real multivariate site-specific test data of multiple soil parameters and one numerical example with known multivariate distribution are presented to illustrate and demonstrate the proposed method. The results indicate that the proposed method performs well in characterizing the multivariate PDF of multiple soil parameters. The role of informative prior knowledge is that it can produce more accurate results than non-informative prior knowledge and thus reduce the demand for the multivariate site-specific test data of multiple soil parameters.

STS-BN: An efficient Bayesian network method for detecting causal SNPs.

Background: The identification of the causal SNPs of complex diseases in large-scale genome-wide association analysis is beneficial to the studies of pathogenesis, prevention, diagnosis and treatment of these diseases. However, existing applicable methods for large-scale data suffer from low accuracy. Developing powerful and accurate methods for detecting SNPs associated with complex diseases is highly desired. Results: We propose a score-based two-stage Bayesian network method to identify causal SNPs of complex diseases for case-control designs. This method combines the ideas of constraint-based methods and score-and-search methods to learn the structure of the disease-centered local Bayesian network. Simulation experiments are conducted to compare this new algorithm with several common methods that can achieve the same function. The results show that our method improves the accuracy and stability compared to several common methods. Our method based on Bayesian network theory results in lower false-positive rates when all correct loci are detected. Besides, real-world data application suggests that our algorithm has good performance when handling genome-wide association data. Conclusion: The proposed method is designed to identify the SNPs related to complex diseases, and is more accurate than other methods which can also be adapted to large-scale genome-wide analysis studies data.

Large Hybrid Time-Varying Parameter VARs

Time-varying parameter VARs with stochastic volatility are routinely used for structural analysis and forecasting in settings involving a few endogenous variables. Applying these models to high-dimensional datasets has proved to be challenging due to intensive computations and over-parameterization concerns. We develop an efficient Bayesian sparsification method for a class of models we call hybrid TVP-VARs—VARs with time-varying parameters in some equations but constant coefficients in others. Specifically, for each equation, the new method automatically decides whether the VAR coefficients and contemporaneous relations among variables are constant or time-varying. Using U.S. datasets of various dimensions, we find evidence that the parameters in some, but not all, equations are time varying. The large hybrid TVP-VAR also forecasts better than many standard benchmarks.

Comparison between the Hamiltonian Monte Carlo method and the Metropolis–Hastings method for coseismic fault model estimation

A rapid source fault estimation and quantitative assessment of the uncertainty of the estimated model can elucidate the occurrence mechanism of earthquakes and inform disaster damage mitigation. The Bayesian statistical method that addresses the posterior distribution of unknowns using the Markov chain Monte Carlo (MCMC) method is significant for uncertainty assessment. The Metropolis–Hastings method, especially the Random walk Metropolis–Hastings (RWMH), has many applications, including coseismic fault estimation. However, RWMH exhibits a trade-off between the transition distance and the acceptance ratio of parameter transition candidates and requires a long mixing time, particularly in solving high-dimensional problems. This necessitates a more efficient Bayesian method. In this study, we developed a fault estimation algorithm using the Hamiltonian Monte Carlo (HMC) method, which is considered more efficient than the other MCMC method, but its applicability has not been sufficiently validated to estimate the coseismic fault for the first time. HMC can conduct sampling more intelligently with the gradient information of the posterior distribution. We applied our algorithm to the 2016 Kumamoto earthquake (MJMA 7.3), and its sampling converged in 2 × 104 samples, including 1 × 103 burn-in samples. The estimated models satisfactorily accounted for the input data; the variance reduction was approximately 88%, and the estimated fault parameters and event magnitude were consistent with those reported in previous studies. HMC could acquire similar results using only 2% of the RWMH chains. Moreover, the power spectral density (PSD) of each model parameter's Markov chain showed this method exhibited a low correlation with the subsequent sample and a long transition distance between samples. These results indicate HMC has advantages in terms of chain length than RWMH, expecting a more efficient estimation for a high-dimensional problem that requires a long mixing time or a problem using nonlinear Green’s function, which has a large computational cost.Graphical

An efficient Bayesian inversion method for seepage parameters using a data-driven error model and an ensemble of surrogates considering the interactions between prediction performance indicators

The Bayesian method has been increasingly applied to the inversion of seepage parameters owing to its superiority of considering the uncertainty in the inversion process. However, most of the current Bayesian inversion studies only focus on parameter uncertainty, ignoring the model structure error. In addition, existing research has mostly adopted a single machine learning algorithm or an ensemble of surrogates based on a single prediction performance indicator as a substitute for the seepage forward model and has not considered the interactions between multiple prediction performance indicators, thereby leading to poor accuracy. To address these issues, this study proposed an efficient Bayesian inversion method for seepage parameters using a data-driven error model and an ensemble of surrogates considering the interactions between prediction performance indicators. A data-driven error model based on Gaussian process regression was integrated into the Bayesian inversion model to modify the likelihood function for dealing with the model structure error. For determining the weight coefficients of the ensemble surrogates, the improved Dempster-Shafer (D-S) evidence theory based on the Hellinger distance and Deng entropy was proposed to fuse multiple prediction performance indicators and consider their interactions. Further, an ensemble of surrogates in conjunction with support vector regression, Kriging, and multivariate adaptive regression splines was constructed through weighted summation. The validity and accuracy of the proposed method were verified by applying it to a real hydropower station in China. The results showed that the proposed method can significantly improve the accuracy and efficiency of Bayesian inversion of seepage parameters. The proposed method therefore serves as a new basis for the inversion of seepage parameters and can be applied to parameter inversion in other related fields.

Fast and Flexible Bayesian Inference in Time-varying Parameter Regression Models

In this article, we write the time-varying parameter (TVP) regression model involving K explanatory variables and T observations as a constant coefficient regression model with KT explanatory variables. In contrast with much of the existing literature which assumes coefficients to evolve according to a random walk, a hierarchical mixture model on the TVPs is introduced. The resulting model closely mimics a random coefficients specification which groups the TVPs into several regimes. These flexible mixtures allow for TVPs that feature a small, moderate or large number of structural breaks. We develop computationally efficient Bayesian econometric methods based on the singular value decomposition of the KT regressors. In artificial data, we find our methods to be accurate and much faster than standard approaches in terms of computation time. In an empirical exercise involving inflation forecasting using a large number of predictors, we find our models to forecast better than alternative approaches and document different patterns of parameter change than are found with approaches which assume random walk evolution of parameters.

Mining heterogeneous clinical notes by multi-modal latent topic model

Latent knowledge can be extracted from the electronic notes that are recorded during patient encounters with the health system. Using these clinical notes to decipher a patient's underlying comorbidites, symptom burdens, and treatment courses is an ongoing challenge. Latent topic model as an efficient Bayesian method can be used to model each patient's clinical notes as "documents" and the words in the notes as "tokens". However, standard latent topic models assume that all of the notes follow the same topic distribution, regardless of the type of note or the domain expertise of the author (such as doctors or nurses). We propose a novel application of latent topic modeling, using multi-note topic model (MNTM) to jointly infer distinct topic distributions of notes of different types. We applied our model to clinical notes from the MIMIC-III dataset to infer distinct topic distributions over the physician and nursing note types. Based on manual assessments made by clinicians, we observed a significant improvement in topic interpretability using MNTM modeling over the baseline single-note topic models that ignore the note types. Moreover, our MNTM model led to a significantly higher prediction accuracy for prolonged mechanical ventilation and mortality using only the first 48 hours of patient data. By correlating the patients' topic mixture with hospital mortality and prolonged mechanical ventilation, we identified several diagnostic topics that are associated with poor outcomes. Because of its elegant and intuitive formation, we envision a broad application of our approach in mining multi-modality text-based healthcare information that goes beyond clinical notes. Code available at https://github.com/li-lab-mcgill/heterogeneous_ehr.

PyClone-VI: scalable inference of clonal population structures using whole genome data

BackgroundAt diagnosis tumours are typically composed of a mixture of genomically distinct malignant cell populations. Bulk sequencing of tumour samples coupled with computational deconvolution can be used to identify these populations and study cancer evolution. Existing computational methods for populations deconvolution are slow and/or potentially inaccurate when applied to large datasets generated by whole genome sequencing data.ResultsWe describe PyClone-VI, a computationally efficient Bayesian statistical method for inferring the clonal population structure of cancers. We demonstrate the utility of the method by analyzing data from 1717 patients from PCAWG study and 100 patients from the TRACERx study.ConclusionsOur proposed method is 10–100× times faster than existing methods, while providing results which are as accurate. Software implementing our method is freely available https://github.com/Roth-Lab/pyclone-vi.

Efficient Bayesian FFT method for damage detection using ambient vibration data with consideration of uncertainty

Efficient Bayesian FFT method for damage detection using ambient vibration data with consideration of uncertainty

Bayesian estimation of stochastic tail index from high-frequency financial data

Tails of the return distribution of an asset are informative about the (financial) risk behavior of that asset. Stochastic tail index (STI) models are designed to quantify riskiness by estimating a time-varying tail index from the distribution of extreme values using high-frequency financial data. In this paper, we propose a computationally efficient Bayesian estimation method for STI models based on the recent advances in band and sparse matrix algorithms. We then show how the deviance information criterion (DIC) can be calculated from the integrated likelihood function for model comparison exercises. In a Monte Carlo study, we investigate the finite sample properties of the Bayesian estimator as well as the performance of two DIC measures. Our results show that the Bayesian estimator performs sufficiently well and the DIC measures based on the integrated likelihood function are useful for model selection exercises. In an empirical application, we illustrate calculation of the tail index using high-frequency data on IBM stock returns. Our estimation results indicate that the daily tail index of the return distribution of IBM stock has a time-varying feature: It tends to decline when there are large losses, whereas it tends to increase when there are small losses.

Choosing between identification schemes in noisy-news models

Abstract This paper is about identifying structural shocks in noisy-news models using structural vector autoregressive moving average (SVARMA) models. We develop a new identification scheme and efficient Bayesian methods for estimating the resulting SVARMA. We discuss how our identification scheme differs from the one which is used in existing theoretical and empirical models. Our main contributions lie in the development of methods for choosing between identification schemes. We estimate specifications with up to 20 variables using US macroeconomic data. We find that our identification scheme is preferred by the data, particularly as the size of the system is increased and that noise shocks generally play a negligible role. However, small models may overstate the importance of noise shocks.

Bayesian estimation of spatially varying soil parameters with spatiotemporal monitoring data

The characterization of in situ ground conditions is essential for geotechnical practice. The probabilistic estimation of soil parameters can be achieved via updating with monitoring data within the Bayesian framework. The estimation of spatially varying soil parameters is seldom undertaken with time-variant monitoring data. In this study, an efficient Bayesian method is presented for the estimation of spatially varied saturated hydraulic conductivity ks of unsaturated soil slope with spatiotemporal monitoring data. The computationally cheap surrogate model of the adaptive sparse polynomial chaos expansion method is adopted to approximate the transient numerical model. Markov chain Monte Carlo method is used for the probabilistic estimation of basic random variables. Based on the hypothetical cases, the effects of monitoring frequency and stage are studied. The errors and the uncertainties of the estimated ks fields are increased with the decreasing monitoring frequency. Bayesian estimation of spatial variability is more accurate when using the later stage of monitoring data. The estimated method is further verified with a real case study by the comparison of borehole data, dynamic probe test (DPT) data, and field monitoring data. The distribution of the soil types acquired from boreholes is reflected in the estimated ks. The estimated field of ks has a certain agreement with the borehole log and DPT measurements and can reproduce the spatial variability of the site to an acceptable degree.

Large-scale local causal inference of gene regulatory relationships

Gene regulatory networks play a crucial role in controlling an organism's biological processes, which is why there is significant interest in developing computational methods that are able to extract their structure from high-throughput genetic data. Many of these computational methods are designed to infer individual regulatory relationships among genes from data on gene expression. We propose a novel efficient Bayesian method for discovering local causal relationships among triplets of (normally distributed) variables. In our approach, we score covariance structures for each triplet in one go and incorporate available background knowledge in the form of priors to derive posterior probabilities over local causal structures. Our method is flexible in the sense that it allows for different types of causal structures and assumptions. We apply our approach to the task of learning causal regulatory relationships among genes. We show that the proposed algorithm produces stable and conservative posterior probability estimates over local causal structures that can be used to derive an honest ranking of the most meaningful regulatory relationships. We demonstrate the stability and efficacy of our method both on simulated data and on real-world data from an experiment on yeast.