Monte Carlo Sampling Algorithm Research Articles

Efficient Trans-Dimensional Bayesian Inversion of C-Response Data from Geomagnetic Observatory and Satellite Magnetic Data

To investigate deep Earth information, researchers often utilize geomagnetic observatories and satellite data to obtain the conversion function of geomagnetic sounding, C-response data, and employ traditional inversion techniques to reconstruct subsurface structures. However, the traditional gradient-based inversion produces geophysical models with artificial structure constraint enforced subjectively to guarantee a unique solution. This method typically requires the model parameterization knowledge a priori (e.g., based on personal preference) without uncertainty estimation. In this paper, we apply an efficient trans-dimensional (trans-D) Bayesian algorithm to invert C-response data from observatory and satellite geomagnetic data for the electrical conductivity structure of the Earth’s mantle, with the model parameterization treated as unknown and determined by the data. In trans-D Bayesian inversion, the posterior probability density (PPD) represents a complete inversion solution, based on which useful inversion inferences about the model can be made with the requirement of high-dimensional integration of PPD. This is realized by an efficient reversible-jump Markov-chain Monte Carlo (rjMcMC) sampling algorithm based on the birth/death scheme. Within the trans-D Bayesian algorithm, the model parameter is perturbated in the principal-component parameter space to minimize the effect of inter-parameter correlations and improve the sampling efficiency. A parallel tempering scheme is applied to guarantee the complete sampling of the multiple model space. Firstly, the trans-D Bayesian inversion is applied to invert C-response data from two synthetic models to examine the resolution of the model structure constrained by the data. Then, C-response data from geomagnetic satellites and observatories are inverted to recover the global averaged mantle conductivity structure and the local mantle structure with quantitative uncertainty estimation, which is consistent with the data.

Bayesian neural network and Bayesian physics-informed neural network via variational inference for seismic petrophysical inversion

Deep-learning methods are being successfully applied to seismic inversion and reservoir characterization problems; however, the uncertainty quantification process has not been fully studied. We investigate probabilistic approaches for seismic petrophysical inversion by using Bayesian formulations of neural networks. In particular, Bayesian neural network via variational inference (BNN-VI) aims to estimate the probability distribution of the training parameters of the neural network, which are difficult to compute with traditional methods, by proposing a family of densities and finding the candidates that are close to the target. Variational inference (VI) has the advantage of providing a faster tool than Markov chain Monte Carlo sampling algorithms. However, for accurate predictions, BNN-VI requires large training data that are not always available in geophysical inverse problems. Hence, we develop a method that combines a Bayesian approach with physics-informed neural networks (PINNs) via VI, namely Bayesian PINNs via VI (BPINN-VI), and we apply it to a geophysical inverse problem for the estimation of petrophysical properties from seismic data. The method is based on a Bayesian network approach in which the objective function is defined by the Kullback-Leibler divergence. In addition, the physical relations between data (seismic measurements) and model variables (petrophysical properties) are embedded into neural networks through a physics-dependent loss term. We test our method on a synthetic prestack seismic data set and a real data set including an oil-saturated clastic reservoir in the North Sea with a sequence of sand and shale layers. The outcome is the most likely model of petrophysical properties. Compared with BNN-VI, the BPINN-VI results show higher correlations and [Formula: see text] coefficients, lower mean-square errors, as well as lower uncertainty and higher lateral continuity.

Predicting Colloidal Interaction Parameters from Small-Angle X-ray Scattering Curves Using Artificial Neural Networks and Markov Chain Monte Carlo Sampling.

Small-angle X-ray scattering (SAXS) is a characterization technique that allows for the study of colloidal interactions by fitting the structure factor of the SAXS profile with a selected model and closure relation. However, the applicability of this approach is constrained by the limited number of existing models that can be fitted analytically, as well as the narrow operating range for which the models are valid. In this work, we demonstrate a proof of concept for using an artificial neural network (ANN) trained on SAXS curves obtained from Monte Carlo (MC) simulations to predict values of the effective macroion valency (Z eff) and the Debye length (κ-1) for a given SAXS profile. This ANN, which was trained on 200,000 simulated SAXS curves, was able to predict values of Z eff and κ-1 for a test set containing 25,000 simulated SAXS curves, where most predicted values had errors smaller than 20%. Subsequently, an ANN was used as a surrogate model in a Markov chain Monte Carlo sampling algorithm to obtain maximum a posteriori estimates of Z eff and κ-1, as well as the associated confidence intervals and correlations between Z eff and κ-1 for an experimentally obtained SAXS profile.

Joint modeling of varying-disease-state longitudinal ordinal data and time-to-event data with application to alzheimer’s disease

Joint models are routinely used in clinical trials to fit longitudinal and survival data simulta- neously in an integrated fashion. We propose a joint model that incorporates state-specific trajectories for fitting the longitudinal ordinal response and time-to-event data, focusing on its application to Alzheimer’s Disease (AD). The proposed joint model effectively captures the fluctuating cognitive conditions observed before and after the transition between two disease states. By integrating longitudinal data into the survival sub-model through shared trajectories, we can improve the fit of the survival data. A Markov chain Monte Carlo (MCMC) sampling algorithm is developed to carry out Bayesian computation. A variation of the Deviance Information Criterion is developed to assess the fit of each component of the joint model as well as the contribution of the longitudinal data in fitting the survival data. A variation of the concordance (C) index is further derived to assess the discriminatory and predictive performance of the longitudinal sub-model. An in-depth analysis of the real data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database is carried out to demonstrate the applicability of the proposed methodology. Journal of Statistical Research 2024, Vol. 58, No. 1, pp. 49-73.

Calibrating the Discrete Boundary Conditions of a Dynamic Simulation: A Combinatorial Approximate Bayesian Computation Sequential Monte Carlo (ABC-SMC) Approach.

This paper presents a novel adaptation of the conventional approximate Bayesian computation sequential Monte Carlo (ABC-SMC) sampling algorithm for parameter estimation in the presence of uncertainties, coined combinatorial ABC-SMC. Inference of this type is used in situations where there does not exist a closed form of the associated likelihood function, which is replaced by a simulating model capable of producing artificial data. In the literature, conventional ABC-SMC is utilised to perform inference on continuous parameters. The novel scheme presented here has been developed to perform inference on parameters that are high-dimensional binary, rather than continuous. By altering the form of the proposal distribution from which to sample candidates in subsequent iterations (referred to as waves), high-dimensional binary variables may be targeted and inferred by the scheme. The efficacy of the proposed scheme is demonstrated through application to vibration data obtained in a structural dynamics experiment on a fibre-optic sensor simulated as a finite plate with uncertain boundary conditions at its edges. Results indicate that the method provides sound inference on the plate boundary conditions, which is validated through subsequent application of the method to multiple vibration datasets. Comparisons between appropriate forms of the metric function used in the scheme are also developed to highlight the effect of this element in the schemes convergence.

Optimal dispatching of regional power grid considering vehicle network interaction.

When large-scale electric vehicles are connected to the grid for unordered charging, it will seriously affect the stability and security of the power system. To solve this problem, this paper proposes a regional power network optimization scheduling method considering vehicle network interaction. Initially, based on the user behavior characteristics and charging and discharging characteristics of electric vehicles, a charging and discharging behavior model of electric vehicles was established. Based on the Monte Carlo sampling algorithm, the scheduling upper and lower limits of each scheduling cycle of electric vehicles were described, and the scheduling potential of each scheduling cycle of electric vehicles was obtained. Then, the electricity price is then used as an incentive parameter to guide EV users to charge during periods of low electricity prices and participate in discharge during periods of peak electricity prices. Aiming at the highest economic efficiency, the best consumption effect of new energy and the smoothest demand-side power curve of regional power grid, a three-objective optimal dispatching model was established. In the later stage, uncertainty factors are taken into consideration by introducing the concept of interval numbers, and an interval multi-objective optimization dispatching model is established. The two dispatching models are solved by NSGA-II algorithm and improved NSGA-II algorithm, and the Pareto solution set is obtained. Finally, based on the analytic Hierarchy Process (AHP), the optimal scheduling scheme is determined. The Monte Carlo sampling method is used to simulate the user side charging demand, and the effectiveness of this method is verified. In addition, the results of the interval multi-objective optimization model and the deterministic multi-objective optimization model are compared, and it is proved that the solution results of the interval multi-objective model are more adaptive, practical and robust to the uncertain factors.

Hopsy - a methods marketplace for convex polytope sampling in Python.

Effective collaboration between developers of Bayesian inference methods and users is key to advance our quantitative understanding of biosystems. We here present hopsy, a versatile open-source platform designed to provide convenient access to powerful Markov chain Monte Carlo sampling algorithms tailored to models defined on convex polytopes (CP). Based on the high-performance C++ sampling library HOPS, hopsy inherits its strengths and extends its functionalities with the accessibility of the Python programming language. A versatile plugin-mechanism enables seamless integration with domain-specific models, providing method developers with a framework for testing, benchmarking, and distributing CP samplers to approach real-world inference tasks. We showcase hopsy by solving common and newly composed domain-specific sampling problems, highlighting important design choices. By likening hopsy to a marketplace, we emphasize its role in bringing together users and developers, where users get access to state-of-the-art methods, and developers contribute their own innovative solutions for challenging domain-specific inference problems. Sources, documentation and a continuously updated list of sampling algorithms are available at https://jugit.fz-juelich.de/IBG-1/ModSim/hopsy, with Linux, Windows and MacOS binaries at https://pypi.org/project/hopsy/.

Inference of Constant-Stress Model of Fréchet Distribution under a Maximum Ranked Set Sampling with Unequal Samples

This paper explores the inference for a constant-stress accelerated life test under a ranked set sampling scenario. When the lifetime of products follows the Fréchet distribution, and the failure times are collected under a maximum ranked set sampling with unequal samples, classical and Bayesian approaches are proposed, respectively. Maximum likelihood estimators along with the existence and uniqueness of model parameters are established, and the corresponding asymptotic confidence intervals are constructed based on asymptotic theory. Under squared error loss, Bayesian estimation and highest posterior density confidence intervals are provided, and an associated Monte-Carlo sampling algorithm is proposed for complex posterior computation. Finally, extensive simulation studies are conducted to demonstrate the performance of different methods, and a real-data example is also presented for applications.

Proteomic Barcoding Platform for Macromolecular Screening and Delivery.

Engineered macromolecules offer compelling means for the therapy of conventionally undruggable interactions in human disease. However, their efficacy is limited by barriers to tissue and intracellular delivery. Inspired by recent advances in molecular barcoding and evolution, we developed BarcodeBabel, a generalized method for the design of libraries of peptide barcodes suitable for high-throughput mass spectrometry proteomics. Combined with PeptideBabel, a Monte Carlo sampling algorithm for the design of peptides with evolvable physicochemical properties and sequence complexity, we developed a barcoded library of cell penetrating peptides (CPPs) with distinct physicochemical features. Using quantitative targeted mass spectrometry, we identified CPPS with improved nuclear and cytoplasmic delivery exceeding hundreds of millions of molecules per human cell while maintaining minimal membrane disruption and negligible toxicity in vitro. These studies provide a proof of concept for peptide barcoding as a homogeneous high-throughput method for macromolecular screening and delivery. BarcodeBabel and PeptideBabel are available open-source from https://github.com/kentsisresearchgroup/.

A Semiparametric Bayesian Approach to Heterogeneous Spatial Autoregressive Models.

Many semiparametric spatial autoregressive (SSAR) models have been used to analyze spatial data in a variety of applications; however, it is a common phenomenon that heteroscedasticity often occurs in spatial data analysis. Therefore, when considering SSAR models in this paper, it is allowed that the variance parameters of the models can depend on the explanatory variable, and these are called heterogeneous semiparametric spatial autoregressive models. In order to estimate the model parameters, a Bayesian estimation method is proposed for heterogeneous SSAR models based on B-spline approximations of the nonparametric function. Then, we develop an efficient Markov chain Monte Carlo sampling algorithm on the basis of the Gibbs sampler and Metropolis-Hastings algorithm that can be used to generate posterior samples from posterior distributions and perform posterior inference. Finally, some simulation studies and real data analysis of Boston housing data have demonstrated the excellent performance of the proposed Bayesian method.

Molecular isotopologue measurements toward super star clusters and the relation to their ages in NGC 253 with ALCHEMI

Context. Determining the evolution of the CNO isotopes in the interstellar medium (ISM) of starburst galaxies can yield important constraints on the ages of super star clusters (SSCs), or on other aspects and factors contributing to their evolution, such as the initial mass function (IMF). Due to the time-dependent nature of the abundances of isotopes within the ISM – as they are supplied from processes such as nucleosynthesis or chemical fractionation –, this provides the opportunity to test whether or not isotope ratios trace the ages of highly star-forming regions, such as SSCs. Aims. The goal of this study is to investigate whether the isotopic variations in SSC regions within NGC 253 are correlated with their different ages as derived from stellar population modelling. Methods. We measured abundance ratios of CO, HCN, and HCO+ isotopologues in six regions containing SSCs within NGC 253 using high-spatial-resolution (1.6″, ∼28 pc) data from the ALCHEMI (ALma Comprehensive High-resolution Extragalactic Molecular Inventory) ALMA Large program. We then analysed these ratios using RADEX radiative transfer modelling, with the parameter space sampled using the nested sampling Monte Carlo algorithm MLFriends. These abundance ratios were then compared to ages predicted in each region via the fitting of observed star-formation tracers (such as Brγ) to Starburst99 starburst stellar population evolution models. Results. We determined the isotopic column density ratios across multiple regions of SSC activity in NGC 253 using non-LTE radiative transfer modelling. We do not find any significant trend with age for the CO and HCN isotopologue ratios on timescales of the ages of the SSC* regions observed. However, HCO+ may show a correlation with age over these timescales in 12C/13C. Conclusions. The driving factors of these ratios within SSCs could be the IMF or fractionation effects. To further probe these effects in SSCs over time, a larger sample of SSCs must be observed spanning a larger age range.

Bayesian mixed-effect higher-order hidden Markov models with applications to predictive healthcare using electronic health records

The disease progression dynamics observed in electronic health records often reflect patients’ health condition evolution, holding the promise of enabling the development of clinical predictive models. These dynamics, however, generally display significant variability among patients, due to some critical factors (e.g., gender and age) and patient-level heterogeneity. Moreover, future health state may not only depend on the current state, but also more distant history states due to the complicated disease progression. To capture this complex transition behavior and address mixed effects in clinical prediction problems, we propose a novel and flexible Bayesian Mixed-Effect Higher-Order Hidden Markov Model (MHOHMM), and develop a classifier based on MHOHMMs. A range of MHOHMMs are designed to capture different data structures and the optimal one is identified by using the k-fold cross-validation approach. An effective two-stage Markov chain Monte Carlo (MCMC) sampling algorithm is designed for model inference. A simulation study is conducted to evaluate the performance of the proposed sampling algorithm and the MHOHMM-based classification method. The practical utility of the proposed framework is demonstrated by a case study on the acute hypotensive episode prediction for intensive care unit patients. Our results show that the MHOHMM-based framework provides good prediction performance.

Inference for dependent complementary competing risks model from an inverted Kumaraswamy distribution under ranked set sampling

AbstractRanked set sampling (RSS) has been proved to be an efficient sampling design for parametric and nonparametric inference. This paper explores inference for a maximum RSS procedure with unequal samples (MRSSU) with multiple dependent failure causes. When the lifetimes of units are characterized by a proposed complementary competing risks model, classical likelihood and Bayesian approaches are discussed for parameters and reliability estimation. Maximum likelihood estimators of model parameters and associated existence and uniqueness are established, and approximate confidence intervals are constructed using asymptotic theory and delta methods. With respect to general flexible priors, Bayesian estimates of interested quantities are also performed and a Monte Carlo sampling algorithm is proposed for complex posterior computation. Additionally, when extra historical information between the competing risks parameters is available, likelihood and Bayesian estimation are also studied under an order restriction case. Different methods are compared based on extensive simulation studies, and another real data example is demonstrated for application propose.

Bayesian Structural Time Series Model for Forecasting the Composite Stock Price Index in Indonesia

One of the models that can be used to predict time series data is the Bayesian Structural Time Series (BSTS) model. The BSTS model is a more modern model and can handle data movement better. In the BSTS model, the Markov Chain Monte Carlo (MCMC) sampling algorithm is used to simulate the posterior distribution, which smoothes the forecasting results over a large number of potential models using Bayesian averaging models. The purpose of this study was to obtain the best BSTS model for Composite Stock Price Index (CSPI) data in Indonesia based on the state component and the number of MCMC iterations, and obtain forecasting results for CSPI value in Indonesia for the next 24 months, namely the period July 2023 to June 2024. The results obtained are based on a comparison of the R-square values in the model, the BSTS model with local linear trend and seasonal state components, and the number of MCMC iterations n = 5 00 is the best BSTS model that can be used for forecasting the CSPI value in Indonesia with an R-square value of 99.96%. The results of forecasting the CSPI value in Indonesia for the period July 2023 to June 2024 range from 6589 to 6760, with the lowest forecasting value in October 2023 and the highest in March 2023.

Rayleigh Wave Dispersion and Inversion for Shallow Surface with a High-velocity Rigid Pavement

We investigate the rigid pavement influence on phase velocity dispersion of the Rayleigh wave by using numerical modeling. An overestimation of Rayleigh wave phase velocity is observed more than 10% deviation caused by high-velocity rigid pavement. Based on the generalized reflection and transmission (R/T) coefficient method, the linear eigenvalue problem is solved for vertically heterogeneous media with a rigid pavement structure. The secular function demonstrates the multi-valuedness which is different phase velocity at the same frequency for the single Rayleigh mode. To address mode misidentification problem, we adopt a new misfit function for multimodal dispersion inversion. A Markov chain Monte Carlo (MCMC) sampling algorithm is used to solve dispersion inversion. The synthetic test shows that Rayleigh wave dispersion inversion incorporating a pavement layer can improve estimation accuracy of shear wave velocity.

Research on new energy vehicle charging prediction based on Monte Carlo algorithm and its impact on distribution network

With the vigorous promotion of new energy policies, the large-scale charging of new energy vehicles has put forward higher requirements for the safety and stability of the distribution network. Based on the daily driving habits and charging patterns of new energy vehicles, a Monte Carlo sampling algorithm was used to establish a charging load model for new energy vehicles. The model analyzed the driving range, charging load, and time related parameters of new energy vehicles. By analyzing the law of daily charging power of new energy vehicles, the overall trend of charging load of new energy vehicles is obtained. Combined with the daily electricity consumption law of the distribution network, the total load of the distribution network is obtained, and the degree of impact on the distribution network is analyzed. This provides direction for the scheduling of future electric vehicle charging behavior and the construction of related supporting facilities, and provides strong guidance for the optimization and upgrading of the distribution network.

A latent functional approach for modeling the effects of multidimensional exposures on disease risk.

Understanding the relationships between exposure and disease incidence is an important problem in environmental epidemiology. Typically, a large number of these exposures are measured, and it is found either that a few exposures transmit risk or that each exposure transmits a small amount of risk, but, taken together, these may pose a substantial disease risk. Further, these exposure effects can be nonlinear. We develop a latent functional approach, which assumes that the individual effect of each exposure can be characterized as one of a series of unobserved functions, where the number of latent functions is less than or equal to the number of exposures. We propose Bayesian methodology to fit models with a large number of exposures and show that existing Bayesian group LASSO approaches are a special case of the proposed model. An efficient Markov chain Monte Carlo sampling algorithm is developed for carrying out Bayesian inference. The deviance information criterion is used to choose an appropriate number of nonlinear latent functions. We demonstrate the good properties of the approach using simulation studies. Further, we show that complex exposure relationships can be represented with only a few latent functional curves. The proposed methodology is illustrated with an analysis of the effect of cumulative pesticide exposure on cancer risk in a large cohort of farmers.

Bayesian prognosis analysis of human papillomavirus-associated head and neck cancer using hierarchical Dirichlet process mixture models

The incidence and prognosis of Head and Neck Cancer (HNC) depend heavily on patients’ Human PapillomaVirus (HPV) status. Prognosis analysis of HPV-associated HNC is of clinical importance because in-depth understanding of the survival distribution is valuable for designing more informed treatment strategies. In this article, we develop a novel Hierarchical Dirichlet Process Weibull Mixture Model (HDP-WMM) to study the prognosis of HNC patients given their HPV status. The HDP-WMM is capable of simultaneously characterizing the survival distributions of grouped data and capturing the dependence among different groups. Moreover, the HDP-WMM can identify clusters of patients based on their outcomes, providing additional information for exploring patient subtypes. Effective Markov chain Monte Carlo sampling algorithms are designed for model inference and function estimation. The clustering structure is identified by summarizing the posterior samples of the data random partition using the Bayesian cluster analysis tool. A simulation study is designed to validate the performance of the proposed inference methods. The practical utility of the proposed HDP-WMM is demonstrated by a case study on prognosis analysis of HPV-associated HNC. Our results show that the Bayesian HDP-WMM achieves satisfactory performance on estimating the survival functions and clustering patients based on their outcomes.

Reservoir interwell connectivity estimation from small datasets using a probabilistic data driven approach and uncertainty quantification

Over the past decade, various artificial intelligence types have made significant progress in petroleum reservoir modeling, from machine learning to deep learning. These data-driven proxies mostly use the available data as features and targets to find patterns between them or make a function approximation. This study mainly summarizes the estimation of reservoir inter-well connectivity by combining physics and data-driven. In addition, polynomial chaos expansion (PCE) has been used to quantify the uncertainty of inter-well connectivity. The contribution of this work is dedicated to a probabilistic data-driven, namely Bayesian inference applied to an artificial neural network and a reduced physics called capacitance resistance modeling (CRM). The main advantages of using such a probabilistic model are the quantification of uncertainty, which is well suited for decision-making in small datasets. As well as prior information, BBNs are often more appropriately calibrated than a classical neural network since its predicted uncertainty is more similar to the observed errors, and it is not necessary to use separate cross-validation. We implement a Markov chain Monte Carlo sampling algorithm; this approach evaluates the CRM parameters over time under waterflooding – dynamic parameters determination. Additionally, we have conducted a statistical inference to quantify the uncertainties associated with BNN modeling. Maximum a posteriori, a Bayesian optimization method, has also been used to compare estimated parameters by history-matching reservoirs' liquid production rates. An evaluation of the effectiveness of the proposed method was conducted on a synthetic and real reservoir consisting of a small and medium dataset. The results show that, when a small dataset is used, BBNs match the cumulative liquid production rate more accurately than MAP inference and the deterministic CRM, where an error of 5.87%, 10.94%, and 9.38% is achieved, respectively. In the case of the medium dataset, BNN methods give similar results. PCE is investigated in the case study with a small sample of data. Our results motivate us to perform sensitivity analysis on parameters to improve the robustness of the model in such cases.

Being Bayesian about learning Gaussian Bayesian networks from incomplete data

We propose a Bayesian model averaging (BMA) approach for inferring the structure of Gaussian Bayesian networks (BNs) from incomplete data, i.e. from data with missing values. Our method builds on the ‘Bayesian metric for Gaussian networks having score equivalence’ (BGe score) and we make the assumption that the unobserved data points are ‘missing completely at random’. We present a Markov Chain Monte Carlo sampling algorithm that allows for simultaneously sampling directed acyclic graphs (DAGs) as well as the values of the unobserved data points. We empirically cross-compare the network reconstruction accuracy of the new BMA approach with two non-Bayesian approaches for dealing with incomplete BN data, namely the classical structural Expectation Maximisation (EM) approach and the more recently proposed node average likelihood (NAL) method. For the empirical evaluation we use synthetic data from a benchmark Gaussian BN and real wet-lab protein phosphorylation data from the RAF signalling pathway.