Bayesian Inference Algorithm Research Articles

Forecasting the power of higher order weak-lensing statistics with automatically differentiable simulations

Aims. We present the fully differentiable physical Differentiable Lensing Lightcone (DLL) model, designed for use as a forward model in Bayesian inference algorithms that require access to derivatives of lensing observables with respect to cosmological parameters. Methods. We extended the public FlowPM N-body code, a particle-mesh N-body solver, while simulating the lensing lightcones and implementing the Born approximation in the Tensorflow framework. Furthermore, DLL is aimed at achieving high accuracy with low computational costs. As such, it integrates a novel hybrid physical-neural (HPN) parameterization that is able to compensate for the small-scale approximations resulting from particle-mesh schemes for cosmological N-body simulations. We validated our simulations in the context of the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) against high-resolution κTNG-Dark simulations by comparing both the lensing angular power spectrum and multiscale peak counts. We demonstrated its ability to recover lensing Cℓ up to a 10% accuracy at ℓ = 1000 for sources at a redshift of 1, with as few as ∼0.6 particles per Mpc h−1. As a first-use case, we applied this tool to an investigation of the relative constraining power of the angular power spectrum and peak counts statistic in an LSST setting. Such comparisons are typically very costly as they require a large number of simulations and do not scale appropriately with an increasing number of cosmological parameters. As opposed to forecasts based on finite differences, these statistics can be analytically differentiated with respect to cosmology or any systematics included in the simulations at the same computational cost of the forward simulation. Results. We find that the peak counts outperform the power spectrum in terms of the cold dark matter parameter, Ωc, as well as on the amplitude of density fluctuations, σ8, and the amplitude of the intrinsic alignment signal, AIA.

Efficient multifidelity likelihood-free Bayesian inference with adaptive computational resource allocation

Likelihood-free Bayesian inference algorithms are popular methods for inferring the parameters of complex stochastic models with intractable likelihoods. These algorithms characteristically rely heavily on repeated model simulations. However, whenever the computational cost of simulation is even moderately expensive, the significant burden incurred by likelihood-free algorithms leaves them infeasible for many practical applications. The multifidelity approach has been introduced in the context of approximate Bayesian computation to reduce the simulation burden of likelihood-free inference without loss of accuracy, by using the information provided by simulating computationally cheap, approximate models in place of the model of interest. In this work we demonstrate that multifidelity techniques can be applied in the general likelihood-free Bayesian inference setting. Analytical results on the optimal allocation of computational resources to simulations at different levels of fidelity are derived, and subsequently implemented practically. We provide an adaptive multifidelity likelihood-free inference algorithm that learns the relationships between models at different fidelities and adapts resource allocation accordingly, and demonstrate that this algorithm produces posterior estimates with near-optimal efficiency.

Fast and robust Bayesian inference using Gaussian processes with GPry

We present the GPry algorithm for fast Bayesian inference of general (non-Gaussian) posteriors with a moderate number of parameters. GPry does not need any pre-training, special hardware such as GPUs, and is intended as a drop-in replacement for traditional Monte Carlo methods for Bayesian inference. Our algorithm is based on generating a Gaussian Process surrogate model of the log-posterior, aided by a Support Vector Machine classifier that excludes extreme or non-finite values. An active learning scheme allows us to reduce the number of required posterior evaluations by two orders of magnitude compared to traditional Monte Carlo inference. Our algorithm allows for parallel evaluations of the posterior at optimal locations, further reducing wall-clock times. We significantly improve performance using properties of the posterior in our active learning scheme and for the definition of the GP prior. In particular we account for the expected dynamical range of the posterior in different dimensionalities. We test our model against a number of synthetic and cosmological examples. GPry outperforms traditional Monte Carlo methods when the evaluation time of the likelihood (or the calculation of theoretical observables) is of the order of seconds; for evaluation times of over a minute it can perform inference in days that would take months using traditional methods. GPry is distributed as an open source Python package (pip install gpry) and can also be found at https://github.com/jonaselgammal/GPry.

Estimation of genetic parameters for the tick and hemoparasite burden in Angus cattle.

The study was conducted with the objective of estimating genetic and phenotypic parameters for tick (CRM) and Babesia bigemina (IBBi), Babesia bovis (IBBo), and Anaplasma marginale (IAM) burden in Angus female breed in Brazil. The sample group was composed of Angus females raised in herds located in a region of endemic instability for cattle tick fever in the state of Rio Grande Sul (RS), Brazil. The variance components were estimated using Bayesian inference and Gibbs sampling algorithm, considering a multi-trait animal model. Heritability estimates showed values of low magnitude, ranging from 0.03 (IBBo) to 0.16 (CRM), while repeatability estimates ranged between 0.07 (IBBo) and 0.21 (CRM). Regarding the genetic correlation estimates, the values showed low (-0.01 for IBBo × IAM) to moderate (0.55 between IBBi × IAM) magnitudes. The results indicate that it is possible to use tick count and hemoparasite infection levels as selection criteria, with small genetic gains.

Static Behavior Prediction of Concrete Truss Arch Bridge Based on Dynamic Test Data and Bayesian Inference

The field load test is a direct and effective method for evaluating the performance of bridge structures. However, the existing bridge static load tests on site are costly, inefficient, and obstruct traffic; moreover, improper loading may also cause some damage to the bridge structure. This paper proposes a random model update method based on bridge dynamic load tests and the Bayesian inference as an alternative to the static load test. The Gaussian process model was used instead of the finite element model to reduce the cost of model calculation. Furthermore, choose the Markov Chain Monte Carlo (MCMC) method based on delay rejection adaptive Metropolis algorithm for Bayesian inference to improve the speed of the Bayesian method inferring the posterior probability density of updated parameters. First, the parameters to be updated for the bridge structure analysis model were determined based on the global sensitivity analysis method. Second, a uniform design sampling method was used to establish the Gaussian process optimization model to update the random model of the bridge structure. Finally, a reinforced concrete truss arch bridge was used to verify the correctness of the static load results of the bridge predicted by the random model update method based on dynamic load testing and Bayesian inference. The results show that the predicted results of the bridge static load test based on the dynamic load test and Bayesian reasoning method have an excellent agreement with the measured results, and this method can effectively overcome the adverse effects of the existing bridge static load test.

Multiple Introduction Events of Cultivated Diospyros Kaki L. of and Native Diospyros Species in Taiwan Inferred by Low Copy Nuclear Markers

Diospyros kaki L. is one of the most important fruits in Taiwan. Cultivated D. kaki cultivars are classified into four types, according to their loss of astringency in ripening fruit and their change in flesh color, namely the Pollination-Constant Non-Astringent (PCNA), the Pollination Variant Non-Astringent (PVNA), the Pollination Variant Astringent (PVA) and the Pollination Constant Astringent (PCA). Recently, persimmon breeders are paying more attention to rootstock selection for successful grafting and agricultural production. To identify suitable rootstock and to trace the origins of extant native Diospyros species in Taiwan, the 10 known native Diospyros species together with four D. kaki cultivars were analyzed by using two low-copy, nuclear-encoded DNA markers, ncpGS and PHYA. There was moderate to strong support for the major nodes within the phylogenetic tree, obtained from both the Maximum-likelihood or the Bayesian inference methods. The constructed D. kaki phylogeny using Maximum Likelihood (ML) and Bayesian Inference (BI) algorithms formed three clades. The most basal clade (clade A) comprised species distributed in the South East Asia , then species distributed in East Asia (clade B), and then D. kaki cultivars (clade C). The extant native Taiwan Diospyros species nested into clade A and B, suggesting a northward migration pattern of native Taiwan Diospyros species. In addition, the species D. oldhamii formed a sister clade with the D. kaki cultivars, suggesting that D. oldhamii may be a compatible rootstock. Lastly, D. kaki cultivars sampled from growers that were acquaintances formed corresponding subclades within clade C (C-1, C-2 and C-3), suggesting a high discrimination ability of the ncpGS and PHYA molecular markers used here.

An uncertainty quantification framework for logistic regression based geospatial natural hazard modeling

There is a class of data-driven global natural hazard predictive models that take advantage of broadly available geospatial proxies. These data-driven geospatial models have been commonly used for landslides and are becoming more available in recent years for liquefaction. Logistic regression is the most common method for predicting these ground failure occurrences. These models do not often include robust quantification of uncertainties although they are widely used in the pre-disaster planning and post-disaster response around the world. Taking the logistic regression based global geospatial liquefaction model (GGLM) (Zhu et al., 2017) as an example, we propose an uncertainty quantification (UQ) framework that consists of characterization of different sources of uncertainty, model sensitivity analysis, and forward uncertainty propagation. In this study, we have identified the main sources of uncertainty in such predictive models as parameter estimation uncertainty, modeling error, and geospatial input uncertainty. A Bayesian inference algorithm is used to quantify the posterior distribution of model parameters and quantify model parameter estimation uncertainties which are found to be negligible when a large amount of data is used in the parameter estimation process. Modeling errors are characterized based on the observed residuals between model predictions and measurements and by fitting a Gaussian distribution to the liquefaction probability residuals. The geospatial input uncertainties are characterized using the literature and expert judgment and propagated into model output. Second, we investigate the sensitivity of model output to different uncertain inputs and find that the variance of model output is largely controlled by the geospatial input uncertainties and model errors. Last, we propose an approximate forward uncertainty propagation method, which provides comparable results to a Monte Carlo simulation-based method with better computational efficiency. The proposed UQ framework provides a measure of uncertainty on model predictions and can be applied to any logistic-regression models and other geospatial modeling problems.

3D integral imaging depth estimation of partially occluded objects using mutual information and Bayesian optimization.

Integral imaging (InIm) is useful for passive ranging and 3D visualization of partially-occluded objects. We consider 3D object localization within a scene and in occlusions. 2D localization can be achieved using machine learning and non-machine learning-based techniques. These techniques aim to provide a 2D bounding box around each one of the objects of interest. A recent study uses InIm for the 3D reconstruction of the scene with occlusions and utilizes mutual information (MI) between the bounding box in this 3D reconstructed scene and the corresponding bounding box in the central elemental image to achieve passive depth estimation of partially occluded objects. Here, we improve upon this InIm method by using Bayesian optimization to minimize the number of required 3D scene reconstructions. We evaluate the performance of the proposed approach by analyzing different kernel functions, acquisition functions, and parameter estimation algorithms for Bayesian optimization-based inference for simultaneous depth estimation of objects and occlusion. In our optical experiments, mutual-information-based depth estimation with Bayesian optimization achieves depth estimation with a handful of 3D reconstructions. To the best of our knowledge, this is the first report to use Bayesian optimization for mutual information-based InIm depth estimation.

A wiener-based remaining useful life prediction method with multiple degradation patterns

Remaining useful life prediction is a critical step in prognostics and health management. Wiener process has been widely used in RUL prediction studies for degradation modelling. However, current wiener-based remaining life prediction methods usually include single degradation pattern, which limits the universality of the methods to cases with unclear degradation trends. To overcome this drawback, a novel remaining useful life prediction method considering multiple degradation patterns is developed in this paper. In the offline phase, the initial degradation parameters are estimated with an improved approximation algorithm combing least squares regression and maximum likelihood estimation. In the online phase, three degradation patterns are considered and the Bayesian inference and expectation maximization algorithm are applied to estimate the model parameters based on the online monitored degradation values. The remaining useful life prediction of each pattern is obtained as probability density function according to the property of the wiener process. To evaluate the performance of each degradation pattern and to determine the optimal pattern in real time, a fitness evaluation algorithm integrating dynamic time warping and feature-similarity is developed. The simulation signals and experimental datasets of aero engines and cutting tools are introduced to verify the effectiveness of the proposed method, and some widely used prediction methods are also applied as comparison. The results of the proposed method show higher accuracy than state-of-the art methods, which demonstrate the superiority of the proposed method.

A new approach for dynamic reliability analysis of reactor protection system for HPR1000

Traditional static methods are mostly adopted to analyze the reliability of reactor protection system (RPS) for nuclear power plant (NPP). However, they cannot characterize its dynamic interaction, time correlation, and uncertainty. In order to solve this problem, dynamic fault tree (DFT) analysis was firstly utilized to create the DFT for the RPS to characterize its dynamic interaction. Then, dynamic Bayesian network (DBN) was used to create the DBN model based upon the DFT for the RPS to characterize its dynamic interaction, time correlation, and uncertainty. Furthermore, Latin hypercube sampling (LHS) was employed to define a new Bayesian inference algorithm. Finally, the defined algorithm was applied in a RPS for Hualong-1 (HPR1000) in East China to conduct its dynamic prediction and sensitivity analyses through the Bayesian forward and backward inferences, and a new approach for dynamic reliability analysis of the RPS for HPR1000 was proposed. The research results showed that the proposed approach could be used for the dynamic reliability analysis of the RPS.

Sequential Bayesian inference for agent-based models with application to the Chinese business cycle

Agent-based models (ABMs), which model the evolving system through the interactive behavior of agents, are often challenged by inaccurate estimates. However, estimates provided by traditional approaches are sensitive to initial values due to irregular and multimodal likelihood functions. In this paper, we extend the Sequential Monte Carlo (SMC) algorithm for Bayesian posterior inference to ABMs, and used the kernel density estimation (KDE) to approximate the likelihood function. Our simulation results demonstrate that this method delivers advantages for parameters estimation in addressing multimodal likelihood, especially for large samples. Applying this approach to China’s macroeconomic data, we find that exogenous shocks in the agent-based model play a small role in explaining China’s business cycle. The SMC combined with KDE is straightforward to implement in empirical studies and yields estimates that align with the intuitions of ABMs, by mitigating estimation errors and reducing efficiency losses.

Scalable Variational Inference for Low-Rank Spatiotemporal Receptive Fields.

An important problem in systems neuroscience is to characterize how a neuron integrates sensory inputs across space and time. The linear receptive field provides a mathematical characterization of this weighting function and is commonly used to quantify neural response properties and classify cell types. However, estimating receptive fields is difficult in settings with limited data and correlated or high-dimensional stimuli. To overcome these difficulties, we propose a hierarchical model designed to flexibly parameterize low-rank receptive fields. The model includes gaussian process priors over spatial and temporal components of the receptive field, encouraging smoothness in space and time. We also propose a new temporal prior, temporal relevance determination, which imposes a variable degree of smoothness as a function of time lag. We derive a scalable algorithm for variational Bayesian inference for both spatial and temporal receptive field components and hyperparameters. The resulting estimator scales to high-dimensional settings in which full-rank maximum likelihood or a posteriori estimates are intractable. We evaluate our approach on neural data from rat retina and primate cortex and show that it substantially outperforms a variety of existing estimators. Our modeling approach will have useful extensions to a variety of other high-dimensional inference problems with smooth or low-rank structure.

Improvement of measurement accuracy using Bayesian inference - Reduction of instrumentation effort in an axial compressor

As the next generation of turbomachinery components becomes more sensitive to instrumentation intrusiveness, a reduction of the number of measurement devices required for the evaluation of performance is a possible and cost-effective way to mitigate the arising of non-mastered experimental errors. A hybrid methodology that couples experimental techniques with modeling techniques through a Bayesian data-driven framework is employed to reduce the instrumentation effort. A numerical model is employed to provide an initial belief of the flow, which is then updated based on undersampled experimental observations by a Bayesian inference algorithm. The goal of the present work is to showcase the proposed hybrid methodology and demonstrate its partial application through Gaussian Process regression in reducing the instrumentation effort and testing time at the outlet of a low aspect ratio axial compressor stage representative of the last stage of a high-pressure compressor. Preliminary results show an accurate reconstruction of the mean flow field with a direct uncertainty quantification provided by the Bayesian approach.

Efficient Data Augmentation Techniques for Some Classes of State Space Models

Data augmentation improves the convergence of iterative algorithms, such as the EM algorithm and Gibbs sampler by introducing carefully designed latent variables. In this article, we first propose a data augmentation scheme for the first-order autoregression plus noise model, where optimal values of working parameters introduced for recentering and rescaling of the latent states, can be derived analytically by minimizing the fraction of missing information in the EM algorithm. The proposed data augmentation scheme is then utilized to design efficient Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of some non-Gaussian and nonlinear state space models, via a mixture of normals approximation coupled with a block-specific reparametrization strategy. Applications on simulated and benchmark real data sets indicate that the proposed MCMC sampler can yield improvements in simulation efficiency compared with centering, noncentering and even the ancillarity-sufficiency interweaving strategy.

JAX-COSMO: An End-to-End Differentiable and GPU Accelerated Cosmology Library

We present jax-cosmo, a library for automatically differentiable cosmological theory calculations. It uses the JAX library, which has created a new coding ecosystem, especially in probabilistic programming. As well as batch acceleration, just-in-time compilation, and automatic optimization of code for different hardware modalities (CPU, GPU, TPU), JAX exposes an automatic differentiation (autodiff) mechanism. Thanks to autodiff, jax-cosmo gives access to the derivatives of cosmological likelihoods with respect to any of their parameters, and thus enables a range of powerful Bayesian inference algorithms, otherwise impractical in cosmology, such as Hamiltonian Monte Carlo and Variational Inference. In its initial release, jax-cosmo implements background evolution, linear and non-linear power spectra (using halofit or the Eisenstein and Hu transfer function), as well as angular power spectra with the Limber approximation for galaxy and weak lensing probes, all differentiable with respect to the cosmological parameters and their other inputs. We illustrate how autodiff can be a game-changer for common tasks involving Fisher matrix computations, or full posterior inference with gradient-based techniques. In particular, we show how Fisher matrices are now fast, exact, no longer require any fine tuning, and are themselves differentiable. Finally, using a Dark Energy Survey Year 1 3x2pt analysis as a benchmark, we demonstrate how jax-cosmo can be combined with Probabilistic Programming Languages to perform posterior inference with state-of-the-art algorithms including a No U-Turn Sampler, Automatic Differentiation Variational Inference,and Neural Transport HMC. We further demonstrate that Normalizing Flows using Neural Transport are a promising methodology for model validation in the early stages of analysis. <em>The software and documentation referred to in this paper can be found <a href="https://github.com/DifferentiableUniverseInitiative/jax-cosmo-paper/">here</a></em>.

Efficient and generalizable tuning strategies for stochastic gradient MCMC

Stochastic gradient Markov chain Monte Carlo (SGMCMC) is a popular class of algorithms for scalable Bayesian inference. However, these algorithms include hyperparameters such as step size or batch size that influence the accuracy of estimators based on the obtained posterior samples. As a result, these hyperparameters must be tuned by the practitioner and currently no principled and automated way to tune them exists. Standard Markov chain Monte Carlo tuning methods based on acceptance rates cannot be used for SGMCMC, thus requiring alternative tools and diagnostics. We propose a novel bandit-based algorithm that tunes the SGMCMC hyperparameters by minimizing the Stein discrepancy between the true posterior and its Monte Carlo approximation. We provide theoretical results supporting this approach and assess various Stein-based discrepancies. We support our results with experiments on both simulated and real datasets, and find that this method is practical for a wide range of applications.

Bayesian Algorithms for Joint Estimation of Brain Activity and Noise in Electromagnetic Imaging.

Simultaneously estimating brain source activity and noise has long been a challenging task in electromagnetic brain imaging using magneto- and electroencephalography. The problem is challenging not only in terms of solving the NP-hard inverse problem of reconstructing unknown brain activity across thousands of voxels from a limited number of sensors, but also for the need to simultaneously estimate the noise and interference. We present a generative model with an augmented leadfield matrix to simultaneously estimate brain source activity and sensor noise statistics in electromagnetic brain imaging (EBI). We then derive three Bayesian inference algorithms for this generative model (expectation-maximization (EBI-EM), convex bounding (EBI-Convex) and fixed-point (EBI-Mackay)) to simultaneously estimate the hyperparameters of the prior distribution for brain source activity and sensor noise. A comprehensive performance evaluation for these three algorithms is performed. Simulations consistently show that the performance of EBI-Convex and EBI-Mackay updates is superior to that of EBI-EM. In contrast to the EBI-EM algorithm, both EBI-Convex and EBI-Mackay updates are quite robust to initialization, and are computationally efficient with fast convergence in the presence of both Gaussian and real brain noise. We also demonstrate that EBI-Convex and EBI-Mackay update algorithms can reconstruct complex brain activity with only a few trials of sensor data, and for resting-state data, achieving significant improvement in source reconstruction and noise learning for electromagnetic brain imaging.

Bayesian inference in ring attractor networks

Working memories are thought to be held in attractor networks in the brain. These attractors should keep track of the uncertainty associated with each memory, so as to weigh it properly against conflicting new evidence. However, conventional attractors do not represent uncertainty. Here, we show how uncertainty could be incorporated into an attractor, specifically a ring attractor that encodes head direction. First, we introduce a rigorous normative framework (the circular Kalman filter) for benchmarking the performance of a ring attractor under conditions of uncertainty. Next, we show that the recurrent connections within a conventional ring attractor can be retuned to match this benchmark. This allows the amplitude of network activity to grow in response to confirmatory evidence, while shrinking in response to poor-quality or strongly conflicting evidence. This "Bayesian ring attractor" performs near-optimal angular path integration and evidence accumulation. Indeed, we show that a Bayesian ring attractor is consistently more accurate than a conventional ring attractor. Moreover, near-optimal performance can be achieved without exact tuning of the network connections. Finally, we use large-scale connectome data to show that the network can achieve near-optimal performance even after we incorporate biological constraints. Our work demonstrates how attractors can implement a dynamic Bayesian inference algorithm in a biologically plausible manner, and it makes testable predictions with direct relevance to the head direction system as well as any neural system that tracks direction, orientation, or periodic rhythms.

Simultaneous Channel Estimation and Localization of Terahertz Massive MIMO Systems via Bayesian Tensor Decomposition

In this letter, a Bayesian tensor-based scheme is proposed for simultaneous channel estimation and localization (SCEAL) in wideband terahertz (THz) massive multiple-input multiple-output (MIMO) systems with hybrid analog-digital architectures. Considering the beam squint effect, we construct the received training signal into a sixth-order tensor model, including the angle-of-arrival (AOA)/angle-of-departure (AOD), time delay, and path gains. Then, a real-valued Bayesian inference algorithm is developed to fit the constructed tensor model for SCEAL. The proposed algorithm achieves SCEAL with and without line-of-sight (LOS) path, and offers higher accuracy compared to the state-of-the-art algorithms. Simulations verify the effectiveness of the proposed algorithm.

Implementation of Bayesian inference MCMC algorithm in phylogenetic analysis of Dipterocarpaceae family

Dipterocarpaceae is one of the most prominent plant families, with more than 500 members of species. This family mostly used timber plants for housing, making ships, decking, and primary materials for making furniture. In Indonesia, many Dipterocarpaceae species have morphological similarities and are challenging to recognize in the field. As a result, the classification process becomes difficult and even results are inconsistent when viewed only from the morphology. This research will analyze the phylogenetic tree of Dipterocarpaceae based on the chloroplast matK gene. The aim of the research is to classify the phylogenetics tree of Dipterocarpaceae family using Bayesian inference algorithm. This research used the chloroplast gene instead of morphological characters which has more accurate. The analysis steps are collecting data, modifying the structure sequence name, sequence alignment, constructing tree by using Markov Chain Monte Carlo (MCMC) from Bayesian Inference, and evaluating and analyzing the phylogenetic tree. The results showed that the tree constructed based on the gene is different from the tree based on morphology. Based on the morphological, Dipterocarpus should be in the Dipterocarpeae tribe but based on the similarity of its genes, Dipterocarpus is more similar to the Shoreae tribe.