Hierarchical Bayesian Inference Research Articles

Marginal unbiased score expansion and application to CMB lensing

We present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G $1500\text{ }\text{ }{\mathrm{deg}}^{2}$ survey, corresponding to a latent dimensionality of $\ensuremath{\sim}6\text{ }\text{ }\mathrm{million}$ and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only $\ensuremath{\sim}10%$. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

Rational arbitration between statistics and rules in human sequence processing.

Detecting and learning temporal regularities is essential to accurately predict the future. A long-standing debate in cognitive science concerns the existence in humans of a dissociation between two systems, one for handling statistical regularities governing the probabilities of individual items and their transitions, and another for handling deterministic rules. Here, to address this issue, we used finger tracking to continuously monitor the online build-up of evidence, confidence, false alarms and changes-of-mind during sequence processing. All these aspects of behaviour conformed tightly to a hierarchical Bayesian inference model with distinct hypothesis spaces for statistics and rules, yet linked by a single probabilistic currency. Alternative models based either on a single statistical mechanism or on two non-commensurable systems were rejected. Our results indicate that a hierarchical Bayesian inference mechanism, capable of operating over distinct hypothesis spaces for statistics and rules, underlies the human capability for sequence processing.

Generalized hierarchical Bayesian inference for fatigue life prediction based on multi-parameter Weibull models

This article proposes the use of generalized hierarchical Bayesian inference for fatigue life prediction based on general multi-parameter Weibull models. First, a five-parameter Weibull model for corrosion-stress-life (C-S-N) is introduced; neglecting the influence of corrosion, the model degenerates into a three-parameter Weibull model for stress-life (S-N). To predict the fatigue life based on the observation data, a three-layer hierarchical Bayesian structure for these Weibull models is established, and the posterior joint PDF of the parameters and hyperparameters in the generalized hierarchical Bayesian model (GHBM) is derived. Gibbs sampling is employed to obtain posterior samples for parameters and hyperparameters using their full conditional distribution, which is simplified using a Markov blanket based on the probabilistic dependences of the parameters and hyperparameters. Subsequently, three groups of fatigue data for steel wires with different corrosion degrees are used to validate the GHBM, considering both noninformative and informative priors. When considering conventional fatigue data (i.e., without corrosion), the fatigue life prediction obtained from the GHBM and the maximum likelihood estimation (MLE) are compared, along with the results obtained from traditional lognormal models. The results indicate that the scatter in fatigue life prediction for the corroded specimens using the GHBM becomes smaller when the informative priors for the parameters are considered in the Weibull model for C-S-N. For conventional fatigue data, the fatigue life predictions using the Weibull model for S-N are similar for both the GHBM with the noninformative priors and the MLE, which are more conservative when compared with the results obtained from the traditional lognormal models. The proposed method in this paper can be used as a general approach in fatigue life prediction.

Task-induced neural covariability as a signature of approximate Bayesian learning and inference.

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called "differential correlations" as the observer's internal model learns the stimulus distribution, and the observer's behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject's internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.

Performance-based ranking of porous materials for PSA carbon capture under the uncertainty of experimental data

The question we address in this study is how the uncertainty in equilibrium adsorption data and other properties of an adsorbent affects its ranking in screening studies for carbon capture using pressure swing adsorption (PSA). Construction of a model for a PSA simulation is normally based on a single set of experimental data, which leads to a single energy penalty-productivity Pareto front, used as the performance indicator for a given material. In reality, experimental measurements performed on the same material by different groups show a significant degree of scattering. How this scattering propagates to the process level is not known. To answer the question posed in this study, we consider the case of zeolite 13X, a well-known benchmark for carbon capture studies. Using curated experimental data from the literature, we develop a probabilistic isotherm model using hierarchical Bayesian inference. We then combine detailed process optimisation and surrogate models to explore the impact of material-level uncertainty on the behaviour of Pareto fronts. We observe that the performance of zeolite 13X is more accurately represented by a cloud of possible Pareto fronts rather than by a single Pareto front, and that characteristics of this cloud are mostly determined by the uncertainty in the adsorption data. Given the observed variability in process-level performance, materials ranking should be approached with caution as it is very likely that clouds of two distinct materials can overlap.

Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of Partial Differential Equations

Robust physics discovery is of great interest for many scientific and engineering fields. Inspired by the principle that a representative model is the simplest one among all possible models, a new model selection criterion considering both model’s Parsimony and Sparsity is proposed. A Parsimony-Enhanced Sparse Bayesian Learning (PeSBL) method is developed for discovering the governing Partial Differential Equations (PDEs) of nonlinear dynamical systems. Compared with the conventional Sparse Bayesian Learning (SBL) method, the PeSBL method promotes parsimony of the learned model in addition to its sparsity. In this method, the parsimony of model terms is evaluated using their locations in the prescribed candidate library, for the first time, considering the increased complexity with the power of polynomials and the order of spatial derivatives. Subsequently, model parameters are updated through Bayesian inference with raw data. This procedure aims to reduce the error associated with the possible loss of information in data preprocessing and numerical differentiation prior to sparse regression. Results of numerical case studies indicate that the governing PDEs of many canonical dynamical systems can be correctly identified using the proposed PeSBL method from highly noisy data (up to 50% in the current study). Next, the proposed methodology is extended for stochastic PDE learning in which both model parameters and modeling errors are considered as random variables. Hierarchical Bayesian Inference (HBI) is integrated with the proposed framework for stochastic PDE learning from a population of observations. Finally, the proposed PeSBL is demonstrated for system response prediction with uncertainties and anomaly detection. Codes of all demonstrated examples in this study are available on the website: https://github.com/ymlasu.

Exploring the millimetre emission in nearby galaxies: Analysis of the edge-on galaxy NGC 891

New observations of the edge-on galaxy NGC 891, at 1.15 and 2mm obtained with the IRAM 30-m telescope and the NIKA2 camera, within the framework of the IMEGIN (Interpreting the Millimetre Emission of Galaxies with IRAM and NIKA2) Large Program, are presented in this work. By using multiwavelength maps (from the mid-IR to the cm wavelengths) we perform SED fitting in order to extract the physical properties of the galaxy on both global and local (~kpc) scales. For the interpretation of the observations we make use of a state-of-the-art SED fitting code, HerBIE (HiERarchical Bayesian Inference for dust Emission). The observations indicate a galaxy morphology, at mm wavelengths, similar to that of the cold dust emission traced by submm observations and to that of the molecular gas. The contribution of the radio emission at the NIKA2 bands is very small (negligible at 1.15 mm and ~ 10% at 2 mm) while it dominates the total energy budget at longer wavelengths (beyond 5 mm). On local scales, the distribution of the free-free emission resembles that of the dust thermal emission while the distribution of the synchrotron emission shows a deficiency along the major axis of the disc of the galaxy.

Population Properties of Neutron Stars in the Coalescing Compact Binaries

Abstract We perform a hierarchical Bayesian inference to investigate the population properties of the coalescing compact binaries involving at least one neutron star (NS). With the current gravitational-wave (GW) observation data, we can rule out none of the double Gaussian, single Gaussian, and uniform NS mass distribution models, though a specific double Gaussian model inferred from the Galactic NSs is found to be slightly more preferred. The mass distribution of black holes (BHs) in the neutron star–black hole (NSBH) population is found to be similar to that in the Galactic X-ray binaries. Additionally, the ratio of the merger rate densities between NSBHs and BNSs is estimated to be ∼3:7. The spin properties of the binaries, though constrained relatively poorly, play a nontrivial role in reconstructing the mass distribution of NSs and BHs. We find that a perfectly aligned spin distribution can be ruled out, while a purely isotropic distribution of spin orientation is still allowed. To evaluate the feasibility of reliably determining the population properties of NSs in the coalescing compact binaries with upcoming GW observations, we perform simulations with a mock population. We find that with 100 detections (including BNSs and NSBHs) the mass distribution of NSs can be well determined, and the fraction of BNSs can also be accurately estimated.

A probabilistic model to evaluate the resilience of unattended machinery plants in autonomous ships

Over the next few years, digitalization and automation are expected to be key drivers for maritime transport innovation to be key drivers for maritime transportation innovation. This revolutionary shift in the shipping industry will heavily impact the reliability of the machinery which is intended to be operated remotely with minimum support from humans. Despite a large amount of research into autonomous navigation and control systems in maritime transportation, the evaluation of unattended engine rooms has received very little attention. For autonomous vessels to be effective during their unmanned mission, it is essential for the engine room understand its health condition and self-manage performance. The unattended machinery plant (UMP) should be resilient enough to have the ability to survive and recover from unexpected perturbations, disruptions, and operational degradations. Otherwise, the system may require unplanned maintenance or the operation will stop. Therefore, the UMP must continue its operation without human intervention and safely return the ship to port. This paper aims to develop a machine learning-based model to predict an UMP's performance and estimate how long the engine room can operate without human assistance. A Random Process Tree is used to model failures in the unattended components, while a Hierarchical Bayesian Inference is adopted to facilitate the prediction of unknown parameters in the process. A probabilistic Bayesian Network developed and evaluated the dependent relationship between active and standby components to assess the effect of redundant units in the performance of unattended machinery. The present framework will provide helpful additional information to evaluate the associate uncertainties and predict the untoward events that put the engine room at risk. The results highlight the model's ability to predict the UMP's trusted operation period and evaluate an unattended engine room's resilience. A real case study of a merchant vessel used for short sea shipping in European waters is considered to demonstrate the model's application.

Black Hole Gravitational Potential Enhanced Fallback Accretion onto the Nascent Lighter Compact Object: Tentative Evidence in the O3 Run Data of LIGO/Virgo

Abstract In a binary system, the gravitational potential of the primary black hole may play an important role in enhancing the fallback accretion onto the lighter compact object newly formed in the second supernova explosion. As a result, the final masses of the binary compact objects would be correlated, as suggested recently by Safarzadeh & Wysocki. In this work, we analyze the mass distribution of four gravitational-wave events, which are characterized by both a small mass ratio and a low mass (≤5M ⊙) of the light component, and find tentative evidence for a mass correlation among the objects. To evaluate the feasibility of testing such a hypothesis with upcoming observations, we carry out simulations with a mock population and perform Bayesian hierarchical inference for the mass distribution. We find that with dozens of low mass ratio events, whether there exists correlation in the component mass distributions or not can be robustly tested and the correlation, if it exists, can be well determined.

Discriminating between different scenarios for the formation and evolution of massive black holes with LISA

Electromagnetic observations have provided strong evidence for the existence of massive black holes in the center of galaxies, but their origin is still poorly known. Different scenarios for the formation and evolution of massive black holes lead to different predictions for their properties and merger rates. LISA observations of coalescing massive black hole binaries could be used to reverse engineer the problem and shed light on these mechanisms. In this paper, we introduce a pipeline based on hierarchical Bayesian inference to infer the mixing fraction between different theoretical models by comparing them to LISA observations of massive black hole mergers. By testing this pipeline against simulated LISA data, we show that it allows us to accurately infer the properties of the massive black hole population as long as our theoretical models provide a reliable description of the Universe. We also show that measurement errors, including both instrumental noise and weak lensing errors, have little impact on the inference.

Hierarchical Bayesian models of transcriptional and translational regulation processes with delays.

Simultaneous recordings of gene network dynamics across large populations have revealed that cell characteristics vary considerably even in clonal lines. Inferring the variability of parameters that determine gene dynamics is key to understanding cellular behavior. However, this is complicated by the fact that the outcomes and effects of many reactions are not observable directly. Unobserved reactions can be replaced with time delays to reduce model dimensionality and simplify inference. However, the resulting models are non-Markovian, and require the development of new inference techniques. We propose a non-Markovian, hierarchical Bayesian inference framework for quantifying the variability of cellular processes within and across cells in a population. We illustrate our approach using a delayed birth-death process. In general, a distributed delay model, rather than a popular fixed delay model, is needed for inference, even if only mean reaction delays are of interest. Using in silico and experimental data we show that the proposed hierarchical framework is robust and leads to improved estimates compared to its non-hierarchical counterpart. We apply our method to data obtained using time-lapse microscopy and infer the parameters that describe the dynamics of protein production at the single cell and population level. The mean delays in protein production are larger than previously reported, have a coefficient of variation of around 0.2 across the population, and are not strongly correlated with protein production or growth rates. Accompanying code in Python is available at https://github.com/mvcortez/Bayesian-Inference. kresimir.josic@gmail.com or jaekkim@kaist.ac.kr or cbskust@korea.ac.kr. Supplementary data are available at Bioinformatics online.

Inferring the dynamics of mutated hematopoietic stem and progenitor cells induced by IFNα in myeloproliferative neoplasms

AbstractClassical BCR-ABL–negative myeloproliferative neoplasms (MPNs) are clonal disorders of hematopoietic stem cells (HSCs) caused mainly by recurrent mutations in genes encoding JAK2 (JAK2), calreticulin (CALR), or the thrombopoietin receptor (MPL). Interferon α (IFNα) has demonstrated some efficacy in inducing molecular remission in MPNs. To determine factors that influence molecular response rate, we evaluated the long-term molecular efficacy of IFNα in patients with MPN by monitoring the fate of cells carrying driver mutations in a prospective observational and longitudinal study of 48 patients over more than 5 years. We measured the clonal architecture of early and late hematopoietic progenitors (84 845 measurements) and the global variant allele frequency in mature cells (409 measurements) several times per year. Using mathematical modeling and hierarchical Bayesian inference, we further inferred the dynamics of IFNα-targeted mutated HSCs. Our data support the hypothesis that IFNα targets JAK2V617F HSCs by inducing their exit from quiescence and differentiation into progenitors. Our observations indicate that treatment efficacy is higher in homozygous than heterozygous JAK2V617F HSCs and increases with high IFNα dose in heterozygous JAK2V617F HSCs. We also found that the molecular responses of CALRm HSCs to IFNα were heterogeneous, varying between type 1 and type 2 CALRm, and a high dose of IFNα correlates with worse outcomes. Our work indicates that the long-term molecular efficacy of IFNα implies an HSC exhaustion mechanism and depends on both the driver mutation type and IFNα dose.

Hierarchical, Memory-Based Movement Models for Translocated Elk (Cervus canadensis)

The use of spatial memory is well-documented in many animal species and has been shown to be critical for the emergence of spatial learning. Adaptive behaviors based on learning can emerge thanks to an interdependence between the acquisition of information over time and movement decisions. The study of how spatio-ecological knowledge is constructed throughout the life of an individual has not been carried out in a quantitative and comprehensive way, hindered by the lack of knowledge of the information an animal already has of its environment at the time monitoring begins. Identifying how animals use memory to make beneficial decisions is fundamental to developing a general theory of animal movement and space use. Here we propose several mobility models based on memory and perform hierarchical Bayesian inference on 11-month trajectories of 21 elk after they were released in a completely new environment. Almost all the observed animals exhibited preferential returns to previously visited patches, such that memory and random exploration phases occurred. Memory decay was mild or negligible over the study period. The fact that individual elk rapidly become used to a relatively small number of patches was consistent with the hypothesis that they seek places with predictable resources and reduced mortality risks such as predation.

Minimal physicalism as a scale-free substrate for cognition and consciousness.

Theories of consciousness and cognition that assume a neural substrate automatically regard phylogenetically basal, nonneural systems as nonconscious and noncognitive. Here, we advance a scale-free characterization of consciousness and cognition that regards basal systems, including synthetic constructs, as not only informative about the structure and function of experience in more complex systems but also as offering distinct advantages for experimental manipulation. Our “minimal physicalist” approach makes no assumptions beyond those of quantum information theory, and hence is applicable from the molecular scale upwards. We show that standard concepts including integrated information, state broadcasting via small-world networks, and hierarchical Bayesian inference emerge naturally in this setting, and that common phenomena including stigmergic memory, perceptual coarse-graining, and attention switching follow directly from the thermodynamic requirements of classical computation. We show that the self-representation that lies at the heart of human autonoetic awareness can be traced as far back as, and serves the same basic functions as, the stress response in bacteria and other basal systems.

Cluster strong lensing with hierarchical inference

Context. Lensing by galaxy clusters is a versatile probe of cosmology and extragalactic astrophysics, but the accuracy of some of its predictions is limited by the simplified models adopted to reduce the (otherwise intractable) number of degrees of freedom. Aims. We aim to explore cluster lensing models in which the parameters of all cluster member galaxies are free to vary around some common scaling relations with non-zero scatter and deviate significantly from these relations if, and only if, the data require this. Methods. We devised a Bayesian hierarchical inference framework that enables the determination of all lensing parameters and the scaling relation hyperparameters, including intrinsic scatter, from lensing constraints and (if given) stellar kinematic measurements. We achieve this through BAYESLENS, a purpose-built wrapper around common parametric lensing codes that can sample the full posterior on parameters and hyperparameters; we release BAYESLENS with this paper. Results. We ran functional tests of our code against simple mock cluster lensing datasets with realistic uncertainties. The parameters and hyperparameters are recovered within their 68% credibility ranges and the positions of all the “observed” multiple images are accurately reproduced by the BAYELENS best-fit model, without over-fitting. Conclusions. We have shown that an accurate description of cluster member galaxies is attainable, despite a large number of degrees of freedom, through fast and tractable inference. This extends beyond the most updated cluster lensing models. The precise impact on studies of cosmography, galaxy evolution, and high-redshift galaxy populations can then be quantified on real galaxy clusters. While other sources of systematics exist and may be significant in real clusters, our results show that the contribution of intrinsic scatter in cluster member populations can now be controlled.

Hierarchical Bayesian Uncertainty Quantification for a Model of the Red Blood Cell

Simulations of blood flows in microfluidic devices and physiological systems are gaining importance in complementing experimental and clinical studies. The predictive capabilities of these simulations hinge on the parameters of the red blood cell (RBC) model that are usually calibrated from experimental data. However, these parameter values may vary drastically when calibrated using different experimental quantities or experimental settings. In turn, the results of existing blood flow simulations largely depend on the utilized parameters that have been chosen to validate a particular experiment. We suggest a revision to this type of model calibration to properly integrate experimental data in the computational models and accordingly inform their predictions. In this context, we introduce the calibration of a popular RBC model using data-driven, hierarchical Bayesian inference. We employ data from classical experiments of RBC stretching by optical tweezers and tank treading in shear flows, and distinguish the calibration of the model parameters through single-level and hierarchical Bayesian uncertainty quantification. We find that the optimal model parameters depend not only on the data used for the inference but also on the way the data are used in the inference process. Single-level Bayesian models predict well the data used in their calibration, but are inferior to the hierarchical Bayesian model at predicting previously unseen data. This work demonstrates that the proper integration of experimental data is essential for the development of a robust and transferable RBC model. We believe that the present study can serve as a prototype across scientific fields, in revising the integration of computational models and heterogeneous experimental data.

Searching for ultralight bosons within spin measurements of a population of binary black hole mergers

Ultralight bosons can form clouds around rotating black holes if their Compton wavelength is comparable to the black hole size. The boson cloud spins down the black hole through a process called superradiance, lowering the black hole spin to a characteristic value. It has been suggested that spin measurements of the black holes detected by ground-based gravitational-wave detectors can be used to constrain the mass of ultralight bosons. Unfortunately, a measurement of the individual black hole spins is often uncertain, resulting in inconclusive results. Instead, we use hierarchical Bayesian inference to combine information from multiple gravitational-wave sources and obtain stronger constraints. We show that hundreds of high signal-to-noise ratio gravitational-wave detections are enough to exclude (confirm) the existence of non-interacting bosons in the mass range $\left[10^{-13},3\times 10^{-12}\right]~\rm{eV}$ $\left([10^{-13},10^{-12}]~\rm{eV}\right)$. The precise number depends on the distribution of black hole spins at formation and the mass of the boson. From the few uninformative spin measurements of binary black hole mergers detected by LIGO and Virgo in their first two observing runs, we cannot draw statistically significant conclusions.

Hierarchical Bayesian Models to Estimate the Number of Losses of Separation between Aircraft in Flight

Air transport is considered to be the safest mode of mass transportation. Air traffic management (ATM) systems constitute one of the fundamental pillars that contribute to these high levels of safety. In this paper we wish to answer two questions: (i) What is the underlying safety level of ATM systems in Europe? and (ii) What is the dispersion, that is, how far does each ATM service provider deviate from this underlying safety level? To do this, we develop four hierarchical Bayesian inference models that allow us to infer and predict the common rate of occurrence of SMIs, as well as the specific rates of occurrence for each air navigation service provider (ANSP). This study shows the usefulness of hierarchical structures when it comes to obtaining parameters that enable risk to be quantified effectively. The models developed have been found to be useful in explaining and predicting the safety performance of 29 European ATM systems with common regulations and work procedures, but with different circumstances and numbers of aircraft, each managing traffic of differing complexity.

A multinomial process tree for reliability assessment of machinery in autonomous ships

Maritime Autonomous Surface Ships have received a significant amount of attention in recent projects. They promise a reduction in marine accidents and mitigation of human errors. Most of the ongoing research effort is directed toward autonomous navigation and cybersecurity. However, the importance of a machinery plant in the engine room that can operate reliably without human attendance is hardly investigated. To prevent failures in such systems and extend the interval between required human interventions, it is essential to improve their reliability. This paper aims to present a systematic approach to evaluate the reliability of an autonomous system under the influence of uncertain disruptions and to predict failure rates of unattended machinery plants. A Multinomial Process Tree is used to model failures in the main failure-sensitive components. Hierarchical Bayesian Inference is adopted to facilitate the prediction of frequencies of disruptive events and estimate the entire system's failure rate. The outcome of this research enables design strategies to improve the reliability of autonomous ships and prevent Fatal Technical Failure during the operation. This allows assessing whether a given machinery plant is sufficiently reliable to be used on unmanned ships. A case study is considered to demonstrate the application of the presented method.