Likelihood Evaluation Research Articles

Inferring Binary Properties from Gravitational-Wave Signals

This review provides a conceptual and technical survey of methods for parameter estimation of gravitational-wave signals in ground-based interferometers such as Laser Interferometer Gravitational-Wave Observatory (LIGO) and Virgo. We introduce the framework of Bayesian inference and provide an overview of models for the generation and detection of gravitational waves from compact binary mergers, focusing on the essential features that are observable in the signals. Within the traditional likelihood-based paradigm, we describe various approaches for enhancing the efficiency and robustness of parameter inference. This includes techniques for accelerating likelihood evaluations, such as heterodyne/relative binning, reduced-order quadrature, multibanding, and interpolation. We also cover methods to simplify the analysis to improve convergence, via reparameterization, importance sampling, and marginalization. We end with a discussion of recent developments in the application of likelihood-free (simulation-based) inference methods to gravitational-wave data analysis.

Fast Bayesian Inference for Spatial Mean-Parameterized Conway–Maxwell–Poisson Models

Count data with complex features arise in many disciplines, including ecology, agriculture, criminology, medicine, and public health. Zero inflation, spatial dependence, and non-equidispersion are common features in count data. There are currently two classes of models that allow for these features—the mode-parameterized Conway–Maxwell–Poisson (COMP) distribution and the generalized Poisson model. However both require the use of either constraints on the parameter space or a parameterization that leads to challenges in interpretability. We propose spatial mean-parameterized COMP models that retain the flexibility of these models while resolving the above issues. We use a Bayesian spatial filtering approach in order to efficiently handle high-dimensional spatial data and we use reversible-jump MCMC to automatically choose the basis vectors for spatial filtering. The COMP distribution poses two additional computational challenges—an intractable normalizing function in the likelihood and no closed-form expression for the mean. We propose a fast computational approach that addresses these challenges by, respectively, introducing an efficient auxiliary variable algorithm and pre-computing key approximations for fast likelihood evaluation. We illustrate the application of our methodology to simulated and real datasets, including Texas HPV-cancer data and US vaccine refusal data. Supplementary materials for this article are available online.

Scalable Bayesian uncertainty quantification with data-driven priors for radio interferometric imaging

Abstract Next-generation radio interferometers like the Square Kilometer Array have the potential to unlock scientific discoveries thanks to their unprecedented angular resolution and sensitivity. One key to unlocking their potential resides in handling the deluge and complexity of incoming data. This challenge requires building radio interferometric (RI) imaging methods that can cope with the massive data sizes and provide high-quality image reconstructions with uncertainty quantification (UQ). This work proposes a method coined QuantifAI to address UQ in RI imaging with data-driven (learned) priors for high-dimensional settings. Our model, rooted in the Bayesian framework, uses a physically motivated model for the likelihood. The model exploits a data-driven convex prior potential, which can encode complex information learned implicitly from simulations and guarantee the log-concavity of the posterior. We leverage probability concentration phenomena of high-dimensional log-concave posteriors to obtain information about the posterior, avoiding MCMC sampling techniques. We rely on convex optimisation methods to compute the MAP estimation, which is known to be faster and better scale with dimension than MCMC strategies. QuantifAI allows us to compute local credible intervals and perform hypothesis testing of structure on the reconstructed image. We propose a novel fast method to compute pixel-wise uncertainties at different scales, which uses 3 and 6 orders of magnitude less likelihood evaluations than other UQ methods like length of the credible intervals and Monte Carlo posterior sampling, respectively. We demonstrate our method by reconstructing RI images in a simulated setting and carrying out fast and scalable UQ, which we validate with MCMC sampling. Our method shows an improved image quality and more meaningful uncertainties than the benchmark method based on a sparsity-promoting prior. QuantifAI’s source code is available from https://github.com/astro-informatics/QuantifAI.

Solving Bayesian inverse problems with expensive likelihoods using constrained Gaussian processes and active learning

Solving inverse problems using Bayesian methods can become prohibitively expensive when likelihood evaluations involve complex and large scale numerical models. A common approach to circumvent this issue is to approximate the forward model or the likelihood function with a surrogate model. But also there, due to limited computational resources, only a few training points are available in many practically relevant cases. Thus, it can be advantageous to model the additional uncertainties of the surrogate in order to incorporate the epistemic uncertainty due to limited data. In this paper, we develop a novel approach to approximate the log likelihood by a constrained Gaussian process based on prior knowledge about its boundedness. This improves the accuracy of the surrogate approximation without increasing the number of training samples. Additionally, we introduce a formulation to integrate the epistemic uncertainty due to limited training points into the posterior density approximation. This is combined with a state of the art active learning strategy for selecting training points, which allows to approximate posterior densities in higher dimensions very efficiently. We demonstrate the fast convergence of our approach for a benchmark problem and infer a random field that is discretized by 30 parameters using only about 1000 model evaluations. In a practically relevant example, the parameters of a reduced lung model are calibrated based on flow observations over time and voltage measurements from a coupled electrical impedance tomography simulation.

Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows.

Pulsar timing arrays perform Bayesian posterior inference with expensive Markov chain MonteCarlo (MCMC) methods. Given a dataset of ∼10-100 pulsars and O(10^{3}) timing residuals each, producing a posterior distribution for the stochastic gravitational wave background (SGWB) can take days to a week. The computational bottleneck arises because the likelihood evaluation required for MCMC is extremely costly when considering the dimensionality of the search space. Fortunately, generating simulated data is fast, so modern simulation-based inference techniques can be brought to bear on the problem. In this Letter, we demonstrate how conditional normalizing flows trained on simulated data can be used for extremely fast and accurate estimation of the SGWB posteriors, reducing the sampling time from weeks to a matter of seconds.

Impact of individual and work factors on the heat-related consequences among power plant workers in a hot region

ABSTRACT Prolonged exposure to hot environments increases the probability of heat load that may cause occupational heat strain to workers. This study investigates the impact of individual and work-related factors on the heat-related consequences among power plant workers in a hot region. This cross-sectional study was conducted in 2020. The collecting data tool was a validated self-reported 26 item questionnaire and completed in 534 individuals. We used logistic regression, Adjusted odds ratio (AOR) and maximum likelihood evaluations for data analysis. The findings indicated that age, work environment, physical work demands and drinking fluids during work hours are significant with heat exposure perception, heat-related symptoms, and work performance (P-value<0.05). Further, the male workers aged 40-49 are more prone to heat-related symptoms (AOR 1.34, 95% CI 1.18-2.13). The importance of addressing heat stress in occupational settings is necessary and informing strategies to help workers adapt to heat in hot workplaces.

Spectral neural approximations for models of transcriptional dynamics

The advent of high-throughput transcriptomics provides an opportunity to advance mechanistic understanding of transcriptional processes and their connections to cellular function at an unprecedented, genome-wide scale. These transcriptional systems, which involve discrete stochastic events, are naturally modeled using chemical master equations (CMEs), which can be solved for probability distributions to fit biophysical rates that govern system dynamics. While CME models have been used as standards in fluorescence transcriptomics for decades to analyze single-species RNA distributions, there are often no closed-form solutions to CMEs that model multiple species, such as nascent and mature RNA transcript counts. This has prevented the application of standard likelihood-based statistical methods for analyzing high-throughput, multi-species transcriptomic datasets using biophysical models. Inspired by recent work in machine learning to learn solutions to complex dynamical systems, we leverage neural networks and statistical understanding of system distributions to produce accurate approximations to a steady-state bivariate distribution for a model of the RNA life cycle that includes nascent and mature molecules. The steady-state distribution to this simple model has no closed-form solution and requires intensive numerical solving techniques: our approach reduces likelihood evaluation time by several orders of magnitude. We demonstrate two approaches, whereby solutions are approximated by 1) learning the weights of kernel distributions with constrained parameters or 2) learning both weights and scaling factors for parameters of kernel distributions. We show that our strategies, denoted by kernel weight regression and parameter-scaled kernel weight regression, respectively, enable broad exploration of parameter space and can be used in existing likelihood frameworks to infer transcriptional burst sizes, RNA splicing rates, and mRNA degradation rates from experimental transcriptomic data.

Bidirectional machine learning-assisted sensitivity-based stochastic searching approach for groundwater DNAPL source characterization.

In this study, we designed a machine learning-based parallel global searching method using the Bayesian inversion framework for efficient identification of dense non-aqueous phase liquid (DNAPL) source characteristics and contaminant transport parameters in groundwater. Swarm intelligence organized hybrid-kernel extreme learning machine (SIO-HKELM) was proposed to approximate the forward and inverse input-output correlation with a high accuracy using the DNAPL transport numerical simulation model. An adaptive inverse-HKELM was established for preliminary estimation of the source characteristics and contaminant transport parameters to correct prior information and generate high-quality initial starting points of parallel searching. A local accurate forward-HKELM surrogate of the numerical model was embedded in the searching system for avoiding repetitive CPU-demanding likelihood evaluations. A sensitivity-based Metropolis criterion (MC), incorporating the dynamic particle swarm optimization (SD-PSO) algorithm, was developed for improving the search ergodicity and realizing precise inversion of all the unknown variables with drastic variations in sensitivity to the likelihood function. Results showed that the generalization capability and robustness of SIO-HKELM were superior to those of the traditional machine learning methods, including KELM and support vector regression (SVR), and it sufficiently approximated the forward and inverse input-output mapping of the numerical model with testing determination coefficients of 0.9944 and 0.6440, respectively. With high-quality prior information and initial starting points generated by the adaptive inverse-HKELM feed approach, the uncertainty in the inversion outputs was reduced, and the searching process rapidly converged to reasonable posterior distributions in around 60 iterations. Compared with the widely used multichain Markov chain Monte Carlo (MCMC) approach, the parallel searching lines generated by SD-PSO-MC adequately covered the searching space, and the "equifinality" effect was more effectively restrained by reducing the relative errors of all the point estimations to less than 8%. Therefore, the real source information reflected by the statistical characteristics of the SD-PSO-MC inversion outputs was more precise than that obtained using the multichain MCMC approach.

Improving difficult inference tasks and likelihood evaluations with neural networks

Improving difficult inference tasks and likelihood evaluations with neural networks

Kernel-Based Particle Filtering for Scalable Inference in Partially Observed Boolean Dynamical Systems

This paper addresses the inference challenges associated with a class of hidden Markov models with binary state variables, known as partially observed Boolean dynamical systems (POBDS). POBDS have demonstrated remarkable success in modeling the ON and OFF dynamics of genes, microbes, and bacteria in systems biology, as well as in network security to represent the propagation of attacks among interconnected elements. Despite existing optimal and approximate inference solutions for POBDS, scalability remains a significant issue due to the computational cost associated with likelihood evaluations and the exploration of extensive parameter spaces. To overcome these challenges, this paper proposes a kernel-based particle filtering approach for large-scale inference of POBDS. Our method employs a Gaussian process (GP) to efficiently represent the expensive-to-evaluate likelihood function across the parameter space. The likelihood evaluation is approximated using a particle filtering technique, enabling the GP to account for various sources of uncertainty, including limited likelihood evaluations. Leveraging the GP’s predictive behavior, a Bayesian optimization strategy is derived for effectively seeking parameters yielding the highest likelihood, minimizing the overall computational burden while balancing exploration and exploitation. The proposed method’s performance is demonstrated using two biological networks: the mammalian cell-cycle network and the T-cell large granular lymphocyte leukemia network.

Accelerating Markov Chain Monte Carlo sampling with diffusion models

Global fits of physics models require efficient methods for exploring high-dimensional and/or multimodal posterior functions. We introduce a novel method for accelerating Markov Chain Monte Carlo (MCMC) sampling by pairing a Metropolis-Hastings algorithm with a diffusion model that can draw global samples with the aim of approximating the posterior. We briefly review diffusion models in the context of image synthesis before providing a streamlined diffusion model tailored towards low-dimensional data arrays. We then present our adapted Metropolis-Hastings algorithm which combines local proposals with global proposals taken from a diffusion model that is regularly trained on the samples produced during the MCMC run. Our approach leads to a significant reduction in the number of likelihood evaluations required to obtain an accurate representation of the Bayesian posterior across several analytic functions, as well as for a physical example based on a global analysis of parton distribution functions. Our method is extensible to other MCMC techniques, and we briefly compare our method to similar approaches based on normalizing flows. A code implementation can be found at https://github.com/NickHunt-Smith/MCMC-diffusion.

Accelerating galaxy dynamical modeling using a neural network for joint lensing and kinematic analyses

Strong gravitational lensing is a powerful tool to provide constraints on galaxy mass distributions and cosmological parameters, such as the Hubble constant, H0. Nevertheless, inference of such parameters from images of lensing systems is not trivial as parameter degeneracies can limit the precision in the measured lens mass and cosmological results. External information on the mass of the lens, in the form of kinematic measurements, is needed to ensure a precise and unbiased inference. Traditionally, such kinematic information has been included in the inference after the image modeling, using spherical Jeans approximations to match the measured velocity dispersion integrated within an aperture. However, as spatially resolved kinematic measurements become available via IFU data, more sophisticated dynamical modeling is necessary. Such kinematic modeling is expensive, and constitutes a computational bottleneck that we aim to overcome with our Stellar Kinematics Neural Network (SKiNN). SKiNN emulates axisymmetric modeling using a neural network, quickly synthesizing from a given mass model a kinematic map that can be compared to the observations to evaluate a likelihood. With a joint lensing plus kinematic framework, this likelihood constrains the mass model at the same time as the imaging data. We show that SKiNN’s emulation of a kinematic map is accurate to a considerably better precision than can be measured (better than 1% in almost all cases). Using SKiNN speeds up the likelihood evaluation by a factor of ~200. This speedup makes dynamical modeling economical, and enables lens modelers to make effective use of modern data quality in the JWST era.

Comprehensive safety risk evaluation of fireworks production enterprises using the frequency-based ANP and BPNN

The fireworks industry has long struggled with the problem of safety. Scientific, reasonable, and operable evaluation models are prerequisites of reducing risk. Based on the data from over 100 fireworks production safety accidents in China from 2010 to 2022, two evaluation models were established from the perspective of safety risk definition. Firstly, a weight calculation derivative method, the frequency-based analytic network process (ANP), was proposed creatively. This method optimized the importance ranking index calculation process in the ANP by considering the causal frequency of risk factors in the historical accident samples, thus determining how much each indicator affects the likelihood of accidents. Secondly, utilizing the historical accident samples as the dataset, a back propagation neural network (BPNN) model was developed to extract the mathematical relationship between each risk factor and the severity of accident consequence. Finally, the frequency-based ANP and BPNN models were combined to determine the safety risk level of the fireworks production enterprises. Meanwhile, the safety evaluation research samples were used as the comparison set for empirical study with historical accident samples, involving 100 fireworks production enterprises in China evaluated from 2017 to 2020. The significance result of zero shows that there is a statistically significant difference between the likelihood evaluation results of the accident and non-accident companies. Additionally, the severity evaluation model exhibits an excellent result, revealing a classification accuracy of 98.21 %, a mean square error of 8.97 × 10−4, a percent bias of 1.24 %, and a correlation coefficient and Nash-Sutcliffe efficiency coefficient both of 0.96. The frequency-based ANP and BPNN models integrate self-learning, self-adaptive, and fuzzy information processing, obtaining more accurate and objective evaluation results. This work provides a new strategy for the promotion and application of artificial intelligence in the field of safety risk evaluation, thus offering real-time safety risk evaluation and decision support of the safety management for the enterprises.

Analytical advances alleviate model misspecification in non-Brownian multivariate comparative methods.

Adams and Collyer argue that contemporary multivariate (Gaussian) phylogenetic comparative methods are prone to favouring more complex models of evolution and sometimes rotation invariance can be an issue. Here we dissect the concept of rotation invariance and point out that, depending on the understanding, this can be an issue with any method that relies on numerical instead of analytical estimation approaches. We relate this to the ongoing discussion concerning phylogenetic principal component analysis. Contrary to what Adams and Collyer found, we do not observe a bias against the simpler Brownian motion process in simulations when we use the new, improved, likelihood evaluation algorithm employed by mvSLOUCH, which allows for studying much larger phylogenies and more complex model setups.

Fast likelihood evaluation using meshfree approximations for reconstructing compact binary sources

Fast likelihood evaluation using meshfree approximations for reconstructing compact binary sources

Partition function approach to non-Gaussian likelihoods: physically motivated convergence criteria for Markov chains

ABSTRACT Non-Gaussian distributions in cosmology are commonly evaluated with Monte Carlo Markov chain methods, as the Fisher matrix formalism is restricted to the Gaussian case. The Metropolis–Hastings algorithm will provide samples from the posterior distribution after a burn-in period, and the corresponding convergence is usually quantified with the Gelman–Rubin criterion. In this paper, we investigate the convergence of the Metropolis–Hastings algorithm by drawing analogies to statistical Hamiltonian systems in thermal equilibrium for which a canonical partition sum exists. Specifically, we quantify virialization, equipartition, and thermalization of Hamiltonian Monte Carlo Markov chains for a toy model and for the likelihood evaluation for a simple dark energy model constructed from supernova data. We follow the convergence of these criteria to the values expected in thermal equilibrium, in comparison to the Gelman–Rubin criterion. We find that there is a much larger class of physically motivated convergence criteria with clearly defined target values indicating convergence. As a numerical tool, we employ physics-informed neural networks for speeding up the sampling process.

Bayesian model calibration for diblock copolymer thin film self-assembly using power spectrum of microscopy data and machine learning surrogate

Identifying parameters of computational models from experimental data, or model calibration, is fundamental for assessing and improving the predictability and reliability of computer simulations. In this work, we propose a method for Bayesian calibration of models that predict morphological patterns of diblock copolymer (Di-BCP) thin film self-assembly while accounting for various sources of uncertainties in pattern formation and data acquisition. This method extracts the azimuthally-averaged power spectrum (AAPS) of the top-down microscopy characterization of Di-BCP thin film patterns as summary statistics for Bayesian inference of model parameters via the pseudo-marginal method. We derive the analytical and approximate form of a conditional likelihood for the AAPS of image data. We demonstrate that AAPS-based image data reduction retains the mutual information, particularly on important length scales, between image data and model parameters while being relatively agnostic to the aleatoric uncertainties associated with the random long-range disorder of Di-BCP patterns. Additionally, we propose a phase-informed prior distribution for Bayesian model calibration. Furthermore, reducing image data to AAPS enables us to efficiently build surrogate models to accelerate the proposed Bayesian model calibration procedure. We present the formulation and training of two multi-layer perceptrons for approximating the parameter-to-spectrum map, which enables fast integrated likelihood evaluations. We validate the proposed Bayesian model calibration method through numerical examples, for which the neural network surrogate delivers a fivefold reduction of the number of model simulations performed for a single calibration task.

Nautilus: boosting Bayesian importance nested sampling with deep learning

ABSTRACT We introduce a novel approach to boost the efficiency of the importance nested sampling (INS) technique for Bayesian posterior and evidence estimation using deep learning. Unlike rejection-based sampling methods such as vanilla nested sampling (NS) or Markov chain Monte Carlo (MCMC) algorithms, importance sampling techniques can use all likelihood evaluations for posterior and evidence estimation. However, for efficient importance sampling, one needs proposal distributions that closely mimic the posterior distributions. We show how to combine INS with deep learning via neural network regression to accomplish this task. We also introduce nautilus, a reference open-source python implementation of this technique for Bayesian posterior and evidence estimation. We compare nautilus against popular NS and MCMC packages, including emcee, dynesty, ultranest, and pocomc, on a variety of challenging synthetic problems and real-world applications in exoplanet detection, galaxy SED fitting and cosmology. In all applications, the sampling efficiency of nautilus is substantially higher than that of all other samplers, often by more than an order of magnitude. Simultaneously, nautilus delivers highly accurate results and needs fewer likelihood evaluations than all other samplers tested. We also show that nautilus has good scaling with the dimensionality of the likelihood and is easily parallelizable to many CPUs.

Conditional density estimation using population Monte Carlo based approximate Bayesian computation

Most statistical methods require likelihood evaluation to draw a statistical inference. However, in some situations, likelihood evaluation becomes difficult analytically or computationally. Different likelihood-free methods are available that eliminate the need to compute the likelihood function. Approximate Bayesian Computation (ABC) is a framework that implements likelihood-free inference and replaces the likelihood evaluation with simulations by using forward modeling. The goal of ABC methods is to approximate the posterior distribution. However, posterior approximation via ABC methods is still considerably expensive for high dimensions. ABC requires many simulations that become computationally infeasible for complex models. Here, a technique is proposed that combines a somewhat more efficient form of ABC (Population Monte Carlo, PMC) with a Conditional Density Estimation (CDE) approach. The proposed framework provides an estimation of the posterior distribution which is referred to as PMC-CDE. A simulation study is performed that provides empirical evidence to show the efficiency of PMC-CDE in terms of integrated squared error loss. Furthermore, real-life datasets manifest the application of the proposed method.

Importance nested sampling with normalising flows

Abstract We present an improved version of the nested sampling algorithm nessai in which the core algorithm is modified to use importance weights. In the modified algorithm, samples are drawn from a mixture of normalising flows and the requirement for samples to be independently and identically distributed (i.i.d.) according to the prior is relaxed. Furthermore, it allows for samples to be added in any order, independently of a likelihood constraint, and for the evidence to be updated with batches of samples. We call the modified algorithm i-nessai. We first validate i-nessai using analytic likelihoods with known Bayesian evidences and show that the evidence estimates are unbiased in up to 32 dimensions. We compare i-nessai to standard nessai for the analytic likelihoods and the Rosenbrock likelihood, the results show that i-nessai is consistent with nessai whilst producing more precise evidence estimates. We then test i-nessai on 64 simulated gravitational-wave signals from binary black hole coalescence and show that it produces unbiased estimates of the parameters. We compare our results to those obtained using standard nessai and dynesty and find that i-nessai requires 2.68 and 13.3 times fewer likelihood evaluations to converge, respectively. We also test i-nessai of an 80 s simulated binary neutron star signal using a reduced-order-quadrature basis and find that, on average, it converges in 24 min, whilst only requiring 1.01 × 10 6 likelihood evaluations compared to 1.42 × 10 6 for nessai and 4.30 × 10 7 for dynesty. These results demonstrate that i-nessai is consistent with nessai and dynesty whilst also being more efficient.