Bayesian Posteriors Research Articles

Synthetic Likelihood in Misspecified Models

Bayesian synthetic likelihood is a widely used approach for conducting Bayesian analysis in complex models where evaluation of the likelihood is infeasible but simulation from the assumed model is tractable. We analyze the behavior of the Bayesian synthetic likelihood posterior when the assumed model differs from the actual data generating process. We demonstrate that the Bayesian synthetic likelihood posterior can display a wide range of nonstandard behaviors depending on the level of model misspecification, including multimodality and asymptotic non-Gaussianity. Our results suggest that likelihood tempering, a common approach for robust Bayesian inference, fails for synthetic likelihood whilst recently proposed robust synthetic likelihood approaches can ameliorate this behavior and deliver reliable posterior inference under model misspecification. All results are illustrated using a simple running example. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Robustly encoding certainty in a metastable neural circuit model.

Localized persistent neural activity can encode delayed estimates of continuous variables. Common experiments require that subjects store and report the feature value (e.g., orientation) of a particular cue (e.g., oriented bar on a screen) after a delay. Visualizing recorded activity of neurons along their feature tuning reveals activity bumps whose centers wander stochastically, degrading the estimate over time. Bump position therefore represents the remembered estimate. Recent work suggests bump amplitude may represent estimate certainty reflecting a probabilistic population code for a Bayesian posterior. Idealized models of this type are fragile due to the fine tuning common to constructed continuum attractors in dynamical systems. Here we propose an alternative metastable model for robustly supporting multiple bump amplitudes by extending neural circuit models to include quantized nonlinearities. Asymptotic projections of circuit activity produce low-dimensional evolution equationsfor the amplitude and position of bump solutions in response to external stimuli and noise perturbations. Analysis of reduced equationsaccurately characterizes phase variance and the dynamics of amplitude transitions between stable discrete values. More salient cues generate bumps of higher amplitude which wander less, consistent with experiments showing certainty correlates with more accurate memories.

Automatic Parallel Tempering Markov Chain Monte Carlo with Nii-C

Due to the high dimensionality or multimodality that is common in modern astronomy, sampling Bayesian posteriors can be challenging. Several publicly available codes based on different sampling algorithms can solve these complex models, but the execution of the code is not always efficient or fast enough. The article introduces a C language general-purpose code, Nii-C, that implements a framework of automatic parallel tempering Markov Chain Monte Carlo. Automatic in this context means that the parameters that ensure an efficient parallel tempering process can be set by a control system during the inital stages of a sampling process. The autotuned parameters consist of two parts, the temperature ladders of all parallel tempering Markov Chains and the proposal distributions for all model parameters across all parallel tempering chains. In order to reduce dependencies in the compilation process and increase the code’s execution speed, Nii-C code is constructed entirely in the C language and parallelized using the message-passing interface protocol to optimize the efficiency of parallel sampling. These implementations facilitate rapid convergence in the sampling of high-dimensional and multimodal distributions, as well as the expeditious code execution time. The Nii-C code can be used in various research areas to trace complex distributions due to its high sampling efficiency and quick execution speed. This article presents a few applications of the Nii-C code.

An Empirical Model For Intrinsic Alignments: Insights From Cosmological Simulations

We extend current models of the halo occupation distribution (HOD) to include a flexible, empirical framework for the forward modeling of the intrinsic alignment (IA) of galaxies. A primary goal of this work is to produce mock galaxy catalogs for the purpose of validating existing models and methods for the mitigation of IA in weak lensing measurements. This technique can also be used to produce new, simulation-based predictions for IA and galaxy clustering. Our model is probabilistically formulated, and rests upon the assumption that the orientations of galaxies exhibit a correlation with their host dark matter (sub)halo orientation or with their position within the halo. We examine the necessary components and phenomenology of such a model by considering the alignments between (sub)halos in a cosmological dark matter only simulation. We then validate this model for a realistic galaxy population in a set of simulations in the Illustris-TNG suite. We create an HOD mock with Illustris-like correlations using our method, constraining the associated IA model parameters, with the between our model’s correlations and those of Illustris matching as closely as 1.4 and 1.1 for orientation–position and orientation–orientation correlation functions, respectively. By modeling the misalignment between galaxies and their host halo, we show that the 3-dimensional two-point position and orientation correlation functions of simulated (sub)halos and galaxies can be accurately reproduced from quasi-linear scales down to . We also find evidence for environmental influence on IA within a halo. Our publicly-available software provides a key component enabling efficient determination of Bayesian posteriors on IA model parameters using observational measurements of galaxy-orientation correlation functions in the highly nonlinear regime.

Bayesian F -statistic-based parameter estimation of continuous gravitational waves from known pulsars

We present a new method and implementation to obtain Bayesian posteriors on the amplitude parameters {h0,cosι,ψ,ϕ0} of continuous gravitational waves emitted by known pulsars. This approach leverages the well-established F-statistic framework and software. We further explore the benefits of employing a likelihood function that is analytically marginalized over ϕ0, which avoids signal degeneracy problems in the ψ−ϕ0 subspace. The method is tested on simulated signals, hardware injections in Advanced-LIGO detector data, and by performing percentile-percentile self-consistency tests of the posteriors via Monte-Carlo simulations. We apply our methodology to PSR J1526-2744, a recently discovered millisecond pulsar. We find no evidence for a signal and obtain a Bayesian upper limit h095% on the gravitational-wave amplitude of approximately 7×10−27, comparable with a previous frequentist upper limit. Published by the American Physical Society 2024

Linking Frequentist and Bayesian Change-Point Methods

We show that the two-stage minimum description length (MDL) criterion widely used to estimate linear change-point (CP) models corresponds to the marginal likelihood of a Bayesian model with a specific class of prior distributions. This allows results from the frequentist and Bayesian paradigms to be bridged together. Thanks to this link, one can rely on the consistency of the number and locations of the estimated CPs and the computational efficiency of frequentist methods, and obtain a probability of observing a CP at a given time, compute model posterior probabilities, and select or combine CP methods via Bayesian posteriors. Furthermore, we adapt several CP methods to take advantage of the MDL probabilistic representation. Based on simulated data, we show that the adapted CP methods can improve structural break detection compared to state-of-the-art approaches. Finally, we empirically illustrate the usefulness of combining CP detection methods when dealing with long time series and forecasting.

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models.In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. The practical aspects of the approach are demonstrated with inversion problems in structural dynamics.

Poster Session: Bayesian inference and adaptation in neural responses.

Many of the perceptual changes induced by adaptation can be captured in a Bayesian framework, in which adaptation renormalizes neural responses to weak or strong stimuli, consistent with correcting the likelihood to match the prior (Emery and Webster VSS 2020). We explored what this model implies about Bayesian estimation at the level of an individual neuron or channel. The prior corresponds to what the neuron expects to see, but the adaptation causes responses to match what it expects. If the decoder does not "know" the neuron's adaptation state (Series et al 2009), these adjustments effectively flatten priors for dimensions the neuron adapts to, preserving expectations only for information that it does not adapt to, such as how signals deviate from the predicted value. Bayesian-like estimates of these deviations can be computed within the neuron because the shape of the prior is instantiated in the neuron's contrast response function (CRF). For example, for a Gaussian prior the corresponding CRF is a cumulative Gaussian. Steeper CRFs bias responses more toward the mean. The biases approximate but do not equal the Bayesian posterior. If, like adaptation, the decoding does not account for this approximation, then the interpreted stimulus value is distorted relative to the Bayes estimate. This may underlie the distortions evident in some perceptual dimensions, such as nonlinearities in perceived contrast.

Comparative analyses of covariance matrix adaptation and iterative ensemble smoother on high-dimensional inverse problems in high-resolution groundwater modeling

Parameter estimation is an inverse problem which is crucial to reliable groundwater model predictions and management. Numerous techniques have been developed to address this challenging problem. This study aims to compare the performance of a stochastic optimization method, covariance matrix adaptation-evolution strategy (CMA-ES), and a data assimilation method, Levenberg-Marquardt based iterative ensemble smoother (ES-LM), on solving inverse problems in groundwater modeling. This study also presents a parallelization strategy to accelerate their implementation. The comparative analysis involves synthetic inverse problems and a real-world case study of calibrating a high-resolution groundwater model. Results from the synthetic problems suggest that both the CMA-ES and the ES-LM are able to achieve desirable data fitting, but the ES-LM generally exhibits greater efficiency than the CMA-ES. Also, with appropriate stopping criteria, both methods can perform well on propagating uncertainty of model predictions. However, both methods can underestimate the uncertainty of model parameters derived from Bayesian posterior. The underestimation is particularly pronounced by the ES-LM when using a small ensemble size, whereas uncertainty quantification by the CMA-ES is less affected by population size. Moreover, calibration of the highly parameterized groundwater model in Louisiana and Southwest Mississippi indicates that the ES-LM can outperform the CMA-ES for high-dimensional inverse problems regarding both data match and computational costs. With proper ensemble size, the two methods can produce comparable uncertainty of model parameters and predictions. This study contributes to the scientific understanding and efficient application of the CMA-ES and the ES-LM.

Morphological Parameters and Associated Uncertainties for 8 Million Galaxies in the Hyper Suprime-Cam Wide Survey

We use the Galaxy Morphology Posterior Estimation Network (GaMPEN) to estimate morphological parameters and associated uncertainties for ∼8 million galaxies in the Hyper Suprime-Cam Wide survey with z ≤ 0.75 and m ≤ 23. GaMPEN is a machine-learning framework that estimates Bayesian posteriors for a galaxy’s bulge-to-total light ratio (L B /L T ), effective radius (R e ), and flux (F). By first training on simulations of galaxies and then applying transfer learning using real data, we trained GaMPEN with <1% of our data set. This two-step process will be critical for applying machine-learning algorithms to future large imaging surveys, such as the Rubin-Legacy Survey of Space and Time, the Nancy Grace Roman Space Telescope, and Euclid. By comparing our results to those obtained using light profile fitting, we demonstrate that GaMPEN’s predicted posterior distributions are well calibrated (≲5% deviation) and accurate. This represents a significant improvement over light profile fitting algorithms, which underestimate uncertainties by as much as ∼60%. For an overlapping subsample, we also compare the derived morphological parameters with values in two external catalogs and find that the results agree within the limits of uncertainties predicted by GaMPEN. This step also permits us to define an empirical relationship between the Sérsic index and L B /L T that can be used to convert between these two parameters. The catalog presented here represents a significant improvement in size (∼10×), depth (∼4 mag), and uncertainty quantification over previous state-of-the-art bulge+disk decomposition catalogs. With this work, we also release GaMPEN’s source code and trained models, which can be adapted to other data sets.

Reputation and Partial Default

This paper presents a continuous-time reputation model of sovereign debt allowing for both varying levels of partial default and full default. In it, a government can be a nonstrategic commitment type or a strategic opportunistic type, and a government’s reputation is its equilibrium Bayesian posterior of being the commitment type. Our equilibrium has that for bond levels reachable by both types without defaulting, bigger partial defaults (or bigger haircuts for bond holders) imply higher interest rates for subsequent bond issuances, as in the data. (JEL D83, E32, E43, G12, H63)

Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inference.

We combine amortized neural posterior estimation with importance sampling for fast and accurate gravitational-wave inference. We first generate a rapid proposal for the Bayesian posterior using neural networks, and then attach importance weights based on the underlying likelihood and prior. This provides (1)a corrected posterior free from network inaccuracies, (2)a performance diagnostic (the sample efficiency) for assessing the proposal and identifying failure cases, and (3)an unbiased estimate of the Bayesian evidence. By establishing this independent verification and correction mechanism we address some of the most frequent criticisms against deep learning for scientific inference. We carry out a large study analyzing 42 binary black hole mergers observed by LIGO and Virgo with the SEOBNRv4PHM and IMRPhenomXPHM waveform models. This shows a median sample efficiency of ≈10% (2 orders of magnitude better than standard samplers) as well as a tenfold reduction in the statistical uncertainty in the log evidence. Given these advantages, we expect a significant impact on gravitational-wave inference, and for this approach to serve as a paradigm for harnessing deep learning methods in scientific applications.

Bayesian learning of parameterised quantum circuits

Currently available quantum computers suffer from constraints including hardware noise and a limited number of qubits. As such, variational quantum algorithms that utilise a classical optimiser in order to train a parameterised quantum circuit have drawn significant attention for near-term practical applications of quantum technology. In this work, we take a probabilistic point of view and reformulate the classical optimisation as an approximation of a Bayesian posterior. The posterior is induced by combining the cost function to be minimised with a prior distribution over the parameters of the quantum circuit. We describe a dimension reduction strategy based on a maximum a posteriori point estimate with a Laplace prior. Experiments on the Quantinuum H1-2 computer show that the resulting circuits are faster to execute and less noisy than the circuits trained without the dimension reduction strategy. We subsequently describe a posterior sampling strategy based on stochastic gradient Langevin dynamics. Numerical simulations on three different problems show that the strategy is capable of generating samples from the full posterior and avoiding local optima.

Probabilistic mass-mapping with neural score estimation

Context.Weak lensing mass-mapping is a useful tool for accessing the full distribution of dark matter on the sky, but because of intrinsic galaxy ellipticies, finite fields, and missing data, the recovery of dark matter maps constitutes a challenging, ill-posed inverse problemAims.We introduce a novel methodology that enables the efficient sampling of the high-dimensional Bayesian posterior of the weak lensing mass-mapping problem, relying on simulations to define a fully non-Gaussian prior. We aim to demonstrate the accuracy of the method to simulated fields, and then proceed to apply it to the mass reconstruction of the HST/ACS COSMOS field.Methods.The proposed methodology combines elements of Bayesian statistics, analytic theory, and a recent class of deep generative models based on neural score matching. This approach allows us to make full use of analytic cosmological theory to constrain the 2pt statistics of the solution, to understand any differences between this analytic prior and full simulations from cosmological simulations, and to obtain samples from the full Bayesian posterior of the problem for robust uncertainty quantification.Results.We demonstrate the method in theκTNG simulations and find that the posterior mean significantly outperfoms previous methods (Kaiser–Squires, Wiener filter, Sparsity priors) both for the root-mean-square error and in terms of the Pearson correlation. We further illustrate the interpretability of the recovered posterior by establishing a close correlation between posterior convergence values and the S/N of the clusters artificially introduced into a field. Finally, we apply the method to the reconstruction of the HST/ACS COSMOS field, which yields the highest-quality convergence map of this field to date.Conclusions.We find the proposed approach to be superior to previous algorithms, scalable, providing uncertainties, and using a fully non-Gaussian prior.

Two Methods for Data Assimilation of Wind Direction

Wind direction observations are instrumental weather records that hold promise for improving historical weather reanalyses and extending them deeper into the past. Two methods are developed for assimilating wind direction observations. The first uses a linear observation model with Gaussian additive error, and is thus amenable to use in standard EnKF and variational frameworks. The second is nonlinear and non-Gaussian, and is based on a two-step approach for sampling from the Bayesian posterior. Both methods are tested in the context of an idealized two-dimensional model of turbulent fluid dynamics. The nonlinear, non-Gaussian method assimilating only wind direction observations performs as well as an EnKF assimilating only pressure observations, whereas the first method based on the linear model provides no benefit when assimilating only wind direction observations. The method based on the linear model performs well when paired with other observations, e.g. of pressure, since it performs best when the forecast of wind direction is not far from correct.

Some Approximation Results for Bayesian Posteriors that Involve the Hurwitz–Lerch Zeta Distribution

Some Approximation Results for Bayesian Posteriors that Involve the Hurwitz–Lerch Zeta Distribution

Bayesian parameter estimation for targeted anisotropic gravitational-wave background

Extended sources of the stochastic gravitational backgrounds have been conventionally searched on the spherical harmonics bases. The analysis during the previous observing runs by the ground-based gravitational-wave detectors, such as LIGO and Virgo, have yielded the constraints on the angular power spectrum ${C}_{\ensuremath{\ell}}$, yet it lacks the capability of estimating other parameters such as a spectral index. In this paper, we introduce an alternative Bayesian formalism to search for such stochastic signals with a particular distribution of anisotropies on the sky. This approach provides a Bayesian posterior of model parameters and also enables selection tests among different signal models. While the conventional analysis fixes the highest angular scale a priori, here we show a more systematic and quantitative way to determine the cutoff scale based on a Bayes factor, which depends on the amplitude and the angular scale of observed signals. Also, we analyze the third observing runs of LIGO and Virgo for the population of millisecond pulsars and obtain the 95% constraints of the signal amplitude, $\ensuremath{\epsilon}&lt;2.7\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}8}$.

Statistical inference via conditional Bayesian posteriors in high-dimensional linear regression

We propose a new method under the Bayesian framework to perform valid inference for low dimensional parameters in high dimensional linear models under sparsity constraints. Our approach is to use surrogate Bayesian posteriors based on partial regression models to remove the effect of high dimensional nuisance variables. We name the final distribution we used to conduct inference “conditional Bayesian posterior” as it is a surrogate posterior constructed conditional on quasi posterior distributions of other parameters and does not admit a fully Bayesian interpretation. Unlike existing Bayesian regularization methods, our method can be used to quantify the estimation uncertainty for arbitrarily small signals and therefore does not require variable selection consistency to guarantee its validity. Theoretically, we show that the resulting Bayesian credible intervals achieve desired coverage probabilities in the frequentist sense. Methodologically, our proposed Bayesian framework can easily incorporate popular Bayesian regularization procedures such as those based on spike and slab priors and horseshoe priors to facilitate high accuracy estimation and inference. Numerically, our proposed method rectifies the uncertainty underestimation of Bayesian shrinkage approaches and has a comparable empirical performance with state-of-the-art frequentist methods based on extensive simulation studies and a real data analysis.

Pathwise concentration bounds for Bayesian beliefs

We show that Bayesian posteriors concentrate on the outcome distributions that approximately minimize the Kullback–Leibler divergence from the empirical distribution, uniformly over sample paths, even when the prior does not have full support. This generalizes Diaconis and Freedman's (1990) uniform convergence result to, e.g., priors that have finite support, are constrained by independence assumptions, or have a parametric form that cannot match some probability distributions. The concentration result lets us provide a rate of convergence for Berk's (1966) result on the limiting behavior of posterior beliefs when the prior is misspecified. We provide a bound on approximation errors in “anticipated‐utility” models, and extend our analysis to outcomes that are perceived to follow a Markov process.

SICRET: Supernova Ia Cosmology with truncated marginal neural Ratio EsTimation

ABSTRACT Type Ia supernovae (SNe Ia), standardizable candles that allow tracing the expansion history of the Universe, are instrumental in constraining cosmological parameters, particularly dark energy. State-of-the-art likelihood-based analyses scale poorly to future large data sets, are limited to simplified probabilistic descriptions, and must explicitly sample a high-dimensional latent posterior to infer the few parameters of interest, which makes them inefficient. Marginal likelihood-free inference, on the other hand, is based on forward simulations of data, and thus can fully account for complicated redshift uncertainties, contamination from non-SN Ia sources, selection effects, and a realistic instrumental model. All latent parameters, including instrumental and survey-related ones, per object and population-level properties, are implicitly marginalized, while the cosmological parameters of interest are inferred directly. As a proof of concept, we apply truncated marginal neural ratio estimation (TMNRE), a form of marginal likelihood-free inference, to bahamas, a Bayesian hierarchical model for salt parameters. We verify that TMNRE produces unbiased and precise posteriors for cosmological parameters from up to 100 000 SNe Ia. With minimal additional effort, we train a network to infer simultaneously the ${\sim}100\, 000$ latent parameters of the supernovae (e.g. absolute brightnesses). In addition, we describe and apply a procedure that utilizes local amortization of the inference to convert the approximate Bayesian posteriors into frequentist confidence regions with exact coverage. Finally, we discuss the planned improvements to the model that are enabled by using a likelihood-free inference framework, like selection effects and non-Ia contamination.