Approximate Bayesian Computation Methods Research Articles

A likelihood-free approach towards Bayesian modeling of degradation growths using mixed-effects regression

Mixed-effects regression models are widely applicable for predicting degradation growths in structural components. The Bayesian inference method is used to estimate the regression parameters when the degradation data are confounded by measurement and parameter uncertainties. The Gibbs sampler (GS), commonly used for this purpose, works when the regression errors are assumed as normally distributed that allows for the analytical formulation of the likelihood function. In case of a more general regression error distribution (e.g., mixture models), the likelihood becomes analytically intractable and computationally expensive to a degree that any likelihood-based Bayesian inference scheme (e.g., GS, Metropolis-Hastings sampler) can no longer be used for solving a practical problem.This paper proposes a practical likelihood-free approach for parameter estimation based on the approximate Bayesian computation (ABC) method. The ABC method implements forward simulation coupled with a rejection mechanism to sample from a target posterior distribution thereby eliminating the need to evaluate the likelihood function. The advantages of the proposed method are illustrated by analyzing degradation data obtained from a Canadian nuclear power plant.

Reliability Correction is Key for Robust Kepler Occurrence Rates

Abstract The Kepler DR25 planet candidate catalog was produced using an automated method of planet candidate identification based on various tests. These tests were tuned to obtain a reasonable but arbitrary balance between catalog completeness and reliability. We produce new catalogs with differing balances of completeness and reliability by varying these tests, and study the impact of these alternative catalogs on occurrence rates. We find that if there is no correction for reliability, different catalogs give statistically inconsistent occurrence rates, while if we correct for both completeness and reliability, we get statistically consistent occurrence rates. This is a strong indication that correction for completeness and reliability is critical for the accurate computation of occurrence rates. Additionally, we find that this result is the same whether using Bayesian Poisson-likelihood Markov Chain Monte Carlo or Approximate Bayesian Computation methods. We also examine the use of a Robovetter disposition score cut as an alternative to reliability correction, and find that while a score cut does increase the reliability of the catalog, it is not as accurate as performing a full reliability correction. We get the same result when performing a reliability correction with and without a score cut. Therefore removing low-score planets removes data without providing any advantage, and should be avoided when possible. We make our alternative catalogs publicly available, and propose that these should be used as a test of occurrence rate methods, with the requirement that a method should provide statistically consistent occurrence rates for all these catalogs.

The origin of the Afro-Mediterranean cypresses: Evidence from genetic analysis

Only few woody species of North Africa have been the subject of biogeographical studies, despite the importance of this region, in the Mediterranean area, for the presence of hotspots of endemism and biodiversity, and of refugia. Here, we present research on the Afro-Mediterranean Cupressus spp., which have a discontinuous and fragmented distribution, resulting from population dynamics experienced during their life history. We aim to retrace their demographic history, by combining Bayesian clustering and approximate Bayesian computation (ABC) methods. The fragmentation of the cypress gene pool dates back to Pliocene, when the Mediterranean and the North African lineages separated from a common ancestor. The separation of C. atlantica and C. dupreziana dates back to the Pleistocene, when the history of C. dupreziana as relic species started. Finally, C. sempervirens var. numidica is likely endemic of Tunisia, belonging to the cypress gene pool of the central Mediterranean since the Pleistocene.

Bayesian Inference of a Parametric Random Spheroid from its Orthogonal Projections

The paper focuses on a new method for the inference of a parametric random spheroid from the observations of its 2D orthogonal projections. Such a stereological problem is well-known from the literature when the projections come from only one deterministic spheroid. Nevertheless, when the spheroid is random itself, the estimation of its distribution is not straightforward. From a theoretical viewpoint, it is shown that the semi-axes of the spheroid and the ones of the projected ellipses are linked through a random polynomial of degree two which admits two real random positive roots. The likelihood can be formulated in terms of the coefficients of the random polynomial, but is not analytically tractable. Assuming that the random spheroid is parameterized by a set of parameters θreal, an approximation of the maximum a posteriori is used to estimate θreal. The estimator is based on the so-called approximate Bayesian computation method and a kernel density technique. As an illustration, the case of a spheroids population, whose semi-major axis follows a gamma distribution and the flattening coefficient a truncated normal distribution, is studied. The numerical results demonstrate that the bias of the estimator is very low, with a reasonable variance, both for the first and the second order moments of the semi-axes. The proposed method enables to recover some 3D morphological characteristics of a population of independent and identically distributed spheroids thanks to the only observations of its projected ellipses.

Novel Bayesian framework for calibration of spatially distributed physical-based landslide prediction models.

Abstract This study presents a novel Bayesian framework for statistical calibration of spatially distributed physical-based landslide prediction models. The calibration process is formulated in a statistical setting with the model parameters simulated as spatially variable with random fields and the model calibration defined within the Bayesian framework. The implementation of such calibration process is challenging due to large numbers of calibration parameters and high-dimensional likelihood functions, which are central in establishing a relation between observations and the corresponding model predictions. The former challenge was resolved by reformulating the Bayesian updating problem as an equivalent reliability problem and solving it with efficient reliability methods. The latter challenge was resolved by developing novel lower-dimensional approximate likelihood formulations, suitable for the interpretation of landslide initiation zones, based on the Approximate Bayesian Computation method. The novelties of the proposed approach stem from describing landslide model parameters as spatially variable, development of a statistical framework to calibrate landslide prediction models, and introduction of approximate likelihood formulations.

Improved approximate Bayesian computation methods via empirical likelihood

Improved approximate Bayesian computation methods via empirical likelihood

Approximate Bayesian computation (ABC) method for estimating parameters of the gamma process using noisy data

The stochastic gamma process (GP) has emerged as a versatile model of degradation processes affecting the performance of engineering structures and components. The GP model captures the temporal uncertainty in the degradation process, whereas the Bayesian approach is used to account for uncertainty in the model parameters. Although the conceptual approach to GP parameter estimation is straightforward, its practical implementation is quite challenging. The reason is that degradation data required for parameter estimation are invariably contaminated by measurement noise, which turns the likelihood function into a high-dimensional multivariate integral. To circumvent this difficulty, the paper presents a novel and practical approach based on the approximate Bayesian computation (ABC) technique. The ABC method is a simulation-based approach that does not require an explicit formulation of the likelihood function. Instead, ABC uses an “accept-reject” mechanism to perform the posterior computation. To reduce the rejection rate, the Markov chain Monte Carlo (MCMC) approach is used within the ABC algorithm. The proposed ABC-MCMC algorithm is shown to be computationally more efficient than the traditional likelihood-based method without compromising the numerical accuracy. A case study involving the modelling of corrosion in nuclear piping systems is presented, which confirms the practical usefulness of the proposed method.

Using approximate Bayesian inference for a "steps and turns" continuous-time random walk observed at regular time intervals.

The study of animal movement is challenging because movement is a process modulated by many factors acting at different spatial and temporal scales. In order to describe and analyse animal movement, several models have been proposed which differ primarily in the temporal conceptualization, namely continuous and discrete time formulations. Naturally, animal movement occurs in continuous time but we tend to observe it at fixed time intervals. To account for the temporal mismatch between observations and movement decisions, we used a state-space model where movement decisions (steps and turns) are made in continuous time. That is, at any time there is a non-zero probability of making a change in movement direction. The movement process is then observed at regular time intervals. As the likelihood function of this state-space model turned out to be intractable yet simulating data is straightforward, we conduct inference using different variations of Approximate Bayesian Computation (ABC). We explore the applicability of this approach as a function of the discrepancy between the temporal scale of the observations and that of the movement process in a simulation study. Simulation results suggest that the model parameters can be recovered if the observation time scale is moderately close to the average time between changes in movement direction. Good estimates were obtained when the scale of observation was up to five times that of the scale of changes in direction. We demonstrate the application of this model to a trajectory of a sheep that was reconstructed in high resolution using information from magnetometer and GPS devices. The state-space model used here allowed us to connect the scales of the observations and movement decisions in an intuitive and easy to interpret way. Our findings underscore the idea that the time scale at which animal movement decisions are made needs to be considered when designing data collection protocols. In principle, ABC methods allow to make inferences about movement processes defined in continuous time but in terms of easily interpreted steps and turns.

Adaptive Approximate Bayesian Computation Tolerance Selection

Approximate Bayesian Computation (ABC) methods are increasingly used for inference in situations in which the likelihood function is either computationally costly or intractable to evaluate. Extensions of the basic ABC rejection algorithm have improved the computational efficiency of the procedure and broadened its applicability. The ABC – Population Monte Carlo (ABC-PMC) approach has become a popular choice for approximate sampling from the posterior. ABC-PMC is a sequential sampler with an iteratively decreasing value of the tolerance, which specifies how close the simulated data need to be to the real data for acceptance. We propose a method for adaptively selecting a sequence of tolerances that improves the computational efficiency of the algorithm over other common techniques. In addition we define a stopping rule as a by-product of the adaptation procedure, which assists in automating termination of sampling. The proposed automatic ABC-PMC algorithm can be easily implemented and we present several examples demonstrating its benefits in terms of computational efficiency.

Reverse engineering directed gene regulatory networks from transcriptomics and proteomics data of biomining bacterial communities with approximate Bayesian computation and steady-state signalling simulations

BackgroundNetwork inference is an important aim of systems biology. It enables the transformation of OMICs datasets into biological knowledge. It consists of reverse engineering gene regulatory networks from OMICs data, such as RNAseq or mass spectrometry-based proteomics data, through computational methods. This approach allows to identify signalling pathways involved in specific biological functions. The ability to infer causality in gene regulatory networks, in addition to correlation, is crucial for several modelling approaches and allows targeted control in biotechnology applications.MethodsWe performed simulations according to the approximate Bayesian computation method, where the core model consisted of a steady-state simulation algorithm used to study gene regulatory networks in systems for which a limited level of details is available. The simulations outcome was compared to experimentally measured transcriptomics and proteomics data through approximate Bayesian computation.ResultsThe structure of small gene regulatory networks responsible for the regulation of biological functions involved in biomining were inferred from multi OMICs data of mixed bacterial cultures. Several causal inter- and intraspecies interactions were inferred between genes coding for proteins involved in the biomining process, such as heavy metal transport, DNA damage, replication and repair, and membrane biogenesis. The method also provided indications for the role of several uncharacterized proteins by the inferred connection in their network context.ConclusionsThe combination of fast algorithms with high-performance computing allowed the simulation of a multitude of gene regulatory networks and their comparison to experimentally measured OMICs data through approximate Bayesian computation, enabling the probabilistic inference of causality in gene regulatory networks of a multispecies bacterial system involved in biomining without need of single-cell or multiple perturbation experiments. This information can be used to influence biological functions and control specific processes in biotechnology applications.

Using approximate Bayesian computation for estimating parameters in the cue-based retrieval model of sentence processing

A commonly used approach to parameter estimation in computational models is the so-called grid search procedure: the entire parameter space is searched in small steps to determine the parameter value that provides the best fit to the observed data. This approach has several disadvantages: first, it can be computationally very expensive; second, one optimal point value of the parameter is reported as the best fit value; we cannot quantify our uncertainty about the parameter estimate. In the main journal article that this methods article accompanies (Jäger et al., 2020, Interference patterns in subject-verb agreement and reflexives revisited: A large-sample study, Journal of Memory and Language), we carried out parameter estimation using Approximate Bayesian Computation (ABC), which is a Bayesian approach that allows us to quantify our uncertainty about the parameter's values given data. This customization has the further advantage that it allows us to generate both prior and posterior predictive distributions of reading times from the cue-based retrieval model of Lewis and Vasishth, 2005.•Instead of the conventional method of using grid search, we use Approximate Bayesian Computation (ABC) for parameter estimation in the [4] model.•The ABC method of parameter estimation has the advantage that the uncertainty of the parameter can be quantified.

Genetic susceptibility to severe childhood asthma and rhinovirus-C maintained by balancing selection in humans for 150 000years.

Selective pressures imposed by pathogens have varied among human populations throughout their evolution, leading to marked inter-population differences at some genes mediating susceptibility to infectious and immune-related diseases. Here, we investigated the evolutionary history of a common polymorphism resulting in a Y529 versus C529 change in the cadherin related family member 3 (CDHR3) receptor which underlies variable susceptibility to rhinovirus-C infection and is associated with severe childhood asthma. The protective variant is the derived allele and is found at high frequency worldwide (69-95%). We detected genome-wide significant signatures of natural selection consistent with a rapid increase of the haplotypes carrying the allele, suggesting that non-neutral processes have acted on this locus across all human populations. However, the allele has not fixed in any population despite multiple lines of evidence suggesting that the mutation predates human migrations out of Africa. Using an approximate Bayesian computation method, we estimate the age of the mutation while explicitly accounting for past demography and positive or frequency-dependent balancing selection. Our analyses indicate a single emergence of the mutation in anatomically modern humans ~150 000years ago and indicate that balancing selection has maintained the beneficial allele at high equilibrium frequencies worldwide. Apart from the well-known cases of the MHC and ABO genes, this study provides the first evidence that negative frequency-dependent selection plausibly acted on a human disease susceptibility locus, a form of balancing selection compatible with typical transmission dynamics of communicable respiratory viruses that might exploit CDHR3.

Estimation of the within-herd transmission rates of bovine viral diarrhoea virus in extensively grazed beef cattle herds

Many research groups have developed mathematical models to simulate the dynamics of BVDV infections in cattle herds. However, most models use estimates for within-herd BVDV transmission rates that are either based on expert opinion or adapted from other dairy herd simulation models presented in the literature. There is currently little information on the transmission rates for BVDV in extensively grazed beef herds partly due to the logistical challenges in obtaining longitudinal data of individual animal’s seroconversion, and it may not be appropriate to apply the same transmission rates from intensive dairy herds given the significant differences in herd demographics and management. To address this knowledge gap, we measured BVDV antibody levels in 15 replacement heifers in each of 75 New Zealand beef breeding farms after their first calving and again at pregnancy scanning or weaning to check for seroconversion. Among these, data from 9 farms were used to infer the within-herd BVDV transmission rate with an approximate Bayesian computation method. The most probable within-herd BVDV transmission rate was estimated as 0.11 per persistently infected (PI) animal per day with a 95% highest posterior density interval between 0.03 and 0.34. This suggests that BVDV transmission in extensively grazed beef herds is generally slower than in dairy herds where the transmission rate has been estimated at 0.50 per PI animal per day and therefore may not be sufficient to ensure that all susceptible breeding females gain adequate immunity to the virus before the risk period of early pregnancy for generating new PI calves.

Identification of winter moth (Operophtera brumata) refugia in North Africa and the Italian Peninsula during the last glacial maximum.

Numerous studies have shown that the genetic diversity of species inhabiting temperate regions has been shaped by changes in their distributions during the Quaternary climatic oscillations. For some species, the genetic distinctness of isolated populations is maintained during secondary contact, while for others, admixture is frequently observed. For the winter moth (Operophtera brumata), an important defoliator of oak forests across Europe and northern Africa, we previously determined that contemporary populations correspond to genetic diversity obtained during the last glacial maximum (LGM) through the use of refugia in the Iberian and Aegean peninsulas, and to a lesser extent the Caucasus region. Missing from this sampling were populations from the Italian peninsula and from North Africa, both regions known to have played important roles as glacial refugia for other species. Therefore, we genotyped field‐collected winter moth individuals from southern Italy and northwestern Tunisia—the latter a region where severe oak forest defoliation by winter moth has recently been reported—using polymorphic microsatellite. We reconstructed the genetic relationships of these populations in comparison to moths previously sampled from the Iberian and Aegean peninsulas, the Caucasus region, and western Europe using genetic distance, Bayesian clustering, and approximate Bayesian computation (ABC) methods. Our results indicate that both the southern Italian and the Tunisian populations are genetically distinct from other sampled populations, and likely originated in their respective refugium during the LGM after diverging from a population that eventually settled in the Iberian refugium. These suggest that winter moth populations persisted in at least five Mediterranean LGM refugia. Finally, we comment that outbreaks by winter moth in northwestern Tunisia are not the result of a recent introduction of a nonnative species, but rather are most likely due to land use or environmental changes.

Spectral density-based and measure-preserving ABC for partially observed diffusion processes. An illustration on Hamiltonian SDEs

Approximate Bayesian computation (ABC) has become one of the major tools of likelihood-free statistical inference in complex mathematical models. Simultaneously, stochastic differential equations (SDEs) have developed to an established tool for modelling time-dependent, real-world phenomena with underlying random effects. When applying ABC to stochastic models, two major difficulties arise: First, the derivation of effective summary statistics and proper distances is particularly challenging, since simulations from the stochastic process under the same parameter configuration result in different trajectories. Second, exact simulation schemes to generate trajectories from the stochastic model are rarely available, requiring the derivation of suitable numerical methods for the synthetic data generation. To obtain summaries that are less sensitive to the intrinsic stochasticity of the model, we propose to build up the statistical method (e.g. the choice of the summary statistics) on the underlying structural properties of the model. Here, we focus on the existence of an invariant measure and we map the data to their estimated invariant density and invariant spectral density. Then, to ensure that these model properties are kept in the synthetic data generation, we adopt measure-preserving numerical splitting schemes. The derived property-based and measure-preserving ABC method is illustrated on the broad class of partially observed Hamiltonian type SDEs, both with simulated data and with real electroencephalography data. The derived summaries are particularly robust to the model simulation, and this fact, combined with the proposed reliable numerical scheme, yields accurate ABC inference. In contrast, the inference returned using standard numerical methods (Euler–Maruyama discretisation) fails. The proposed ingredients can be incorporated into any type of ABC algorithm and directly applied to all SDEs that are characterised by an invariant distribution and for which a measure-preserving numerical method can be derived.

Convolutional neural network for seismic impedance inversion

We have addressed the geophysical problem of obtaining an elastic model of the subsurface from recorded normal-incidence seismic data using convolutional neural networks (CNNs). We train the network on synthetic full-waveform seismograms generated using Kennett’s reflectivity method on earth models that were created under rock-physics modeling constraints. We use an approximate Bayesian computation method to estimate the posterior distribution corresponding to the CNN prediction and to quantify the uncertainty related to the predictions. In addition, we test the robustness of the network in predicting impedances of previously unobserved earth models when the input to the network consisted of seismograms generated using: (1) earth models with different spatial correlations (i.e. variograms), (2) earth models with different facies proportions, (3) earth models with different underlying rock-physics relations, and (4) source-wavelet phase and frequency different than in the training data. Results indicate that the predictions of the trained network are susceptible to facies proportions, the rock-physics model, and source-wavelet parameters used in the training data set. Finally, we apply CNN inversion on the Volve field data set from offshore Norway. P-wave impedance [Formula: see text] inverted for the Volve data set using CNN showed a strong correlation (82%) with the [Formula: see text] log at a well.

A review of approximate Bayesian computation methods via density estimation: Inference for simulator‐models

AbstractThis paper provides a review of approximate Bayesian computation (ABC) methods for carrying out Bayesian posterior inference, through the lens of density estimation. We describe several recent algorithms and make connection with traditional approaches. We show advantages and limitations of models based on parametric approaches and we then draw attention to developments in machine learning, which we believe have the potential to make ABC scalable to higher dimensions and may be the future direction for research in this area.This article is categorized under: Algorithms and Computational Methods < Algorithms Statistical and Graphical Methods of Data Analysis < Bayesian Methods and Theory Statistical Models < Simulation Models

Local dimension reduction of summary statistics for likelihood-free inference

Approximate Bayesian computation (ABC) and other likelihood-free inference methods have gained popularity in the last decade, as they allow rigorous statistical inference for complex models without analytically tractable likelihood functions. A key component for accurate inference with ABC is the choice of summary statistics, which summarize the information in the data, but at the same time should be low-dimensional for efficiency. Several dimension reduction techniques have been introduced to automatically construct informative and low-dimensional summaries from a possibly large pool of candidate summaries. Projection-based methods, which are based on learning simple functional relationships from the summaries to parameters, are widely used and usually perform well, but might fail when the assumptions behind the transformation are not satisfied. We introduce a localization strategy for any projection-based dimension reduction method, in which the transformation is estimated in the neighborhood of the observed data instead of the whole space. Localization strategies have been suggested before, but the performance of the transformed summaries outside the local neighborhood has not been guaranteed. In our localization approach the transformation is validated and optimized over validation datasets, ensuring reliable performance. We demonstrate the improvement in the estimation accuracy for localized versions of linear regression and partial least squares, for three different models of varying complexity.

Robust approximate Bayesian inference

We discuss an approach for deriving robust posterior distributions from M-estimating functions using Approximate Bayesian Computation (ABC) methods. In particular, we use M-estimating functions to construct suitable summary statistics in ABC algorithms. The theoretical properties of the robust posterior distributions are discussed. Special attention is given to the application of the method to linear mixed models. Simulation results and an application to a clinical study demonstrate the usefulness of the method. An R implementation is also provided in the robustBLME package.

Approximate Bayesian estimation of coevolutionary arms races.

Exaggerated traits involved in species interactions have long captivated the imagination of evolutionary biologists and inspired the durable metaphor of the coevolutionary arms race. Despite decades of research, however, we have only a handful of examples where reciprocal coevolutionary change has been rigorously established as the cause of trait exaggeration. Support for a coevolutionary mechanism remains elusive because we lack generally applicable tools for quantifying the intensity of coevolutionary selection. Here we develop an approximate Bayesian computation (ABC) approach for estimating the intensity of coevolutionary selection using population mean phenotypes of traits mediating interspecific interactions. Our approach relaxes important assumptions of a previous maximum likelihood approach by allowing gene flow among populations, variable abiotic environments, and strong coevolutionary selection. Using simulated data, we show that our ABC method accurately infers the strength of coevolutionary selection if reliable estimates are available for key background parameters and ten or more populations are sampled. Applying our approach to the putative arms race between the plant Camellia japonica and its seed predatory weevil, Curculio camelliae, provides support for a coevolutionary hypothesis but fails to preclude the possibility of unilateral evolution. Comparing independently estimated selection gradients acting on Camellia pericarp thickness with values simulated by our model reveals a correlation between predicted and observed selection gradients of 0.941. The strong agreement between predicted and observed selection gradients validates our method.