Bayesian Computation Techniques Research Articles

Dissociating visuo-spatial and verbal working memory: It\u2019s all in the features

Echoing many of the themes of the seminal work of Atkinson and Shiffrin (The Psychology of Learning and Motivation, 2; 89–195, 1968), this paper uses the feature model (Nairne, Memory & Cognition, 16, 343–352, 1988; Nairne, Memory & Cognition, 18; 251–269, 1990; Neath & Nairne, Psychonomic Bulletin & Review, 2; 429–441, 1995) to account for performance in working-memory tasks. The Brooks verbal and visuo-spatial matrix tasks were performed alone, with articulatory suppression, or with a spatial suppression task; the results produced the expected dissociation. We used approximate Bayesian computation techniques to fit the feature model to the data and showed that the similarity-based interference process implemented in the model accounted for the data patterns well. We then fit the model to data from Guérard and Tremblay (2008, Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 556–569); the latter study produced a double dissociation while calling upon more typical order reconstruction tasks. Again, the model performed well. The findings show that a double dissociation can be modelled without appealing to separate systems for verbal and visuo-spatial processing. The latter findings are significant as the feature model had not been used to model this type of dissociation before; importantly, this is also the first time the model is quantitatively fit to data. For the demonstration provided here, modularity was unnecessary if two assumptions were made: (1) the main difference between spatial and verbal working-memory tasks is the features that are encoded; (2) secondary tasks selectively interfere with primary tasks to the extent that both tasks involve similar features. It is argued that a feature-based view is more parsimonious (see Morey, 2018, Psychological Bulletin, 144, 849–883) and offers flexibility in accounting for multiple benchmark effects in the field.

Bayesian Analysis of Accumulated Damage Models in Lumber Reliability

ABSTRACTWood products that are subjected to sustained stress over a period of long duration may weaken, and this effect must be considered in models for the long-term reliability of lumber. The damage accumulation approach has been widely used for this purpose to set engineering standards. In this article, we revisit an accumulated damage model and propose a Bayesian framework for analysis. For parameter estimation and uncertainty quantification, we adopt approximation Bayesian computation (ABC) techniques to handle the complexities of the model. We demonstrate the effectiveness of our approach using both simulated and real data, and apply our fitted model to analyze long-term lumber reliability under a stochastic live loading scenario. Code is available at https://github.com/wongswk/abc-adm.

Calibration of forest 137Cs cycling model ”FoRothCs” via approximate Bayesian computation based on 6-year observations from plantation forests in Fukushima

Predicting the environmental fate of 137Cs in forest ecosystems along with the concentrations of 137Cs in tree parts are important for the managements of radioactively contaminated forests. In this study, we calibrate the Forest RothC and Cs model (FoRothCs), a forest ecosystem 137Cs dynamics model, using observational data obtained over six years from four forest sites with different levels of 137Cs contamination from Fukushima Prefecture. To this end, we applied an approximate Bayesian computation (ABC) technique based on the observed 137Cs concentrations (Bq kg−1) of five compartments (leaf, branch, stem, litter, and soil) in a Japanese cedar plantation. The environmental decay (increment) constants of the five compartments were used as the summary statistics (i.e., the metric for model performance) to infer the five parameters related to 137Cs transfer processes in FoRothCs. The ABC technique successfully reconciled the model outputs with the observed trends in 137Cs concentrations at all sites during the study period. Furthermore, the estimated parameters are in agreement with the literature values (e.g., the root uptake rates of 137Cs). Our study demonstrates that model calibration with ABC based on the trends in 137Cs concentrations of multi compartments is useful for reducing the prediction uncertainty of 137Cs dynamics in forest ecosystems.

Inference-based assessment of parameter identifiability in nonlinear biological models

As systems approaches to the development of biological models become more mature, attention is increasingly focusing on the problem of inferring parameter values within those models from experimental data. However, particularly for nonlinear models, it is not obvious, either from inspection of the model or from the experimental data, that the inverse problem of parameter fitting will have a unique solution, or even a non-unique solution that constrains the parameters to lie within a plausible physiological range. Where parameters cannot be constrained they are termed 'unidentifiable'. We focus on gaining insight into the causes of unidentifiability using inference-based methods, and compare a recently developed measure-theoretic approach to inverse sensitivity analysis to the popular Markov chain Monte Carlo and approximate Bayesian computation techniques for Bayesian inference. All three approaches map the uncertainty in quantities of interest in the output space to the probability of sets of parameters in the input space. The geometry of these sets demonstrates how unidentifiability can be caused by parameter compensation and provides an intuitive approach to inference-based experimental design.

Approximate Bayesian computation techniques for optical characterization of nanoparticle clusters.

Characterization of nanoparticle aggregates from observed scattered light leads to a highly complex inverse problem. Even the forward model is so complex that it prohibits the use of classical likelihood-based inference methods. In this study, we compare four so-called likelihood-free methods based on approximate Bayesian computation (ABC) that requires only numeric simulation of the forward model without the need of evaluating a likelihood. In particular, rejection, Markov chain Monte Carlo, population Monte Carlo, and adaptive population Monte Carlo (APMC) are compared in terms of accuracy. In the current model, we assume that the nanoparticle aggregates are mutually well separated and made up of particles of same size. Filippov's particle-cluster algorithm is used to generate aggregates, and discrete dipole approximation is used to estimate scattering behavior. It is found that the APMC algorithm is superior to others in terms of time and acceptance rates, although all algorithms produce similar posterior distributions. Using ABC techniques and utilizing unpolarized light experiments at 266nm wavelength, characterization of soot aggregates is performed with less than 2nm deviation in nanoparticle radius and 3-4 deviation in number of nanoparticles forming the monodisperse aggregates. Promising results are also observed for the polydisperse aggregate with log-normal particle size distribution.

Simulation-based adjustment after exploratory biomarker subgroup selection in phase II.

As part of the evaluation of phase II trials, it is common practice to perform exploratory subgroup analyses with the aim of identifying patient populations with a beneficial treatment effect. When investigating targeted therapies, these subgroups are typically defined by biomarkers. Promising results may lead to the decision to select the respective subgroup as the target population for a subsequent phase III trial. However, a selection based on a large observed treatment effect may potentially induce an upwards-bias leading to over-optimistic expectations on the success probability of the phase III trial. We describe how Approximate Bayesian Computation techniques can be used to derive a simulation-based bias adjustment method in this situation. Recommendations for the implementation of the approach are given. Simulation studies show that the proposed method reduces bias substantially compared with the maximum likelihood estimator. The procedure is illustrated with data from an oncology trial. Copyright © 2017 John Wiley & Sons, Ltd.

Bayesian estimation of agent-based models

We consider Bayesian inference techniques for agent-based (AB) models, as an alternative to simulated minimum distance (SMD). Three computationally heavy steps are involved: (i) simulating the model, (ii) estimating the likelihood and (iii) sampling from the posterior distribution of the parameters. Computational complexity of AB models implies that efficient techniques have to be used with respect to points (ii) and (iii), possibly involving approximations. We first discuss non-parametric (kernel density) estimation of the likelihood, coupled with Markov chain Monte Carlo sampling schemes. We then turn to parametric approximations of the likelihood, which can be derived by observing the distribution of the simulation outcomes around the statistical equilibria, or by assuming a specific form for the distribution of external deviations in the data. Finally, we introduce Approximate Bayesian Computation techniques for likelihood-free estimation. These allow embedding SMD methods in a Bayesian framework, and are particularly suited when robust estimation is needed. These techniques are first tested in a simple price discovery model with one parameter, and then employed to estimate the behavioural macroeconomic model of De Grauwe (2012), with nine unknown parameters.

Bayesian identification of bacterial strains from sequencing data.

Rapidly assaying the diversity of a bacterial species present in a sample obtained from a hospital patient or an environmental source has become possible after recent technological advances in DNA sequencing. For several applications it is important to accurately identify the presence and estimate relative abundances of the target organisms from short sequence reads obtained from a sample. This task is particularly challenging when the set of interest includes very closely related organisms, such as different strains of pathogenic bacteria, which can vary considerably in terms of virulence, resistance and spread. Using advanced Bayesian statistical modelling and computation techniques we introduce a novel pipeline for bacterial identification that is shown to outperform the currently leading pipeline for this purpose. Our approach enables fast and accurate sequence-based identification of bacterial strains while using only modest computational resources. Hence it provides a useful tool for a wide spectrum of applications, including rapid clinical diagnostics to distinguish among closely related strains causing nosocomial infections. The software implementation is available at https://github.com/PROBIC/BIB.

Efficient Bayesian Computation by Proximal Markov Chain Monte Carlo: When Langevin Meets Moreau

Modern imaging methods rely strongly on Bayesian inference techniques to solve challenging imaging problems. Currently, the predominant Bayesian computation approach is convex optimization, which scales very efficiently to high-dimensional image models and delivers accurate point estimation results. However, in order to perform more complex analyses, for example, image uncertainty quantification or model selection, it is necessary to use more computationally intensive Bayesian computation techniques such as Markov chain Monte Carlo methods. This paper presents a new and highly efficient Markov chain Monte Carlo methodology to perform Bayesian computation for high-dimensional models that are log-concave and nonsmooth, a class of models that is central in imaging sciences. The methodology is based on a regularized unadjusted Langevin algorithm that exploits tools from convex analysis, namely, Moreau--Yoshida envelopes and proximal operators, to construct Markov chains with favorable convergence properties. ...

Ill-Posed? Not a Problem: Inferring Mechanisms of Action from Amyloid Formation Kinetics using Approximate Bayesian Computation

Amyloid formation is implicated in a number of human diseases, and is thought to proceed via a nucleation-dependent polymerization mechanism wherein small concentrations of “nuclei” form initially, followed by the rapid, highly favorable addition of further monomer to nuclei/fibril ends. This type of behavior results in highly cooperative, sigmoidal fibrillization kinetics. In many cases, experimenters wish to relate changes in the observed kinetics, for example in response to small molecule effectors, to specific mechanistic steps along this pathway. However, fitting kinetic fibril formation data to a complex model including explicit rate constants results in an ill-posed problem, with a vast number of potential solutions. The amount of uncertainty remaining in parameters calculated from these models, which arises both from experimental noise and high levels of covariance in parameters, is often unclear.Here, we demonstrate the utility of combining these explicit mathematical models with approximate Bayesian computational (ABC) techniques to assign mechanistic roles to small molecule effectors of amyloid fibril formation. We show even when exact rate constants cannot be extracted from bulk amyloid formation kinetic data, the ratios and products of these rate constants can be recovered and used to assign mechanistic effects and their relative magnitudes with confidence. Furthermore, ABC provides a robust method for visualizing uncertainty remaining in the model parameters, regardless of its origin. We apply these methods to the problem of heparin-mediated tau polymerization, which displays complex kinetic behavior not amenable to analysis by more traditional methods. Our studies indicate that heparin's effect cannot be explained by enhancement of nucleation alone, as has been previously proposed. These methods are applicable to a wide range of systems, as models can be easily adapted to account for new reactions and reversibility.

Bayesian nonparametric forecasting of monotonic functional time series

We propose a Bayesian nonparametric approach to modelling and predicting a class of functional time series with application to energy markets, based on fully observed, noise-free functional data. Traders in such contexts conceive profitable strategies if they can anticipate the impact of their bidding actions on the aggregate demand and supply curves, which in turn need to be predicted reliably. Here we propose a simple Bayesian nonparametric method for predicting such curves, which take the form of monotonic bounded step functions. We borrow ideas from population genetics by defining a class of interacting particle systems to model the functional trajectory, and develop an implementation strategy which uses ideas from Markov chain Monte Carlo and approximate Bayesian computation techniques and allows to circumvent the intractability of the likelihood. Our approach shows great adaptation to the degree of smoothness of the curves and the volatility of the functional series, proves to be robust to an increase of the forecast horizon and yields an uncertainty quantification for the functional forecasts. We illustrate the model and discuss its performance with simulated datasets and on real data relative to the Italian natural gas market.

A Novel Statistical Model to Estimate Host Genetic Effects Affecting Disease Transmission.

There is increasing recognition that genetic diversity can affect the spread of diseases, potentially affecting plant and livestock disease control as well as the emergence of human disease outbreaks. Nevertheless, even though computational tools can guide the control of infectious diseases, few epidemiological models can simultaneously accommodate the inherent individual heterogeneity in multiple infectious disease traits influencing disease transmission, such as the frequently modeled propensity to become infected and infectivity, which describes the host ability to transmit the infection to susceptible individuals. Furthermore, current quantitative genetic models fail to fully capture the heritable variation in host infectivity, mainly because they cannot accommodate the nonlinear infection dynamics underlying epidemiological data. We present in this article a novel statistical model and an inference method to estimate genetic parameters associated with both host susceptibility and infectivity. Our methodology combines quantitative genetic models of social interactions with stochastic processes to model the random, nonlinear, and dynamic nature of infections and uses adaptive Bayesian computational techniques to estimate the model parameters. Results using simulated epidemic data show that our model can accurately estimate heritabilities and genetic risks not only of susceptibility but also of infectivity, therefore exploring a trait whose heritable variation is currently ignored in disease genetics and can greatly influence the spread of infectious diseases. Our proposed methodology offers potential impacts in areas such as livestock disease control through selective breeding and also in predicting and controlling the emergence of disease outbreaks in human populations.

Nonparametric Estimation of the Probability of Detection of Flaws in an Industrial Component, from Destructive and Nondestructive Testing Data, Using Approximate Bayesian Computation.

We consider the problem of estimating the probability of detection (POD) of flaws in an industrial steel component. Modeled as an increasing function of the flaw height, the POD characterizes the detection process; it is also involved in the estimation of the flaw size distribution, a key input parameter of physical models describing the behavior of the steel component when submitted to extreme thermodynamic loads. Such models are used to assess the resistance of highly reliable systems whose failures are seldom observed in practice. We develop a Bayesian method to estimate the flaw size distribution and the POD function, using flaw height measures from periodic in-service inspections conducted with an ultrasonic detection device, together with measures from destructive lab experiments. Our approach, based on approximate Bayesian computation (ABC) techniques, is applied to a real data set and compared to maximum likelihood estimation (MLE) and a more classical approach based on Markov Chain Monte Carlo (MCMC) techniques. In particular, we show that the parametric model describing the POD as the cumulative distribution function (cdf) of a log-normal distribution, though often used in this context, can be invalidated by the data at hand. We propose an alternative nonparametric model, which assumes no predefined shape, and extend the ABC framework to this setting. Experimental results demonstrate the ability of this method to provide a flexible estimation of the POD function and describe its uncertainty accurately.

Simulation-based estimation of mean and standard deviation for meta-analysis via Approximate Bayesian Computation (ABC).

BackgroundWhen conducting a meta-analysis of a continuous outcome, estimated means and standard deviations from the selected studies are required in order to obtain an overall estimate of the mean effect and its confidence interval. If these quantities are not directly reported in the publications, they must be estimated from other reported summary statistics, such as the median, the minimum, the maximum, and quartiles.MethodsWe propose a simulation-based estimation approach using the Approximate Bayesian Computation (ABC) technique for estimating mean and standard deviation based on various sets of summary statistics found in published studies. We conduct a simulation study to compare the proposed ABC method with the existing methods of Hozo et al. (2005), Bland (2015), and Wan et al. (2014).ResultsIn the estimation of the standard deviation, our ABC method performs better than the other methods when data are generated from skewed or heavy-tailed distributions. The corresponding average relative error (ARE) approaches zero as sample size increases. In data generated from the normal distribution, our ABC performs well. However, the Wan et al. method is best for estimating standard deviation under normal distribution. In the estimation of the mean, our ABC method is best regardless of assumed distribution.ConclusionABC is a flexible method for estimating the study-specific mean and standard deviation for meta-analysis, especially with underlying skewed or heavy-tailed distributions. The ABC method can be applied using other reported summary statistics such as the posterior mean and 95 % credible interval when Bayesian analysis has been employed.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0055-5) contains supplementary material, which is available to authorized users.

REGRESSION ANALYSIS WITH LINKED DATA: PROBLEMS AND POSSIBLE SOLUTIONS.

In this paper we have described and extended some recent proposals on a general Bayesian methodology for performing record linkage and making inference using the resulting matched units. In particular, we have framed the record linkage process into a formal statistical model which comprises both the matching variables and the other variables included at the inferential stage. This way, the researcher is able to account for the matching process uncertainty in inferential procedures based on probabilistically linked data, and at the same time, he/she is also able to generate a feedback propagation of the information between the working statistical model and the record linkage stage. We have argued that this feedback effect is both essential to eliminate potential biases that otherwise would characterize the resulting linked data inference, and able to improve record linkage performances. The practical implementation of the procedure is based on the use of standard Bayesian computational techniques, such as Markov Chain Monte Carlo algorithms. Although the methodology is quite general, we have restricted our analysis to the popular and important case of multiple linear regression set-up for expository convenience.

A simulation approach for change-points on phylogenetic trees.

We observe n sequences at each of m sites and assume that they have evolved from an ancestral sequence that forms the root of a binary tree of known topology and branch lengths, but the sequence states at internal nodes are unknown. The topology of the tree and branch lengths are the same for all sites, but the parameters of the evolutionary model can vary over sites. We assume a piecewise constant model for these parameters, with an unknown number of change-points and hence a transdimensional parameter space over which we seek to perform Bayesian inference. We propose two novel ideas to deal with the computational challenges of such inference. Firstly, we approximate the model based on the time machine principle: the top nodes of the binary tree (near the root) are replaced by an approximation of the true distribution; as more nodes are removed from the top of the tree, the cost of computing the likelihood is reduced linearly in n. The approach introduces a bias, which we investigate empirically. Secondly, we develop a particle marginal Metropolis-Hastings (PMMH) algorithm, that employs a sequential Monte Carlo (SMC) sampler and can use the first idea. Our time-machine PMMH algorithm copes well with one of the bottle-necks of standard computational algorithms: the transdimensional nature of the posterior distribution. The algorithm is implemented on simulated and real data examples, and we empirically demonstrate its potential to outperform competing methods based on approximate Bayesian computation (ABC) techniques.

Scalable inference for Markov processes with intractable likelihoods

Bayesian inference for Markov processes has become increasingly relevant in recent years. Problems of this type often have intractable likelihoods and prior knowledge about model rate parameters is often poor. Markov Chain Monte Carlo (MCMC) techniques can lead to exact inference in such models but in practice can suffer performance issues including long burn-in periods and poor mixing. On the other hand approximate Bayesian computation techniques can allow rapid exploration of a large parameter space but yield only approximate posterior distributions. Here we consider the combined use of approximate Bayesian computation and MCMC techniques for improved computational efficiency while retaining exact inference on parallel hardware.

Inference of the Properties of the Recombination Process from Whole Bacterial Genomes

Patterns of linkage disequilibrium, homoplasy, and incompatibility are difficult to interpret because they depend on several factors, including the recombination process and the population structure. Here we introduce a novel model-based framework to infer recombination properties from such summary statistics in bacterial genomes. The underlying model is sequentially Markovian so that data can be simulated very efficiently, and we use approximate Bayesian computation techniques to infer parameters. As this does not require us to calculate the likelihood function, the model can be easily extended to investigate less probed aspects of recombination. In particular, we extend our model to account for the bias in the recombination process whereby closely related bacteria recombine more often with one another. We show that this model provides a good fit to a data set of Bacillus cereus genomes and estimate several recombination properties, including the rate of bias in recombination. All the methods described in this article are implemented in a software package that is freely available for download at http://code.google.com/p/clonalorigin/.

Assessing parameter identifiability in phylogenetic models using data cloning.

The success of model-based methods in phylogenetics has motivated much research aimed at generating new, biologically informative models. This new computer-intensive approach to phylogenetics demands validation studies and sound measures of performance. To date there has been little practical guidance available as to when and why the parameters in a particular model can be identified reliably. Here, we illustrate how Data Cloning (DC), a recently developed methodology to compute the maximum likelihood estimates along with their asymptotic variance, can be used to diagnose structural parameter nonidentifiability (NI) and distinguish it from other parameter estimability problems, including when parameters are structurally identifiable, but are not estimable in a given data set (INE), and when parameters are identifiable, and estimable, but only weakly so (WE). The application of the DC theorem uses well-known and widely used Bayesian computational techniques. With the DC approach, practitioners can use Bayesian phylogenetics software to diagnose nonidentifiability. Theoreticians and practitioners alike now have a powerful, yet simple tool to detect nonidentifiability while investigating complex modeling scenarios, where getting closed-form expressions in a probabilistic study is complicated. Furthermore, here we also show how DC can be used as a tool to examine and eliminate the influence of the priors, in particular if the process of prior elicitation is not straightforward. Finally, when applied to phylogenetic inference, DC can be used to study at least two important statistical questions: assessing identifiability of discrete parameters, like the tree topology, and developing efficient sampling methods for computationally expensive posterior densities.

Efficient Approximate Bayesian Computation Coupled With Markov Chain Monte Carlo Without Likelihood

Approximate Bayesian computation (ABC) techniques permit inferences in complex demographic models, but are computationally inefficient. A Markov chain Monte Carlo (MCMC) approach has been proposed (Marjoram et al. 2003), but it suffers from computational problems and poor mixing. We propose several methodological developments to overcome the shortcomings of this MCMC approach and hence realize substantial computational advances over standard ABC. The principal idea is to relax the tolerance within MCMC to permit good mixing, but retain a good approximation to the posterior by a combination of subsampling the output and regression adjustment. We also propose to use a partial least-squares (PLS) transformation to choose informative statistics. The accuracy of our approach is examined in the case of the divergence of two populations with and without migration. In that case, our ABC-MCMC approach needs considerably lower computation time to reach the same accuracy than conventional ABC. We then apply our method to a more complex case with the estimation of divergence times and migration rates between three African populations.