Model Choice Methods Research Articles

Direct simulation Monte Carlo on petaflop supercomputers and beyond

The gold-standard definition of the Direct Simulation Monte Carlo (DSMC) method is given in the 1994 book by Bird [Molecular Gas Dynamics and the Direct Simulation of Gas Flows (Clarendon Press, Oxford, UK, 1994)], which refined his pioneering earlier papers in which he first formulated the method. In the intervening 25 years, DSMC has become the method of choice for modeling rarefied gas dynamics in a variety of scenarios. The chief barrier to applying DSMC to more dense or even continuum flows is its computational expense compared to continuum computational fluid dynamics methods. The dramatic (nearly billion-fold) increase in speed of the largest supercomputers over the last 30 years has thus been a key enabling factor in using DSMC to model a richer variety of flows, due to the method’s inherent parallelism. We have developed the open-source SPARTA DSMC code with the goal of running DSMC efficiently on the largest machines, both current and future. It is largely an implementation of Bird’s 1994 formulation. Here, we describe algorithms used in SPARTA to enable DSMC to operate in parallel at the scale of many billions of particles or grid cells, or with billions of surface elements. We give a few examples of the kinds of fundamental physics questions and engineering applications that DSMC can address at these scales.

Implications of uniformly distributed, empirically informed priors for phylogeographical model selection: a reply to Hickerson et al.

Establishing that a set of population-splitting events occurred at the same time can be a potentially persuasive argument that a common process affected the populations. Recently, Oaks etal. () assessed the ability of an approximate-Bayesian model-choice method (msBayes) to estimate such a pattern of simultaneous divergence across taxa, to which Hickerson etal. () responded. Both papers agree that the primary inference enabled by the method is very sensitive to prior assumptions and often erroneously supports shared divergences across taxa when prior uncertainty about divergence times is represented by a uniform distribution. However, the papers differ about the best explanation and solution for this problem. Oaks etal. () suggested the method's behavior was caused by the strong weight of uniformly distributed priors on divergence times leading to smaller marginal likelihoods (and thus smaller posterior probabilities) of models with more divergence-time parameters (Hypothesis 1); they proposed alternative prior probability distributions to avoid such strongly weighted posteriors. Hickerson etal. () suggested numerical-approximation error causes msBayes analyses to be biased toward models of clustered divergences because the method's rejection algorithm is unable to adequately sample the parameter space of richer models within reasonable computational limits when using broad uniform priors on divergence times (Hypothesis 2). As a potential solution, they proposed a model-averaging approach that uses narrow, empirically informed uniform priors. Here, we use analyses of simulated and empirical data to demonstrate that the approach of Hickerson etal. () does not mitigate the method's tendency to erroneously support models of highly clustered divergences, and is dangerous in the sense that the empirically derived uniform priors often exclude from consideration the true values of the divergence-time parameters. Our results also show that the tendency of msBayes analyses to support models of shared divergences is primarily due to Hypothesis 1, whereas Hypothesis 2 is an untenable explanation for the bias. Overall, this series of papers demonstrates that if our prior assumptions place too much weight in unlikely regions of parameter space such that the exact posterior supports the wrong model of evolutionary history, no amount of computation can rescue our inference. Fortunately, as predicted by fundamental principles of Bayesian model choice, more flexible distributions that accommodate prior uncertainty about parameters without placing excessive weight in vast regions of parameter space with low likelihood increase the method's robustness and power to detect temporal variation in divergences.

Adaptive ABC model choice and geometric summary statistics for hidden Gibbs random fields

Selecting between different dependency structures of hidden Markov random field can be very challenging, due to the intractable normalizing constant in the likelihood. We answer this question with approximate Bayesian computation (ABC) which provides a model choice method in the Bayesian paradigm. This comes after the work of Grelaud et al. (2009) who exhibited sufficient statistics on directly observed Gibbs random fields. But when the random field is latent, the sufficiency falls and we complement the set with geometric summary statistics. The general approach to construct these intuitive statistics relies on a clustering analysis of the sites based on the observed colors and plausible latent graphs. The efficiency of ABC model choice based on these statistics is evaluated via a local error rate which may be of independent interest. As a byproduct we derived an ABC algorithm that adapts the dimension of the summary statistics to the dataset without distorting the model selection.

A transdimensional approximate Bayesian computation using the pseudo-marginal approach for model choice

When the likelihood functions are either unavailable analytically or are computationally cumbersome to evaluate, it is impossible to implement conventional Bayesian model choice methods. Instead, approximate Bayesian computation (ABC) or the likelihood-free method can be used in order to avoid direct evaluation of the intractable likelihoods. This paper proposes a new Markov chain Monte Carlo (MCMC) method for model choice. This method is based on the pseudo-marginal approach and is appropriate for situations where the likelihood functions for the competing models are intractable. This method proposes jumps between the models with different dimensionalities without matching the dimensionalities. Therefore, it enables the construction of a flexible proposal distribution. The proposal distribution used in this paper is convenient to implement and works well in the context of ABC. Because the posterior model probabilities can be estimated simultaneously, it is expected that the proposed method will be useful, especially when the number of competing models is large. In the simulation study, a comparison between the proposed and existing methods is presented. The method is then applied to the model choice problem for an exchange return model.

An improved approximate-Bayesian model-choice method for estimating shared evolutionary history.

BackgroundTo understand biological diversification, it is important to account for large-scale processes that affect the evolutionary history of groups of co-distributed populations of organisms. Such events predict temporally clustered divergences times, a pattern that can be estimated using genetic data from co-distributed species. I introduce a new approximate-Bayesian method for comparative phylogeographical model-choice that estimates the temporal distribution of divergences across taxa from multi-locus DNA sequence data. The model is an extension of that implemented in msBayes.ResultsBy reparameterizing the model, introducing more flexible priors on demographic and divergence-time parameters, and implementing a non-parametric Dirichlet-process prior over divergence models, I improved the robustness, accuracy, and power of the method for estimating shared evolutionary history across taxa.ConclusionsThe results demonstrate the improved performance of the new method is due to (1) more appropriate priors on divergence-time and demographic parameters that avoid prohibitively small marginal likelihoods for models with more divergence events, and (2) the Dirichlet-process providing a flexible prior on divergence histories that does not strongly disfavor models with intermediate numbers of divergence events. The new method yields more robust estimates of posterior uncertainty, and thus greatly reduces the tendency to incorrectly estimate models of shared evolutionary history with strong support.

Accuracy of density functional theory in the prediction of carbon dioxide adsorbent materials

Density functional theory (DFT) has become the computational method of choice for modeling and characterization of carbon dioxide adsorbents, a broad family of materials which at present are urgently sought after for environmental applications. The description of polar carbon dioxide (CO(2)) molecules in low-coordinated environments like surfaces and porous materials, however, may be challenging for local and semi-local DFT approximations. Here, we present a thorough computational study in which the accuracy of DFT methods in describing the interactions of CO(2) with model alkali-earth-metal (AEM, Ca and Li) decorated carbon structures, namely anthracene (C(14)H(10)) molecules, is assessed. We find that gas-adsorption energies and equilibrium structures obtained with standard (i.e. LDA and GGA), hybrid (i.e. PBE0 and B3LYP) and van der Waals exchange-correlation functionals of DFT dramatically differ from the results obtained with second-order Møller-Plesset perturbation theory (MP2), an accurate computational quantum chemistry method. The major disagreements found can be mostly rationalized in terms of electron correlation errors that lead to wrong charge-transfer and electrostatic Coulomb interactions between CO(2) and AEM-decorated anthracene molecules. Nevertheless, we show that when the concentration of AEM atoms in anthracene is tuned to resemble as closely as possible the electronic structure of AEM-decorated graphene (i.e. an extended two-dimensional material), hybrid exchange-correlation DFT and MP2 methods quantitatively provide similar results.

IS YOUR PHYLOGENY INFORMATIVE? MEASURING THE POWER OF COMPARATIVE METHODS

Phylogenetic comparative methods may fail to produce meaningful results when either the underlying model is inappropriate or the data contain insufficient information to inform the inference. The ability to measure the statistical power of these methods has become crucial to ensure that data quantity keeps pace with growing model complexity. Through simulations, we show that commonly applied model choice methods based on information criteria can have remarkably high error rates; this can be a problem because methods to estimate the uncertainty or power are not widely known or applied. Furthermore, the power of comparative methods can depend significantly on the structure of the data. We describe a Monte Carlo-based method which addresses both of these challenges, and show how this approach both quantifies and substantially reduces errors relative to information criteria. The method also produces meaningful confidence intervals for model parameters. We illustrate how the power to distinguish different models, such as varying levels of selection, varies both with number of taxa and structure of the phylogeny. We provide an open-source implementation in the pmc ("Phylogenetic Monte Carlo") package for the R programming language. We hope such power analysis becomes a routine part of model comparison in comparative methods.

IS YOUR PHYLOGENY INFORMATIVE? MEASURING THE POWER OF COMPARATIVE METHODS

Phylogenetic comparative methods may fail to produce meaningful results when either the underlying model is inappropriate or the data contain insufficient information to inform the inference. The ability to measure the statistical power of these methods has become crucial to ensure that data quantity keeps pace with growing model complexity. Through simulations, we show that commonly applied model choice methods based on information criteria can have remarkably high error rates; this can be a problem because methods to estimate the uncertainty or power are not widely known or applied. Furthermore, the power of comparative methods can depend significantly on the structure of the data. We describe a Monte Carlo-based method which addresses both of these challenges, and show how this approach both quantifies and substantially reduces errors relative to information criteria. The method also produces meaningful confidence intervals for model parameters. We illustrate how the power to distinguish different models, such as varying levels of selection, varies both with number of taxa and structure of the phylogeny. We provide an open-source implementation in the pmc (“Phylogenetic Monte Carlo”) package for the R programming language. We hope such power analysis becomes a routine part of model comparison in comparative methods.

ABC likelihood-free methods for model choice in Gibbs random fields

Gibbs random fields (GRF) are polymorphous statistical models that can be used to analyse different types of dependence, in particular for spatially correlated data. However, when those models are faced with the challenge of selecting a dependence structure from many, the use of standard model choice methods is hampered by the unavailability of the normalising constant in the Gibbs likelihood. In particular, from a Bayesian perspective, the computation of the posterior probabilities of the models under competition requires special likelihood-free simulation techniques like the Approximate Bayesian Computation (ABC) algorithm that is intensively used in population genetics. We show in this paper how to implement an ABC algorithm geared towards model choice in the general setting of Gibbs random fields, demonstrating in particular that there exists a sufficient statistic across models. The accuracy of the approximation to the posterior probabilities can be further improved by importance sampling on the distribution of the models. The practical aspects of the method are detailed through two applications, the test of an iid Bernoulli model versus a first-order Markov chain, and the choice of a folding structure for two proteins.

ABC methods for model choice in Gibbs random fields

We consider the problem of model selection within the class of Gibbs random fields. In a Bayesian framework, this choice relies on the evaluation of the posterior probabilities of all models. We define an extended parameter setting, including the model index and show the existence of a corresponding sufficient statistic made of the conjunction of the sufficient statistics of all models. We use this statistic to derive an ABC algorithm. To cite this article: A. Grelaud et al., C. R. Acad. Sci. Paris, Ser. I 347 (2009).

Bayesian models identify specific lymphoproliferative disorders associated with hepatitis C virus infection

Bayesian models identify specific lymphoproliferative disorders associated with hepatitis C virus infection

Supertrees join the mainstream of phylogenetics

Supertree methods are fairly widely used to build comprehensive phylogenies for particular groups, but concerns remain over the adequacy of existing approaches. Steel and Rodrigo recently introduced a statistical model of incongruence between trees, allowing maximum-likelihood supertree inference. This approach to supertree construction will enable hypothesis-testing and model-choice methods that are now routine in sequence phylogenetics to be applied in this setting, and might form an important part of future phylogenetic inference from genomic data.

Interpolation of spatial and spatio-temporal Gaussian fields using Gaussian Markov random fields

This paper considers interpolation on a lattice of covariance-based Gaussian Random Field models (Geostatistics models) using Gaussian Markov Random Fields (GMRFs) (conditional autoregression models). Two methods for estimating the GMRF parameters are considered. One generalises maximum likelihood for complete data, and the other ensures a better correspondence between fitted and theoretical correlations for higher lags. The methods can be used both for spatial and spatio-temporal data. Some different cross-validation methods for model choice are compared. The predictive ability of the GMRF is demonstrated by a simulation study, and an example using a real image is considered.

PREDATOR AND PREY SPACE USE: DRAGONFLIES AND TADPOLES IN AN INTERACTIVE GAME

Predator and prey spatial distributions have important population and community level consequences. However, little is known either theoretically or empirically about behavioral mechanisms that underlie the spatial patterns that emerge when predators and prey freely interact. We examined the joint space use and behavioral rules governing movement of freely interacting groups of odonate (dragonfly) predators and two size classes of anuran (tadpole) prey in arenas containing two patches with different levels of the prey's resource. Predator and prey movement and space use was quantified both when they were apart and together. When apart from predators, large tadpoles strongly preferred the high resource patch. When apart from prey, dragonflies weakly preferred the high resource patch. When together, large prey shifted to a uniform distribution, while predators strongly preferred the high resource patch. These patterns qualitatively fit the predictions of several three trophic level, ideal free distribution models. In contrast, the space use of small prey and predators did not deviate from uniform. Three measures of joint space use (spatial correlations, overlap, and co-occurrence) concurred in suggesting that prey avoidance of predators was more important than predator attraction to prey in determining overall spatial patterns. To gain additional insight into behavioral mechanisms, we used a model selection approach to identify behavioral movement rules that can potentially explain the observed, emergent patterns of space use. Prey were more likely to leave patches with more predators and more conspecific competitors; resources had relatively weak effects on prey movements. In contrast, predators were more likely to leave patches with low resources (that they do not consume) and more competing predators; prey had relatively little effect on predator movements. These results highlight the importance of investigating freely interacting predators and prey, the potential for simple game theory models to predict joint spatial distributions, and the utility of using model choice methods to identify potential key factors that govern movement.

Bayesian Model Choice in Exponential Survival Models

ABSTRACT This article introduces four survival models: simple model, change point model, finite mixture model, and survival fraction model. Among these models, in order to choice the best model, we proposed a model choice method using approaches of Gelfand and Ghosh (1998). Then to avoid the computational difficulties, data augmentation method (Tanner and Wong, 1987) and Gibbs sampler (Gelfand and Smith, 1990) are employed. Our methodology is applied to simulated data and Stangl data On-impramint Hydrochloride data. The results indicate that our methodology is not sensitive for prior and weight of criterion.

Comparison of model-free kinetic methods for modeling the cure kinetics of commercial phenol–formaldehyde resins

For many industrial processes it is important to model the cure kinetics of phenol–formaldehyde resoles. Yet the applicability of common model-free kinetic algorithms for the cure of phenolic resins is not known. In this study the ability of the Friedman, Vyazovkin and Kissinger–Akahira–Sunose (KAS) model-free-kinetics algorithms to model and predict the cure kinetics of commercial resoles is compared. The Friedman and Vyazovkin methods generate consistent activation energy dependences on conversion compared to the KAS method. In addition, the activation energy dependency on conversion is of higher amplitude with these two methods than with the KAS method. Hence, the Friedman and Vyazovkin methods are more adequate for revealing the cure steps of commercial PF resoles. Conversely, the KAS algorithm is easily amenable to dynamic cure predictions compared to the Friedman and Vyazovkin methods. Isothermal cure is equally well predicted with the three. As a result, the KAS algorithm is the method of choice for modeling and predicting the cure kinetics of commercial phenolic resoles under various temperature programs.

지수 생존 모형에서의 베이지안 모형 선택

본 논문에서는 지수생존 모형의 형태들로써 단순 지수모형, 변환 점 지수모형과 유한 혼합 지수모형 등 세 가지 모형을 소개한다. 이러한 모형들 중에서, 최적의 모형을 찾기 위하여 Gelfand와 Ghosh(1998)의 방법을 이용한 모형 선택 방법을 제안한다. 이때, 계산상의 어려움을 피하기 위하여 자료 확장 기법(Tanner와 Wong, 1987)과 깁스 샘플러(Gelfand와 Smith, 1990)를 사용하였다. 제안된 베이지안 방법을 설명하기 위하여 모의 실험 자료와Stangl의 항 우울제 자료에 적용한다. 모형 선택 방법은 사전 분포와 모형 선택 기준의 가중치에 민감하지 않다는 것을 제한된 우리의 실험으로 알 수 있었다. We introduce three types of exponential survival models, such as simple model, change-point model and finite mixture model in this paper. Among these models, in order to choose the best model, the model choice method is proposed using Gelfand and Ghosh(1998)'s idea. Then to avoid the computational difficulties, data augmentation method (Tanner and Wong, 1987) and Gibbs sampler (Gelfand and Smith, 1990) are employed. Our methodology is applied to both simulated data and Stangl (1991)'s On-impramint Hydrochloride data.

Markov Chain Monte Carlo Methods for Computing Bayes Factors

The problem of calculating posterior probabilities for a collection of competing models and associated Bayes factors continues to be a formidable challenge for applied Bayesian statisticians. Current approaches that take advantage of modern Markov chain Monte Carlo computing methods include those that attempt to sample over some form of the joint space created by the model indicators and the parameters for each model, others that sample over the model space alone, and still others that attempt to estimate the marginal likelihood of each model directly (because the collection of these is equivalent to the collection of model probabilities themselves). We review several methods and compare them in the context of three examples: a simple regression example, a more challenging hierarchical longitudinal model, and a binary data latent variable model. We find that the joint model-parameter space search methods perform adequately but can be difficult to program and tune, whereas the marginal likelihood methods often are less troublesome and require less additional coding. Our results suggest that the latter methods may be most appropriate for practitioners working in many standard model choice settings, but the former remain important for comparing models of varying dimension (e.g., multiple changepoint models) or models whose parameters cannot easily be updated in relatively few blocks. We caution, however, that all methods we compared require significant human and computer effort, and this suggests that less formal Bayesian model choice methods may offer a more realistic alternative in many cases.