Bayesian Model Fit Research Articles

Support for the Time-Varying Drift Rate Model of Perceptual Discrimination in Dynamic and Static Noise Using Bayesian Model-Fitting Methodology.

The drift-diffusion model (DDM) is a common approach to understanding human decision making. It considers decision making as accumulation of evidence about visual stimuli until sufficient evidence is reached to make a decision (decision boundary). Recently, Smith and colleagues proposed an extension of DDM, the time-varying DDM (TV-DDM). Here, the standard simplification that evidence accumulation operates on a fully formed representation of perceptual information is replaced with a perceptual integration stage modulating evidence accumulation. They suggested that this model particularly captures decision making regarding stimuli with dynamic noise. We tested this new model in two studies by using Bayesian parameter estimation and model comparison with marginal likelihoods. The first study replicated Smith and colleagues' findings by utilizing the classical random-dot kinomatogram (RDK) task, which requires judging the motion direction of randomly moving dots (motion discrimination task). In the second study, we used a novel type of stimulus designed to be like RDKs but with randomized hue of stationary dots (color discrimination task). This study also found TV-DDM to be superior, suggesting that perceptual integration is also relevant for static noise possibly where integration over space is required. We also found support for within-trial changes in decision boundaries ("collapsing boundaries"). Interestingly, and in contrast to most studies, the boundaries increased with increasing task difficulty (amount of noise). Future studies will need to test this finding in a formal model.

Under-Fitting and Over-Fitting: The Performance of Bayesian Model Selection and Fit Indices in SEM

We extended current knowledge by examining the performance of several Bayesian model fit and comparison indices through a simulation study using the confirmatory factor analysis. Our goal was to determine whether commonly implemented Bayesian indices can detect specification errors. Specifically, we wanted to uncover any differences in detecting under-fitting or over-fitting a model. We examined a conventional Bayesian fit index (the posterior predictive p-value), approximate Bayesian fit indices (Bayesian RMSEA, CFI, and TLI), and model comparison indices (BIC and DIC). We varied the type and severity of model mis-specification, sample size, and priors. We focused on the ability of these indices to detect model under- or over-fitting. We provide practical advice for applied researchers regarding how to assess and compare models using these common indices implemented in the Bayesian framework.

Performance of Model Fit and Selection Indices for Bayesian Piecewise Growth Modeling with Missing Data

The Bayesian piecewise growth model (PGM) is a useful class of models for analyzing nonlinear change processes that consist of distinct growth phases. In applications of Bayesian PGMs, it is important to accurately capture growth trajectories and carefully consider knot placements. The presence of missing data is another challenge researchers commonly encounter. To address these issues, one could use model fit and selection indices to detect misspecified Bayesian PGMs, and should give care to the potential impact of missing data on model evaluation. Here we conducted a simulation study to examine the impact of model misspecification and missing data on the performance of Bayesian model fit and selection indices (PPP-value, BCFI, BTLI, BRMSEA, BIC, and DIC), with an additional focus on prior sensitivity. Results indicated that (a) increasing the degree of model misspecification and amount of missing data aggravated the performance of indices in detecting misfit, and (b) different prior specifications had negligible impact on model assessment. We provide practical guidelines for researchers to facilitate effective implementation of Bayesian PGMs.

A point mass proposal method for Bayesian state-space model fitting

State-space models (SSMs) are commonly used to model time series data where the observations depend on an unobserved latent process. However, inference on the model parameters of an SSM can be challenging, especially when the likelihood of the data given the parameters is not available in closed-form. One approach is to jointly sample the latent states and model parameters via Markov chain Monte Carlo (MCMC) and/or sequential Monte Carlo approximation. These methods can be inefficient, mixing poorly when there are many highly correlated latent states or parameters, or when there is a high rate of sample impoverishment in the sequential Monte Carlo approximations. We propose a novel block proposal distribution for Metropolis-within-Gibbs sampling on the joint latent state and parameter space. The proposal distribution is informed by a deterministic hidden Markov model (HMM), defined such that the usual theoretical guarantees of MCMC algorithms apply. We discuss how the HMMs are constructed, the generality of the approach arising from the tuning parameters, and how these tuning parameters can be chosen efficiently in practice. We demonstrate that the proposed algorithm using HMM approximations provides an efficient alternative method for fitting state-space models, even for those that exhibit near-chaotic behavior.

A Goodness of Fit Test for a Survival and Count Bayesian Joint Model: In the Presence of Clusters

Bayesian statistical model fitting was an uncommon approach until recently, causing a lack of assessment techniques for these models. However, with the enhancement of computational facilities and advanced estimation techniques, Bayesian models have become popular. Though there are developed goodness of fit (GOF) tests available for classical multilevel models including joint modelling of mixed responses, there is no suitable model based GOF test to be applied on such a model which is fitted under a Bayesian framework. Therefore, this study focused on developing a suitable GOF test for multilevel Bayesian joint models having survival and count responses which are two frequently occurring data types in many fields. The novel test is developed mainly based on four classical GOF tests, including the well-known Hosmer-Lemeshow test and, the Bayesian concepts such as Bayesian credible intervals and regions. In addition, a simulation study has been used to examine the properties of the GOF test together with an application to a real-life example. The novel test performed well in terms of power and acceptable in terms of Type I error rates. Overall, the test performed well with small sample sizes.

The Accuracy of Bayesian Model Fit Indices in Selecting Among Multidimensional Item Response Theory Models.

Item response theory (IRT) models are often compared with respect to predictive performance to determine the dimensionality of rating scale data. However, such model comparisons could be biased toward nested-dimensionality IRT models (e.g., the bifactor model) when comparing those models with non-nested-dimensionality IRT models (e.g., a unidimensional or a between-item-dimensionality model). The reason is that, compared with non-nested-dimensionality models, nested-dimensionality models could have a greater propensity to fit data that do not represent a specific dimensional structure. However, it is unclear as to what degree model comparison results are biased toward nested-dimensionality IRT models when the data represent specific dimensional structures and when Bayesian estimation and model comparison indices are used. We conducted a simulation study to add clarity to this issue. We examined the accuracy of four Bayesian predictive performance indices at differentiating among non-nested- and nested-dimensionality IRT models. The deviance information criterion (DIC), a commonly used index to compare Bayesian models, was extremely biased toward nested-dimensionality IRT models, favoring them even when non-nested-dimensionality models were the correct models. The Pareto-smoothed importance sampling approximation of the leave-one-out cross-validation was the least biased, with the Watanabe information criterion and the log-predicted marginal likelihood closely following. The findings demonstrate that nested-dimensionality IRT models are not automatically favored when the data represent specific dimensional structures as long as an appropriate predictive performance index is used.

Big problems in spatio-temporal disease mapping: Methods and software

Background and objective: Fitting spatio-temporal models for areal data is crucial in many fields such as cancer epidemiology. However, when data sets are very large, many issues arise. The main objective of this paper is to propose a general procedure to analyze high-dimensional spatio-temporal areal data, with special emphasis on mortality/incidence relative risk estimation.Methods: We present a pragmatic and simple idea that permits hierarchical spatio-temporal models to be fitted when the number of small areas is very large. Model fitting is carried out using integrated nested Laplace approximations over a partition of the spatial domain. We also use parallel and distributed strategies to speed up computations in a setting where Bayesian model fitting is generally prohibitively time-consuming or even unfeasible.Results:Using simulated and real data, we show that our method outperforms classical global models. We implement the methods and algorithms that we develop in the open-source R package bigDM where specific vignettes have been included to facilitate the use of the methodology for non-expert users.Conclusions: Our scalable methodology proposal provides reliable risk estimates when fitting Bayesian hierarchical spatio-temporal models for high-dimensional data.

Modeling nation-wide U.S. swine movement networks at the resolution of the individual premises

The spread of infectious livestock diseases is a major cause for concern in modern agricultural systems. In the dynamics of the transmission of such diseases, movements of livestock between herds play an important role. When constructing mathematical models used for activities such as forecasting epidemic development, evaluating mitigation strategies, or determining important targets for disease surveillance, including between-premises shipments is often a necessity. In the United States (U.S.), livestock shipment data is not routinely collected, and when it is, it is not readily available and mostly concerned with between-state shipments. To bridge this gap in knowledge and provide insight into the complete livestock shipment network structure, we have developed the U.S. Animal Movement Model (USAMM). Previously, USAMM has only existed for cattle shipments, but here we present a version for domestic swine. This new version of USAMM consists of a Bayesian model fit to premises demography, county-level livestock industry variables, and two limited data sets of between-state swine movements. The model scales up the data to simulate nation-wide networks of both within- and between-state shipments at the level of individual premises. Here we describe this shipment model in detail and subsequently explore its usefulness with a rudimentary predictive model of the prevalence of porcine epidemic diarrhea virus (PEDv) across the U.S. Additionally, in order to promote further research on livestock disease and other topics involving the movements of swine in the U.S., we also make 250 synthetic premises-level swine shipment networks with complete coverage of the entire conterminous U.S. freely available to the research community as a useful surrogate for the absent shipment data.

Fishing gear entanglement threatens recovery of critically endangered North Atlantic right whales

AbstractNorth Atlantic right whales frequently become entangled in fishing gear, which can negatively affect their reproductive output and probability of survival. We estimated individual whale health from a hierarchical Bayesian model fit to photographic indices of health. We reviewed 696 whales sighted from 1980 to 2011 and assigned 1196 entanglement events to 573 individuals in six categories of increasing injury severity and estimated monthly median health scores (0–100 scale) for the duration of their life within the study period. We then quantified the relationship between entanglement injury events and their severity with survival, reproduction, and population health. Severe entanglements resulted in worse health for all whales—males and females with severe injuries were eight times more likely to die than males with minor injuries. Females with severe injuries that survived had the lowest birth rates. Though the relationship between entanglement and fecundity was complex, we found that as the health of reproductively active females declined, their calving intervals increased. Unimpacted whale health scores declined significantly over three decades, 1980s, 1990s, and 2000s, suggesting food limitations may be contributing to population‐wide health declines. Decadal health scores of entangled whales showed a more notable reduction in health suggesting a clear and perhaps synergistic effect.

Sensitivity of Bayesian Model Fit Indices to the Prior Specification of Latent Growth Models

Longitudinal research often involves relatively small samples and missing values. Under these conditions, Bayesian estimation can still result in accurate parameter estimates for latent growth models (LGMs). However, researchers were limited in their options for assessing model fit. Several new (approximate) model fit indices have been introduced into the Bayesian structural equation modeling framework. Through a simulation study, we examined the performance of these indices for model fit and selection with a latent growth model (LGM). Specifically, this study was designed to assess the impact of different prior specifications on the fit indices across several sample sizes and missing data conditions. Findings suggested that priors that diverge from the population values can interfere with model fit and selection assessment, making correctly specified models appear misspecified. In addition, the approximate fit indices may be more suited for model selection rather than model fit assessment. Implications for applied researchers are discussed.

Placental functional assessment and its relationship to adverse pregnancy outcome: comparison of intravoxel incoherent motion (IVIM) MRI, T2-relaxation time, and umbilical artery Doppler ultrasound.

Early identification of placental insufficiency can lead to appropriate treatment selections and can improve neonates' outcomes. Possible contributions of magnetic resonance imaging (MRI) have been suggested. To evaluate the prognostic capabilities of placental intravoxel incoherent motion (IVIM) parameters and T2-relaxation time, and their correlation with fetal growth and adverse outcomes, comparing umbilical artery (UmA) pulsatility index (PI). A total of 68 singleton pregnancies at 24-40 weeks of gestation underwent placental MRI and were reviewed retrospectively. UmA-PI was measured using Doppler ultrasound by obstetricians. IVIM parameters (Dfast, Dslow, and f) were calculated with a Bayesian model fitting. First, the associations between gestational age (GA) with placental IVIM parameters, T2-relaxation time, and placental thickness (PT) were evaluated. Second, IVIM parameters, T2 value (Z-score), PT (Z-score), and UmA-PI (Z-score) were compared between ( 1) those delivering small for gestational age (SGA) and appropriate for gestational age (AGA) neonates, ( 2) emergency cesarean section (ECS), and non-ECS, and ( 3) preterm birth and full-term birth. Low birth weight was observed in 15/68 cases (22%). GA was significantly associated only with T2-relaxation time and PT. SGA was significantly associated with T2 value (Z-score), f, and UmA-PI (Z-score). In the ECS groups, T2 value (Z-score), f, and Dfast were significantly lower than those in non-ECS groups. All IVIM parameters and T2 values (Z-score) showed significantly lower scores in the preterm birth group. Placental f and T2 value (Z-score) had significant associations with low birth weight and clinical adverse outcomes and could be potential imaging biomarkers of placental insufficiency.

Predicting the growth of the amphibian chytrid fungus in varying temperature environments.

Environmental temperature is a crucial abiotic factor that influences the success of ectothermic organisms, including hosts and pathogens in disease systems. One example is the amphibian chytrid fungus, Batrachochytrium dendrobatidis (Bd), which has led to widespread amphibian population declines. Understanding its thermal ecology is essential to effectively predict outbreaks. Studies that examine the impact of temperature on hosts and pathogens often do so in controlled constant temperatures. Although varying temperature experiments are becoming increasingly common, it is unrealistic to test every temperature scenario. Thus, reliable methods that use constant temperature data to predict performance in varying temperatures are needed. In this study, we tested whether we could accurately predict Bd growth in three varying temperature regimes, using a Bayesian hierarchical model fit with constant temperature Bd growth data. We fit the Bayesian hierarchical model five times, each time changing the thermal performance curve (TPC) used to constrain the logistic growth rate to determine how TPCs influence the predictions. We then validated the model predictions using Bd growth data collected from the three tested varying temperature regimes. Although all TPCs overpredicted Bd growth in the varying temperature regimes, some functional forms performed better than others. Varying temperature impacts on disease systems are still not well understood and improving our understanding and methodologies to predict these effects could provide insights into disease systems and help conservation efforts.

Agent-Based Models for Collective Animal Movement: Proximity-Induced State Switching

Animal movement is a complex phenomenon where individual movement patterns can be influenced by a variety of factors including the animal’s current activity, available terrain and habitat, and locations of other animals. Motivated by modeling grizzly bear movement in the Greater Yellowstone Ecosystem, this article presents an agent-based model represented in a state-space framework for collective animal movement. The novel contribution of this work is a collective animal movement model that captures interactions between animals that can trigger changes in movement patterns, such as when a dominant grizzly bear may cause another subordinate bear to temporarily leave an area. The modeling framework enables learning different movement patterns through a state-space representation with particle-MCMC methods for fully Bayesian model fitting and the prediction of future animal movement behaviors.Supplementary materials accompanying this paper appear online.

Advances in Bayesian Model Fit Evaluation for Structural Equation Models

ABSTRACT In this article, we discuss the Posterior Predictive P-value (PPP) method in the presence of missing data, the Bayesian adaptation of the approximate fit indices RMSEA, CFI and TLI, as well as the Bayesian adaptation of the Wald test for nested models. Simulation studies are presented. We also illustrate how these new methods can be used to build BSEM models.

Integrating Quality of Life and Survival Outcomes in Cardiovascular Clinical Trials: Results From the PARTNER Trial

Background Survival and health status (eg, symptoms and quality of life) are key outcomes in clinical trials of heart failure treatment. However, health status can only be recorded on survivors, potentially biasing treatment effect estimates when there is differential survival across treatment groups. Joint modeling of survival and health status can address this bias. Methods and Results We analyzed patient-level data from the PARTNER 1B trial (Placement of Aortic Transcatheter Valves) of transcatheter aortic valve replacement versus standard care. Health status was quantified with the Kansas City Cardiomyopathy Questionnaire (KCCQ) at randomization, 1, 6, and 12 months. We compared hazard ratios for survival and mean differences in KCCQ scores at 12 months using several models: the original growth curve model for KCCQ scores (ignoring death), separate Bayesian models for survival and KCCQ scores, and a Bayesian joint longitudinal-survival model fit to either 12 or 30 months of survival follow-up. The benefit of transcatheter aortic valve replacement on 12-month KCCQ scores was greatest in the joint-model fit to all survival data (mean difference, 33.7 points; 95% credible intervals [CrI], 24.2-42.4), followed by the joint-model fit to 12 months of survival follow-up (32.3 points; 95% CrI, 22.5-41.5), a Bayesian model without integrating death (30.4 points; 95% CrI, 21.4-39.3), and the original growth curve model (26.0 points; 95% CI, 18.7-33.3). At 12 months, the survival benefit of transcatheter aortic valve replacement was also greater in the joint model (hazard ratio, 0.50; 95% CrI, 0.32-0.73) than in the nonjoint Bayesian model (0.54; 95% CrI, 0.37-0.75) or the original Kaplan-Meier estimate (0.55; 95% CI, 0.40-0.74). Conclusions In patients with severe symptomatic aortic stenosis and prohibitive surgical risk, the estimated benefits of transcatheter aortic valve replacement on survival and health status compared with standard care were greater in joint Bayesian models than other approaches.

Variation in tree growth sensitivity to moisture across a water-limited forest landscape

Water availability acts as a major constraint on productivity in many sub-humid forest regions. Precipitation can be an important limiting factor for tree growth in such areas, but the strength of the relationship can vary by habitat and species, as well as with tree size and local hydrology. We quantified the influence of past weather conditions on the growth of two conifer species (Pinus contorta and Picea glauca) across a water-limited forest landscape in western Canada. The two species differ in moisture requirements and are segregated across a local elevational gradient, and so we expected them to exhibit different sensitivities to precipitation. We also expected that larger trees and those more distant from creeks would have a stronger response to precipitation. A hierarchical Bayesian model fit to the annual ring widths of 387 trees showed that historical precipitation from 1951 to 2016 had a positive overall effect on radial growth. The magnitude of precipitation effects on radial tree growth varied with creek proximity (a proxy for the soil moisture provided by an elevated water table in the valley bottom) and tree size. Precipitation had a greater positive influence on the growth of larger P. glauca trees, as well as individuals of both species at far and intermediate distances from creeks. Precipitation had a weaker but still positive effect on P. glauca trees growing close to creeks. Tree growth rates may change with the predicted greater inter-annual variability of precipitation under climate change, but the magnitude of these responses appear to vary by species, size, and creek proximity. Overall changes in tree growth are expected to be relatively small as trees are well-adapted to cope with the variation in water availability across a moisture-limited landscape.

The relative efficiency of time-to-progression and continuous measures of cognition in presymptomatic Alzheimer's disease

IntroductionClinical trials on preclinical Alzheimer’s disease are challenging because of the slow rate of disease progression. We use a simulation study to demonstrate that models of repeated cognitive assessments detect treatment effects more efficiently than models of time to progression. MethodsMultivariate continuous data are simulated from a Bayesian joint mixed-effects model fit to data from the Alzheimer’s Disease Neuroimaging Initiative. Simulated progression events are algorithmically derived from the continuous assessments using a random forest model fit to the same data. ResultsWe find that power is approximately doubled with models of repeated continuous outcomes compared with the time-to-progression analysis. The simulations also demonstrate that a plausible informative missing data pattern can induce a bias that inflates treatment effects, yet 5% type I error is maintained. DiscussionGiven the relative inefficiency of time to progression, it should be avoided as a primary analysis approach in clinical trials of preclinical Alzheimer’s disease.

The Impact of Moderate Priors For Bayesian Estimation and Testing of Item Factor Analysis Models When Maximum Likelihood is Unsuitable

In psychological research, available data are often insufficient to estimate item factor analysis (IFA) models using traditional estimation methods, such as maximum likelihood (ML) or limited information estimators. Bayesian estimation with common-sense, moderately informative priors can greatly improve efficiency of parameter estimates and stabilize estimation. There are a variety of methods available to evaluate model fit in a Bayesian framework; however, past work investigating Bayesian model fit assessment for IFA models has assumed flat priors, which have no advantage over ML in limited data settings. In this paper, we evaluated the impact of moderately informative priors on ability to detect model misfit for several candidate indices: posterior predictive checks based on the observed score distribution, leave-one-out cross-validation, and widely available information criterion (WAIC). We found that although Bayesian estimation with moderately informative priors is an excellent aid for estimating challenging IFA models, methods for testing model fit in these circumstances are inadequate.

A guide to Bayesian model checking for ecologists

AbstractChecking that models adequately represent data is an essential component of applied statistical inference. Ecologists increasingly use hierarchical Bayesian statistical models in their research. The appeal of this modeling paradigm is undeniable, as researchers can build and fit models that embody complex ecological processes while simultaneously accounting for observation error. However, ecologists tend to be less focused on checking model assumptions and assessing potential lack of fit when applying Bayesian methods than when applying more traditional modes of inference such as maximum likelihood. There are also multiple ways of assessing the fit of Bayesian models, each of which has strengths and weaknesses. For instance, BayesianPvalues are relatively easy to compute, but are well known to be conservative, producingPvalues biased toward 0.5. Alternatively, lesser known approaches to model checking, such as prior predictive checks, cross‐validation probability integral transforms, and pivot discrepancy measures may produce more accurate characterizations of goodness‐of‐fit but are not as well known to ecologists. In addition, a suite of visual and targeted diagnostics can be used to examine violations of different model assumptions and lack of fit at different levels of the modeling hierarchy, and to check for residual temporal or spatial autocorrelation. In this review, we synthesize existing literature to guide ecologists through the many available options for Bayesian model checking. We illustrate methods and procedures with several ecological case studies including (1) analysis of simulated spatiotemporal count data, (2) N‐mixture models for estimating abundance of sea otters from an aircraft, and (3) hidden Markov modeling to describe attendance patterns of California sea lion mothers on a rookery. We find that commonly used procedures based on posterior predictivePvalues detect extreme model inadequacy, but often do not detect more subtle cases of lack of fit. Tests based on cross‐validation and pivot discrepancy measures (including the “sampled predictivePvalue”) appear to be better suited to model checking and to have better overall statistical performance. We conclude that model checking is necessary to ensure that scientific inference is well founded. As an essential component of scientific discovery, it should accompany most Bayesian analyses presented in the literature.

Making the most of sparse data to estimate density of a rare and threatened species: a case study with the fosa, a little‐studied Malagasy carnivore

AbstractSparse detections in camera trap surveys commonly hinder density estimation for threatened species. By combining detections across multiple surveys, or using informative priors in Bayesian model fitting, researchers can improve parameter estimation from sparse capture–recapture data. Using a spatial mark–resight model that incorporates site‐level heterogeneity in the spatial scale parameter via a hierarchical process and prior information, we estimated the density of a threatened carnivore (fosa, Cryptoprocta ferox) from multiple sparse datasets collected during extensive camera trapping surveys in northeastern Madagascar (2008–2015). Our objectives were to estimate density for six sites, examine the response of fosa density and movement to habitat degradation, monitor annual density trends across 7 years at two sites, and estimate fosa abundance in the Makira–Masoala protected area complex. We obtained a mean of 16.1 (se = 0.52; range = 2–49) fosa detections and three observers identified a mean of 3.62 (se = 0.09; range = 1–8) marked individuals per survey. Fosa daily baseline encounter rate was very low (λ0 = 0.004; 0.003–0.006) and density/movement estimates were similar across forest types. Density estimates at resurveyed sites suggested annual variability in density, with estimates trending lower during the final surveys [e.g. D = 0.39 (0.14–1.11) versus 0.08 (0.05–0.31) individuals per km2]. We estimated fosa abundance across the Makira–Masoala region to be 1061 (95% HPDI: 596–1780) adult individuals. On the basis of our estimate and the size of the region, we believe Makira–Masoala harbors a significant portion of the global fosa population. The conservation and management of rare species is commonly limited due to lack of population estimates. By combining detections across surveys, we overcame estimation issues and obtained valuable information on a threatened carnivore, allowing us to better assess its status and prioritize conservation actions. We advocate for practical use of sparse datasets for such data‐deficient species.