Prior Specification Research Articles

Bayesian hierarchical modeling: an introduction and reassessment

With the recent development of easy-to-use tools for Bayesian analysis, psychologists have started to embrace Bayesian hierarchical modeling. Bayesian hierarchical models provide an intuitive account of inter- and intraindividual variability and are particularly suited for the evaluation of repeated-measures designs. Here, we provide guidance for model specification and interpretation in Bayesian hierarchical modeling and describe common pitfalls that can arise in the process of model fitting and evaluation. Our introduction gives particular emphasis to prior specification and prior sensitivity, as well as to the calculation of Bayes factors for model comparisons. We illustrate the use of state-of-the-art software programs Stan and brms. The result is an overview of best practices in Bayesian hierarchical modeling that we hope will aid psychologists in making the best use of Bayesian hierarchical modeling.

Evaluating false positive rates of standard and hierarchical measures of metacognitive accuracy

A key aspect of metacognition is metacognitive accuracy, i.e., the degree to which confidence judgments differentiate between correct and incorrect trials. To quantify metacognitive accuracy, researchers are faced with an increasing number of different methods. The present study investigated false positive rates associated with various measures of metacognitive accuracy by hierarchical resampling from the confidence database to accurately represent the statistical properties of confidence judgements. We found that most measures based on the computation of summary-statistics separately for each participant and subsequent group-level analysis performed adequately in terms of false positive rate, including gamma correlations, meta-d′, and the area under type 2 ROC curves. Meta-d′/d′ is associated with a false positive rate even below 5%, but log-transformed meta-d′/d′ performs adequately. The false positive rate of HMeta-d depends on the study design and on prior specification: For group designs, the false positive rate is above 5% when independent priors are placed on both groups, but the false positive rate is adequate when a prior was placed on the difference between groups. For continuous predictor variables, default priors resulted in a false positive rate below 5%, but the false positive rate was not distinguishable from 5% when close-to-flat priors were used. Logistic mixed model regression analysis is associated with dramatically inflated false positive rates when random slopes are omitted from model specification. In general, we argue that no measure of metacognitive accuracy should be used unless the false positive rate has been demonstrated to be adequate.

Order-based structure learning without score equivalence

Summary We propose an empirical Bayes formulation of the structure learning problem, where the prior specification assumes that all node variables have the same error variance, an assumption known to ensure the identifiability of the underlying causal directed acyclic graph. To facilitate efficient posterior computation, we approximate the posterior probability of each ordering by that of a best directed acyclic graph model, which naturally leads to an order-based Markov chain Monte Carlo algorithm. Strong selection consistency for our model in high-dimensional settings is proved under a condition that allows heterogeneous error variances, and the mixing behaviour of our sampler is theoretically investigated. Furthermore, we propose a new iterative top-down algorithm, which quickly yields an approximate solution to the structure learning problem and can be used to initialize the Markov chain Monte Carlo sampler. We demonstrate that our method outperforms other state-of-the-art algorithms under various simulation settings, and conclude the paper with a single-cell real-data study illustrating practical advantages of the proposed method.

Mkite: A distributed computing platform for high-throughput materials simulations

Advances in high-throughput simulation (HTS) software enabled computational databases and big data to become common resources in materials science. However, while computational power is increasingly larger, software packages orchestrating complex workflows in heterogeneous environments are scarce. This paper introduces mkite, a Python package for performing HTS in distributed computing environments. The mkite toolkit is built with the client–server pattern, decoupling production databases from client runners. When used in combination with message brokers, mkite enables any available client to perform calculations without prior hardware specification on the server side. Furthermore, the software enables the creation of complex workflows with multiple inputs and branches, facilitating the exploration of combinatorial chemical spaces. Software design principles are discussed in detail, highlighting the usefulness of decoupling simulations and data management tasks to diversify simulation environments. To exemplify how mkite handles simulation workflows of combinatorial systems, case studies on zeolite synthesis and surface catalyst discovery are provided. Finally, key differences with other atomistic simulation workflows are outlined. The mkite suite can enable HTS in distributed computing environments, simplifying workflows with heterogeneous hardware and software, and helping deployment of calculations at scale.

Evidential Calibration of Confidence Intervals

We present a novel and easy-to-use method for calibrating error-rate based confidence intervals to evidence-based support intervals. Support intervals are obtained from inverting Bayes factors based on a parameter estimate and its standard error. A k support interval can be interpreted as “the observed data are at least k times more likely under the included parameter values than under a specified alternative.” Support intervals depend on the specification of prior distributions for the parameter under the alternative, and we present several types that allow different forms of external knowledge to be encoded. We also show how prior specification can to some extent be avoided by considering a class of prior distributions and then computing so-called minimum support intervals which, for a given class of priors, have a one-to-one mapping with confidence intervals. We also illustrate how the sample size of a future study can be determined based on the concept of support. Finally, we show how the bound for the Type I error rate of Bayes factors leads to a bound for the coverage of support intervals. An application to data from a clinical trial illustrates how support intervals can lead to inferences that are both intuitive and informative.

A Hierarchical Expected Improvement Method for Bayesian Optimization

The Expected Improvement (EI) method, proposed by Jones, Schonlau, andWelch, is a widely used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model for optimization, which results in suboptimal solutions even with large sample sizes. To address this, we propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration of the optimization space. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian optimization procedure, while avoiding expensive Markov-chain Monte Carlo sampling steps. We prove the global convergence of HEI over a broad function space, and establish near-minimax convergence rates under certain prior specifications. Numerical experiments show the improvement of HEI over existing Bayesian optimization methods, for synthetic functions and a semiconductor manufacturing optimization problem. Supplementary materials for this article are available online.

Use of days alive without life support and similar count outcomes in randomised clinical trials – an overview and comparison of methodological choices and analysis methods

BackgroundDays alive without life support (DAWOLS) and similar outcomes that seek to summarise mortality and non-mortality experiences are increasingly used in critical care research. The use of these outcomes is challenged by different definitions and non-normal outcome distributions that complicate statistical analysis decisions.MethodsWe scrutinized the central methodological considerations when using DAWOLS and similar outcomes and provide a description and overview of the pros and cons of various statistical methods for analysis supplemented with a comparison of these methods using data from the COVID STEROID 2 randomised clinical trial. We focused on readily available regression models of increasing complexity (linear, hurdle-negative binomial, zero–one-inflated beta, and cumulative logistic regression models) that allow comparison of multiple treatment arms, adjustment for covariates and interaction terms to assess treatment effect heterogeneity.ResultsIn general, the simpler models adequately estimated group means despite not fitting the data well enough to mimic the input data. The more complex models better fitted and thus better replicated the input data, although this came with increased complexity and uncertainty of estimates. While the more complex models can model separate components of the outcome distributions (i.e., the probability of having zero DAWOLS), this complexity means that the specification of interpretable priors in a Bayesian setting is difficult.Finally, we present multiple examples of how these outcomes may be visualised to aid assessment and interpretation.ConclusionsThis summary of central methodological considerations when using, defining, and analysing DAWOLS and similar outcomes may help researchers choose the definition and analysis method that best fits their planned studies.Trial registrationCOVID STEROID 2 trial, ClinicalTrials.gov: NCT04509973, ctri.nic.in: CTRI/2020/10/028731.

Intersectional sociodemographic and neurological relationships in the naming ability of persons with post-stroke aphasia

IntroductionSignificant attention has been given to the role of brain function and disruption in determining performance on naming tasks among individuals with aphasia. However, scholarly pursuit of a neurological explanation has overlooked the fundamental cornerstone of individual health—the underlying social, economic, and environmental factors that shape how they live, work, and age, also known as the social determinants of health (SDOH). This study examines the correlation between naming performance and these underlying factors. MethodsIndividual level data from the 2010 Moss Aphasia Psycholinguistic Project Database (MAPPD) was matched with the 2009–2011 Medical Expenditure Panel Survey (MEPS) using a propensity score algorithm based on functional, health, and demographic characteristics. Multilevel, generalized, nonlinear regression models were applied to the resulting data set to assess the correlation between the Boston Naming Test (BNT) percentile score and age, income, sex, race, household size, marital status, aphasia type, and region of residence. Poisson regression models with bootstrapped standard errors were used to estimate these relationships ResultsDiscrete dependent variable estimation with non-normal prior specification included individual level (age, marital status, years of education), socioeconomic (family income), health (aphasia type), household (family size), and environmental (region of residence) characteristics. Regression results indicated that, relative to individuals with Wernicke's, individuals with Anomic (0.74, SE = 0.0008) and Conduction (0.42, SE = 0.0009) aphasia performed better on the BNT. While age at the time of testing was not significantly correlated, higher income level (0.15, SE = 0.0003) and larger family size (0.002, SE = 0.002) was associated with higher BNT score percentiles. Finally, Black persons with aphasia (PWA) (-0.0124, SE = 0.0007) had lower average score percentiles when other factors were held constant. ConclusionsThe findings reported here suggest higher income and larger family size are associated with better outcomes. As expected, aphasia type was significantly associated with naming outcomes. However, poorer performance by Black PWA and individuals with low income suggests that SDOH can play a critical role (positive and negative) in naming impairment in some populations with aphasia.

Bayesian prior modeling in vector autoregressions via the Yule-Walker equations

In multivariate time series analysis, the Yule-Walker method refers to a system of equations relating the cross-covariances of a stationary vector autoregressive (VAR) model with the matrices of the autoregressive coefficients and the covariance matrix of the noise, both of which are unknown to be estimated. In Bayesian inference of VAR models, one of the key problems is the setting of the prior distributions on these unknown parameters. The Yule-Walker equations are used here to develop a novel prior specification that exploits the reparameterization of the unknowns in terms of the mean, the cross-covariances, and the covariance of the process. Further, the cross-covariance matrices are separated out in terms of the standard deviations and the correlations. All these new quantities are easier to handle because it is more common to have prior information on the mean and the correlation structure of a multiple time series rather than the underlying autoregressive coefficients and the white noise process. The proposed prior specification is suitable for both non informative and informative settings. Through the Yule-Walker based prior, parameter estimation and structure learning of the stationary VAR models are performed via Markov chain Monte Carlo methods. The methodology is illustrated via some synthetic data sets, a benchmark example, and an environmental time series.

Incorporating biological knowledge in analyses of environmental mixtures and health.

A key goal of environmental health research is to assess the risk posed by mixtures of pollutants. As epidemiologic studies of mixtures can be expensive to conduct, it behooves researchers to incorporate prior knowledge about mixtures into their analyses. This work extends the Bayesian multiple index model (BMIM), which assumes the exposure-response function is a nonparametric function of a set of linear combinations of pollutants formed with a set of exposure-specific weights. The framework is attractive because it combines the flexibility of response-surface methods with the interpretability of linear index models. We propose three strategies to incorporate prior toxicological knowledge into construction of indices in a BMIM: (a) imposing directional homogeneity constraints on the weights, (b) structuring index weights by exposure transformations, and (c) placing informative priors on the index weights. We propose a novel prior specification that combines spike-and-slab variable selection with an informative Dirichlet distribution based on relative potency factors often derived from previous toxicological studies. In simulations we show that the proposed priors improve inferences when prior information is correct and can protect against misspecification suffered by naïve toxicological models when prior information is incorrect. Moreover, different strategies may be mixed-and-matched for different indices to suit available information (or lack thereof). We demonstrate the proposed methods on an analysis of data from the National Health and Nutrition Examination Survey and incorporate prior information on relative chemical potencies obtained from toxic equivalency factors available in the literature.

Applying Monte Carlo Simulations to a Small Data Analysis of a Case of Economic Growth in COVID-19 Times.

Studies on the going-on COVID-19 pandemic face small sample issues. In this context, Bayesian estimation is considered a viable alternative to frequentist estimation. Demonstrating the Bayesian approach's advantage in dealing with this problem, our research conducted a case study concerning ASEAN economic growth during the COVID-19 pandemic. By using Monte Carlo standard errors and interval hypothesis testing to check parameter bias within a Bayesian MCMC simulation study, the author obtained significant conclusions as follows: first, in insufficient sample sizes, in contrast to frequentist estimation, the Bayesian framework can offer meaningful results, that is, expansionary monetary and contractionary fiscal policies are positively associated with economic growth; second, in the face of a small sample, by incorporating more information into prior distributions for the model parameters, Bayesian Monte Carlo simulations perform so far better than naïve Bayesian and frequentist estimation; third, in case of a correctly specified prior, the inferences are robust to different prior specifications. The author strongly recommends applying specific informative priors to Bayesian analyses, particularly in small sample investigations.

Bayesian Analysis of Multivariate Matched Proportions with Sparse Response

Multivariate matched proportions (MMP) data appear in a variety of contexts including post-market surveillance of adverse events in pharmaceuticals, disease classification, and agreement between care providers. It consists of multiple sets of paired binary measurements taken on the same subject. While recent work proposes methods to address the complexities of MMP data, the issue of sparse response, where no or very few “yes” responses are recorded for one or more sets, is unaddressed. The presence of sparse response sets results in the underestimation of variance components, loss of coverage, and lowered power in existing methods. Bayesian methods, which have not previously been considered for MMP data, provide a useful framework when sparse responses are present. In particular, the Bayesian probit model in combination with mean model prior specifications provides an elegant solution to the problem of variance underestimation. We examine a multivariate probit-based approach using hierarchical horseshoe-like priors along with a Bayesian functional principal component analysis (FPCA) to model the latent covariance. We show that our approach performs well on MMP data with sparse responses and outperforms existing methods. In a re-examination of a study on the system of care (SOC) framework for children with mental and behavioral disorders, we are able to provide a more complete picture of the relationships in the data. Our analysis provides additional insights into the functioning on the SOC that a previous univariate analysis missed.

Bayesian cluster analysis.

Bayesian cluster analysis offers substantial benefits over algorithmic approaches by providing not only point estimates but also uncertainty in the clustering structure and patterns within each cluster. An overview of Bayesian cluster analysis is provided, including both model-based and loss-based approaches, along with a discussion on the importance of the kernel or loss selected and prior specification. Advantages are demonstrated in an application to cluster cells and discover latent cell types in single-cell RNA sequencing data to study embryonic cellular development. Lastly, we focus on the ongoing debate between finite and infinite mixtures in a model-based approach and robustness to model misspecification. While much of the debate and asymptotic theory focuses on the marginal posterior of the number of clusters, we empirically show that quite a different behaviour is obtained when estimating the full clustering structure. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Bayesian model comparison for mortality forecasting

Abstract Stochastic models are appealing for mortality forecasting in their ability to generate intervals that quantify uncertainties underlying the forecasts. We present a fully Bayesian implementation of the age-period-cohort-improvement (APCI) model with overdispersion, which is compared with the Lee–Carter model with cohorts. We show that naive prior specification can yield misleading inferences, where we propose Laplace prior as an elegant solution. We also perform model averaging to incorporate model uncertainty. Our findings indicate that the APCI model offers better fit and forecast for England and Wales data spanning 1961–2002. Our approach also allows coherent inclusion of multiple sources of uncertainty, producing well-calibrated probabilistic intervals.

Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics (with Discussion)

Bayesian models based on the Dirichlet process and other stick-breaking priors have been proposed as core ingredients for clustering, topic modeling, and other unsupervised learning tasks. However, due to the flexibility of these models, the consequences of prior choices can be opaque. And so prior specification can be relatively difficult. At the same time, prior choice can have a substantial effect on posterior inferences. Thus, considerations of robustness need to go hand in hand with nonparametric modeling. In the current paper, we tackle this challenge by exploiting the fact that variational Bayesian methods, in addition to having computational advantages in fitting complex nonparametric models, also yield sensitivities with respect to parametric and nonparametric aspects of Bayesian models. In particular, we demonstrate how to assess the sensitivity of conclusions to the choice of concentration parameter and stick-breaking distribution for inferences under Dirichlet process mixtures and related mixture models. We provide both theoretical and empirical support for our variational approach to Bayesian sensitivity analysis.

Generalized fiducial factor: An alternative to the Bayes factor for forensic identification of source problems

One formulation of forensic identification of source problems is to determine the source of trace evidence, for instance, glass fragments found on a suspect for a crime. The current state of the science is to compute a Bayes factor comparing the marginal distribution of measurements of trace evidence under two competing propositions for whether or not the unknown source evidence originated from a specific source. The obvious problem with such an approach is the ability to tailor the prior distributions (placed on the features/parameters of the statistical model for the measurements of trace evidence) in favor of the defense or prosecution which is further complicated by the fact that the typical number of measurements of trace evidence is typically sufficiently small that prior choice/specification has a strong influence on the value of the Bayes factor. To remedy this problem of prior specification and choice, we develop an alternative to the Bayes factor, within the framework of generalized fiducial inference, that we term a generalized fiducial factor. Furthermore, we demonstrate empirically, on synthetic and real Netherlands Forensic Institute casework data, deficiencies in Bayes factor and classical/frequentist likelihood ratio approaches.

Illustrating the Value of Prior Predictive Checking for Bayesian Structural Equation Modeling

A unique feature of Bayesian estimation is the inclusion of prior knowledge through prior distributions. These prior distributions can benefit or impair many components of the ensuing analysis. Priors are especially important to assess in the context of structural equation models (SEMs), which often carry data and modeling complexities where priors can be particularly influential. In this article, we illustrate a statistical approach to assess the impact of our prior specifications: the prior predictive checking procedure. We introduce a comprehensive prior predictive checking workflow that organizes the procedure into clear steps, and we relate this workflow to the SEM framework. Through three examples, we demonstrate how the workflow can aid researchers in more fully understanding different aspects of their prior specification within SEM. Code and additional resources are provided in the online Supplemental Materials to facilitate future application of the prior predictive checking procedure within the SEM framework.

Joint distribution properties of fully conditional specification under the normal linear model with normal inverse-gamma priors

Fully conditional specification (FCS) is a convenient and flexible multiple imputation approach. It specifies a sequence of simple regression models instead of a potential complex joint density for missing variables. However, FCS may not converge to a stationary distribution. Many authors have studied the convergence properties of FCS when priors of conditional models are non-informative. We extend to the case of informative priors. This paper evaluates the convergence properties of the normal linear model with normal-inverse gamma priors. The theoretical and simulation results prove the convergence of FCS and show the equivalence of prior specification under the joint model and a set of conditional models when the analysis model is a linear regression with normal inverse-gamma priors.

Bayesian hierarchical models for adaptive basket trial designs.

Basket trials evaluate a single drug targeting a single genetic variant in multiple cancer cohorts. Empirical findings suggest that treatment efficacy across baskets may be heterogeneous. Most modern basket trial designs use Bayesian methods. These methods require the prior specification of at least one parameter that permits information sharing across baskets. In this study, we provide recommendations for selecting a prior for scale parameters for adaptive basket trials by using Bayesian hierarchical modeling. Heterogeneity among baskets attracts much attention in basket trial research, and substantial heterogeneity challenges the basic assumption of exchangeability of Bayesian hierarchical approach. Thus, we also allowed each stratum-specific parameter to be exchangeable or nonexchangeable with similar strata by using data observed in an interim analysis. Through a simulation study, we evaluated the overall performance of our design based on statistical power and type I error rates. Our research contributes to the understanding of the properties of Bayesian basket trial designs.

DIVI: Dynamically Interactive Visualization.

Dynamically Interactive Visualization (DIVI) is a novel approach for orchestrating interactions within and across static visualizations. DIVI deconstructs Scalable Vector Graphics charts at runtime to infer content and coordinate user input, decoupling interaction from specification logic. This decoupling allows interactions to extend and compose freely across different tools, chart types, and analysis goals. DIVI exploits positional relations of marks to detect chart components such as axes and legends, reconstruct scales and view encodings, and infer data fields. DIVI then enumerates candidate transformations across inferred data to perform linking between views. To support dynamic interaction without prior specification, we introduce a taxonomy that formalizes the space of standard interactions by chart element, interaction type, and input event. We demonstrate DIVI's usefulness for rapid data exploration and analysis through a usability study with 13 participants and a diverse gallery of dynamically interactive visualizations, including single chart, multi-view, and cross-tool configurations.