Bayes Factor Test Research Articles

Default Bayes Factors for Testing the (In)equality of Several Population Variances

Testing the (in)equality of variances is an important problem in many statistical applications. We develop default Bayes factor tests to assess the (in)equality of two or more population variances, as well as a test for whether a population variance equals a specific value. The resulting test can be used to check assumptions for commonly used procedures such as the t-test or ANOVA, or test substantive hypotheses concerning variances directly. We show that our Bayes factor fulfills a number of desiderata. Researchers may have directed hypotheses such as σ12>σ22, they may want to extend H0 to have a null-region, or wish to combine hypotheses about equality with hypotheses about inequality, for example σ12=σ22>(σ32,σ42). We extend our Bayes factor test to allow for these deviations from our proposed default and illustrate it on a number of practical examples. Our procedure is implemented in the R package bfvartest.

The Bayes factor, HDI-ROPE, and frequentist equivalence tests can all be reverse engineered-Almost exactly-From one another: Reply to Linde et al. (2021).

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST," the Bayesian "HDI-ROPE," and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence" (p. 1). We redo the simulation study of Linde et al. (2021) in its entirety but with the different procedures calibrated to have the same predetermined maximum Type I error rate. Our results suggest that, when calibrated in this way, the Bayes factor, HDI-ROPE, and frequentist equivalence tests all have similar-almost exactly-Type II error rates. In general any advocating for frequentist testing as better or worse than Bayesian testing in terms of empirical findings seems dubious at best. If one decides on which underlying principle to subscribe to in tackling a given problem, then the method follows naturally. Bearing in mind that each procedure can be reverse-engineered from the others (at least approximately), trying to use empirical performance to argue for 1 approach over another seems like tilting at windmills. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Safety and Tolerability of tDCS across Different Ages, Sexes, Diagnoses, and Amperages: A Randomized Double-Blind Controlled Study.

Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique with substantial evidence for its safety and tolerability in adults. However, less than 5% of published tDCS research is in pediatrics. Our primary objective was to investigate tDCS safety, tolerability, and acceptability in a sample of children and adults. We hypothesized that children and adults would be equal with regard to tDCS safety, tolerability, and acceptability. We tested this hypothesis using a Bayesian approach. Sixty participants aged 6-45 (balanced for sex) participated in a randomized double-blind controlled trial. They were randomly assigned to two ten-minute tDCS sessions with varying amperages and electrode locations. The primary outcome measure of this study was the intensity of 13 potential side effects evaluated at six different time points spanning two weeks. Independent sample Bayes factor tests were conducted between children/adults, males/females, clinical/healthy, and low/high amperage groups. As predicted, there was moderate support for the null hypothesis in all between-group analyses. There were no serious adverse events or dropouts, and the number needed to treat for an additional harmful outcome was 23. This study provided evidence supporting the overall short-term safety, tolerability, and acceptability of tDCS including amperages up to 2 mA and different electrode placements.

Examining Measurement Invariance in Bayesian Item Response Theory Models: A Simulation Study

The aim of the study is to determine a measurement invariance cut-off point based on item parameter differences in Bayesian Item Response Theory Models. Within this scope, the Bayes factor is estimated for testing measurement invariance. For this purpose, a simulation study is conducted. The data were generated in the R software for each simulation condition under the one-parameter logistic model for 10 binary (1-0 scored) items. The invariance test was performed for various group sizes (n=500, 1000, 1500 and 2000) and difficulty parameters (dk=0, dk=0.1, dk=0.3, dk=0.5 and dk=0.7). The Bayesian analyzes were performed on the WINBUGS using the codes written in the R. A Bayes factor that provides evidence for measurement invariance was calculated depending on the parameter differences. The Savage–Dickey density ratio, one of the MCMC sampling schemas, was used to calculate the Bayes factor. As a result, if the item parameter difference is dk=0.3 and group sizes are 1500 or larger, the measurement invariance cannot be achieved. However, for small sample sizes (n=1000 or less) measurement invariance interpretation should be done carefully. When the dk=0.5, there are invariant items only in n=500. According to Bayes factor test results, evidence has been produced when dk=0.7 that measurement invariance cannot be achieved.

Objective Bayesian Approach for Binary Hypothesis Testing of Multivariate Gaussian Observations

This paper proposes an objective Bayesian detector for the binary hypothesis testing problem in which the observation vectors are IID and follow multivariate Gaussian distribution under both the null and alternative hypotheses. The mean vector and covariance matrix under each hypothesis are not known, they also do not have special structure other than the covariance matrices are partially ordered in the positive definite cone with their difference positive semi-definite. An example of such a problem is the detection of Gaussian random signal in Gaussian noise. Non-informative priors are imposed on the unknown parameters for computing the marginals and evaluating the Bayes factor test statistic, where the prior for the covariance matrix is applied to the Cholesky factorization of the precision matrix. For the null hypothesis, we propose conjugate priors for the unknowns. For the alternative hypothesis, we propose uniform prior on the mean vector, and a class of objective priors that encompasses the Jeffreys, Independence Jeffreys, Geisser and Cornfield, Haar Measure and Reference priors for the precision matrix. The proposed priors enable closed-form expressions for the marginals and lead to an objective Bayes factor having the posteriors mainly governed by the data. The proposed detector is analyzed for justifying the priors used, deriving the theoretical moments, and approximating the false alarm and detection probability densities by the shifted Gamma distributions. The developed detector exhibits improvement in detection performance over the energy detector (ED) and the generalized likelihood ratio test (GLRT).

Efficiency in sequential testing: Comparing the sequential probability ratio test and the sequential Bayes factor test

In a sequential hypothesis test, the analyst checks at multiple steps during data collection whether sufficient evidence has accrued to make a decision about the tested hypotheses. As soon as sufficient information has been obtained, data collection is terminated. Here, we compare two sequential hypothesis testing procedures that have recently been proposed for use in psychological research: Sequential Probability Ratio Test (SPRT; Psychological Methods, 25(2), 206–226, 2020) and the Sequential Bayes Factor Test (SBFT; Psychological Methods, 22(2), 322–339, 2017). We show that although the two methods have different philosophical roots, they share many similarities and can even be mathematically regarded as two instances of an overarching hypothesis testing framework. We demonstrate that the two methods use the same mechanisms for evidence monitoring and error control, and that differences in efficiency between the methods depend on the exact specification of the statistical models involved, as well as on the population truth. Our simulations indicate that when deciding on a sequential design within a unified sequential testing framework, researchers need to balance the needs of test efficiency, robustness against model misspecification, and appropriate uncertainty quantification. We provide guidance for navigating these design decisions based on individual preferences and simulation-based design analyses.

Bayes factor testing of equality and order constraints on measures of association in social research

Measures of association play a central role in the social sciences to quantify the strength of a linear relationship between the variables of interest. In many applications researchers can translate scientific expectations to hypotheses with equality and/or order constraints on these measures of association. In this paper a Bayes factor test is proposed for testing multiple hypotheses with constraints on the measures of association between ordinal and/or continuous variables, possibly after correcting for certain covariates. This test can be used to obtain a direct answer to the research question how much evidence there is in the data for a social science theory relative to competing theories. The stand-alone software package ‘BCT’ allows users to apply the methodology in an easy manner. The methodology will also be available in the R package ‘BFpack’. An empirical application from leisure studies about the associations between life, leisure and relationship satisfaction and an application about the differences about egalitarian justice beliefs across countries are used to illustrate the methodology.

On the Measure-Theoretic Premises of Bayes Factor and Full Bayesian Significance Tests: a Critical Reevaluation

The Full Bayesian Significance Test (FBST) and the Bayesian evidence value recently have received increasing attention across a variety of sciences including psychology. Ly and Wagenmakers (2021) have provided a critical evaluation of the method and concluded that it suffers from four problems which are mostly attributed to the asymptotic relationship of the Bayesian evidence value to the frequentist p-value. While Ly and Wagenmakers (2021) tackle an important question about the best way of statistical hypothesis testing in the cognitive sciences, it is shown in this paper that their arguments are based on a specific measure-theoretic premise. The identified problems hold only under a specific class of prior distributions which are required only when adopting a Bayes factor test. However, the FBST explicitly avoids this premise, which resolves the problems in practical data analysis. In summary, the analysis leads to the more important question whether precise point null hypotheses are realistic for scientific research, and a shift towards the Hodges-Lehmann paradigm may be an appealing solution when there is doubt on the appropriateness of a precise hypothesis.

The Molecular Epidemiology and Transmission Dynamics of HIV Type 1 in a General Population Cohort in Uganda.

The General Population Cohort (GPC) in south-western Uganda has a low HIV-1 incidence rate (<1%). However, new infections continue to emerge. In this research, 3796 HIV-1 pol sequences (GPC: n = 1418, non-GPC sites: n = 1223, Central Uganda: n = 1010 and Eastern Uganda: n = 145) generated between 2003–2015 were analysed using phylogenetic methods with demographic data to understand HIV-1 transmission in this cohort and inform the epidemic response. HIV-1 subtype A1 was the most prevalent strain in the GPC area (GPC and non-GPC sites) (39.8%), central (45.9%) and eastern (52.4%) Uganda. However, in the GPC alone, subtype D was the predominant subtype (39.1%). Of the 524 transmission clusters identified by Cluster Picker, all large clusters (≥5 individuals, n = 8) involved individuals from the GPC. In a multivariate analysis, clustering was strongly associated with being female (adjusted Odds Ratio, aOR = 1.28; 95% CI, 1.06–1.54), being >25 years (aOR = 1.52; 95% CI, 1.16–2.0) and being a resident in the GPC (aOR = 6.90; 95% CI, 5.22–9.21). Phylogeographic analysis showed significant viral dissemination (Bayes Factor test, BF > 3) from the GPC without significant viral introductions (BF < 3) into the GPC. The findings suggest localized HIV-1 transmission in the GPC. Intensifying geographically focused combination interventions in the GPC would contribute towards controlling HIV-1 infections.

There is Nothing Magical about Bayesian Statistics: An Introduction to Epistemic Probabilities in Data Analysis for Psychology Starters

This paper is a reader-friendly introduction to Bayesian inference applied to psychological science. We begin by explaining the difference between frequentist and epistemic interpretations of probability that underpin respectively frequentist and Bayesian statistics. We use a concrete example—a student wondering whether s/he carries the virus statisticus malignum—to explain how both approaches are different one from another. We illustrate Bayesian inference with intuitive examples, before introducing the mathematical framework. Different schools of thoughts and recommendations are discussed to illustrate how to use priors in Bayes Factor testing. We discuss how psychology could benefit from a greater reliance on Bayesian methods. Finally, we illustrate how to compute Bayes Factors analyses with real data and provide the R code.

Macroeconomic Growth in Vietnam Transitioned to Market: An Unrestricted VES Framework

The Vietnamese economy has increased at high speed over the transformation decades; however, most recent studies on the economic growth of this country used the Cobb-Douglas or CES (Constant Elasticity of Substitution) production functions, which are unable to explore the relationship between the elasticity of capital-labour substitution and development process, and hence, are not relevant to accessing a dynamic economic system. For that reason, this study is conducted to specify an unrestricted VES (Variable Elasticity of Substitution) production function in a one-sector growth model of Vietnam, highlighted by two characteristics: successful transition from plan to market and rapid progress. The VES is given preference over the CES and the Cobb-Douglas having the elasticity of substitution between capital and labour varying with economic development. By employing a Bayesian nonlinear regression through MCMC methods, the study reported the following findings: (1) the above-unity variable elasticity of capital-labour substitution in an aggregate unrestricted VES function specified for Vietnam shows that the model generates the possibility of endogenous economic growth; (2) the capital share tends to increase, while the labour share faces a downward trend along with the development of Vietnam; (3) the VES is empirically proven through a Bayes factor test to be superior to the CES and Cobb-Douglas for analysis of the growth process of Vietnam, an emerging transition economy.

Valid p-values and expectations of p-values revisited

We focus on valid definitions of p-values. A valid p-value (VpV) statistic can be used to make a prefixed level- $$ \alpha $$ decision. In this context, Kolmogorov–Smirnov goodness-of-fit tests and the normal two-sample problem are considered. We examine an issue regarding the goodness-of-fit testability based on a single observation. We exemplify constructions of new test procedures, advocating practical reasons to implement VpV mechanisms. The VpV framework induces an extension of the conventional expected p-value (EPV) tool for measuring the performance of a test. Associating the EPV concept with the receiver operating characteristic (ROC) curve methodology, a well-established biostatistical approach, we propose a Youden’s index-based optimality to derive critical values of tests. In these terms, the significance level $$ \alpha = 0.05 $$ is suggested. We introduce partial EPV’s to characterize properties of tests including their unbiasedness. We provide the intrinsic relationship between the Bayes Factor (BF) test statistic and the BF of test statistics.

Reexamining the Effect of Gustatory Disgust on Moral Judgment: A Multilab Direct Replication of Eskine, Kacinik, and Prinz (2011)

Eskine, Kacinik, and Prinz’s (2011) influential experiment demonstrated that gustatory disgust triggers a heightened sense of moral wrongness. We report a large-scale multisite direct replication of this study conducted by labs in the Collaborative Replications and Education Project. Subjects in each sample were randomly assigned to one of three beverage conditions: bitter (disgusting), control (neutral), or sweet. Then, subjects made a series of judgments about the moral wrongness of the behavior depicted in six vignettes. In the original study ( N = 57), drinking the bitter beverage led to higher ratings of moral wrongness than did drinking the control or sweet beverage; a contrast between the bitter condition and the other two conditions was significant among conservative ( n = 19) but not liberal ( n = 25) subjects. In the current project, random-effects meta-analyses across all subjects ( N = 1,137, k = 11 studies), conservative subjects ( n = 142, k = 5), and liberal subjects ( n = 635, k = 9) revealed standardized overall effect sizes across replications that were smaller than reported in the original study. Some were in the opposite of the predicted direction; all had 95% confidence intervals containing zero, and all were smaller than the effect size the original authors could have meaningfully detected. Results of linear mixed-effects regressions revealed that drinking the bitter beverage led to higher ratings of moral wrongness than did drinking the control beverage but not the sweet beverage. Bayes factor tests revealed greater relative support for the null than for the replication hypothesis. The overall pattern provides little to no support for the theory that physical disgust via taste perception harshens judgments of moral wrongness.

Research waste: why we need to rethink meta-analysis

A key factor in reducing Research Waste is to make the scientific process more efficient by taking previous studies into account in prioritizing, designing and interpreting new research. However, conventional methods for meta-analysis do not allow that study series size and meta-analysis timing depend on previous results in the study series. Hence, our methods do not permit that promising initial results are more likely to develop into (large) series of studies than their disappointing counterparts or that conclusive studies are more likely to trigger a meta-analysis than not so noteworthy findings. Since efficient accumulation of scientific knowledge needs such dependencies, it introduces Accumulation Bias, a term introduced in this paper to study all possible dependencies potentially involved in meta-analysis. Fortunately, all dependencies characterized by our Accumulation Bias framework are statistically manageable by testing meta-analyses with Safe Tests. These tests are an extension of the Test Martingales introduced in Shafer, Shen, Vereshchagin & Vovk (2011) and they include some Bayes Factor tests (e.g. Bayesian T-test), but certainly not all. Safe Tests are very flexible towards many efficient decision procedures that expose meta-analyses to new dependencies. Safe Tests are easily interpretable in comparison to standard nullhypothesis significance testing. Safe Tests also pave the way to interesting research into Safe Estimation that counteracts empirical phenomena like inflated, Proteus, and fading effects in meta-analysis. We introduce Safe Tests for meta-analysis that are ready to use and thus allow valid assessment of previous studies to reduce Research Waste.

Bayes Factor Testing of Multiple Intraclass Correlations

The intraclass correlation plays a central role in modeling hierarchically structured data, such as educational data, panel data, or group-randomized trial data. It represents relevant information concerning the between-group and within-group variation. Methods for Bayesian hypothesis tests concerning the intraclass correlation are proposed to improve decision making in hierarchical data analysis and to assess the grouping effect across different group categories. Estimation and testing methods for the intraclass correlation coefficient are proposed under a marginal modeling framework where the random effects are integrated out. A class of stretched beta priors is proposed on the intraclass correlations, which is equivalent to shifted F priors for the between groups variances. Through a parameter expansion it is shown that this prior is conditionally conjugate under the marginal model yielding efficient posterior computation. A special improper case results in accurate coverage rates of the credible intervals even for minimal sample size and when the true intraclass correlation equals zero. Bayes factor tests are proposed for testing multiple precise and order hypotheses on intraclass correlations. These tests can be used when prior information about the intraclass correlations is available or absent. For the noninformative case, a generalized fractional Bayes approach is developed. The method enables testing the presence and strength of grouped data structures without introducing random effects. The methodology is applied to a large-scale survey study on international mathematics achievement at fourth grade to test the heterogeneity in the clustering of students in schools across countries and assessment cycles.

Simple Bayesian testing of scientific expectations in linear regression models

Scientific theories can often be formulated using equality and order constraints on the relative effects in a linear regression model. For example, it may be expected that the effect of the first predictor is larger than the effect of the second predictor, and the second predictor is expected to be larger than the third predictor. The goal is then to test such expectations against competing scientific expectations or theories. In this paper, a simple default Bayes factor test is proposed for testing multiple hypotheses with equality and order constraints on the effects of interest. The proposed testing criterion can be computed without requiring external prior information about the expected effects before observing the data. The method is implemented in R-package called ‘lmhyp’ which is freely downloadable and ready to use. The usability of the method and software is illustrated using empirical applications from the social and behavioral sciences.

Phylogenomics provides a robust topology of the major cnidarian lineages and insights on the origins of key organismal traits

BackgroundThe phylogeny of Cnidaria has been a source of debate for decades, during which nearly all-possible relationships among the major lineages have been proposed. The ecological success of Cnidaria is predicated on several fascinating organismal innovations including stinging cells, symbiosis, colonial body plans and elaborate life histories. However, understanding the origins and subsequent diversification of these traits remains difficult due to persistent uncertainty surrounding the evolutionary relationships within Cnidaria. While recent phylogenomic studies have advanced our knowledge of the cnidarian tree of life, no analysis to date has included genome-scale data for each major cnidarian lineage.ResultsHere we describe a well-supported hypothesis for cnidarian phylogeny based on phylogenomic analyses of new and existing genome-scale data that includes representatives of all cnidarian classes. Our results are robust to alternative modes of phylogenetic estimation and phylogenomic dataset construction. We show that two popular phylogenomic matrix construction pipelines yield profoundly different datasets, both in the identities and in the functional classes of the loci they include, but resolve the same topology. We then leverage our phylogenetic resolution of Cnidaria to understand the character histories of several critical organismal traits. Ancestral state reconstruction analyses based on our phylogeny establish several notable organismal transitions in the evolutionary history of Cnidaria and depict the ancestral cnidarian as a solitary, non-symbiotic polyp that lacked a medusa stage. In addition, Bayes factor tests strongly suggest that symbiosis has evolved multiple times independently across the cnidarian radiation.ConclusionsCnidaria have experienced more than 600 million years of independent evolution and in the process generated an array of organismal innovations. Our results add significant clarification on the cnidarian tree of life and the histories of some of these innovations. Further, we confirm the existence of Acraspeda (staurozoans plus scyphozoans and cubozoans), thus reviving an evolutionary hypothesis put forward more than a century ago.

Probing dark energy in the scope of a Bianchi type I spacetime

It is well known that flat FRW metric is a special case of Bianchi type I spacetime. In this paper, we use 38 Hubble parameter, $H(z)$, measurements at intermediate redshifts $0.07\leq z\leq 2.36$ and its joint combination with the latest \textgravedbl joint light curves\textacutedbl (JLA) sample, comprised of 740 type Ia supernovae in the redshift range $z \epsilon [0.01, 1.30]$ to constrain the parameters of Bianchi type I dark energy model. We also use the same datasets to constrain flat $\Lambda$CDM Model. In both cases, we specifically address the expansion rate $H_{0}$ as well as the transition redshift $z_{t}$ determinations out of these measurements. In both Models we found that using joint combination of datasets gives rise to lower values for model parameters. Also to compare the considered cosmologies, we have made Akaike information criterion (AIC) and Bayes factor ($\Psi$) tests.

Network Autocorrelation Modeling: A Bayes Factor Approach for Testing (Multiple) Precise and Interval Hypotheses

Currently available (classical) testing procedures for the network autocorrelation can only be used for falsifying a precise null hypothesis of no network effect. Classical methods can be neither used for quantifying evidence for the null nor for testing multiple hypotheses simultaneously. This article presents flexible Bayes factor testing procedures that do not have these limitations. We propose Bayes factors based on an empirical and a uniform prior for the network effect, respectively, first. Next, we develop a fractional Bayes factor where a default prior is automatically constructed. Simulation results suggest that the first two Bayes factors show superior performance and are the Bayes factors we recommend. We apply the recommended Bayes factors to three data sets from the literature and compare the results to those coming from classical analyses using p values. R code for efficient computation of the Bayes factors is provided.

Bayes Factor Covariance Testing in Item Response Models.

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.