A Bayesian Perspective on Intervention Research: Using Prior Information in the Development of Social and Health Programs

Abstract

Objective: By presenting a simulation study that compares Bayesian and classical frequentist approaches to research design, this paper describes and demonstrates a Bayesian perspective on intervention research. Method: Using hypothetical pilot-study data where an effect size of 0.2 had been observed, we designed a 2-arm trial intended to compare an intervention with a control condition (e.g., usual services). We determined the trial sample size by a power analysis with a Type I error probability of 2.5% (1-sided) at 80% power. Following a Monte-Carlo computational algorithm, we simulated 1 million outcomes for this study and then compared the performance of the Bayesian perspective with the performance of the frequentist analytic perspective. Treatment effectiveness was assessed using a frequentist t-test and an empirical Bayesian t-test. Statistical power was calculated as the criterion for comparison of the 2 approaches to analysis. Results: In the simulations, the classical frequentist t-test yielded 80% power as designed. However, the Bayesian approach yielded 92% power. Conclusion: Holding sample size constant, a Bayesian analytic approach can improve power in intervention research. A Bayesian approach may also permit smaller samples holding power constant. Using a Bayesian analytic perspective could reduce design demands in the developmental experimentation that typifies intervention research.