Crowdsourcing prior information to improve study design and data analysis.

Jeffrey S Chrabaszcz,Michael R Dougherty,Joe W Tidwell

doi:10.1371/journal.pone.0188246

Jeffrey S Chrabaszcz, Michael R Dougherty + Show 1 more

Open Access

https://doi.org/10.1371/journal.pone.0188246

Copy DOI

Abstract

Though Bayesian methods are being used more frequently, many still struggle with the best method for setting priors with novel measures or task environments. We propose a method for setting priors by eliciting continuous probability distributions from naive participants. This allows us to include any relevant information participants have for a given effect. Even when prior means are near-zero, this method provides a principle way to estimate dispersion and produce shrinkage, reducing the occurrence of overestimated effect sizes. We demonstrate this method with a number of published studies and compare the effect of different prior estimation and aggregation methods.

Highlights

IntroductionLimitations on time and funding constrain sample sizes and thereby reduce the information available from any individual study
Research activities trade off between accuracy of estimates and cost of information
Aggregating participant belief into a prior distribution on an effect size is intractable without further information in any of these scenarios

Summary

Introduction

Limitations on time and funding constrain sample sizes and thereby reduce the information available from any individual study. Additional, extra-experimental information regarding effect sizes would allow researchers to plan appropriate sample sizes and improve Bayesian analyses by constraining and informing prior distributions. Non-uniform (i.e., informed) prior distributions are often criticized for their subjectivity and may be difficult to estimate in the absence of relevant literature. While empirical Bayesian methods are con tinually improving [1], these methods avoid the near-universal benefit to analytical efficiency gained from including additional prior information. Infinite and uniform prior distributions like those traditionally favored by empirical Bayesians can be too informative, granting excess credibility to extreme (and sometimes impossible) values of estimated parameters [2]. For sparse data or small sample sizes, uniform prior distributions can produce inappropriate parameter estimates

Objectives

Results

Discussion

Conclusion