A Meta-Analytic Re-Appraisal of the Framing Effect

Abstract

Abstract. We reevaluated and reanalyzed the data of Kühberger’s (1998) meta-analysis on framing effects in risky decision making by using p-curve. This method uses the distribution of only significant p-values to correct the effect size, thus taking publication bias into account. We found a corrected overall effect size of d = 0.52, which is considerably higher than the effect reported by Kühberger (d = 0.31). Similarly to the original analysis, most moderators proved to be effective, indicating that there is not the risky-choice framing effect. Rather, the effect size varies with different manipulations of the framing task. Taken together, the p-curve analysis shows that there are reliable risky-choice framing effects, and that there is no evidence of intense p-hacking. Comparing the corrected estimate to the effect size reported in the Many Labs Replication Project (MLRP) on gain-loss framing (d = 0.60) shows that the two estimates are surprisingly similar in size. Finally, we conducted a new meta-analysis of risk framing experiments published in 2016 and again found a similar effect size (d = 0.56). Thus, although there is discussion on the adequate explanation for framing effects, there is no doubt about their existence: risky-choice framing effects are highly reliable and robust. No replicability crisis there.