Sample Size Calculation in Economic Experiments

Abstract

Clinical studies and economic experiments are often conducted with randomized controlled trials. In clinical studies, power calculations are carried out as a standard. But what’s about economic experiments? After describing the basic idea of the calculation procedure in a brief tutorial, I tackle the practice of sample size calculations in the field of experimental economics by considering the publications of 5 economic journals in the period 2000–2018. These are two top-ranked economic journals (Quarterly Journals of Economics and American Economic Review), the leading field journals in the area of experimental economics (Experimental Economics) and behavioral sciences (Journal of Economic Behavior and Organization), and a leading field journal in environmental economics (Environmental and Resource Economics). In contrast to clinical drug trials, sample size calculations have rarely been carried out by experimental economists. But the number of power calculations has slightly increased in recent years, especially in the top-ranked journals of economics. However, this can be partly explained by the fact that field experiments (in which scholars pay more attention to power analyses than in lab experiments these days) play an important role in these journals.