A Note on Norms in Experimental Economics

Abstract

Many prominent recent contributions to economic theory [Laibson 1997; Camerer 2003; Charness and Dufwenberg 2006] are motivated by the findings of economics experiments, and are then advanced by further experimentation [Houser and Kurzban 2002; Xiao and Houser 2005]. This iterative process indicates a sense of trust both in experiments and in the experimenters who conduct them. This trust hinges on two key norms that guide the design, analysis, and reporting of economic experiments in both the lab and the field. These norms help ensure adherence to the key ethical standard of "transparency" [Levy and Peart this issue]. Norms here refer to informal rules guiding the decisions of scholars who use designed investigations to inform economic hypotheses. The first key norm for experimental economists is simplicity: experimentalists adopt the simplest design that can address their research hypothesis. An experimental design is "simple" if it provides (a) ease of replicability and (b) straightforward interpretations of outcomes. The importance of the former will be discussed further below. The latter point (b) is not meant to suggest that sophisticated econometric or statistical analysis is or should be avoided in experimental economics. Rather, it means that the analysis procedures should be appropriate and defensible given the design, and that the design should be chosen in view of such considerations. The second norm is that all published data and procedures be made publicly available. In particular, the published results must be supported by a discussion that would allow the procedures to be replicated. The procedural discussion includes not just a description of the formal experimetrics procedures [Houser 2008], but also the procedures by which the data were generated. Usually, this entails publishing at least the experiment's instructions, either in the journal or on a website that the published article references.