Moving experimental psychology online: How to obtain high quality data when we can’t see our participants

Abstract

The past 10 years have seen rapid growth of online (web-based) data collection across the behavioural sciences. Despite the many important contributions of such studies, some researchers have concerns about the reduction in experimental control when research moves outside of laboratory conditions. This paper provides an accessible overview of the issues that can adversely affect data quality in online experiments, with particular focus on cognitive studies of memory and language. I provide checklists for researchers setting up such experiments to help improve data quality. These recommendations focus on three key aspects of experimental design: the technology choices made by researchers and participants, participant recruitment methods, and the performance of participants during experiments. I argue that ensuring high data quality for online experiments requires significant effort prior to data collection to maintain the credibility of our rapidly expanding evidence base. With such safeguards in place, online experiments will continue to provide important, paradigm-changing opportunities across the behavioural sciences.