Conducting Web-Based Experiments for Numerical Cognition Research.

Arnold R Kochari

doi:10.5334/joc.85

Abstract

It is becoming increasingly popular and straightforward to collect data in cognitive psychology through web-based studies. In this paper, I review issues around web-based data collection for the purpose of numerical cognition research. Provided that the desired type of data can be collected through a web-browser, such online studies offer numerous advantages over traditional forms of physical lab-based data collection, such as gathering data from larger sample sizes in shorter time-windows and easier access to non-local populations. I then present results of two replication studies that employ classical paradigms in numerical cognition research: the number-size congruity paradigm and comparison to a given standard, which also included a masked priming manipulation. In both replications, reaction times and error rates were comparable to original, physical lab-based studies. Consistent with the results of original studies, a distance effect, a congruity effect, and a priming effect were observed. Data collected online thus offers a level of reliability comparable to data collected in a physical lab when it comes to questions in numerical cognition.

Highlights

Web-based data collection, whereby participants take part in a research study remotely from their own computer, has gained prominence in psychology in the past decade
The general reaction time (RT) level was approximately within the 500–650 ms range, which falls with the normal range of RTs for this paradigm
I excluded all trials in which the reaction time was too short (

Summary

Introduction

Web-based data collection, whereby participants take part in a research study remotely from their own computer, has gained prominence in psychology in the past decade. This is due to its clear advantages: easy access to larger and more diverse samples and speed of data collection (see e.g., Gosling & Mason, 2015; Stewart, Chandler, & Paolacci, 2017; Woods, Velasco, Levitan, Wan, & Spence, 2015, for reviews). A number of tools have been developed to facilitate the building of experiments with careful timing of experimental stimuli display and accurate response recording within web-browsers. This has made web-based data collection suitable for many experimental paradigms typical in cognitive psychology. Being able to reach a culturally diverse set of participants (e.g., with different traditions of teaching children to count) and to collect data in small variations of the same set-up (e.g., multiplication problems with differing levels of difficulty) are advantages that are especially useful in answering questions in numerical cognition

Methods

Results

Discussion

Conclusion