Abstract
Animal cognition research often involves small and idiosyncratic samples. This can constrain the generalizability and replicability of a study’s results and prevent meaningful comparisons between samples. However, there is little consensus about what makes a strong replication or comparison in animal research. We apply a resampling definition of replication to answer these questions in Part 1 of this article, and, in Part 2, we focus on the problem of representativeness in animal research. Through a case study and a simulation study, we highlight how and when representativeness may be an issue in animal behavior and cognition research and show how the representativeness problems can be viewed through the lenses of, i) replicability, ii) generalizability and external validity, iii) pseudoreplication and, iv) theory testing. Next, we discuss when and how researchers can improve their ability to learn from small sample research through, i) increasing heterogeneity in experimental design, ii) increasing homogeneity in experimental design, and, iii) statistically modeling variation. Finally, we describe how the strongest solutions will vary depending on the goals and resources of individual research programs and discuss some barriers towards implementing them.
Highlights
Animal cognition research often involves small and idiosyncratic samples
This article shows how concerns about replicability, representativeness, comparison and theory testing, and pseudoreplication are all related through the lens of sampling
Part 2 of the article focuses on representativeness and asks how concerned researchers should be with the problem of non-representative sampling in animal research
Summary
A study is labeled a replication because it is the same in some regards to a previous experiment. The resampling definition of replication states that a replication study is a study that resamples from the same populations of experimental units, treatments, measurements, and settings that an original study could have sampled from, relative to the claim being tested (Machery, 2020; Nosek & Errington, 2020). This is outlined, which is adapted from Machery (2020). An experiment samples from: A population of experimental units, e.g., a population of a species in captivity
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have