Most Psychological Researchers Assume Their Samples Are Ergodic: Evidence From a Year of Articles in Three Major Journals

Abstract

Conventional statistics methods in most psychological research, such as null-hypothesis significance tests (NHSTs), use aggregated values (i.e., the sample means) of group behaviours to make inferences about individuals. Such inferences are possibly erroneous because groups of humans rarely, if ever, constitute an ergodic system. To assume ergodicity without checking is to commit the ‘ergodic fallacy’. The aim of the current study was to examine the prevalence of this error in contemporary psychological research. We analysed three highly cited ‘Q1’ journals in the fields of clinical, educational and cognitive psychology for statements that indicated this error. As hypothesized, the ergodic fallacy was found in the vast majority of the papers investigated here. We also hypothesised that the prevalence of this error would be highest in cognitive psychology papers because this field typically assesses theoretical claims about universal cognitive mechanisms, whereas clinical and educational psychology are more concerned with empirically supported interventions. This hypothesis was also supported by our results. Nonetheless, the prevalence of the ergodic fallacy was still high in all fields. Implications are discussed with respect to the reporting of research findings and the validity of theories in psychology.