How representative are neuroimaging samples? Large-scale evidence for trait anxiety differences between fMRI and behaviour-only research participants.

Caroline J Charpentier ,Karin Roelofs,Michael Browning,Verena Ly,Erdem Pulcu,Roshan Cools,Aniek Fransen,Nadine Wanke,Gundula Zerbes,Vincent Valton,Christian Grillon,Luiz Pessoa,Valentina Krenz,Alexander Kaltenboeck,Yumeya Yamamori,Alessio Giarrizzo,Lars Schwabe,Paul Faulkner,Anahit Mkrtchian,Níall Lally,Felix Kalbe,Ioannis Sarigiannidis,Susannah E Murphy ,Marieke S Tollenaar ,Eva Pool ,Franziska Magdalena Kausche ,Quentin J M Huys ,Anna Lena Cremer ,C J Harmer ,Henry W Chase ,Kelly A Morrow ,Jonathan P Roiser ,Lisa Marieke Kluen ,Oliver J Robinson ,Lynn Paul ,John P O’doherty

doi:10.1093/scan/nsab057

Abstract

Over the past three decades, functional magnetic resonance imaging (fMRI) has become crucial to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into fMRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, a behavioural effect discovered in a non-imaging context not replicating in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (N = 272), that fMRI participants differ from behaviour-only participants on one fundamental individual difference variable: trait anxiety. Analysing trait anxiety scores and possible confounding variables from healthy volunteers across multiple institutions (N = 3317), we found robust support for lower trait anxiety in fMRI study participants, consistent with a sampling or self-selection bias. The bias was larger in studies that relied on phone screening (compared with full in-person psychiatric screening), recruited at least partly from convenience samples (compared with community samples), and in pharmacology studies. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures or sampling strategies to mitigate this bias.

Highlights

Neuroimaging methods, such as functional MRI, have been fundamental to the emergence of cognitive neuroscience as a research field
Confirming our hypothesis and preliminary data, the difference in trait anxiety between functional MRI (fMRI) and behavioural studies was significant in the larger sample, albeit with a smaller, but nonnegligible, effect size (t-test assuming unequal variance: T3180=6.41, P
The distribution of trait anxiety scores across the two study contexts (Fig. 1B) indicates that the difference is driven by a larger proportion of individuals in fMRI studies scoring between 30 and 40, and a larger proportion of individuals in behavioural studies scoring above 45

Summary

Introduction

Neuroimaging methods, such as functional MRI (fMRI), have been fundamental to the emergence of cognitive neuroscience as a research field. Given that many studies in cognitive neuroscience involve a behavioural piloting phase to assess behavioural effects, followed by an fMRI scanning phase to assess neural mechanisms, it is important to ensure that individuals who volunteer to participate in each study context exhibit similar profiles and can be characterized by similar population distributions. This is especially relevant for studies in which effects that are present (and replicate) outside the scanner (Bolton and Robinson, 2017) fail to replicate (Garibbo et al, 2019) inside the scanning environment. While there is evidence that physical characteristics of the scanning environment, such as acoustic noise (Hommel et al, 2012; Skouras et al, 2013; Kobald et al, 2016), can affect cognitive and affective processes, their neural basis, and hormonal responses (Gossett et al, 2018), poor generalizability across testing contexts could be due, in part, to unanticipated biases in study recruitment

Methods

Results

Conclusion