A "Goldilocks zone" for mind-wandering reports? A secondary data analysis of how few thought probes are enough for reliable and valid measurement.

Abstract

Mind-wandering assessment relies heavily on the thought probe technique as a reliable and valid method to assess momentary task-unrelated thought (TUT), but there is little guidance available to help researchers decide how many probes to include within a task. Too few probes may lead to unreliable measurement, but too many probes might artificially disrupt normal thought flow and produce reactive effects. Is there a "Goldilocks zone" for how few thought probes can be used to reliably and validly assess individual differences in mind-wandering propensity? We address this question by reanalyzing two published datasets (Study 1, n = 541; Study 2, ns ≈ 260 per condition) in which thought probes were presented in multiple tasks. Our primary analyses randomly sampled probes in increments of two for each subject in each task. A series of confirmatory factor analyses for each probe "bin" size tested whether the latent correlations between TUT rate and theoretically relevant constructs like working memory capacity, attention-control ability, disorganized schizotypy, and retrospective self-reported mind wandering changed as more probes assessed the TUT rate. TUT rates were remarkably similar across increasing probe-bin sizes and zero-order correlations within and between tasks stabilized at 8-10 probes; moreover, TUT-rate correlations with other latent variables stabilized at about 8 thought probes. Our provisional recommendation (with caveats) is that researchers may use as few as 8 thought probes in prototypical cognitive tasks to gain reliable and valid information about individual differences in TUT rate.