Disappearing dissociations in experimental psychology: Using state-trace analysis to test for multiple processes

Abstract

Dissociations have served as a key source of evidence for theory development in experimental psychology. Claims about the existence of multiple distinct psychological processes or systems are often based on demonstrations that manipulations such as working memory load, mood or instructions have differential effects on task performance. For example, a manipulation may have a larger effect on performance in one task, and a smaller or no detectable effect in another, as identified by statistical models like analysis of variance. However, inferring distinct underlying processes based on such interaction effects can be misleading. Such an inference depends on the strong – and probably false – assumption that underlying psychological variables map linearly onto the observable dependent variables. Fortunately, state-trace analysis offers an alternative approach to test for multiple underlying variables, avoiding the linearity assumption. We apply state-trace analysis to databases of studies from reasoning and from category learning that have been cited as evidence for qualitatively distinct processes. We show that many of the dissociations thought to reflect the operation of distinct processes disappear against the stricter criteria of state-trace analysis. We argue that it is important for experiments to be designed with state-trace analysis in mind, and highlight the need for the development and more widespread use of similar techniques. This will lead to a more rigorous foundation for theoretical claims about distinct underlying psychological mechanisms.