Analyzability, ad hoc restrictions, and excessive flexibility of evidence-accumulation models: reply to two critical commentaries.

Abstract

Jones and Dzhafarov (2014) proved the linear ballistic accumulator (LBA) and diffusion model (DM) of speeded choice become unfalsifiable if 2 assumptions are removed: that growth rate variability between trials follows a Gaussian distribution and that this distribution is invariant under certain experimental manipulations. The former assumption is purely technical and has never been claimed as a theoretical commitment, and the latter is logically and empirically suspect. Heathcote, Wagenmakers, and Brown (2014) questioned the distinction between theoretical and technical assumptions and argued that only the predictions of the whole model matter. We respond that it is valuable to understand how a model's predictions depend on each of its assumptions to know what is critical to an explanation and to generalize principles across phenomena or domains. Smith, Ratcliff, and McKoon (2014) claimed unfalsifiability of the generalized DM relies on parameterizations with negligible diffusion and proposed a theoretical commitment to simple growth-rate distributions. We respond that a lower bound on diffusion would be a new, ad hoc assumption, and restrictions on growth-rate distributions are only theoretically justified if one supplies a model of what determines growth-rate variability. Finally, we summarize a simulation of the DM that retains the growth-rate invariance assumption, requires the growth-rate distribution to be unimodal, and maintains a contribution of diffusion as large as in past fits of the standard model. The simulation demonstrates mimicry between models with different theoretical assumptions, showing the problems of excess flexibility are not limited to the cases to which Smith et al. objected. (PsycINFO Database Record (c) 2014 APA, all rights reserved).