Who's in and who's out? Inclusion criteria, model evaluation, and the treatment of exceptions in case series

Abstract

Several years ago, Eleanor Saffran, Nadine Martin, and the two of us began our collaborative project using aphasic speech errors to test production models that had been inspired by the errors of unimpaired speakers. At the time, though, one of us nearly balked upon seeing the data. Compared to everyday speech errors, transcripts of aphasic speech, even in controlled situations such as picture naming or word repetition, revealed an overwhelming amount of between- and even within-patient variation. To the seasoned CN researchers among us, this was no surprise. The researcher's art consists in finding ways to deal with the variability. The case series method is valuable because it allows the researcher to embrace, and ultimately, to tame the neuropsychological variability in service of testing theory. We are pleased that several distinguished CN researchers, some of whom have considerably more experience than we do with the method, generally agreed with the positive assessment of case series that we presented in Schwartz and Dell (2010). But commenters offered specific recommendations about the method, and pointed to lacunae in our paper. Is there is a right way to do a case series? In Schwartz and Dell (2010), we offered examples of case series studies in our work and in other research on topics that we had some familiarity with to show what one could do with the method, and made some recommendations about what one should do. The commenters augmented both the “could” and the “should” parts. Nickels, Howard, and Best remind us that case series could be used in treatment studies, and Shallice and Buiatti point to studies in which the selection of cases is based on anatomical rather than behavioural criteria. The studies they refer to seem to be clear examples of case series methodology. And they also remind us once more that it is hard to draw the boundaries between classic CN in which the goal is to draw inferences about normal cognition and the many other reasons for studying impaired cognition. A lot of the commentary concerned how one could (or should) treat individual differences in CN. Lambon-Ralph, Patterson, and Plaut consider the possibility that these differences can be due to pre-morbid factors explicable within the theory (e.g. the triangle theory of reading). They and Bub argue that appealing to this kind of individual variation had been “close to taboo” in the Aristotelian inspired single-subject approach to CN. We can agree with them that case series methods are well suited to the investigation of the contribution to performance of post-morbid individual differences (and pre-morbid differences, when data are available). Goldrick, however, tells us that appeals to individual differences are often too pat. When assumed to arise pre-morbidly, they usually lack pre-morbid supporting data. When assessed post-morbidly and simply attributed to differing degrees of damage,