Note: procedure for training judges to score tapes of aphasic speech.

Abstract

Packman and Ingham (1978) obtained variable reliability among four judges independently scoring the verbal responses of 5 aphasic subjects into seven categories. They pointed out the considerable discrepancy of their findings and the high reliability between two judges (.98) reported in our study (Goodkin, 1968). They justly indicated that our reports did not include details for training judges to score the aphasic speech, an important omission for those who wish to replicate these procedures. Indeed, proven reliability is essential if the response categories are to be useful in clinical research and practice. The training of judges first involved carefully reviewing the definition of each response category and discussing a variety of examples of responses to be assigned to each category. Then, sample tapes of 10 aphasic subjects, with varying levels of expressive and receptive impairment, were listened to by the judge and the experimenter. The tapes consisted of the aphasic's responses to 10 opemended questions (Goodkin, 1968). As the judge tallied each response into one of the categories, the experimenter gave immediate feedback as to whether the response was scored accurately or whether it would fit more appropriately in another category. With more fluent aphasics it was not possible to keep track of all seven response categories simultaneously. In such cases the tapes were played back two or three times and only two or three categories were scored each time. When there were questions, disagreements, or comments during the training period, the tape was stopped, and the issues were discussed. After an hour or two of such training, the judge began independently to evaluate the tapes of aphasics for the purpose of the study. I agree with Packman and Ingham that without feedback on the judging behavior during a training session, agreement in scoring aphasic speech would not necessarily improve over time.