The Reliability of Diagnostic and Remedial Decisions of Reading Specialists

Abstract

This study empirically assessed whether reading specialists' written diagnoses and written remedial plans were reliable and reliably associated. Eight experienced reading specialists, four trained in Michigan and four trained in Illinois, were observed as they diagnosed three simulated cases of reading difficulty. Two of the cases were thinly disguised versions of the same reading problem. The third represented a different reading problem. The study demonstrated that (1) the bulk of the diagnostic and remedial statements/associations for a given case were made only once; identical cases were not diagnosed as such. (2) Examination of common case information led neither to common diagnoses, common remediations, nor common associations between remediation and diagnosis. Only by aggregating diagnostic and remedial statements/associations across clinicians could the outlines of consensus on each case be discerned. (3) Agreement between individual clinicians on diagnostic and remedial statements seen as characterizing a case ranged from very little to none whatever. (4) Clinicians never followed their stated plans regarding information collection procedures and the writing of the diagnosis and remediation. (5) There was no difference in performance between the Michigan-trained and Illinois-trained subjects.