Myth of the master detective: Reliability of interpretations for Kaufman's "intelligent testing" approach to the WISC–III.

Abstract

Based on the internal-consistency and test-retest reliability coefficients reported for the Wechsler Intelligence Scale for Children-Third Edition (WISC-III), computer simulation methods were used to examine the reliability of interpretations for Kaufman's approach to the WISC-III. Two independent samples of 5,000 cases were generated for each of three sources of variability in scores: content sampling, time sampling, and the combined or additive effect of content and time sampling. Because many additional sources of error were not represented by the model (scoring errors, practice effects), the simulation provided a best case scenario for the evaluation of Kaufman's system. Analyses of decision reliability showed that VIQ-PIQ differences, factor index-score differences, and ipsative profile patterns on the WISC-III could not be interpreted with confidence. Consequently, Kaufman's assertion that the limitations of IQ testing can be overcome through skilled detective work may be viewed as mythology. Implications for intelligent professional practice and public policy in the assessment of children are discussed. Despite decades of controversy, the administration, scoring, and interpretation of IQ tests continue to exert a dominating influence on the professional practice of school psychology (e.g., Stinnett, Havey, & Oehler-Stinnett, 1994). Simply counting the number of children referred and re-evaluated for special education services, more than a million IQ tests are individually administered to school-aged children in the United States each year. Questions about the desirability of this current situation have polarized the field. At one extreme, there has been increasing interest in service delivery models that either minimize or eliminate the need for IQ testing