Item analysis of written expression scoring systems from the PIAT-R and WIAT

Abstract

The Peabody Individual Achievement Test–Revised (PIAT-R) and Wechsler Individual Achievement Test (WIAT) contain measures of written expression. However, these subtests are not theory-based and were assessed with inappropriate psychometric analyses. This study attempted to enhance the study of written expression by reexamining the reliability and validity of the PIAT-R and WIAT Written Expression scoring systems, applying theory and more appropriate statistical analyses. First, items were identified that were the most and least reliable, determined by interrater agreement. Next, the most and least valid items were identified, based on item–total correlations. Subjects included 50 adolescents, men, and women aged 13 to 46 years; raters were three graduate students with experience and training similar to that of the typical test user. Results indicate that seven items were too easy, as virtually all subjects received the maximum score on these items—these items were eliminated. The remaining 24 items were classified as both reliable and valid (9 items), reliable but not valid (4 items), valid with limited reliability (5 items), and neither reliable nor valid (6 items). The WIAT written expression scoring system was found to have more items that were both reliable and valid compared to the PIAT-R scoring system. Items measuring global, rather than specific, content were also found to be more reliable and valid. © 1999 John Wiley & Sons, Inc.