Rasch Analysis as a Technique to Examine the Psychometric Properties of a Career Ability Placement Survey Subtest

Abstract

Rasch analysis offers researchers an innovative method for developing and validating rehabilitation instruments. Despite the increased use of Rasch analysis by allied health researchers, this method continues to be underutilized by rehabilitation counseling professionals. In the rehabilitation counseling profession, vocational evaluators use various instruments, such as the Career Ability Placement Survey (CAPS), to determine client aptitudes and justify job placement recommendations. The purpose of this article is to introduce Rasch analysis as a method for analyzing vocational assessments by examining the psychometric properties of the Word Knowledge (WK) subtest of the CAPS. University undergraduate and graduate students ( n = 227) completed the WK subtest as part of a classroom exercise. The analysis determined item hierarchy, item gaps and redundancies, item-fit statistics (unidimensionality), and person reliability and separation statistics. Although the WK construct demonstrates good psychometric characteristics relating to unidimensionality and to person reliability and separation statistics, several items deviated from the expected ordering pattern. The reordering of items revealed potential item gaps and redundancies as well as poor person—instrument match. Adding and removing items will improve the validity and reliability of the WK subtest psychometrics.