Demonstrating Construct Validity of the American Board of Physical Medicine and Rehabilitation Part I Examination: An Analysis of Dimensionality

Abstract

BackgroundIdeally, high-stakes examinations assess 1 dimension of medical knowledge to produce precise estimates of a candidate’s performance. It has not been reported whether the American Board of Physical Medicine and Rehabilitation Part 1 Certification Examination (ABPMR-CE-1) is unidimensional or not. ObjectiveTo examine the ABPMR-CE-1 to measure how many dimensions it assesses. DesignRetrospective observational study. SettingWe assessed examination results from the 2015 ABPMR-CE-1. ParticipantsA total of 489 deidentified candidates taking the 2015 ABPMR-CE-1. MethodsA 1-parameter Item Response Theory (IRT) measurement model was utilized. A Principal Components Analysis (PCA) of standardized residual correlations was used to detect multidimensionality. Main Outcome MeasureNumber of primary dimensions reflected in the 325 test questions. ResultsThe results of the dimensionality analysis indicated that the ABPMR-CE-1 examination is highly unidimensional from a psychometric perspective. Expert content review of the substantive content of small contrasting clusters of questions provided additional assurance of the unidimensional nature of the examination. ConclusionsThe ABPMR-CE-1 appears indeed to measure a single construct, which suggests a sound structure of the examination. It closely approximates the assumption of statistical unidimensionality. Level of EvidenceNot applicable.