Considerations Around Predicting Physician Assistant National Certifying Exam Scores From PACKRAT®: A Multiprogram Study.

Abstract

The Physician Assistant Clinical Knowledge Rating and Assessment Tool (PACKRAT®) is a known predictor of performance on the Physician Assistant National Certifying Exam (PANCE). It is unknown, however, whether these associations (1) vary across programs; (2) differ by PACKRAT metrics (first-year [PACKRAT 1], second-year [PACKRAT 2], and composite score [arithmetic mean of PACKRAT 1 and PACKRAT 2]); or (3) are modified by demographic or socioeconomic variables. Linear and logistic hierarchical regression models (HRMs) were used to evaluate associations between PACKRAT metrics and (1) continuous PANCE scores and (2) odds of low PANCE performance (LPP), respectively. Likelihood ratio tests were used to evaluate differences in associations between programs and effect modification by demographic and socioeconomic variables. Receiver operating characteristic (ROC) curves were used to examine the sensitivity, specificity, positive predictive values, and negative predictive values for various PACKRAT metrics/cut points. Models were adjusted for demographic and socioeconomic variables. The PACKRAT scores were standardized for each year to the national mean and SD. Adjusted HRMs across 5 programs (n = 1014) found the composite score to have the strongest association, with a 10-percentile-point increase associated with a 22-point (95% confidence interval [CI]: 19-26) increase in PANCE score. The composite score also strongly predicted decrements in odds of LPP (odds ratio: 0.46; 95% CI: 0.38-0.55). Hierarchical regression models and ROC curves identified significant variability in associations among programs. Effect modification was not observed by any investigated variable. The composite score had the largest magnitudes of association with PANCE scores and odds of LPP. The significant difference in association identified between programs suggests that the predictive ability of the exam is not uniform. The lack of effect modification by demographic and socioeconomic variables suggests that associations do not significantly differ by these metrics.