An initial reliability analysis of a patient counseling rubric to objectively measure student pharmacist performance

Abstract

Current literature outlines the documented need for improved communication during patient medication counseling. Although many tools exist, there needs to be a national standardized tool that complies with federal and state law, to objectively measure student pharmacist performance during patient counseling in the community pharmacy setting. The primary objective of this study is to perform an initial analysis of the internal consistency reliability of a patient medication counseling rubric designed with an Indian Health Services theoretical framework. Secondary objectives include measuring changes in student performance over the time of the study. The 18-item rubric was developed to objectively measure student pharmacist performance during patient medication counseling sessions in a 21-h Introductory Pharmacy Practice Experience (IPPE) course. The community-pharmacy-based IPPE patient counseling course evaluates students' communication skills and patient-centered counseling techniques in live and simulated patient counseling sessions. Three pharmacist evaluators assessed a total of 247 student counseling sessions. The rubric's internal consistency reliability was analyzed, and student performance improvement was observed within the course. Students' performance was evaluated as “meets expectations” in most live and simulated sessions. However, an independent groups t-test showed that the mean performance score for the live counseling sessions (2.59, SD = 0.29) was higher (p < 0.001) than that for the simulated counseling sessions (2.35, SD = 0.35). Students' performance in the course improved over three weeks [Week 1: mean (SD) = 2.29 (0.32), Week 2: mean (SD) = 2.44 (0.33), Week 3: mean (SD) = 2.62 (0.29); p < 0.001]. A Tukey-Kramer comparison post hoc test found a significant increase in the mean performance scores between weeks (p < 0.05). The overall internal consistency reliability of the counseling rubric was determined acceptable, with a Cronbach's alpha of 0.75. Further study is required, including the assessment of inter-rater reliability, factor analysis, variable analysis, and use in other states with patient confirmation testing necessary to validate the rubric for use with student pharmacists in the community pharmacy setting.