Effects of Transition from Closed-Book to Open-Book Assessment on Students' Scores in a Pharmacokinetics Course.

Abstract

Closed-book summative assessment of student learning, common in pharmacy education, is challenging to administer in a remote setting due to the need for costly and intrusive monitoring technology. Therefore, open-book assessments without monitoring have been considered an alternative in remote settings. The present study investigated the effects of the transition from in-person closed-book to remote open-book format on the students' scores in different assessment categories in a Pharmacokinetics course. The students' performances in the transition cohort (Transition, n = 96) during the in-person and remote periods were compared with those of an in-person cohort (Control, n = 85) during the same periods. Assessments included take-home assignments, daily quizzes, and progress/final examinations. Whereas the take-home assignments were open-book for cohorts and periods, the quizzes and examinations were open-book only for the Transition cohort during the remote period. Only the quiz/examination questions that were identical for both cohorts were included in the analysis. Statistical analysis by a linear, mixed-effects model indicated that the transition did not have any significant impact on the scores of students in the assignments, which were open-book for both cohorts and both periods. However, there were significant increases in the Transition cohort's scores (mean ± SE) during the remote open-book period in both quizzes (+8.4 ± 1.9%) and examination (+6.8 ± 1.5%) questions, compared with the Control cohort who had in-person closed-book assessments. These differences amounted to Cohen's d-effect sizes of 0.61 and 0.59 for the quiz and examination questions, respectively. It is concluded that when the questions are similar, the students' scores in pharmacokinetic assessments are higher (medium effect size) in a remote open-book format compared with the in-person closed-book format.