Nudging clinical supervisors to provide better in-training assessment reports.

Valérie Dory,Mélanie Mondou,Meredith Young,Beth-Ann Cummings

doi:10.1007/s40037-019-00554-3

Abstract

IntroductionIn-training assessment reports (ITARs) summarize assessment during a clinical placement to inform decision-making and provide formal feedback to learners. Faculty development is an effective but resource-intensive means of improving the quality of completed ITARs. We examined whether the quality of completed ITARs could be improved by ‘nudges’ from the format of ITAR forms.MethodsOur first intervention consisted of placing the section for narrative comments at the beginning of the form, and using prompts for recommendations (Do more, Keep doing, Do less, Stop doing). In a second intervention, we provided a hyperlink to a detailed assessment rubric and shortened the checklist section. We analyzed a sample of 360 de-identified completed ITARs from six disciplines across the three academic years where the different versions of the ITAR were used. Two raters independently scored the ITARs using the Completed Clinical Evaluation Report Rating (CCERR) scale. We tested for differences between versions of the ITAR forms using a one-way ANOVA for the total CCERR score, and MANOVA for the nine CCERR item scores.ResultsChanges to the form structure (nudges) improved the quality of information generated as measured by the CCERR instrument, from a total score of 18.0/45 (SD 2.6) to 18.9/45 (SD 3.1) and 18.8/45 (SD 2.6), p = 0.04. Specifically, comments were more balanced, more detailed, and more actionable compared with the original ITAR.DiscussionNudge interventions, which are inexpensive and feasible, should be included in multipronged approaches to improve the quality of assessment reports.

Highlights

In-training assessment reports (ITARs) summarize assessment during a clinical placement to inform decision-making and provide formal feedback to learners
Workplace-based assessment is traditionally documented at the end of a clinical rotation in an in-training assessment report (ITAR—previously referred to as in-training evaluation report or ITER)[2]
The shift to competency-based medical education is leading to more frequent documentation of specific assessment events, ITARs continue to play a role in synthesizing assessments for decision-making and for providing formal feedback to learners [2]

Summary

Methods

Our first intervention consisted of placing the section for narrative comments at the beginning of the form, and using prompts for recommendations (Do more, Keep doing, Do less, Stop doing). We provided a hyperlink to a detailed assessment rubric and shortened the checklist section. We analyzed a sample of 360 de-identified completed ITARs from six disciplines across the three academic years where the different versions of the ITAR were used. Cummings Undergraduate Medical Education, Department of Medicine, and Institute of Health Sciences Education; Faculty of Medicine, McGill University, Montreal, QC, Canada. Young Department of Medicine and Institute of Health Sciences Education; Faculty of Medicine, McGill University, Montreal, QC, Canada between versions of the ITAR forms using a one-way ANOVA for the total CCERR score, and MANOVA for the nine CCERR item scores

Results

Background

Evaluation plan

Evaluation findings