Use of a Mobile Diary App to Influence Narrative Feedback on Medical Student Evaluations

Abstract

Local Mentor: Wendy Hansen, MD and Brady Reynolds, PhD APGO Advisor: Jody Steinauer, MD OBJECTIVE: To evaluate the impact of using a mobile diary app on the quantity and quality of narrative feedback written on medical student evaluations during medicine, surgery, and ob/gyn rotations at rural and university hospitals. METHODS: Narrative feedback written by 46 faculty members was obtained for 6 months during a cohort study (baseline 1-3, comparison 4-6). The intervention (month 3) instructed participants to keep daily notes on student performance using a diary app on their mobile device. Quantity was determined by a simple word count, compared before and after the intervention. Quality was evaluated by 4 blinded raters using The Quality Improvement Instrument (QI) and the Completed Clinical Evaluation Report Rating (CCERR). The study had 80% power to detect a 30% increase in word count. Scores were compared using a T-test. RESULTS: Baseline evaluations (n=43) contained an average word count of 43 (range 2 to 216). The baseline mean CCERR score was 16.6 and the Quality Improvement score was 2.8. Preliminary results from the post-intervention evaluations (n=45) included an average word count of 45 (range 5 to 169). The mean CCERR and QI scores were 21.4 and 3.4, respectively. No differences were statistically significant at a P<.5. CONCLUSION: Use of a mobile diary app to record daily student performance did not statistically improve the quantity or rated quality of narrative feedback in medical student education. Effort should be focused elsewhere to improve written comments by faculty.