Harnessing the Power of Generative AI for Clinical Summaries: Perspectives From Emergency Physicians

Abstract

ImportanceThe workload of clinical documentation contributes to healthcare costs and professional burnout. The advent of generative AI language models presents a promising solution. The perspective of clinicians may contribute to effective and responsible implementation of such tools. ObjectiveTo evaluate 3 uses for generative AI for clinical documentation in pediatric emergency medicine (PEM), measuring time savings, effort reduction, physician attitudes, and identifying potential risks and barriers. DesignA mixed-methods study. SettingSingle Pediatric Emergency Department. Participants10 PEM attending physicians. InterventionParticipants were asked to write a supervisory note for four clinical scenarios, with varying level of complexity, twice without any assistance and twice with the assistance of ChatGPT Version 4.0. Participants evaluated two additional ChatGPT-generated clinical summaries: a structured handoff and a visit summary for a family, written at an 8th grade reading level. Finally, a semi-structured interview was performed to assess physicians’ perspective on the use of ChatGPT in PEM. Main Outcomes and MeasuresBetween subjects’ comparison of the effort and time taken to complete the supervisory note with and without ChatGPT assistance. Effort was measured using a self-reported Likert scale of 0-10. Physicians’ scoring of and attitude toward the ChatGPT-generated summaries was measured using a 0-10 Likert scale and open-ended questions. Summaries were scored for completeness, accuracy, efficiency, readability, and overall satisfaction. A thematic analysis was performed to analyze the content of the open-ended questions and to identify key themes. ResultsChatGPT yielded a 40% reduction in time and a 33% decrease in effort for supervisory notes in intricate cases, with no discernible impact on simpler ones. ChatGPT-generated summaries for structured handoffs and family-letters were highly rated, ranging from 7.0 to 9.0 out of 10, and most participants favored their inclusion in clinical practice. However, there were several critical reservations, out of which a set of general recommendations for applying ChatGPT to clinical summaries was formulated. ConclusionsPEM attendings in our study perceived that ChatGPT can deliver high-quality summaries while saving time and effort in many scenarios, but not all.