Utility of an Artificial Intelligence Language Model for Post-Operative Patient Instructions Following Facial Trauma

Abstract

Study Design Qualitative Descriptive Study. Objective To evaluate the utility of post-operative instructions (POIs) for facial trauma provided by the language model ChatGPT as compared to those from electronic medical record (EMR) templates and Google search engine. Methods POIs for four common facial trauma procedures (mandibular fracture, maxillary fracture, nasal fracture, and facial laceration) were generated by ChatGPT, extracted from EMR templates, or obtained from Google search engine. The POIs were evaluated by a panel of surgeons using the Patient Education Materials Assessment Tool (PEMAT-P). Results Average PEMAT-P understandability scores across all 3 sources ranged from 55% to 94%. Actionability scores ranged from 33% to 94%, and procedure-specific scores ranged from 50% to 92%. ChatGPT-generated POIs ranged from 82% to 88% for understandability, 60% to 80% for actionability, and 50% to 75% for procedure-specific items. Conclusions ChatGPT has great potential to be a useful adjunct in providing post-operative instructions for patients undergoing facial trauma procedures. Detailed and patient-specific prompts are necessary to output tailored instructions that are accurate, understandable, and actionable.