ChatGPT vs Expert-Guided Care Pathways for Postesophagectomy Symptom Management

Abstract

BackgroundThe objective of this study was to compare generative artificial intelligence–initiated care pathways, using ChatGPT, with expert-guided consensus-initiated care pathways from AskMayoExpert (AME) for symptom management of esophageal cancer patients after esophagectomy. MethodsA formal protocol for development of 9 AME care pathways was followed for specific patient-identified domains after esophagectomy for esophageal cancer. Domain scores were measured and assessed through the Upper Digestive Disease tool. These care pathways were developed by experts validated by a consensus-driven methodology. ChatGPT was used to answer specific questions similar to the AME care pathway on April 9, 2023, and March 28, 2024. To compare outcomes, answers were recorded, and algorithms were compared with a survey tool composed of 5 questions. ResultsBoth modalities were able to provide a clear definition with multidirectional management options for all 9 domains: dysphagia, generalized dumping, gastrointestinal dumping, pain, regurgitation, heartburn, nausea, physical health, and mental health. When provided with a simple prompt, ChatGPT 3.5 failed to provide a comprehensive stepwise approach for providers, any testing recommendations, or any form of triage process. However, ChatGPT 4.0 provided plans, similar to AME care pathways, when a sophisticated prompt was used. ConclusionsGenerative artificial intelligence–initiated care pathways can be used by physicians as a supplementary tool to guide provider management of patients with complex symptoms after esophagectomy. This technology will continue to advance but is currently insufficient to solely guide clinical management of complex patients with severe symptoms.