A web-based, LLM-powered AI symptom summarization tool (ASST) for monitoring of breast cancer treatment toxicity.

Abstract

Listen

Translate article

e13622 Background: Traditional methods for capturing patient-reported outcomes (PROs) are often time-consuming and may lead to underreporting of symptoms by physicians. With advancements in artificial intelligence (AI), particularly large language models (LLMs) like GPT-4, there is potential to revolutionize this aspect of patient care. Methods: This study developed and tested a GPT-4-based web application for monitoring cancer treatment toxicity in breast cancer patients. The application utilized 35 items from the PRO-CTCAE scale to create an interactive form for patients to report treatment-related symptoms. Upon form completion, a natural language summary of the patient’s responses was generated using the GPT-4 API for physician review. Nine radiation oncologists (5 attendings, 4 residents) evaluated the summaries of virtual patients using the adapted Physician Documentation Quality Index (PDQI-9). PDQI-9 consists of 9 items scored using a 5-point Likert scale (1 -not at all- to 5. -extremely-). IRB approval was not required because this study did not use real patient data, in accordance with the 2018 Revised Common Rule Requirements of the NIH. All data used were researcher-generated and non-identifiable. Results: The mean time for textual summary generation was 7 (5.7-9.2) seconds. The AI-Symptom Summarization Tool (ASST) mean scores were 4.25 for accuracy and 4 for thoroughness. Usefulness, organization, comprehensibility, concision, synthesis quality and consistency were all rated between 3 and 4. Overall score was 42/50. No hallucinations (false information made up by the LLM) were found. Conclusions: The GPT-4-based application for cancer treatment toxicity monitoring demonstrates significant promise in enhancing the quality and efficiency of patient care. By automating the documentation of patient-reported symptoms, this tool could allow physicians to focus more time on patient interaction and individualized care. As AI technologies continue to evolve, their integration into clinical practice must be approached with caution, emphasizing augmentation over replacement and the importance of verification to maintain trust in these new tools.