ChatGPT-Assisted Classification of Postoperative Bleeding Following Microinvasive Glaucoma Surgery Using Electronic Health Record Data

Abstract

PurposeTo evaluate the performance of a large language model (LLM) in classifying electronic health record (EHR) text, and to use this classification to evaluate the type and resolution of hemorrhagic events (HE) following micro-invasive glaucoma surgery (MIGS). DesignRetrospective cohort study. ParticipantsEyes from the Bascom Palmer Glaucoma Repository. MethodsEyes that underwent MIGS between July 1, 2014 and February 1, 2022 were analyzed. ChatGPT was used to classify deidentified EHR anterior chamber examination text into HE categories (no hyphema, microhyphema, clot, and hyphema). Agreement between classifications by ChatGPT and a glaucoma specialist was evaluated using Cohen’s Kappa and precision-recall (PR) curve. Time to resolution of HEs was assessed using Cox proportional-hazards models. Goniotomy HE resolution was evaluated by degree of angle treatment (90-179º, 180-269º, 270-360º). Logistic regression was used to identified HE risk factors. Main Outcome MeasuresAccuracy of ChatGPT HE classification and incidence and resolution of HEs. ResultsThe study included 434 goniotomy eyes (368 patients) and 528 Schlemm’s Canal Stent (SCS) eyes (390 patients). ChatGPT facilitated excellent HE classification (Cohen’s kappa 0.93, area under PR curve 0.968). Using ChatGPT classifications, at postoperative day 1, HEs occurred in 67.8% of goniotomy and 25.2% of SCS eyes (p<0.001). The 270-360º goniotomy group had the highest HE rate (84.0%, p<0.001). At postoperative week 1, HEs were observed in 43.4% and 11.3% of goniotomy and SCS eyes respectively (p<0.001). By postoperative month 1, HE rates were 13.3% and 1.3% among goniotomy and SCS eyes respectively (p<0.001). Time to HE resolution differed between the goniotomy angle groups (log-rank p=0.034); median time to resolution was 10, 10, and 15 days for the 90-179º, 180-269º, and 270-360º groups respectively. Risk factor analysis demonstrated greater goniotomy angle was the only significant predictor of HEs (OR for 270-360º: 4.08, p<0.001). ConclusionsLLMs can be effectively used to classify longitudinal EHR free-text exam data with high accuracy, highlighting a promising direction for future LLM-assisted research and clinical decision support. HEs are relatively common, self-resolving complications that occur more often in goniotomy cases and with larger goniotomy treatments. Time to HE resolution differs significantly between goniotomy groups.