Abstract
Purpose: Evaluate a large language model, GPT4 with vision (GPT-4V), for diagnosing vitreoretinal diseases in real-world ophthalmology settings. Methods: A retrospective cross-sectional study at Bascom Palmer Eye Clinic, analyzing patient data from January 2010 to March 2023, assesses GPT-4V’s performance on retinal image analysis and ICD-10 coding across two patient groups: simpler cases (Group A) and complex cases (Group B) requiring more in-depth analysis. Diagnostic accuracy was assessed through open-ended (OEQ) and multiple-choice questions (MCQs) independently verified by three retina specialists. Results: In 256 eyes from 143 patients, GPT4-V demonstrated a 13.7% accuracy for OEQs and 31.3% for MCQs, with ICD-10 code accuracies at 5.5% and 31.3% respectively. Accurately diagnosed posterior vitreous detachment, non-exudative age-related macular degeneration, and retinal detachment. ICD-10 coding was most accurate for non-exudative age-related macular degeneration, central retinal vein occlusion, and macular hole in EOQs, and for posterior vitreous detachment, non-exudative age-related macular degeneration, and retinal detachment in MCQs. No significant difference in diagnostic or coding accuracy was found in Groups A and B. Conclusion: GPT-4V has potential in clinical care and record-keeping, particularly with standardized questions. Its effectiveness in open-ended scenarios is limited, indicating a significant limitation in providing complex medical advice.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Similar Papers
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.