IMG-32. USER EVALUATION OF AN AI-POWERED WEB APPLICATION TO SUPPORT RADIOGRAPHIC INTERPRETATION OF PATIENTS WITH ADAMANTINOMATOUS CRANIOPHARYNGIOMA

Abstract

Abstract BACKGROUND In neuro-oncology, accurately interpreting radiographic images, especially for diagnosing central nervous system (CNS) tumors like Adamantinomatous Craniopharyngioma (ACP), presents formidable challenges. Even with advanced imaging technologies such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT), pinpointing features related to ACP invasiveness remains daunting. METHODS We introduced an interactive web application that utilizes a custom AI model to analyze preoperative MRI and CT images of 50 patients with ACP. This application can predict a 12-point radiographic feature profile and rank patients based on similarity, aiming to support clinicians in making more precise diagnoses. We deployed our application on a hospital-grade radiographic reading terminal. Our application’s utility and user-friendliness were evaluated through a user study involving four neurosurgeons and two neuroradiologists, who constructed 12-point radiographic feature profiles for ten patients within a 30-minute timeframe given patient MRI and CT images. RESULTS Our study subjects reported high self-assessed numeracy (SNS scores ranging from 4 to 5.25) despite varied preferences in engaging with numerical data. Subjects could efficiently learn to navigate our software interface, with an average task completion time of approximately 3 minutes and 25 seconds per patient. Introducing AI into the task workflow resulted in a slight, statistically non-significant increase in completion time, indicating that while AI modifies interaction dynamics, it does not significantly hinder efficiency in task performance. We observed distinct agreement patterns between clinicians and AI prediction, utilizing Fleiss’ Kappa for categorization. AI assistance subtly boosted clinicians’ confidence and slightly lessened the perceived difficulty of interpreting radiographic images. However, these benefits were not statistically significant. CONCLUSIONS Our study highlights the promise of AI in improving neuro-oncology diagnostics, showing that clinicians quickly adapted to an AI-enhanced application, which subtly influenced their radiographic interpretation. This finding underscores the potential for AI to enhance decision-making accuracy in challenging cases like ACP.