Abstract
PurposeThe aim of this study was to implement and evaluate a quality assurance (QA) workflow that leverages natural language processing to rapidly resolve inadvertent discordance between radiologists and an artificial intelligence (AI) decision support system (DSS) in the interpretation of high-acuity CT studies when the radiologist does not engage with AI DSS output. MethodsAll consecutive high-acuity adult CT examinations performed in a health system between March 1, 2020, and September 20, 2022, were interpreted alongside an AI DSS (Aidoc) for intracranial hemorrhage, cervical spine fracture, and pulmonary embolus. CT studies were flagged for this QA workflow if they met three criteria: (1) negative results by radiologist report, (2) a high probability of positive results by the AI DSS, and (3) unviewed AI DSS output. In these cases, an automated e-mail notification was sent to our quality team. If discordance was confirmed on secondary review—an initially missed diagnosis—addendum and communication documentation was performed. ResultsOf 111,674 high-acuity CT examinations interpreted alongside the AI DSS over this 2.5-year time period, the frequency of missed diagnoses (intracranial hemorrhage, pulmonary embolus, and cervical spine fracture) uncovered by this workflow was 0.02% (n = 26). Of 12,412 CT studies prioritized as depicting positive findings by the AI DSS, 0.4% (n = 46) were discordant, unengaged, and flagged for QA. Among these discordant cases, 57% (26 of 46) were determined to be true positives. Addendum and communication documentation was performed within 24 hours of the initial report signing in 85% of these cases. ConclusionsInadvertent discordance between radiologists and the AI DSS occurred in a small number of cases. This QA workflow leveraged natural language processing to rapidly detect, notify, and resolve these discrepancies and prevent potential missed diagnoses.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
