Pre-deployment assessment of an AI model to assist radiologists in chest X-ray detection and identification of lead-less implanted electronic devices for pre-MRI safety screening: realized implementation needs and proposed operational solutions.

Abstract

Chest X-ray (CXR) use in pre-MRI safety screening, such as for lead-less implanted electronic device (LLIED) recognition, is common. To assist CXR interpretation, we "pre-deployed" an artificial intelligence (AI) model to assess (1)accuracies in LLIED-type (and consequently safety-level) identification, (2)safety implications of LLIED nondetections or misidentifications, (3)infrastructural or workflow requirements, and (4)demands related to model adaptation to real-world conditions. A two-tier cascading methodology for LLIED detection/localization and identification on a frontal CXR was applied to evaluate the performance of the original nine-class AI model. With the unexpected early appearance of LLIED types during simulated real-world trialing, retraining of a newer 12-class version preceded retrialing. A zero footprint (ZF) graphical user interface (GUI)/viewer with DICOM-based output was developed for inference-result display and adjudication, supporting end-user engagement and model continuous learning and/or modernization. During model testing or trialing using both the nine-class and 12-class models, robust detection/localization was consistently 100%, with mAP 0.99 from fivefold cross-validation. Safety-level categorization was high during both testing ( and , respectively) and trialing (accuracy 98% and 97%, respectively). LLIED-type identifications by the two models during testing (1)were 98.9% and 99.5% overall correct and (2)consistently showed (1.00 for 8/9 and 9/12 LLIED-types, respectively). Pre-deployment trialing of both models demonstrated overall type-identification accuracies of 94.5% and 95%, respectively. Of the small number of misidentifications, none involved MRI-stringently conditional or MRI-unsafe types of LLIEDs. Optimized ZF GUI/viewer operations led to greater user-friendliness for radiologist engagement. Our LLIED-related AI methodology supports (1)100% detection sensitivity, (2)high identification (including MRI-safety) accuracy, and (3)future model deployment with facilitated inference-result display and adjudication for ongoing model adaptation to future real-world experiences.