Automated detection of steno-occlusive lesion on time-of-flight magnetic resonance angiography: an observer performance study.

Abstract

Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an artificial intelligence model for detecting steno-occlusive lesions in the intracranial arteries. Overall, 138 TOF-MRA images were collected from two institutions, which served as internal (n = 62) and external (n = 76) test sets, respectively. Each study was reviewed by five radiologists (two neuroradiologists and three radiology residents) to compare the usage and non-usage of our proposed artificial intelligence model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed using the area under the Jackknife free-response receiver operating characteristic curve and reading time for comparison. The average area under the Jackknife free-response receiver operating characteristic curve for the five radiologists demonstrated an improvement from 0.70 without artificial intelligence to 0.76 with artificial intelligence (P = .027). Notably, this improvement was most pronounced among the three radiology residents, whose performance metrics increased from 0.68 to 0.76 (P = .002). Despite an increased reading time upon using artificial intelligence, there was no significant change among the readings by radiology residents. Moreover, the use of artificial intelligence resulted in improved inter-observer agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752). Our proposed artificial intelligence model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less-experienced readers may benefit the most from this model.ABBREVIATIONS: AI = Artificial intelligence; AUC = Area under the receiver operating characteristic curve; AUFROC = Area under the Jackknife free-response receiver operating characteristic curve; DL = Deep learning; ICC = Intraclass correlation coefficient; IRB = Institutional Review Boards; JAFROC = Jackknife free-response receiver operating characteristic.