Abstract
Background: Radiological diagnosis is error prone, yet accurate error rates are difficult to study. Current literature typically relies on manual review of scans, likely placing an upper limit on accuracy. AI-assisted retrospective review of finalized CT scans may help quantify error rates better than conventional methods used in prior discrepancy studies. In this fashion, we focus here on quantifying the rate of intracranial hemorrhage (ICH) missed on neuroradiologist (NR) finalized CT reports. Evaluation: A retrospective study compared discrepancies between CT reports and AI-based detection of ICH. All non-contrast CT brain cases were collected from our institution over two months; cases reported as negative for ICH were included (N=1812), while cases reported as suspicious for ICH (N=504) were excluded. Fellowship-trained NRs finalized all reports. An AI-tool trained to detect ICH was then run on the case set. A subset of discrepant “negative by report” but ”positive by AI” cases was isolated (N=36). This subset was then reviewed again by two separate experimenter-NRs and then together as needed to achieve consensus regarding their agreement or disagreement with the AI-based ICH detection. Upon final consensus, 22 cases of ICH detected by AI were missed on finalized reports, yielding an error rate of 4.2% (22/(504+22)). Discussion: This ICH miss rate of 4.2% on finalized NR reports was higher than expected, likely reflecting a higher sensitivity involved in AI-assisted screening of scan/report discrepancies in comparison to conventional manual review. Missed ICH can result in potentially devastating consequences for a patient. Better quantification and characterization of ICH diagnostic misses might offer insight into biases that dispose to error, and help guide improved radiologist performance and better patient care. Conclusion: AI-augmentation may help flag potential discrepancies between scans and their reports, possibly improving quantification of error rates among practicing radiologists. The result further suggests that AI-augmented ICH detection, if used in real time, may improve radiologist detection of ICH that may otherwise get overlooked.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call