Abstract 135: Examining the Information Loss Between Neuroimages and Neuroimaging Reports for Detection of Silent Brain Infarcts and White Matter Disease Using Artificial Intelligence Technologies

Abstract

Background: Real world studies of silent brain infarction (SBI) and white matter disease (WMD) are impeded by challenges in cohort identification. Natural language processing (NLP) from imaging reports may facilitate future studies. However, electronic health records can be heterogeneous and the process of interpreting neuroimages and generating reports can vary. Understanding knowledge representation and relationships between neuroimages and imaging reports is crucial for using NLP to facilitate disease management and cohort identification. Methods: A balanced sample of head neuroimages (CT, MRI) of patients >50 years without clinical histories of symptomatic stroke, TIA, or dementia were obtained at Mayo Clinic and Tufts Medical Center. A team of 4 radiology residents performed report interpretation (RI) on 1000 reports according to a standardized protocol for the presence of SBIs, the presence of WMD, and WMD grade. A random subsample of 400 was doubly read for interrater reliability. For benchmarking, a team of 4 neuroradiologists directly reviewed and described findings on a subsample of 182 images, each doubly read. We assessed interrater reliability for direct review (DR) and RI, and agreement between these 2 information sources. An NLP algorithm was developed to review and extract findings from 1000 imaging reports. Results: For DR, interrater reliability was moderate for SBIs and WMD (k = 0.53, 95% CI 0.43-0.64 and k = 0.47, 95% CI 0.33-0.61) and good for WMD grade (Spearman 0.71, p<0.001). For RI, interrater reliability for SBIs, WMD and WMD grade was good (k = 0.88, 95% CI 0.80-0.97; k = 0.98, 95% CI 0.97-1.00; and Spearman = 0.985, p<0.001, respectively). Agreement between DR and RI was good for SBIs (k = 0.77, 95% CI 0.67-0.86) and WMD (k = 0.65, 95% CI 0.54-0.77). Spearman rank correlations comparing WMD grade interpretation DR to RI was 0.60 (p<0.001). In identifying the presence of SBIs and WMDs, the accuracy of the NLP algorithm was 0.991 and 0.994, respectively. Conclusion: For the presence of SBI and WMD, and WMD grade, agreement between RI and DR was similar to agreement between 2 neuroradiologists directly reviewing neuroimages. It is highly feasible to use NLP to identify patients with SBIs and WMDs for clinical studies.