Using Natural Language Processing to Automatically Identify Dysplasia in Pathology Reports for Patients With Barrett’s Esophagus

Abstract

Identifying dysplasia of Barrett's esophagus (BE) in the electronic medical record (EMR) requires manual abstraction of unstructured data. Natural language processing (NLP) creates structure to unstructured free text. We aimed to develop and validate an NLP algorithm to identify dysplasia in BE patients on histopathology reports with varying report formats in a large integrated EMR system. We randomly selected 600 pathology reports for NLP development and 400 reports for validation from patients with suspected BE in the national Veterans Affairs databases. BE and dysplasia were verified by manual review of the pathology reports. We used NLP software (Clinical Language Annotation, Modeling, and Processing Toolkit; Melax Tech, Houston, TX) to develop an algorithm to identify dysplasia using findings. The algorithm performance characteristics were calculated as recall, precision, accuracy, and F-measure. In the development set of 600 patients, 457 patients had confirmed BE (60 with dysplasia). The NLP identified dysplasia with 98.0% accuracy, 91.7% recall, and 93.2% precision, with an F-measure of 92.4%. All 7 patients with confirmed high-grade dysplasia were classified by the algorithm as having dysplasia. Among the 400 patients in the validation cohort, 230 had confirmed BE (39 with dysplasia). Compared with manual review, the NLP algorithm identified dysplasia with 98.7% accuracy, 92.3% recall, and 100.0% precision, with an F-measure of 96.0%. NLP yielded a high degree of sensitivity and accuracy for identifying dysplasia from diverse types of pathology reports for patients with BE. The application of this algorithm would facilitate research and clinical care in an EMR system with text reports in large data repositories.