Application of natural language processing to assess the performance status documentation quality metric in patients with non–small-cell lung cancer.

Abstract

e13582 Background: Performance status (PS) assessment is important for cancer prognosis and treatment decision-making. PS documentation for stage IV non-small cell lung cancer (NSCLC) is a quality measure for ASCO Quality Oncology Practice Initiative (QOPI) Certification Program. We used natural language processing (NLP) to extract PS from unstructured electronic medical records (EMR) clinical notes in patients with metastatic NSCLC to verify PS documentation compliance and disparities in PS documentation. Methods: We identified oncology progress notes of stage IV NSCLC patients from Harris Health System, which is an integrated comprehensive healthcare system for underserved patients in Houston, TX. All progress notes within 60 days of first oncology clinic visit date were filtered based on visit type and author type. A rule-based regular expression NLP algorithm was used to extract the PS, namely Eastern Standard Cooperative Group (ECOG) or Karnofsky Performance Status (KPS). A total of 400 oncology notes were randomly selected from 400 distinct patients and annotated by two clinician reviewers, and discrepancies were resolved by a senior clinician judge. The annotated notes were split randomly into 200 training notes and 200 validation notes, and the performance of the NLP algorithm was evaluated. We used logistic regression to analyze potential disparity in PS documentation. Results: We identified 15,239 progress notes from 757 stage IV NSCLC patients. The NLP algorithm accuracy validated on the test data was 99% with corresponding positive predictive values (PPVs) for ECOG PS 0, 1, 2, 3, 4 of 100%, 100%, 95%, 94%, 94%, respectively. The NLP algorithm identified 85% of patients with appropriate PS documentation (sensitivity 100%, specificity 91%, positive predictive value 98%, negative predictive value 100%). Logistic regression analyses showed no significant differences in association between PS documentation and race/ethnicity, language, or sex. Conclusions: We developed an internally validated NLP algorithm to extract PS from EMR clinical notes. Despite potential language barriers, the QOPI metric of >75% PS documentation was met in our hospital system without significant disparities in clinical care. [Table: see text]