Development and validation of a natural language processing algorithm using electronic health record data to identify patients with breast cancer with low social support.

Abstract

421 Background: Social support is important to the management of breast cancer treatment. Our team has developed data from electronic health record (EHR) data into structured ‘concept groups’ that will form the basis for the development of EHRsupport, a computable, EHR-based measure of social support. We report the evaluation of these concept groups against chart review as a part of our validation. Methods: We built a natural language processing (NLP) algorithm on clinical notes in 7,989 women diagnosed from January 2006 to September 2021 with invasive breast cancer. We identified and developed 10 concept groups from unstructured data: 1) living situation, 2) marital/partner status, 3) parenthood status, 4) visit support (accompanied patient to ≥1 visit, patient attended alone at ≥1 visit), 5) friends/other support, 6) explicit positive or negative mentions of social support, 7) mention of a deceased person, 8) transportation issues, 9) relationship conflict or stress, and 10) social isolation. We validated concept groups against the charts of 100 patients randomly drawn from the broader patient population (nonoverlapping with the training data set) also around the time (-1 to +3 months) of diagnosis. Results: Concept group data availability ranged from 1.3% social isolation to 98.3% for living situation. Specificity and negative predictive values were moderate to high for all concept groups. Sensitivity and positive predictive values were moderate to high for concept groups with high data availability (Table) and lower for concept groups with low data availability. Conclusions: Data on social support have been available since the advent of Epic in 2006 and our NLP-based algorithm accurately captured data within the EHR that were systematically collected supporting the development of a clinical tool that can be used to identify patients at risk of low social support. EHRsupport concept groups, data availability (n=7,989), and evaluation vs. chart review. Availability (%) Sensitivity Specificity PPV NPV Living situation (e.g., alone or not) 98.3 100 92 61 100 Partner/spouse 92.0 85 100 100 81 Parenthood status 88.8 95 80 93 83 Visit support 82.9 92 77 96 59 Positive mentions social support *46.9 75 100 100 70 Negative mentions social support 33 99 67 96 Friend/other support 46.8 81 81 79 83 Deceased person 39.9 86 97 95 90 Transportation issues 14.6 80 85 22 99 Relationship conflict/stress 7.7 25 95 29 94 *Availability of positive and negative mentions of social support.