P581Electronic health records (EHRs) data validation in atherosclerotic/cardiovascular clinical phenotypes

Abstract

Abstract Background Research efforts to develop strategies to effectively identify patients and reduce the burden of cardiovascular diseases is essential for the future of the health system. Most research studies have used only coded parts of electronic health records (EHRs) for case-detection obtaining missed data cases, reducing study quality and in some case bias findings. Incorporating information from free-text into case-detection through Big Data and Artificial Intelligence techniques improves research quality. Savana has developed EHRead, a powerful technology that applies Natural Language Processing, Machine Learning and Deep Learning, to analyse and automatically extracts highly valuable medical information from unstructured free text contained in the EHR to support research and practice. Purpose We aimed to validate the linguistic accuracy performance of Savana, in terms of Precision (P), Recall (R) and overall performance (F-Score) in the cardiovascular domain since this is one of the most prevalent disease in the general population. This means validating the extent to which the Savana system identifies mentions to atherosclerotic/cardiovascular clinical phenotypes in EHRs. Methods The project was conducted in 3 Spanish sites and the system was validated using a corpus that consisted of 739 EHRs, including the emergency, medical and discharge records, written in free text. These EHRs were randomly selected from the total number of clinical documents generated during the period of 2012–2017 and were fully anonymized to comply with legal and ethical requirements. Two physicians per site reviewed records (randomly selected) and annotated all direct references to atherosclerotic/cardiovascular clinical phenotypes, following the annotation guidelines previously developed. A third physician adjudicated discordant annotations. Savana's performance was automatically calculated using as validation resource the gold standard created by the experts. Results We found good levels of performance achieved by Savana in the identification of mentions to atherosclerotic/cardiovascular clinical phenotypes, yielding an overall P, R, and F-score of 0.97, 0.92, and 0.94, respectively. We also found that going through all the EHRs and identifying the mentions to atherosclerotic/cardiovascular clinical phenotypes, the expert spent ∼ 60h while Savana ∼ 36 min. Conclusion(s) Innovative techniques to identify atherosclerotic/cardiovascular clinical phenotypes could be used to support real world data research and clinical practice. Overall Savana showed a high performance, comparable with those obtained by an expert physician annotator doing the same task. Additionally, a significant reduction of time in using automatic information extraction system was achieved.