Comparison and validation of algorithms for asthma diagnosis in an electronic medical record system

Abstract

BackgroundAsthma is one of the most common chronic health conditions, and to leverage the wealth of data in the electronic medical record (EMR), it is important to be able to accurately identify asthma diagnosis. ObjectiveWe aimed to determine the rule-based algorithm with the most balanced performance for sensitivity and positive predictive value of asthma diagnosis. MethodsWe performed a diagnostic accuracy study of multiple rule-based algorithms intended to identify asthma diagnosis in the EMR. Algorithm performance was validated by manual chart review of 795 charts of patients seen in a multisite, tertiary-level, pulmonary specialty clinic using explicit diagnostic criteria to distinguish asthma cases from controls. ResultsAn asthma diagnosis anywhere in the medical record had a 97% sensitivity and a 77% specificity for asthma (F-score 80) when tested on a validation set of asthma cases and nonasthma respiratory disease from a pulmonary specialty clinic. The most balanced performance was seen with asthma diagnosis restricted to an encounter, hospital problem, or problem list diagnosis with a sensitivity of 94% and specificity of 85% (F-score 84). High sensitivity was achieved with the modified Health Plan Employer Data and Information Set criteria and high specificity was achieved with the NUgene algorithm, an algorithm developed for identifying asthma cases by EMR for genome-wide association studies. ConclusionAsthma diagnosis can be accurately identified for research purposes by restricting to encounter, hospital problem, or problem list diagnosis in a pulmonary specialty clinic. Additional rules lead to steep drop-offs in algorithm sensitivity in our population.