Computable Phenotype Algorithms Research Articles

Developing an automated algorithm for identification of children and adolescents with diabetes using electronic health records from the OneFlorida+ clinical research network.

To develop an automated computable phenotype (CP) algorithm for identifying diabetes cases in children and adolescents usingelectronic health records (EHRs) from the UF Health System. The CP algorithm was iteratively derived based on structured data from EHRs (UF Health System 2012-2020). We randomly selected 536 presumed cases among individuals aged <18 years who had (1) glycated haemoglobin levels ≥ 6.5%; or (2) fasting glucose levels ≥126 mg/dL; or (3) random plasma glucose levels ≥200 mg/dL; or (4) a diabetes-related diagnosis code from an inpatient or outpatient encounter; or (5) prescribed, administered, or dispensed diabetes-related medication. Four reviewers independently reviewed the patient charts to determine diabetes status and type. Presumed cases without type 1 (T1D) or type 2 diabetes (T2D) diagnosis codes were categorized as non-diabetes/other types of diabetes. The rest were categorized as T1D if the most recent diagnosis was T1D, or otherwise categorized as T2D if the most recent diagnosis was T2D. Next, we applied a list of diagnoses and procedures that can determine diabetes type (e.g., steroid use suggests induced diabetes) to correct misclassifications from Step 1. Among the 536 reviewed cases, 159 and 64 had T1D and T2D, respectively. The sensitivity, specificity, and positive predictive values of the CP algorithm were 94%, 98% and 96%, respectively, for T1D and 95%, 95% and 73% for T2D. We developed a highly accurate EHR-based CP for diabetes in youth based on EHR data from UF Health. Consistent with prior studies, T2D was more difficult to identify using these methods.

Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study.

Adverse events associated with vaccination have been evaluated by epidemiological studies and more recently have gained additional attention with the emergency use authorization of several COVID-19 vaccines. As part of its responsibility to conduct postmarket surveillance, the US Food and Drug Administration continues to monitor several adverse events of special interest (AESIs) to ensure vaccine safety, including for COVID-19. This study is part of the Biologics Effectiveness and Safety Initiative, which aims to improve the Food and Drug Administration's postmarket surveillance capabilities while minimizing public burden. This study aimed to enhance active surveillance efforts through a rules-based, computable phenotype algorithm to identify 5 AESIs being monitored by the Center for Disease Control and Prevention for COVID-19 or other vaccines: anaphylaxis, Guillain-Barré syndrome, myocarditis/pericarditis, thrombosis with thrombocytopenia syndrome, and febrile seizure. This study examined whether these phenotypes have sufficiently high positive predictive value (PPV) to ensure that the cases selected for surveillance are reasonably likely to be a postbiologic adverse event. This allows patient privacy, and security concerns for the data sharing of patients who had nonadverse events can be properly accounted for when evaluating the cost-benefit aspect of our approach. AESI phenotype algorithms were developed to apply to electronic health record data at health provider organizations across the country by querying for standard and interoperable codes. The codes queried in the rules represent symptoms, diagnoses, or treatments of the AESI sourced from published case definitions and input from clinicians. To validate the performance of the algorithms, we applied them to electronic health record data from a US academic health system and provided a sample of cases for clinicians to evaluate. Performance was assessed using PPV. With a PPV of 93.3%, our anaphylaxis algorithm performed the best. The PPVs for our febrile seizure, myocarditis/pericarditis, thrombocytopenia syndrome, and Guillain-Barré syndrome algorithms were 89%, 83.5%, 70.2%, and 47.2%, respectively. Given our algorithm design and performance, our results support continued research into using interoperable algorithms for widespread AESI postmarket detection.

A general framework for developing computable clinical phenotype algorithms.

To present a general framework providing high-level guidance to developers of computable algorithms for identifying patients with specific clinical conditions (phenotypes) through a variety of approaches, including but not limited to machine learning and natural language processing methods to incorporate rich electronic health record data. Drawing on extensive prior phenotyping experiences and insights derived from 3 algorithm development projects conducted specifically for this purpose, our team with expertise in clinical medicine, statistics, informatics, pharmacoepidemiology, and healthcare data science methods conceptualized stages of development and corresponding sets of principles, strategies, and practical guidelines for improving the algorithm development process. We propose 5 stages of algorithm development and corresponding principles, strategies, and guidelines: (1) assessing fitness-for-purpose, (2) creating gold standard data, (3) feature engineering, (4) model development, and (5) model evaluation. This framework is intended to provide practical guidance and serve as a basis for future elaboration and extension.

Centralized Interactive Phenomics Resource: an integrated online phenomics knowledgebase for health data users.

Development of clinical phenotypes from electronic health records (EHRs) can be resource intensive. Several phenotype libraries have been created to facilitate reuse of definitions. However, these platforms vary in target audience and utility. We describe the development of the Centralized Interactive Phenomics Resource (CIPHER) knowledgebase, a comprehensive public-facing phenotype library, which aims to facilitate clinical and health services research. The platform was designed to collect and catalog EHR-based computable phenotype algorithms from any healthcare system, scale metadata management, facilitate phenotype discovery, and allow for integration of tools and user workflows. Phenomics experts were engaged in the development and testing of the site. The knowledgebase stores phenotype metadata using the CIPHER standard, and definitions are accessible through complex searching. Phenotypes are contributed to the knowledgebase via webform, allowing metadata validation. Data visualization tools linking to the knowledgebase enhance user interaction with content and accelerate phenotype development. The CIPHER knowledgebase was developed in the largest healthcare system in the United States and piloted with external partners. The design of the CIPHER website supports a variety of front-end tools and features to facilitate phenotype development and reuse. Health data users are encouraged to contribute their algorithms to the knowledgebase for wider dissemination to the research community, and to use the platform as a springboard for phenotyping. CIPHER is a public resource for all health data users available at https://phenomics.va.ornl.gov/ which facilitates phenotype reuse, development, and dissemination of phenotyping knowledge.

Use of Recommended Neurodiagnostic Evaluation Among Patients With Drug-Resistant Epilepsy

Interdisciplinary practice parameters recommend that patients with drug-resistant epilepsy (DRE) undergo comprehensive neurodiagnostic evaluation, including presurgical assessment. Reporting from specialized centers suggests long delays to referral and underuse of surgery; however, longitudinal data are limited to characterize neurodiagnostic evaluation among patients with DRE in more diverse US settings and populations. To examine the rate and factors associated with neurodiagnostic studies and comprehensive evaluation among patients with DRE within 3 US cohorts. A retrospective cross-sectional study was conducted using the Observational Medical Outcomes Partnership Common Data Model including US multistate Medicaid data, commercial claims data, and Columbia University Medical Center (CUMC) electronic health record data. Patients meeting a validated computable phenotype algorithm for DRE between January 1, 2015, and April 1, 2020, were included. No eligible participants were excluded. Demographic and clinical variables were queried. The proportion of patients receiving a composite proxy for comprehensive neurodiagnostic evaluation, including (1) magnetic resonance or other advanced brain imaging, (2) video electroencephalography, and (3) neuropsychological evaluation within 2 years of meeting the inclusion criteria. A total of 33 542 patients with DRE were included in the Medicaid cohort, 22 496 in the commercial insurance cohort, and 2741 in the CUMC database. A total of 31 516 patients (53.6%) were women. The proportion of patients meeting the comprehensive evaluation main outcome in the Medicaid cohort was 4.5% (n = 1520); in the commercial insurance cohort, 8.0% (n = 1796); and in the CUMC cohort, 14.3% (n = 393). Video electroencephalography (24.9% Medicaid, 28.4% commercial, 63.2% CUMC) and magnetic resonance imaging of the brain (35.6% Medicaid, 43.4% commercial, 52.6% CUMC) were performed more regularly than neuropsychological evaluation (13.0% Medicaid, 16.6% commercial, 19.2% CUMC) or advanced imaging (3.2% Medicaid, 5.4% commercial, 13.1% CUMC). Factors independently associated with greater odds of evaluation across all 3 data sets included the number of inpatient and outpatient nonemergency epilepsy visits and focal rather than generalized epilepsy. The findings of this study suggest there is a gap in the use of diagnostic studies to evaluate patients with DRE. Care setting, insurance type, frequency of nonemergency visits, and epilepsy type are all associated with evaluation. A common data model can be used to measure adherence with best practices across a variety of observational data sources.

Using electronic health records to enhance surveillance of diabetes in children, adolescents and young adults: a study protocol for the DiCAYA Network

IntroductionTraditional survey-based surveillance is costly, limited in its ability to distinguish diabetes types and time-consuming, resulting in reporting delays. The Diabetes in Children, Adolescents and Young Adults (DiCAYA) Network seeks...

Demographics, Trends, and Clinical Characteristics of HIV Pre-Exposure Prophylaxis Recipients and People Newly Diagnosed with HIV from Large Electronic Health Records in Florida.

Florida is one of the HIV epicenters with high incidence and marked sociodemographic disparities. We analyzed a decade of statewide electronic health record/claims data-OneFlorida+-to identify and characterize pre-exposure prophylaxis (PrEP) recipients and newly diagnosed HIV cases in Florida. Refined computable phenotype algorithms were applied and a total of 2186 PrEP recipients and 7305 new HIV diagnoses were identified between January 2013 and April 2021. We examined patients' sociodemographic characteristics, stratified by self-reported sex, along with both frequency-driven and expert-selected descriptions of clinical conditions documented within 12 months before the first PrEP use or HIV diagnosis. PrEP utilization rate increased in both sexes; higher rates were observed among males with sex differences widening in recent years. HIV incidence peaked in 2016 and then decreased with minimal sex differences observed. Clinical characteristics were similar between the PrEP and new HIV diagnosis cohorts, characterized by a low prevalence of sexually transmitted infections (STIs) and a high prevalence of mental health and substance use conditions. Study limitations include the overrepresentation of Medicaid recipients, with over 96% of female PrEP users on Medicaid, and the inclusion of those engaged in regular health care. Although PrEP uptake increased in Florida, and HIV incidence decreased, sex disparity among PrEP recipients remained. Screening efforts beyond individuals with documented prior STI and high-risk behavior, especially for females, including integration of mental health care with HIV counseling and testing, are crucial to further equalize PrEP access and improve HIV prevention programs.

Developing a computable phenotype for glioblastoma.

Glioblastoma is the most common malignant brain tumor, and thus it is important to be able to identify patients with this diagnosis for population studies. However, this can be challenging as diagnostic codes are nonspecific. The aim of this study was to create a computable phenotype (CP) for glioblastoma multiforme (GBM) from structured and unstructured data to identify patients with this condition in a large electronic health record (EHR). We used the University of Florida (UF) Health Integrated Data Repository, a centralized clinical data warehouse that stores clinical and research data from various sources within the UF Health system, including the EHR system. We performed multiple iterations to refine the GBM-relevant diagnosis codes, procedure codes, medication codes, and keywords through manual chart review of patient data. We then evaluated the performances of various possible proposed CPs constructed from the relevant codes and keywords. We underwent six rounds of manual chart reviews to refine the CP elements. The final CP algorithm for identifying GBM patients was selected based on the best F1-score. Overall, the CP rule "if the patient had at least 1 relevant diagnosis code and at least 1 relevant keyword" demonstrated the highest F1-score using both structured and unstructured data. Thus, it was selected as the best-performing CP rule. We developed and validated a CP algorithm for identifying patients with GBM using both structured and unstructured EHR data from a large tertiary care center. The final algorithm achieved an F1-score of 0.817, indicating a high performance, which minimizes possible biases from misclassification errors.

Data-driven automated classification algorithms for acute health conditions: applying PheNorm to COVID-19 disease.

Automated phenotyping algorithms can reduce development time and operator dependence compared to manually developed algorithms. One such approach, PheNorm, has performed well for identifying chronic health conditions, but its performance for acute conditions is largely unknown. Herein, we implement and evaluate PheNorm applied to symptomatic COVID-19 disease to investigate its potential feasibility for rapid phenotyping of acute health conditions. PheNorm is a general-purpose automated approach to creating computable phenotype algorithms based on natural language processing, machine learning, and (low cost) silver-standard training labels. We applied PheNorm to cohorts of potential COVID-19 patients from 2 institutions and used gold-standard manual chart review data to investigate the impact on performance of alternative feature engineering options and implementing externally trained models without local retraining. Models at each institution achieved AUC, sensitivity, and positive predictive value of 0.853, 0.879, 0.851 and 0.804, 0.976, and 0.885, respectively, at quantiles of model-predicted risk that maximize F1. We report performance metrics for all combinations of silver labels, feature engineering options, and models trained internally versus externally. Phenotyping algorithms developed using PheNorm performed well at both institutions. Performance varied with different silver-standard labels and feature engineering options. Models developed locally at one site also worked well when implemented externally at the other site. PheNorm models successfully identified an acute health condition, symptomatic COVID-19. The simplicity of the PheNorm approach allows it to be applied at multiple study sites with substantially reduced overhead compared to traditional approaches.

Validation of a Computable Phenotype for Successful Cognitive Aging

AbstractBackgroundStudying successful cognitive aging presents an opportunity to identify factors that may mitigate risk for Alzheimer’s disease and related disorders to inform public health initiatives, community interventions, and policy. Yet, identifying these cohorts is challenging, especially at advanced ages. Commutable phenotype algorithms using electronic health records may serve as an aid to identify cohorts of advanced age adults with intact cognitive function. The purpose of this study was to 1) establish, 2) apply, and 3) validate a machine learning‐based computing algorithm for recruitment of successful aging individuals for clinical cohort study.MethodWe identified EHR‐available variables representing successful aging among individuals 85 years and older (85+) by interviewing a panel comprised of 10 aging experts. Based on the identified variables, we developed a rule‐based computable phenotype algorithm composed of 17 inclusion or exclusion criteria. We applied the computable phenotype algorithm to screen all living persons age 85+ at UF Health and identified those meeting successful aging criteria and endeavored to directly evaluate them.There were 52,841 living 85+ individuals identified at UF Health. Of these 24,024 were identified as successful aged by the computable phenotype algorithm. This sample frame was 58% female, 59% White, and 69% non‐Hispanic. A priori permission to be contacted for research had been obtained on 11,898 of these persons, of whom 470 responded to study announcements and 333 consented to evaluation.The main outcomes measures were individuals identified by the computational algorithm subsequently being validated by virtue of having a modified Telephone Interview for Cognitive Status score &gt;27 and Geriatric Depression Scale &lt;6.ResultsForty‐five percent of all living persons 85+ in the UF Health EHR database were identified by the computable phenotype as successfully aged. Approximately 4% of these persons responded to study announcements and 333 were consented. Of these 333, 218 (65%) were ‘validated’ as meeting successful cognitive aging criteria via direct evaluation (See Figure).ConclusionsThe study validated a computable phenotype algorithm for recruitment of individuals for a successful aging study using large‐scale EHRs. Our study provides proof‐of‐concept of using big data and informatics as aids for recruitment of individuals for prospective cohort studies.

Iron Overload and Treatment Outcome in an Adult Sickle Cell Disease Cohort

Objective. Blood transfusion plays a prominent role in the management of patients with sickle cell disease (SCD) and has a compounded effect on iron overload, especially when simple top-off transfusions are given. Adults with sickle cell anemia (SCA: HbSS or HbSβ 0-thalassemia), often require episodic blood transfusions on a chronic basis and, hence, are at risk of developing transfusional iron overload. We monitored 83 adult SCD patients with a diagnosis of iron overload during routine treatment and follow up at our clinical center. The aim of this study was to observe and describe treatment outcomes for individuals with iron overload seen at the University of Illinois adult sickle cell center between 2010 and 2023. Method. Patients eligible for the study were aged 18 years or above at the time of their initial visit. Patients were identified from our institutional clinical and translational research informatics data warehouse using computable phenotype algorithms as well as SCD outpatient encounter information to improve accuracy of abstracted data. Results of our queries were abstracted in the form of identified patient data and subsequently reviewed in Epic electronic health record (Epic, Verona, WI). We retrospectively studied adult subjects with SCD seen during routine clinical care between 2010 and 2023. Selected subjects had a diagnosis of iron overload, determined by 2 consecutive ferritin levels &amp;gt;1000ng/ml in steady state plus a significant transfusion history. We extracted data on clinical laboratory markers, abdominal imaging, use of iron chelation at baseline and in the final year of follow up, whether patients were receiving chronic erythrapheresis treatment at baseline and at end of follow up, and if alive at the end of the follow up period. Subjects have variable start date and follow up period, beginning at the start of diagnosis of iron overload and ending in June 2023 or last identifiable follow up visit, or mortality. Result. Baseline characteristics of 83 adult patients included in our study are summarized in table 1. Seventy percent of the patients were female and just about everyone had HbSS disease. The females had older median age (39.5year; IQR 32-50 years) than males. The mean ferritin value was significantly higher than the initial measure after a median follows up period of 5.3 years. During the follow up period, 40% of male patients discontinued iron chelation therapy whereas only 5.6% of females did the same. Multivariable analyses were performed for association between change in ferritin measure (between follow up period) with demographic data and other covariates presented in Table 1. Covariates were eliminated by backward selection. Predictors retained in the final model include follow-up period, initial ferritin value, gender, ALT, AST, and albumin. Females had a higher drop in their follow up ferritin, but this was not significant. Neither use of iron chelation therapy at baseline nor erythrapheresis at baseline predicted a change in ferritin value, however we did not collect information on compliance or duration of therapy. Patients followed up for longer duration, or who have higher ALT, had subsequent higher measure for follow up ferritin. Higher albumin predicted a lower ferritin at follow up. The effects of liver markers on ferritin are likely correlative only. After a median follow up period of 5.3years, about 22% of the patients were confirmed to be deceased. Some patients lost to follow up have an indeterminate alive status. Predictors of mortality in multivariate analysis include follow up ferritin value, albumin, and HCV status. Positive HCV status had a strong association with cirrhosis (p=0.001) as 71% of HCV positive individuals had cirrhosis. In turn 53% of the cirrhotic patients died in the follow up period. Conclusion. Here we present an overview of the clinical outcomes of adult patients with SCD and iron overload during a median follow up period of 5.3 years. Despite limitations of a typical retrospective study, our study illustrates the difficulty associated with the management of transfusional iron overload during routine treatment follow up, especially in patients continuing to undergo episodic transfusion treatment. We also continue to underline the low adherence with chelation therapy in adult populations with SCD and iron overload, and the high mortality associated with the condition.

Development and External Validation of a Lymphoma-Specific Venous Thromboembolism Risk Assessment Model

Introduction: Venous thromboembolism (VTE) significantly affects cancer patients undergoing systemic therapy. Existing risk models developed in the pan-cancer population, such as the Khorana score, offer limited discrimination in cancer-specific setting. We aimed to develop and validate a risk assessment model (RAM) tailored for lymphoma patients, accounting for the clinical and sociodemographic heterogeneity across three distinct healthcare systems in the United States. Methods: Electronic health records (EHR) linked to cancer registry 2006-2021 from the Veterans Affairs national healthcare system (VA) with a total of 10,313 lymphoma patients were randomly divided into an 80% derivation cohort and a 20% internal validation cohort. Further external validation was carried out using data from two other healthcare systems - Harris Health System (HHS, N= 854, 2011-2020) and MD Anderson Cancer Center (MDACC, N=1,858, 2017-2020). Patients were included if they had newly diagnosed lymphoma requiring first-line systemic therapy within 1 year of diagnosis. Patients were excluded if they had recent diagnosis of acute VTE within the last 6 months or were prescribed anticoagulant within 1 month before index date. The index date was the time of systemic therapy, and all covariates were extracted on or before the index date. Incident VTE was defined using our published computable phenotype algorithm (PMID 36626707; 37067102). For RAM derivation (Figure 1), 32 candidate predictors were initially chosen based on clinical relevance and EHR data availability. Missing values were imputed via chain random forest. Parsimonious variable categories were selected by a combination of Least Absolute Shrinkage and Selection Operator (LASSO), Recursive Feature Elimination (RFE), and random forest to generate the final logistic regression model. For external validation, complete case analysis was performed using the derived beta coefficients without model refitting. Bootstrapped time-dependent c statistics and calibration curves for 6-month VTE were used to assess discrimination and fit. A pre-determined threshold of 8% VTE at 6-month from the predicted probability was used to stratify high vs. low-risk groups. Competing risk models were used to assess cumulative incidence. Results: At 6 months, the VTE incidence were 5.75% (n=469) in VA derivation and 6.12% (n=124), 8.24% (n=69), and 6.55% (n=112) in the VA, HHS, and MDACC validation cohorts, respectively. The three healthcare systems had different baseline characteristics in demographics, lymphoma types, systemic therapies, and comorbidities. Patients from VA were mostly older males with more comorbidities and remote history of VTE, patients from HHS were mostly uninsured minorities who were younger but with more aggressive histology, while those from MDACC were patients treated at a comprehensive cancer center. The variables in the final RAM included lymphoma type (4 groups ranging from very-low to very-high risk), body mass index, indwelling catheter, cytotoxic vs. targeted therapy, history of VTE, recent hospitalization, and paralysis/immobilization. Table 1 shows the variable prevalence in each cohort. As shown in Figure 1, the new RAM showed promising discrimination for VTE with a C-statistic of 0.69 in the derivation cohort and 0.67, 0.70, 0.72 in the internal and two external validation cohorts. By splitting patients using an 8% predicted risk threshold, the high- vs. low-risk group achieved a clear differentiation in risk stratification despite significant differences in baseline characteristics. This represented a noticeable improvement for lymphoma patients over the Khorana Score, which yielded C-statistics of 0.53-0.56. The overall calibration of the RAM appeared adequate. Conclusions: Using multiple cancer-EHR cohorts with heterogeneous patients and a wide range of lymphoma subtypes, we successfully derived and externally validated an improved RAM for VTE specifically in lymphoma patients requiring early initiation of systemic therapy. Along several well-known patient-specific risk factors, lymphoma type was an important predictor of VTE risk. The new RAM could improve VTE risk stratification and prevention efforts in lymphoma patients.

Characteristics and Predictors of Apparent Treatment-Resistant Hypertension in Real-World Populations Using Electronic Health Record-Based Data.

Apparent treatment-resistant hypertension (aTRH) is defined as uncontrolled blood pressure (BP) despite using ≥3 antihypertensive classes or controlled BP while using ≥4 antihypertensive classes. Patients with aTRH have a higher risk for adverse cardiovascular outcomes compared with patients with controlled hypertension (HTN). Although there have been prior reports on the prevalence, characteristics, and predictors of aTRH, these have been broadly derived from smaller datasets, randomized controlled trials, or closed healthcare systems. We extracted patients with HTN defined by ICD-9 and ICD-10 codes during 1/1/2015-12/31/2018, from 2 large electronic health record databases: the OneFlorida Data Trust (n = 223,384) and Research Action for Health Network (REACHnet) (n = 175,229). We applied our previously validated aTRH and stable controlled HTN computable phenotype algorithms and performed univariate and multivariate analyses to identify the prevalence, characteristics, and predictors of aTRH in these populations. The prevalence of aTRH among patients with HTN in OneFlorida (16.7%) and REACHnet (11.3%) was similar to prior reports. Both populations had a significantly higher proportion of Black patients with aTRH compared with those with stable controlled HTN. aTRH in both populations shared similar significant predictors, including Black race, diabetes, heart failure, chronic kidney disease, cardiomegaly, and higher body mass index. In both populations, aTRH was significantly associated with similar comorbidities, when compared with stable controlled HTN. In 2 large, diverse real-world populations, we observed similar comorbidities and predictors of aTRH as prior studies. In the future, these results may be used to improve healthcare professionals' understanding of aTRH predictors and associated comorbidities.

Evaluation of a Computable Phenotype for Successful Cognitive Aging

ObjectiveTo establish, apply, and evaluate a computable phenotype for the recruitment of individuals with successful cognitive aging. Participants and MethodsInterviews with 10 aging experts identified electronic health record (EHR)-available variables representing successful aging among individuals aged 85 years and older. On the basis of the identified variables, we developed a rule-based computable phenotype algorithm composed of 17 eligibility criteria. Starting September 1, 2019, we applied the computable phenotype algorithm to all living persons aged 85 years and older at the University of Florida Health, which identified 24,024 individuals. This sample was comprised of 13,841 (58%) women, 13,906 (58%) Whites, and 16,557 (69%) non-Hispanics. A priori permission to be contacted for research had been obtained for 11,898 individuals, of whom 470 responded to study announcements and 333 consented to evaluation. Then, we contacted those who consented to evaluate whether their cognitive and functional status clinically met out successful cognitive aging criteria of a modified Telephone Interview for Cognitive Status score of more than 27 and Geriatric Depression Scale of less than 6. The study was completed on December 31, 2022. ResultsOf the 45% of living persons aged 85 years and older included in the University of Florida Health EHR database identified by the computable phenotype as successfully aged, approximately 4% of these responded to study announcements and 333 consented, of which 218 (65%) met successful cognitive aging criteria through direct evaluation. ConclusionThe study evaluated a computable phenotype algorithm for the recruitment of individuals for a successful aging study using large-scale EHRs. Our study provides proof of concept of using big data and informatics as aids for the recruitment of individuals for prospective cohort studies.

Derivation and External Validation of Clinical Risk Assessment Model for Cancer Associated Thrombosis in Two Unique U.S. Healthcare Systems

Introduction: Venous thromboembolism (VTE), defined as pulmonary embolism (PE) and lower extremity deep vein thrombosis (LE-DVT), is a potentially preventable complication for cancer patients undergoing systemic therapy. The Khorana score (KS) is the most used risk assessment model (RAM) to predict venous thromboembolism (VTE) in ambulatory cancer patients. Potential limitations of the score include modest discrimination and small proportion of people developing VTE in the high-risk subgroup. We aimed to derive and externally validate a more precise yet equally RAM for VTE through two unique healthcare systems. Methods: We derived a parsimonious RAM using the cancer registry-linked electronic health records from the Harris Health System (HHS, N=9,916) and validated it using the Veterans Affairs national healthcare system (VA, N=78,892) from 2011-2020. Patients were included if they had their first-ever newly diagnosed cancer and received first-line systemic therapy within 1 year of diagnosis. Index date was the time systemic therapy and all covariates were extracted on or prior to index date. Incident VTE and PE/LE-DVT were defined using our computable phenotype algorithm (PMID: 35647478). When tested against the gold standard of chart abstractor review in a random sampling of 400 patients at each site, the algorithm had a positive predictive value (PPV) of 98% and sensitivity (Sn) of 96% at HHS; and a PPV of 90% and weighted Sn of 82% at VA. For RAM derivation, 34 candidate predictors of VTE were initially chosen by clinicians and 11 final covariates were selected by the LASSO regression via cross-validated optimal lambda.1se. The original KS components were forced into the final model. For external validation, bootstrapped time-dependent c statistics and calibration curves for 6-month VTE and PE/LE-DVT risk were used to assess model discrimination and fit. Dichotomized risk strata using integer scores were created and compared against the KS. Complete data analysis was performed without imputation. Results: The median follow-up for continuous VTE assessment were 11.5 months (IQR 5.7-21.1) for HHS and 14.9 months (IQR 5.9-31.9) for VA. At 6 months, there were 656 (6.6%) VTE (including 448 (4.5%) PE/LE-DVT) and 1,094 (11.0%) deaths in HHS, and 3,209 (4.1%) VTE (including 2,474 (3.1%) PE/LE-DVT) and 13,792 (17.5%) deaths in VA. Due to differences in healthcare systems, the two cohorts had very different baseline characteristics in demographic, cancer, systemic therapy, and co-morbidity. Patients from HHS were younger (median age 55 vs. 67), more diverse by sex (female 56.6% vs. 4.9%) and race/ethnicity (15% vs. 71% White, 52% vs. 5% Hispanic, 27% vs. 5% Black, 6% vs. 2% Asian/Pacific Islander [API]), compared to the VA. When assessed at the time of systemic therapy initiation, the new RAM in derivation cohort included components of the KS with modified cancer subtype, cancer staging, systemic therapy class, history of VTE, history of paralysis/immobility, recent hospitalization, and API race (Table 1). Due to lower incidence of VTE in VA, re-calibration-in-the-large (intercept only) optimized the overall fit of the model in external validation. The c statistic for the VTE outcome was 0.70 in HHS and 0.66 in VA (compared to 0.64 and 0.59, respectively, for KS). Furthermore, the new RAM appropriately reclassified 27% of patients from the previous risk groups and increased the proportion of VTEs in the high-risk group from 36.7% to 68.0% in the validation dataset (Table 2). Conclusions: Using easily extractable data from two independent healthcare systems, we successfully derived and externally validated a RAM for VTE in a diverse population by sex and race with all cancer types (including hematologic malignancy) and receiving all types of modern systemic therapies (including immune checkpoint inhibitor, targeted therapy, endocrine therapy) in addition to chemotherapy. It stratified approximately 50% of patients with all cancer types receiving all types of modern systemic therapy into a high-risk group with a 6-month VTE risk of 7-11%, and the other half into a low-risk group with a corresponding risk of 3%. The RAM had improved performance over the original KS and doubled the number of VTE events in the high-risk stratum. The incorporation of this modified RAM can lead to improved risk stratification and personalized decision-making for pharmacologic thromboprophylaxis. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

Utilization of Natural Language Processing in Venous Thromboembolism Identification

Introduction: Capturing venous thromboembolism (VTE) outcomes using only International Classification of Disease (ICD) codes can lead to misclassification of events and to inaccurate conclusions. Manual chart review, the gold standard, is labor intensive and not always feasible. The current study aims to validate an improved computable VTE phenotype algorithm combining ICD with natural language processing (NLP) in patients undergoing hematopoietic cell transplantation (HCT). Methods: All patients undergoing first allogeneic HCT at Fred Hutchinson Cancer Center (FHCC) from 2006-2019 were included in the current study. To capture as many as possible VTE events, we used a sensitive screening method within one year before and after the transplant date that encompassed 1) all patients with ICD-9 and ICD-10 codes for acute, chronic, or historical pulmonary embolism (PE), deep venous thrombosis (DVT), and phlebitis and thrombophlebitis, and 2) all patients with at least 1 radiology report with a pertinent VTE-related keyword from venous doppler ultrasound, contrast computed tomography, or ventilation perfusion scans. All patients from this screened subset were then reviewed by chart abstractors (JA, KM) to establish the gold standard of incident VTE events, which was defined as the new onset of radiologically confirmed PE, lower extremity DVT, or upper extremity/catheter related DVT within 1-year post-transplant. We then tested the performance of the acute VTE ICD-9/10 codes (selective codes from 415, 451, 453; I26, I80, I82) from inpatient or outpatient encounters from stem cell infusion until 1-year post-transplant. We also tested the utility of a NLP algorithm, a method that has been previously validated in a separate cancer cohort using unstructured radiology impressions (PMID: 35647478). Finally, we compared the performance of combining these two algorithms against the gold standard to report positive predictive value (PPV) and sensitivity (Sn). Results: Among 2,879 patients who underwent allogeneic HCT over 15 years, 740 (26%) met study inclusion criteria. Based on the gold standard of detailed medical record review, 275 (10%) were found to have a radiologically confirmed VTE event within 1-year post-transplant (Figure 1). A further 389 (14%) historical VTE events were confirmed before transplant. The acute VTE ICD-9/10 codes identified 339 patients and the NLP algorithm predicted 245 patients to have VTE (including 155 overlap). The ICD codes alone for acute VTE had an estimated Sn of 73% and PPV of 59%. The NLP radiology algorithm alone for VTE had an estimated Sn of 73% and PPV of 82%. Approximately 3 in 10 VTE events were missed in each algorithm. However, the combination of ICD or NLP identified 245/275 of all VTE events (Sn 89%). The PPVs for ICD+/NLP+, ICD-/NLP+, and ICD+/NLP- were 89%, 64%, and 27%, respectively (Table 1). In summary, those with concordant ICD/NLP prediction had excellent PPVs, and approximately 8% of patients with discordant ICD/NLP (n=234/2,879) would require additional chart review to achieve a final PPV >90% and Sn >90%. Conclusion: In the current study, we found that the sensitivity for either the acute ICD codes or the NLP algorithm alone was sub-optimal (missing 3 in 10), and a combined screen should be considered (missing 1 in 10). The use of ICD-9/10 codes alone for new VTE had poor accuracy in our cohort (PPV of 59%), suggesting that additional features are needed, such as concurrent anticoagulation medications. In contrast, the NLP algorithm was validated with high PPV 82% (89% when combined with positive acute ICD screen) in the current cohort and may not require additional confirmation, though caution should be taken for its usage in other studies without dedicated follow-up where radiology reports are captured and stored in one unified healthcare system. One limitation of the study is the lack of review of patients initially screened negative by either ICD codes or radiology keyword searches. While the initial screen was designed to be highly sensitive, we may have missed small number of true VTE events and the reported Sn in this study represents the best-case scenario. In conclusion, while computable phenotype algorithms represent a promising future for the identification of VTE, a hybrid approach involving manual chart review (for only cases where the ICD and NLP screens disagree) may provide the highest yield and help minimize the labor intensive manual review. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

Venous Thromboembolism Risk in Cancer Patients Receiving First-Line Immune Checkpoint Inhibitor Vs. Chemotherapy

Introduction: Whether immune checkpoint inhibitors (ICIs) are associated with an increased risk of venous thromboembolism (VTE) compared to traditional chemotherapy (chemo) is unclear (PMID: 34572833). While retrospective cohort studies reported higher than expected rate of VTE among patients receiving ICI, these patients were often sicker and having received multiple prior lines of therapy. In recent years, ICIs have been used increasingly in the front line setting for several cancers. In the current study, we assessed the incidence of VTE associated with first-line ICIs vs. chemo among comparable cancer patients. Methods: We performed a retrospective cohort study using the national Veterans Affairs (VA) healthcare system. Patients with de novo incident cancer diagnosis 2016-2020 were identified from the VA Cancer Registry and linked to the VA Corporate Data Warehouse. Patients were excluded if they had benign histology, were not primary VA user, or did not receive systemic therapy. For the primary analysis (chemo vs. ICI +/- chemo), we restricted the population to patients with stage III-IV cancers of lung, skin (melanoma), kidney, bladder, liver (hepatocellular carcinoma), and head/neck who received first-line ICI or first-line chemo within first year of diagnosis (Figure 1). For the secondary analysis (chemo vs. ICI vs. ICI/chemo), we further restricted the population to those with lung cancers as concurrent first-line ICI/chemo administration was only observed in this cohort. Incident VTE and PE/LE-DVT were defined using our computable phenotype algorithm (PMID: 35647478). When tested against the gold standard of chart abstractor review in a random sampling of 400 patients, the algorithm had a positive predictive value of 90% and weighted sensitivity of 82%. The following covariates were extracted at the time of first-line ICI or chemo (index date): age, sex, race, diagnosis year, VA frailty index (proxy for performance status), history of VTE, recent prolonged hospitalization last 3 months, cancer type, cancer stage, time delay between diagnosis and treatment, and other Khorana Score components (body mass index, hemoglobin, white blood cell count, platelet count). To address confounding, we used inverse probability weighting (IPW) with overlap weights by fitting a propensity score (PS) logit model from these baseline covariates (2 groups for primary analysis, 3 groups for secondary analysis). Average potential outcome (APO) for VTE and PE/LE-DVT were estimated at 6-months. Weighted relative risk (RR) with bootstrapped 95% confidence interval (CI) were calculated using weighted logistic regression. IPW analyses were performed using the PSweight package in R. Results: From 2016-2020, 45,253 veterans with new cancer diagnosis requiring systemic therapy met the inclusion criteria of whom 6,068 (13.4%) received ICI with a median continuous follow-up of 334 days. However, only 4,615 (10.2%) received ICI within first year of diagnosis, 2,109 (4.7%) received ICI as the initial regimen, and 1,660 (3.7%) received it for FDA-approved cancers (vs. 6,489 comparable chemo patients) (Figure 1). Despite initial cohort matching, the groups remained imbalanced with standardized mean difference (SMD) >0.1 in age, diagnosis year, cancer type, cancer stage, and time to therapy. After IPW, all covariates were balanced with SMD <0.1. In the primary analysis (Table 1), VTE occurred in 5.55% of chemo group vs. 6.10% of ICI group at 6 months. The weighted RR was 1.10 (95% CI 0.83-1.46) for ICI (+/- chemo) vs. chemo (no ICI). In the secondary analysis restricted only to lung cancers, VTE occurred in 5.83% of chemo group vs. 6.13% of ICI group (no chemo) vs. 8.13% of ICI group (+ chemo) at 6 months. The weighted RR was 1.05 (95% 0.62-1.68) for ICI (no chemo) vs. chemo (no ICI). The weighted RR was 1.39 (95% 0.93-2.07) for ICI (+ chemo) vs. chemo (no ICI). The PE/LE-DVT outcome followed a similar trend. Conclusions: In this PS-weighted analysis of 8,149 patients, we found that first-line ICI and chemo had a similar rates of VTE. Larger studies are needed for the subgroup of lung cancer patients where combination ICI/chemo regimen had a non-significant 39% increase in VTE rate compared to either chemo or ICI alone. Previous retrospective studies likely reported inflated rate of VTE in ICI due to indication bias. Nonetheless, patients receiving ICI had a similar rate of VTE and should be risk stratified as those receiving chemo. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

1262-P: Developing a Computable Phenotype Algorithm for Identification of Children and Adolescents with Diabetes Using Electronic Health Records

The Assessing the Burden of Diabetes by Type in Children, Adolescents, and Young Adults (DiCAYA) Network, a CDC/NIDDK-funded collaborative, aims to create a multi-site electronic health record (EHR) -based diabetes surveillance system. Foundational to the network's efforts is the development of a computable phenotype (CP) algorithm that can identify cases of diabetes. To advance the mission of the DiCAYA network, University of Florida (UF) Health system researchers developed a pilot CP algorithm for identifying diabetes cases in youth. The CP algorithm was iteratively derived based on structured data from EHRs (UF Health system 2012-2020) . We randomly selected 500 presumed cases among individuals &amp;lt; 18 years old who has (1) HbA1c ≥ 6.5%; or (2) fasting glucose ≥ 126 mg/dL; or (3) random plasma glucose ≥ 200 mg/dL; or (4) diabetes-related diagnosis code from an inpatient or outpatient encounter; or (5) prescribed, administered, or dispensed diabetes-related medication. Four reviewers independently reviewed the patient charts to determine diabetes status and type. Presumed cases without type 1 (T1D) or type 2 (T2D) diabetes diagnosis codes were categorized as nondiabetes or other types. The rest were categorized as T1D if the ratio of T1D codes to the sum of T1D and T2D codes was ≥ 0.5, or otherwise categorized as T2D. Next, we applied a list of diagnoses and procedures that can determine diabetes type (e.g., steroid use suggests induced diabetes) to correct misclassifications from step 1. Among the 500 reviewed cases, 159 and 64 had T1D and T2D. The sensitivity, specificity, and positive predictive values of the CP algorithm were 94%, 98%, and 96% for T1D; 95%, 95%, and 73% for T2D. We developed a highly accurate EHR-based CP for diabetes in youth based on EHR data from UF Health. Consistent with prior studies, T2D was more difficult to identify using these methods. A DiCAYA-wide validation and algorithm refinement process will be conducted. Disclosure P.Li: None. M.Prosperi: None. B.E.Dixon: Advisory Panel; Merck Sharp &amp; Dohme Corp. D.Dabelea: None. L.H.Utidjian: None. T.L.Crume: None. L.Thorpe: None. A.D.Liese: None. D.Schatz: Advisory Panel; Abbott Diabetes, Medtronic. M.A.Atkinson: None. M.J.Haller: Advisory Panel; SAB Biotherapeutics , Consultant; MannKind Corporation, Sanofi. E.Spector: None. E.Shenkman: None. J.Bian: None. Y.Guo: None. H.Shao: Board Member; BRAVO4HEALTH, LLC. M.A.Atkinson: None. K.Alkhuzam: None. R.S.Patel: None. W.T.Donahoo: None. S.Bost: None. T.Lyu: None. Y.Wu: None. W.Hogan: None. Funding CDC/NIDDK U18DP006512

1265-P: The Role of Health System Penetration Rate in Estimating Prevalence of Type 1 Diabetes (T1D) in Children and Adolescents Using Electronic Health Records (EHR)

This study aimed to develop a novel method that can accurately estimate the prevalence of T1D in children and adolescents using EHR data. T1D cases among individuals &amp;lt;= 18 years old were identified from the OneFlorida EHR (2018-2020) using a computable phenotype algorithm we have developed. The T1D prevalence was estimated at the zip code level, with numerator being the total T1D cases and the denominator being the number of residents in each zip code area extracted from the census data. The observed health system penetration rates (HSPR) in 20 were estimated as a ratio of total patients who visited the healthcare systems in the OneFlorida network (2018-2020) to the total number of residents in each zip code area in 2019. We applied a joint-point regression to determine the cutoff thresholded where the observed HSPR can produce accurate prevalence estimation. The cutpoint value of 72.0% for the observed HSPR was identified, and the trend line between prevalence and HSPR plateaued in areas with HSPR&amp;gt;72% (slope=0.002, figure) . The estimated T1D prevalence in 20 was 0.243%, which is similar to that of estimated by SEARCH (0.255%) . Our algorithm provides a critical tool for developing a EHR-based diabetes surveillance system in the US. Disclosure P.Li: None. T.Lyu: None. J.Bian: None. Y.Guo: None. E.Shenkman: None. W.T.Donahoo: None. J.Guo: None. H.Shao: Board Member; BRAVO4HEALTH, LLC. Funding U18DP006512 CDC/NIDDK

EHR-based cohort assessment for multicenter RCTs: a fast and flexible model for identifying potential study sites.

ObjectiveThe Recruitment Innovation Center (RIC), partnering with the Trial Innovation Network and institutions in the National Institutes of Health-sponsored Clinical and Translational Science Awards (CTSA) Program, aimed to develop a service line to retrieve study population estimates from electronic health record (EHR) systems for use in selecting enrollment sites for multicenter clinical trials. Our goal was to create and field-test a low burden, low tech, and high-yield method.Materials and MethodsIn building this service line, the RIC strove to complement, rather than replace, CTSA hubs’ existing cohort assessment tools. For each new EHR cohort request, we work with the investigator to develop a computable phenotype algorithm that targets the desired population. CTSA hubs run the phenotype query and return results using a standardized survey. We provide a comprehensive report to the investigator to assist in study site selection.ResultsFrom 2017 to 2020, the RIC developed and socialized 36 phenotype-dependent cohort requests on behalf of investigators. The average response rate to these requests was 73%.DiscussionAchieving enrollment goals in a multicenter clinical trial requires that researchers identify study sites that will provide sufficient enrollment. The fast and flexible method the RIC has developed, with CTSA feedback, allows hubs to query their EHR using a generalizable, vetted phenotype algorithm to produce reliable counts of potentially eligible study participants.ConclusionThe RIC’s EHR cohort assessment process for evaluating sites for multicenter trials has been shown to be efficient and helpful. The model may be replicated for use by other programs.