Use Of Clinical Decision Support Research Articles

Performance evaluation of ChatGPT 4.0 on cardiovascular questions from the medical knowledge self-assessment program

Abstract Background Medical trainees are increasingly using online chat-based artificial intelligence (AI) platforms as supplementary resources for board exam preparation and clinical decision support. Prior studies have evaluated the performance of AI chatbots like ChatGPT on various general standardized tests such as the United States Medical Licensing Examination (USMLE), but little is known about their performance on subspecialty-focused exam questions, particularly related to clinical management and treatment. Objective This study aims to evaluate the performance of ChatGPT version 4.0 on the cardiovascular questions from the Medical Knowledge Self-Assessment Program (MKSAP) 19, a widely used resource for board exam preparation in the United States. Methods We submitted all cardiovascular questions from MKSAP 19 to ChatGPT 4.0, covering a broad range of cardiology topics in a multiple-choice format. Performance was gauged against both the official MKSAP answer key and average trainee scores obtained from the MKSAP website. Out of 129 questions, 4 were invalidated due to post-publication data, and 18 were excluded due to reliance on visual aids, leaving 107 questions for analysis. Results ChatGPT 4.0 correctly answered 93 out of 107 questions, reflecting an 87% accuracy rate, compared to a 60% accuracy rate averaged among all human users for the same questions (p<0.0001). ChatGPT accuracy rates for each question category (e.g., heart failure, electrophysiology, etc.) are provided in the figure. On the 14 questions that ChatGPT answered incorrectly, human users averaged a 47% accuracy rate. All 14 incorrectly answered questions were related to clinical management and treatment (as opposed to diagnosis or epidemiology), and the majority (10/14) involved choosing treatment medications or choosing between medications versus interventions. Conclusion ChatGPT 4.0 surpassed the typical 70% accuracy threshold required for passing the internal medicine board certification but performed less well in answering questions about clinical management and treatment. This performance suggests that ChatGPT 4.0 could be an effective adjunct tool for cardiology education, but raises some concerns about its use in clinical decision support for trainees.

Leveraging Generative AI to Accelerate Biocuration of Medical Actions for Rare Disease.

Structured representations of clinical data can support computational analysis of individuals and cohorts, and ontologies representing disease entities and phenotypic abnormalities are now commonly used for translational research. The Medical Action Ontology (MAxO) provides a computational representation of treatments and other actions taken for the clinical management of patients. Currently, manual biocuration is used to assign MAxO terms to rare diseases, enabling clinical management of rare diseases to be described computationally for use in clinical decision support and mechanism discovery. However, it is challenging to scale manual curation to comprehensively capture information about medical actions for the more than 10,000 rare diseases. We present AutoMAxO, a semi-automated workflow that leverages Large Language Models (LLMs) to streamline MAxO biocuration for rare diseases. AutoMAxO first uses LLMs to retrieve candidate curations from abstracts of relevant publications. Next, the candidate curations are matched to ontology terms from MAxO, Human Phenotype Ontology (HPO), and MONDO disease ontology via a combination of LLMs and post-processing techniques. Finally, the matched terms are presented in a structured form to a human curator for approval. We used this approach to process 4,918 unique medical abstracts and identified annotations for 21 rare genetic diseases, we extracted 18,631 candidate disease-treatment curations, 538 of which were confirmed and transferred to the MAxO annotation dataset. The results of this project underscore the potential of generative AI to accelerate precision medicine by enabling a robust and comprehensive curation of the primary literature to represent information about diseases and procedures in a structured fashion. Although we focused on MAxO in this project, similar approaches could be taken for other biomedical curation tasks.

Enhancing Clinical Decision Support in Nephrology: Addressing Algorithmic Bias Through Artificial Intelligence Governance

There has been a steady rise in the use of clinical decision support (CDS) tools to guide Nephrology, as well as general clinical care. Through guidance set by federal agencies and concerns raised by clinical investigators, there has been an equal rise in understanding whether such tools exhibit algorithmic bias leading to unfairness. This has spurred the more fundamental question of whether sensitive variables such as race should be included in CDS tools. In order to properly answer this question, it is necessary to understand how algorithmic bias arises. We break down three sources of bias encountered when using electronic health record data to develop CDS tools: (1) use of proxy variables, (2) observability concerns and (3) underlying heterogeneity. We discuss how answering the question of whether to include sensitive variables like race often hinges more on qualitative considerations than on quantitative analysis, dependent on the function that the sensitive variable serves. Based on our experience with our own institution’s CDS governance group, we show how health system-based governance committees play a central role in guiding these difficult and important considerations. Ultimately, our goal is to foster a community practice of model development and governance teams that emphasizes consciousness about sensitive variables and prioritizes equity.

Strengthening the use of artificial intelligence within healthcare delivery organizations: balancing regulatory compliance and patient safety.

Surface the urgent dilemma that healthcare delivery organizations (HDOs) face navigating the US Food and Drug Administration (FDA) final guidance on the use of clinical decision support (CDS) software. We use sepsis as a case study to highlight the patient safety and regulatory compliance tradeoffs that 6129 hospitals in the United States must navigate. Sepsis CDS remains in broad, routine use. There is no commercially available sepsis CDS system that is FDA cleared as a medical device. There is no public disclosure of an HDO turning off sepsis CDS due to regulatory compliance concerns. And there is no public disclosure of FDA enforcement action against an HDO for using sepsis CDS that is not cleared as a medical device. We present multiple policy interventions that would relieve the current tension to enable HDOs to utilize artificial intelligence to improve patient care while also addressing FDA concerns about product safety, efficacy, and equity.

Machine learning in cardiac surgery: a narrative review.

Machine learning (ML) is increasingly being utilized to provide data driven solutions to challenges in medicine. Within the field of cardiac surgery, ML methods have been employed as risk stratification tools to predict a variety of operative outcomes. However, the clinical utility of ML in this domain is unclear. The aim of this review is to provide an overview of ML in cardiac surgery, particularly with regards to its utility in predictive analytics and implications for use in clinical decision support. We performed a narrative review of relevant articles indexed in PubMed since 2000 using the MeSH terms "Machine Learning", "Supervised Machine Learning", "Deep Learning", or "Artificial Intelligence" and "Cardiovascular Surgery" or "Thoracic Surgery". ML methods have been widely used to generate pre-operative risk profiles, consistently resulting in the accurate prediction of clinical outcomes in cardiac surgery. However, improvement in predictive performance over traditional risk metrics has proven modest and current applications in the clinical setting remain limited. Studies utilizing high volume, multidimensional data such as that derived from electronic health record (EHR) data appear to best demonstrate the advantages of ML methods. Models trained on post cardiac surgery intensive care unit data demonstrate excellent predictive performance and may provide greater clinical utility if incorporated as clinical decision support tools. Further development of ML models and their integration into EHR's may result in dynamic clinical decision support strategies capable of informing clinical care and improving outcomes in cardiac surgery.

Standardizing default electronic health record tools to improve safety for hospitalized patients with Parkinson's disease.

Electronic Health Record (EHR) systems are often configured to address challenges and improve patient safety for persons with Parkinson's disease (PWP). For example, EHR systems can help identify Parkinson's disease (PD) patients across the hospital by flagging a patient's diagnosis in their chart, preventing errors in medication and dosing through the use of clinical decision support, and supplementing staff education through care plans that provide step-by-step road maps for disease-based care of a specific patient population. However, most EHR-based solutions are locally developed and, thus, difficult to scale widely or apply uniformly across hospital systems. In 2020, the Parkinson's Foundation, a national and international leader in PD research, education, and advocacy, and Epic, a leading EHR vendor with more than 35% market share in the United States, launched a partnership to reduce risks to hospitalized PWP using standardized EHR-based solutions. This article discusses that project which included leadership from physician informaticists, movement disorders specialists, hospital quality officers, the Parkinson's Foundation and members of the Parkinson's community. We describe the best practice solutions developed through this project. We highlight those that are currently available as standard defaults or options within the Epic EHR, discuss the successes and limitations of these solutions, and consider opportunities for scalability in environments beyond a single EHR vendor. The Parkinson's Foundation and Epic launched a partnership to develop best practice solutions in the Epic EHR system to improve safety for PWP in the hospital. The goal of the partnership was to create the EHR tools that will have the greatest impact on outcomes for hospitalized PWP.

Tracking persistent postoperative opioid use: a proof-of-concept study demonstrating a use case for natural language processing

BackgroundLarge language models have been gaining tremendous popularity since the introduction of ChatGPT in late 2022. Perioperative pain providers should leverage natural language processing (NLP) technology and explore pertinent use...

Exchanging genomics reports between pathology labs and medical centers using the Minimal Clinical Oncology Data Element (mCODE) FHIR implementation guide.

e13575 Background: The automated transfer of discrete molecular data from pathology labs into the electronic health record (EHR) is rapidly becoming commonplace and is designed to facilitate clinical decision making. Simple genomic changes are represented in existing EHR data transfer standards, but current data standards lack clinical fields that correspond to treatment insights such as associated medications and potential clinical trials. To address this, the mCODE effort defines a consensus oncology data standard that facilitates lightweight transmission of data from patients with cancer. mCODE’s support of genomic data leverages efforts by the HL7 Clinical Genomics Working Group (HL7 CGWG) focusing on oncology. Here, we describe pilot efforts involving both a pathology partner and academic medical centers to extend mCODE FHIR (Fast Healthcare Interoperability Resources) implementation guide (IG) support of genomic data for data interchange and clinical decision support use cases. Methods: From 2021-2022, validation of the mCODE FHIR IG genomic data elements was performed in CodeX, a FHIR accelerator program. These elements encompassed: lab facility, test order information, pathologist, patient demographics, specimen, molecular studies and therapeutic implications. Genomic FHIR data elements were selected for their conciseness and clinical relevance for inclusion in the mCODE FHIR IG under the supervision of two oncology informaticians. As part of the mCODE FHIR IG pilot, synthetic solid tumor molecular patient reports were generated by Tempus Labs. These synthetic molecular reports were transformed into mCODE-conformant FHIR files for transfer. The mCODE IG can be found at http://hl7.org/fhir/us/mcode/ . Results: Over 50 Tempus data elements were analyzed and found to be readily mapped to mCODE genomics elements. Two categories of data elements required additional transformation: 1. Data elements requiring additional genomic database annotations; 2. FHIR syntax constraints like the concatenation of medications in the same drug class. The mCODE file was transmitted to Vanderbilt and the Children's Hospital LA from Tempus for decoding and downstream uses. This effort further informed the HL7 CGWG of needed IG clarifications for compatibility with other standardization efforts and genomic use cases. Conclusions: mCODE’s support of genomic data elements provides a lightweight genomics interchange standard compatible with an existing pathology lab report. Mapping from existing molecular pathology reports to mCODE required nominal alterations. Furthermore, mCODE’s alignment with HL7 genomics efforts ensure FHIR-based compatibility in other clinical domains and with developing international standards. Continued pathology lab and academic medical center collaboration is essential for mCODE refinement and adoption.

Using sociotechnical theory to understand medication safety work in primary care and prescribers’ use of clinical decision support: a qualitative study

ObjectivesThe concept of safety work draws attention to the intentional work of ensuring safety within care systems. Clinical decision support (CDS) has been designed to enhance medication safety in primary...

Automated feedback modestly improves perioperative treatment adherence of postoperative nausea and vomiting

Study objectiveExtensive evidence demonstrates that medical record modernization and a vast amount of available data have not overcome the gap between recommended and delivered care. This study aimed to evaluate the use of clinical decision support (CDS) in conjunction with feedback (post-hoc reporting) to improve PONV medication administration compliance and postoperative nausea and vomiting (PONV) outcomes. DesignSingle center, prospective observational study between January 1, 2015, and June 30, 2017. SettingPerioperative care at a university-affiliated tertiary care center. Patients57,401 adult patients who received general anesthesia in a non-emergency setting. InterventionA multi-phased intervention that consisted of post-hoc reporting for individual providers by email about PONV occurrences in their patients, followed by directive CDS through preoperative daily case emails that provided therapeutic PONV prophylaxis recommendations based on patients' PONV risk scores. MeasurementCompliance with PONV medication recommendations, as well as hospital rates of PONV were measured. Main resultOver the study period, there was a 5.5% (95% CI, 4.2% to 6.4%; p < 0.001) improvement in the compliance of PONV medication administration along with an 8.7% (95% CI, 7.1% to 10.2%, p < 0.001) reduction in PONV rescue medication administration in the PACU. However, there was no statistically or clinically significant reduction in the prevalence of PONV in the PACU. The prevalence of PONV rescue medication administration decreased during the Intervention Rollout Period (odds ratio 0.95 [per month]; 95% CI, 0.91 to 0.99; p = 0.017), and during the Feedback with CDS Recommendation Period (odds ratio, 0.96 [per month]; 95% CI, 0.94 to 0.99; p = 0.013). ConclusionPONV medication administration compliance modestly improves with CDS in conjunction with post-hoc reporting; however, no improvement in PACU rates of PONV occurred.

Use of Clinical Decision Support to Improve the Laboratory Evaluation of Monoclonal Gammopathies

There is considerable variation in ordering practices for the initial laboratory evaluation of monoclonal gammopathies (MGs) despite clear society guidelines to include serum free light chain (sFLC) testing. We assessed the ability of a clinical decision support (CDS) alert to improve guideline compliance and analyzed its clinical impact. We designed and deployed a targeted CDS alert to educate and prompt providers to order an sFLC assay when ordering serum protein electrophoresis (SPEP) testing. The alert was highly effective at increasing the co-ordering of SPEP and sFLC testing. Preimplementation, 62.8% of all SPEP evaluations included sFLC testing, while nearly 90% of evaluations included an sFLC assay postimplementation. In patients with no prior sFLC testing, analysis of sFLC orders prompted by the alert led to the determination that 28.9% (800/2,769) of these patients had an abnormal κ/λ ratio. In 452 of these patients, the sFLC assay provided the only laboratory evidence of a monoclonal protein. Moreover, within this population, there were numerous instances of new diagnoses of multiple myeloma and other MGs. The CDS alert increased compliance with society guidelines and improved the diagnostic evaluation of patients with suspected MGs.

Use of clinical decision support for antibiotic stewardship in the emergency department and outpatient clinics: An interrupted time-series analysis

To evaluate the impact of implementing clinical decision support (CDS) tools for outpatient antibiotic prescribing in the emergency department (ED) and clinic settings. We performed a before-and-after, quasi-experimental study that employed an interrupted time-series analysis. The study institution was a quaternary, academic referral center in Northern California. We included prescriptions for patients in the ED and 21 primary-care clinics within the same health system. We implemented a CDS tool for azithromycin on March 1, 2020, and a CDS tool for fluoroquinolones (FQs; ie, ciprofloxacin, levofloxacin, and moxifloxacin) on November 1, 2020. The CDS added friction to inappropriate ordering workflows while adding health information technology (HIT) features to easily perform recommended actions. The primary outcome was the number of monthly prescriptions for each antibiotic type, by implementation period (before vs after). Immediately after azithromycin-CDS implementation, monthly rates of azithromycin prescribing decreased significantly in both the ED (-24%; 95% CI, -37% to -10%; P < .001) and outpatient clinics (-47%; 95% CI, -56% to -37%; P < .001). In the first month after FQ-CDS implementation in the clinics, there was no significant drop in ciprofloxacin prescriptions; however, there was a significant decrease in ciprofloxacin prescriptions over time (-5% per month; 95% CI, -6% to -3%; P < .001), suggesting a delayed effect of the CDS. Implementing CDS tools was associated with an immediate decrease in azithromycin prescriptions, in both the ED and clinics. CDS may serve as a valuable adjunct to existing antimicrobial stewardship programs.

Patients with bariatric surgery: Urgent need for accurate registration of the contraindication to enable safe pharmacotherapy in hospital and primary care

To enable the use of automatic clinical decision support for pharmacotherapy in patients with bariatric surgery, it is necessary to register the contraindication "bariatric surgery" in the hospital, general practitioner (GP), and community pharmacy electronic health record systems. The aim of this research was to quantify the correct registration of this contraindication in hospital, GP, and community pharmacy records. Furthermore, we investigated whether the registration status in primary care was dependent on the registration status in the hospital. From patients who underwent bariatric procedures performed in the Albert Schweitzer Hospital (Dordrecht, the Netherlands) between 2018 and 2021, the percentage of registered contraindications in hospital medical records was assessed. Due to feasibility reasons, a subset of the patients' data was created for assessing the percentage of registered contraindications in GP and community pharmacy records. Out of 664 patients who underwent bariatric surgery, the contraindication bariatric surgery was registered in 69.1% of the cases. Out of 552 patients, 28.3% and 25.1% were correctly registered in GP and community pharmacy records, respectively. There was no correlation between registration status in the hospital EHR and registration status in GP practices or community pharmacies. The percentage of correct registration of bariatric surgery in hospital, GP, and community pharmacies is low. To avoid doctors prescribing and pharmacists dispensing drugs to post-bariatric patients without knowing thatthey have undergone this procedure, better registration of the contraindication is required to enable optimal use of clinical decision support systems for the pharmacotherapy of patients after bariatric surgery.

Use of Clinical Decision Support to Improve the Laboratory Evaluation of Monoclonal Gammopathies

Introduction: Since its introduction in 2001, the serum free light chain (sFLC) assay has played an increasingly prominent role in the initial clinical evaluation of patients with suspected monoclonal gammopathies (MGs). This is primarily due to its superior sensitivity for detecting monoclonal proteins especially in light chain disease. In 2009, the International Myeloma Working Group (IMWG) recommended ordering the combination of an SPEP, sFLC and sIFE for initial screening for all MGs. However, there is still significant heterogeneity in sFLC ordering practices. A recent survey of over 700 clinical laboratories found that sFLC were included in the most common screening order sets for the initial evaluation of MGs in only 19.7% of centers. Methods: We performed a retrospective analysis of SPEP and sFLC ordering practices at our institution over a 6-year period (2014-2019). We then designed and implemented a clinical decision support (CDS) alert to improve compliance with IMWG's recommendations. The alert fires when a provider orders an SPEP without a concurrent sFLC order or result in the last 30 days and suggests the ordering of sFLC. We analyzed the impact on ordering practices and diagnostic outcomes over the course of a 10-month period. Results: Our retrospective analysis of over 90,000 SPEP results, associated sFLC orders, and ICD-10 codes revealed that many providers deviated from society guidelines for the use of sFLC in the initial laboratory evaluation of MGs. Prior to the alert 62.8% of all SPEP evaluations included sFLC testing, while post-implementation nearly 90% of evaluations included sFLC (Figure 1). When restricting our analysis to patients with no prior sFLC results, we found that the alert contributed to the clinical care of as many as 2,791 patients. Twenty nine percent (810/2,791) of these patients had an abnormal kappa/lambda ratio (rFLC). In 462 of these patients the rFLC was the only laboratory evidence of a monoclonal protein (Table 1). Moreover, within this population there were numerous instances of new diagnoses of multiple myeloma and other MGs. Conclusions: In the initial laboratory evaluation of MGs, a CDS alert was an effective tool to increase clinical practice compliance with society guidelines. More importantly, the alert prompted sFLC orders that contributed to the diagnosis of numerous MGs that may have been missed or warrant further investigation and follow up. This study will hopefully serve as a useful guide for other institutions addressing the gap between clinical practice and evidence-based guidelines. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

1413. Effect of Automated Identification of Antimicrobial Stewardship Opportunities for Urinary Tract Infections

BackgroundThe treatment of asymptomatic bacteriuria (ASB) does not improve clinical outcomes in most patients and may be associated with an increased risk of adverse events such as Clostridioides difficile infection. A best practice alert (BPA) was created to identify patients with possible ASB for antimicrobial stewardship (AS) review. We aimed to determine whether automated identification of ASB improved the timing of stewardship intervention.MethodsAn electronic health record BPA message to inpatient AS pharmacists was activated on 01/19/2021. The BPA identified inpatients with a new antibiotic order with an associated genitourinary indication and a preceding urinalysis with 0 to 5 WBC/hpf. BPAs were reviewed by an AS pharmacist during weekdays and normal business hours. We retrospectively evaluated the impact of the BPA on time from order to stewardship intervention between a cohort of pre-BPA (01/2020 to 12/2020) and post-BPA (01/20/2021 to 04/10/2021) patients. Included patients met the BPA criteria and had an AS intervention within 7 days of the antibiotic order. We specified interventions that were UTI-related. The median time from antibiotic order entry to any AS intervention was compared pre- to post-BPA using the Mann Whitney U test. Rates of UTI-related interventions were compared with Fisher’s Exact test.Results327 antibiotic orders met BPA criteria and were analyzed: 245 and 82 in the pre- and post-BPA group, respectively. Groups had similar baseline characteristics (Table 1). A total of 33 (27 UTI-related) pre-BPA group and 24 (17 UTI-related) post-BPA group interventions were documented by the AS team. The median time to any intervention was 28 hours (IQR 18-64.5) in the pre-BPA group compared to 13.5 hours (IQR 3.5-28.75) in the post-BPA group (p = 0.03, Figure). The pre-BPA group had a lower rate of UTI-related interventions compared to the post-BPA group (11.0% vs 20.7%, p = .04). ConclusionAutomated identification of antibiotics targeting UTI with urinalysis showing absence of pyuria reduced the time to stewardship intervention and increased rate of UTI-specific interventions. The use of clinical decision support may aid in efficiency of AS review and syndrome-targeted AS impact. Disclosures Rebekah W. Moehring, MD, MPH, UpToDate, Inc. (Other Financial or Material Support, Author Royalties)

Emergency medicine physicians' perspectives on diagnostic accuracy in neurology: a qualitative study.

We sought to understand the knowledge, attitudes, and beliefs of emergency medicine (EM) physicians towards non-specific neurological conditions and the use of clinical decision support (CDS) to improve diagnostic accuracy. We conducted semi-structured interviews of EM physicians at four emergency departments (EDs) affiliated with a single US healthcare system. Interviews were conducted until thematic saturation was achieved. Conventional content analysis was used to identify themes related to EM physicians' perspectives on acute diagnostic neurology; directed content analysis was used to explore views regarding CDS. Each interview transcript was independently coded by two researchers using an iteratively refined codebook with consensus-based resolution of coding differences. We identified two domains regarding diagnostic safety: (1) challenges unique to neurological complaints and (2) challenges in EM more broadly. Themes relevant to neurology included: (1) knowledge gaps and uncertainty, (2) skepticism about neurology, (3) comfort with basic as opposed to detailed neurological examination, and (4) comfort with non-neurological diseases. Themes relevant to diagnostic decision making in the ED included: (1) cognitive biases, (2) ED system/environmental issues, (3) patient barriers, (4) comfort with diagnostic uncertainty, and (5) concerns regarding diagnostic error identification and measurement. Most participating EM physicians were enthusiastic about the potential for well-designed CDS to improve diagnostic accuracy for non-specific neurological complaints. Physicians identified diagnostic challenges unique to neurological diseases as well as issues related more generally to diagnostic accuracy in EM. These physician-reported issues should be accounted for when designing interventions to improve ED diagnostic accuracy.

Clinical decision support systems with team-based care on type 2 diabetes improvement for Medicaid patients: A quality improvement project

BackgroundThe prevalence of clinical inertia, the failure of appropriate treatment intensification in diabetes treatment, is a well-documented worldwide phenomenon. This project addresses the problem of clinical inertia through three interrelated activities, clinical decision support (CDSS), team-based care, and patient engagement in diabetes management. ObjectivesThe purpose of this research is to provide analysis under the State-University Partnership Learning Network regarding the impact of an electronic decision support tool combined with team-based care workflow on provider decision-making and patient outcomes for the treatment of poorly controlled diabetes mellitus (diabetes) among patients receiving Kentucky Medicaid. The objectives of this study are to 1) assess clinical outcomes of type 2 diabetes in the Medicaid population with team-based care using CDSS, 2) evaluate physicians' and pharmacists' experience on CDSS. MethodsThis is a quality improvement project using a mixed-method - longitudinal and control group comparison of outcomes based upon clinical measures and online surveys of providers and pharmacists involved in this project. ResultsPatients treated by providers who changed the treatment regimen to one that either fully or partially followed the recommendation of the CDSS tool had a statistically significant reduction in HbA1c with an average initial HbA1c of 10.1 and the final HbA1c of 8. The online survey of physicians shows that more than 80% of physicians agree the use of CDSS will support improved patient outcomes. The use of a team-based care approach that includes pharmacists in implementing treatment changes was broadly supported by both physicians and pharmacists. ConclusionCDSS combined with team-based care can be effective in reducing HbA1c to targeted therapeutic levels. The use of CDSS provides a way to efficiently assess more than 160 potential frontline drugs and properly accelerate treatment. Consistent with the research literature, the inclusion of pharmacists can play a key role in team-based care to assess treatment alternatives and provide for improvement in outcomes and patient adherence for diabetes. The user surveys show both physicians and pharmacists have a positive attitude toward CDSS.

Developing and Demonstrating the Viability and Availability of the Multilevel Implementation Strategy for Syncope Optimal Care Through Engagement (MISSION) Syncope App: Evidence-Based Clinical Decision Support Tool.

BackgroundSyncope evaluation and management is associated with testing overuse and unnecessary hospitalizations. The 2017 American College of Cardiology/American Heart Association (ACC/AHA) Syncope Guideline aims to standardize clinical practice and reduce unnecessary services. The use of clinical decision support (CDS) tools offers the potential to successfully implement evidence-based clinical guidelines. However, CDS tools that provide an evidence-based differential diagnosis (DDx) of syncope at the point of care are currently lacking.ObjectiveWith input from diverse health systems, we developed and demonstrated the viability of a mobile app, the Multilevel Implementation Strategy for Syncope optImal care thrOugh eNgagement (MISSION) Syncope, as a CDS tool for syncope diagnosis and prognosis.MethodsDevelopment of the app had three main goals: (1) reliable generation of an accurate DDx, (2) incorporation of an evidence-based clinical risk tool for prognosis, and (3) user-based design and technical development. To generate a DDx that incorporated assessment recommendations, we reviewed guidelines and the literature to determine clinical assessment questions (variables) and likelihood ratios (LHRs) for each variable in predicting etiology. The creation and validation of the app diagnosis occurred through an iterative clinician review and application to actual clinical cases. The review of available risk score calculators focused on identifying an easily applied and valid evidence-based clinical risk stratification tool. The review and decision-making factors included characteristics of the original study, clinical variables, and validation studies. App design and development relied on user-centered design principles. We used observations of the emergency department workflow, storyboard demonstration, multiple mock review sessions, and beta-testing to optimize functionality and usability.ResultsThe MISSION Syncope app is consistent with guideline recommendations on evidence-based practice (EBP), and its user interface (UI) reflects steps in a real-world patient evaluation: assessment, DDx, risk stratification, and recommendations. The app provides flexible clinical decision making, while emphasizing a care continuum; it generates recommendations for diagnosis and prognosis based on user input. The DDx in the app is deemed a pragmatic model that more closely aligns with real-world clinical practice and was validated using actual clinical cases. The beta-testing of the app demonstrated well-accepted functionality and usability of this syncope CDS tool.ConclusionsThe MISSION Syncope app development integrated the current literature and clinical expertise to provide an evidence-based DDx, a prognosis using a validated scoring system, and recommendations based on clinical guidelines. This app demonstrates the importance of using research literature in the development of a CDS tool and applying clinical experience to fill the gaps in available research. It is essential for a successful app to be deliberate in pursuing a practical clinical model instead of striving for a perfect mathematical model, given available published evidence. This hybrid methodology can be applied to similar CDS tool development.

A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study.

BackgroundExposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric.ObjectiveThis study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs.MethodsWe used an SDM framework and user-centered design methods to guide the design and usability of DDInteract—an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale.ResultsThe designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5.ConclusionsThis study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology.

A novel method for causal structure discovery from EHR data and its application to type-2 diabetes mellitus

Modern AI-based clinical decision support models owe their success in part to the very large number of predictors they use. Safe and robust decision support, especially for intervention planning, requires causal, not associative, relationships. Traditional methods of causal discovery, clinical trials and extracting biochemical pathways, are resource intensive and may not scale up to the number and complexity of relationships sufficient for precision treatment planning. Computational causal structure discovery (CSD) from electronic health records (EHR) data can represent a solution, however, current CSD methods fall short on EHR data. This paper presents a CSD method tailored to the EHR data. The application of the proposed methodology was demonstrated on type-2 diabetes mellitus. A large EHR dataset from Mayo Clinic was used as development cohort, and another large dataset from an independent health system, M Health Fairview, as external validation cohort. The proposed method achieved very high recall (.95) and substantially higher precision than the general-purpose methods (.84 versus .29, and .55). The causal relationships extracted from the development and external validation cohorts had a high (81%) overlap. Due to the adaptations to EHR data, the proposed method is more suitable for use in clinical decision support than the general-purpose methods.