Clinical Decision Support Design Research Articles

Accurate Dosing Weight: When the 10% Really Matters.

Children are at increased risk of medication-associated adverse events, often due to weight-based dosing errors. We aimed to reduce the proportion of medications that were administered where the dosing weight was ≥ 10% different from the recorded weight. We adopted in-situ usability testing to iteratively improve design of clinical decision support that would enable accurate dosing weight documentation by prompting clinicians to update weight if recorded weight was > 10% different and it had been at least 7 days since the last dosing weight update. The proportion of medication administrations with difference >10% between their recorded weight and dosing weight decreased from 13.1% (56,256/ 429,006) in the baseline period to 9.5% (35,560 / 372,443) in the intervention period (P < 0.001). User-centered design of an interruptive alert improved the accuracy of dosing weights during medication administrations without substantial alert burden. In-situ usability testing is an effective approach to rapidly obtain feedback from frontline users and iterate on the design to effect desired behavior changes.

Sepsis Prediction in Hospitalized Children: Clinical Decision Support Design and Deployment.

Following development and validation of a sepsis prediction model described in a companion article, we aimed to use quality improvement and safety methodology to guide the design and deployment of clinical decision support (CDS) tools and clinician workflows to improve pediatric sepsis recognition in the inpatient setting. CDS tools and sepsis huddle workflows were created to implement an electronic health record-based sepsis prediction model. These were proactively analyzed and refined using simulation and safety science principles before implementation and were introduced across inpatient units during 2020-2021. Huddle compliance, alerts per non-ICU patient days, and days between sepsis-attributable emergent transfers were monitored. Rapid Plan-Do-Study-Act (PDSA) cycles based on user feedback and weekly metric data informed improvement throughout implementation. There were 264 sepsis alerts on 173 patients with an 89% bedside huddle completion rate and 10 alerts per 1000 non-ICU patient days per month. There was no special cause variation in the metric days between sepsis-attributable emergent transfers. An automated electronic health record-based sepsis prediction model, CDS tools, and sepsis huddle workflows were implemented on inpatient units with a relatively low rate of interruptive alerts and high compliance with bedside huddles. Use of CDS best practices, simulation, safety tools, and quality improvement principles led to high utilization of the sepsis screening process.

Evidence-Based Clinical Decision Support to Improve Care for Patients Hospitalized With Acute Myocardial Infarction.

Clinical decision support in the EHR is an innovation that can support guideline adherence in acute myocardial infarction. Cardiac rehabilitation referral and left ventricular systolic function assessment are part of evidence-based clinical practice guidelines associated with reduced morbidity and mortality following acute myocardial infarction. Effective clinical decision support is sustained by evidence-based principles for design and implementation. This quality improvement project evaluated the impact of practice advisories designed using principles of effective clinical decision support design to improve performance of left ventricular systolic function assessment and ambulatory referral to cardiac rehabilitation for patients hospitalized with acute myocardial infarction. Performance in cardiac rehabilitation referral and left ventricular systolic function assessment was measured for a 3-month interval pre- and post-intervention. Pre-implementation, cardiac rehabilitation referral or valid documented reason for non-referral was 80.3%. Rehabilitation referral or documented valid reason for non-referral increased to 98.4% post-implementation (P < .001). Left ventricular systolic function assessment increased from 94.2% to 100% following clinical decision support implementation (P = .120). This quality improvement project supports the positive impact of effective clinical decision support design and implementation to improve outcomes for patients hospitalized with acute myocardial infarction.

Human-centered design of clinical decision support for management of hypertension with chronic kidney disease

BackgroundPrimary care providers face challenges in recognizing and controlling hypertension in patients with chronic kidney disease (CKD). Clinical decision support (CDS) has the potential to aid clinicians in identifying patients who could benefit from medication changes. This study designed an alert to control hypertension in CKD patients using an iterative human-centered design process.MethodsIn this study, we present a human-centered design process employing multiple methods for gathering user requirements and feedback on design and usability. Initially, we conducted contextual inquiry sessions to gather user requirements for the CDS. This was followed by group design sessions and one-on-one formative think-aloud sessions to validate requirements, obtain feedback on the design and layout, uncover usability issues, and validate changes.ResultsThis study included 20 participants. The contextual inquiry produced 10 user requirements which influenced the initial alert design. The group design sessions revealed issues related to several themes, including recommendations and clinical content that did not match providers' expectations and extraneous information on the alerts that did not provide value. Findings from the individual think-aloud sessions revealed that participants disagreed with some recommended clinical actions, requested additional information, and had concerns about the placement in their workflow. Following each step, iterative changes were made to the alert content and design.DiscussionThis study showed that participation from users throughout the design process can lead to a better understanding of user requirements and optimal design, even within the constraints of an EHR alerting system. While raising awareness of design needs, it also revealed concerns related to workflow, understandability, and relevance.ConclusionThe human-centered design framework using multiple methods for CDS development informed the creation of an alert to assist in the treatment and recognition of hypertension in patients with CKD.

Using a Sociotechnical Model to Understand Challenges with Sepsis Recognition among Critically Ill Infants

Abstract Objective The aim of the study is to apply a sociotechnical model to the requirements phase of implementing a machine learning algorithm-based system to support sepsis recognition in the neonatal intensive care unit. Methods We incorporated components from the sociotechnical model, Safety in Engineering for Patient Safety 2.0, in three requirements phase activities: (1) semi-structured interviews, (2) user profiles, and (3) system use cases. Results Thirty-one neonatal intensive care unit clinicians participated in semi-structured interviews (11 nurses, 10 front line ordering clinician, five fellows, and five attending physician). Interview transcripts were coded and then compiled into themes deductively based on components from the sociotechnical model (persons, environment, organization, tasks, tools and technology, collaboration, and outcomes). The interview analysis was used to create four user profiles defining responsibilities in sepsis recognition, team collaboration, and attributes relevant to sepsis recognition. Two user profiles (nurse, front line ordering clinician) included variants based on experience relevant to sepsis recognition. The interview analysis was used to develop three system use cases representing clinical sepsis scenarios. Each use case defines the precondition, actors, and high-level sequence of actions, and includes variants based on sociotechnical works system factors that can complicate sepsis recognition. The interview analysis, user profiles, and use cases serve as the foundation for supporting sociotechnical design to all subsequent human-centered design methods including subject recruitment, formative design, summative user testing, and simulation testing. Conclusion Integration of the sociotechnical model-guided requirements gathering activities, analysis, and deliverables by framing a range of sociotechnical components and the interconnectedness of these components in the broader work system. Applying the sociotechnical model resulted in discovering work system, process, and outcome requirements that would otherwise be difficult to capture, or missed entirely, using traditional requirements gathering methods or approaches to clinical decision support design.

Engaging clinicians early during the development of a graphical user display of an intelligent alerting system at the bedside

BackgroundArtificial Intelligence (AI) is increasingly used to support bedside clinical decisions, but information must be presented in usable ways within workflow. Graphical User Interfaces (GUI) are front-facing presentations for communicating AI outputs, but clinicians are not routinely invited to participate in their design, hindering AI solution potential. PurposeTo inform early user-engaged design of a GUI prototype aimed at predicting future Cardiorespiratory Insufficiency (CRI) by exploring clinician methods for identifying at-risk patients, previous experience with implementing new technologies into clinical workflow, and user perspectives on GUI screen changes. MethodsWe conducted a qualitative focus group study to elicit iterative design feedback from clinical end-users on an early GUI prototype display. Five online focus group sessions were held, each moderated by an expert focus group methodologist. Iterative design changes were made sequentially, and the updated GUI display was presented to the next group of participants. Results23 clinicians were recruited (14 nurses, 4 nurse practitioners, 5 physicians; median participant age ∼35 years; 60% female; median clinical experience 8 years). Five themes emerged from thematic content analysis: trend evolution, context (risk evolution relative to vital signs and interventions), evaluation/interpretation/explanation (sub theme: continuity of evaluation), clinician intuition, and clinical operations. Based on these themes, GUI display changes were made. For example, color and scale adjustments, integration of clinical information, and threshold personalization. ConclusionsEarly user-engaged design was useful in adjusting GUI presentation of AI output. Next steps involve clinical testing and further design modification of the AI output to optimally facilitate clinician surveillance and decisions. Clinicians should be involved early and often in clinical decision support design to optimize efficacy of AI tools.

Mapping Usability Heuristics to Medical Ethical Principles in Clinical Decision Support Design

Human factors professionals voice concerns about the slow and uneven adoption of human-centered design (HCD) principles in healthcare. With good intentions and best efforts, human factors work in healthcare continues, but to understand and to be understood we must share a language that speaks to one another. One way to facilitate this understanding is by bringing attention to how we talk about decision-making approaches in medical ethics and HCD. Both disciplines suggest efforts should be participatory, guided by a set of principles applied thoughtfully, with critical consideration to context and trade-offs. Considering the connections between usability heuristics for design and medical ethical principles may offer a bridge between disciplines and their associated dialects. In this paper, we provide three representations mapping ethical principles to design principles, including usability heuristics: a word tree visualization, narrative passages, and a table format. We believe this mapping demonstrates both that ethical design is inherently human-centric and that human-centered design is ethical design.

Applying Clinical Decision Support Design Best Practices With the Practical Robust Implementation and Sustainability Model Versus Reliance on Commercially Available Clinical Decision Support Tools: Randomized Controlled Trial.

BackgroundLimited consideration of clinical decision support (CDS) design best practices, such as a user-centered design, is often cited as a key barrier to CDS adoption and effectiveness. The application of CDS best practices is resource intensive; thus, institutions often rely on commercially available CDS tools that are created to meet the generalized needs of many institutions and are not user centered. Beyond resource availability, insufficient guidance on how to address key aspects of implementation, such as contextual factors, may also limit the application of CDS best practices. An implementation science (IS) framework could provide needed guidance and increase the reproducibility of CDS implementations.ObjectiveThis study aims to compare the effectiveness of an enhanced CDS tool informed by CDS best practices and an IS framework with a generic, commercially available CDS tool.MethodsWe conducted an explanatory sequential mixed methods study. An IS-enhanced and commercial CDS alert were compared in a cluster randomized trial across 28 primary care clinics. Both alerts aimed to improve beta-blocker prescribing for heart failure. The enhanced alert was informed by CDS best practices and the Practical, Robust, Implementation, and Sustainability Model (PRISM) IS framework, whereas the commercial alert followed vendor-supplied specifications. Following PRISM, the enhanced alert was informed by iterative, multilevel stakeholder input and the dynamic interactions of the internal and external environment. Outcomes aligned with PRISM’s evaluation measures, including patient reach, clinician adoption, and changes in prescribing behavior. Clinicians exposed to each alert were interviewed to identify design features that might influence adoption. The interviews were analyzed using a thematic approach.ResultsBetween March 15 and August 23, 2019, the enhanced alert fired for 61 patients (106 alerts, 87 clinicians) and the commercial alert fired for 26 patients (59 alerts, 31 clinicians). The adoption and effectiveness of the enhanced alert were significantly higher than those of the commercial alert (62% vs 29% alerts adopted, P<.001; 14% vs 0% changed prescribing, P=.006). Of the 21 clinicians interviewed, most stated that they preferred the enhanced alert.ConclusionsThe results of this study suggest that applying CDS best practices with an IS framework to create CDS tools improves implementation success compared with a commercially available tool.Trial RegistrationClinicalTrials.gov NCT04028557; http://clinicaltrials.gov/ct2/show/NCT04028557

Choosing and Using Patient-Reported Outcome Measures in Clinical Practice

The increasing use of patient-reported outcome (PRO) measures is forcing clinicians and health care systems to decide which to select and how to incorporate them into their records and clinical workflows. This overview addresses 3 topics related to these concerns. First, a literature review summarizes key psychometric and practical factors (such as reliability, responsiveness, computer adaptive testing, and interpretability) in choosing PROs for clinical practice. Second, 3 clinical decision support issues are highlighted: gathering PROs, electronic health record effect on providers, and incorporating PROs into clinical decision support design and implementation. Lastly, the salience of crosscutting domains as well as 9 key pragmatic decisions are reviewed. Crosscutting domains are those that are relevant across most medical and mental health conditions, such as the SPADE symptom pentad (sleep problems, pain, anxiety, depression, low energy/fatigue) and physical functioning. The 9 pragmatic decisions include (1) generic vs disease-specific scales; (2) single- vs multidomain scales; (3) universal scales vs user-choice selection; (4) number of domains to measure; (5) prioritization of domains when multiple domains are assessed; (6) action thresholds; (7) clinical purpose (screening vs monitoring); as well as the (8) frequency and (9) logistical aspects of PRO administration.

Informatics and interaction: Applying human factors principles to optimize the design of clinical decision support for sepsis.

In caring for patients with sepsis, the current structure of electronic health record systems allows clinical providers access to raw patient data without imputation of its significance. There are a wide range of sepsis alerts in clinical care that act as clinical decision support tools to assist in early recognition of sepsis; however, there are serious shortcomings in existing health information technology for alerting providers in a meaningful way. Little work has been done to evaluate and assess existing alerts using implementation and process outcomes associated with health information technology displays, specifically evaluating clinician preference and performance. We developed graphical model displays of two popular sepsis scoring systems, quick Sepsis Related Organ Failure Assessment and Predisposition, Infection, Response, Organ Failure, using human factors principles grounded in user-centered and interaction design. Models will be evaluated in a larger research effort to optimize alert design to improve the collective awareness of high-risk populations and develop a relevant point-of-care clinical decision support system for sepsis.

A Human(e) Factor in Clinical Decision Support Systems.

The overwhelming amount, production speed, multidimensionality, and potential value of data currently available—often simplified and referred to as big data —exceed the limits of understanding of the human brain. At the same time, developments in data analytics and computational power provide the opportunity to obtain new insights and transfer data-provided added value to clinical practice in real time. What is the role of the health care professional in collaboration with the data scientist in the changing landscape of modern care? We discuss how health care professionals should provide expert knowledge in each of the stages of clinical decision support design: data level, algorithm level, and decision support level. Including various ethical considerations, we advocate for health care professionals to responsibly initiate and guide interprofessional teams, including patients, and embrace novel analytic technologies to translate big data into patient benefit driven by human(e) values.

Corrigendum to: Pharmacogenomic clinical decision support design and multi-site process outcomes analysis in the eMERGE Network.

Corrigendum to: Pharmacogenomic clinical decision support design and multi-site process outcomes analysis in the eMERGE Network.

Adaptive design of a clinical decision support tool: What the impact on utilization rates means for future CDS research.

OBJECTIVEWe employed an agile, user-centered approach to the design of a clinical decision support tool in our prior integrated clinical prediction rule study, which achieved high adoption rates. To understand if applying this user-centered process to adapt clinical decision support tools is effective in improving the use of clinical prediction rules, we examined utilization rates of a clinical decision support tool adapted from the original integrated clinical prediction rule study tool to determine if applying this user-centered process to design yields enhanced utilization rates similar to the integrated clinical prediction rule study.MATERIALS & METHODS: We conducted pre-deployment usability testing and semi-structured group interviews at 6 months post-deployment with 75 providers at 14 intervention clinics across the two sites to collect user feedback. Qualitative data analysis is bifurcated into immediate and delayed stages; we reported on immediate-stage findings from real-time field notes used to generate a set of rapid, pragmatic recommendations for iterative refinement. Monthly utilization rates were calculated and examined over 12 months.RESULTSWe hypothesized a well-validated, user-centered clinical decision support tool would lead to relatively high adoption rates. Then 6 months post-deployment, integrated clinical prediction rule study tool utilization rates were substantially lower than anticipated based on the original integrated clinical prediction rule study trial (68%) at 17% (Health System A) and 5% (Health System B). User feedback at 6 months resulted in recommendations for tool refinement, which were incorporated when possible into tool design; however, utilization rates at 12 months post-deployment remained low at 14% and 4% respectively.DISCUSSIONAlthough valuable, findings demonstrate the limitations of a user-centered approach given the complexity of clinical decision support.CONCLUSIONStrategies for addressing persistent external factors impacting clinical decision support adoption should be considered in addition to the user-centered design and implementation of clinical decision support.

Pharmacogenomic clinical decision support design and multi-site process outcomes analysis in the eMERGE Network.

To better understand the real-world effects of pharmacogenomic (PGx) alerts, this study aimed to characterize alert design within the eMERGE Network, and to establish a method for sharing PGx alert response data for aggregate analysis. Seven eMERGE sites submitted design details and established an alert logging data dictionary. Six sites participated in a pilot study, sharing alert response data from their electronic health record systems. PGx alert design varied, with some consensus around the use of active, post-test alerts to convey Clinical Pharmacogenetics Implementation Consortium recommendations. Sites successfully shared response data, with wide variation in acceptance and follow rates. Results reflect the lack of standardization in PGx alert design. Standards and/or larger studies will be necessary to fully understand PGx impact. This study demonstrated a method for sharing PGx alert response data and established that variation in system design is a significant barrier for multi-site analyses.

Understanding complex clinical reasoning in infectious diseases for improving clinical decision support design.

BackgroundClinical experts’ cognitive mechanisms for managing complexity have implications for the design of future innovative healthcare systems. The purpose of the study is to examine the constituents of decision complexity and explore the cognitive strategies clinicians use to control and adapt to their information environment.MethodsWe used Cognitive Task Analysis (CTA) methods to interview 10 Infectious Disease (ID) experts at the University of Utah and Salt Lake City Veterans Administration Medical Center. Participants were asked to recall a complex, critical and vivid antibiotic-prescribing incident using the Critical Decision Method (CDM), a type of Cognitive Task Analysis (CTA). Using the four iterations of the Critical Decision Method, questions were posed to fully explore the incident, focusing in depth on the clinical components underlying the complexity. Probes were included to assess cognitive and decision strategies used by participants.ResultsThe following three themes emerged as the constituents of decision complexity experienced by the Infectious Diseases experts: 1) the overall clinical picture does not match the pattern, 2) a lack of comprehension of the situation and 3) dealing with social and emotional pressures such as fear and anxiety. All these factors contribute to decision complexity. These factors almost always occurred together, creating unexpected events and uncertainty in clinical reasoning. Five themes emerged in the analyses of how experts deal with the complexity. Expert clinicians frequently used 1) watchful waiting instead of over- prescribing antibiotics, engaged in 2) theory of mind to project and simulate other practitioners’ perspectives, reduced very complex cases into simple 3) heuristics, employed 4) anticipatory thinking to plan and re-plan events and consulted with peers to share knowledge, solicit opinions and 5) seek help on patient cases.ConclusionThe cognitive strategies to deal with decision complexity found in this study have important implications for design future decision support systems for the management of complex patients.Electronic supplementary materialThe online version of this article (doi:10.1186/s12911-015-0221-z) contains supplementary material, which is available to authorized users.

Heuristics in Managing Complex Clinical Decision Tasks in Experts' Decision Making.

Clinical decision support is a tool to help experts make optimal and efficient decisions. However, little is known about the high level of abstractions in the thinking process for the experts. The objective of the study is to understand how clinicians manage complexity while dealing with complex clinical decision tasks. After approval from the Institutional Review Board (IRB), three clinical experts were interviewed the transcripts from these interviews were analyzed. We found five broad categories of strategies by experts for managing complex clinical decision tasks: decision conflict, mental projection, decision trade-offs, managing uncertainty and generating rule of thumb. Complexity is created by decision conflicts, mental projection, limited options and treatment uncertainty. Experts cope with complexity in a variety of ways, including using efficient and fast decision strategies to simplify complex decision tasks, mentally simulating outcomes and focusing on only the most relevant information. Understanding complex decision making processes can help design allocation based on the complexity of task for clinical decision support design.