Clinical Decision Support Systems Alerts Research Articles

Poor performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction.

Clinical decision support systems (CDSS) are used to identify drugs with potential need for dose modification in patients with renal impairment. ChatGPT holds the potential to be integrated in the electronic health record (EHR) system to give such dosing advices. In this study, we aim to evaluate the performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal impairment. This cross-sectional study was performed at Tergooi Medical Center, the Netherlands. CDSS alerts regarding renal dysfunction were collected from the electronic health record (EHR) during a 2-week period and were presented to ChatGPT and an expert panel. Alerts were presented with and without patient variables. To evaluate the performance, suggested medication interventions were compared. In total, 172 CDDS alerts were generated for 80 patients. Indecisive responses by ChatGPT to alerts were excluded. For alerts presented without patient variables, ChatGPT provided "correct and identical" responses to 19.9%, "correct and different" responses to 26.7%, and "incorrect responses to 53.4% of the alerts. For alerts including patient variables, ChatGPT provided "correct and identical" responses to 16.7%, "correct and different" responses to 16.0%, and "incorrect responses to 67.3% of the alerts. Accuracy was better for newer drugs such as direct oral anticoagulants. The performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction was poor. Based on these results, we conclude that ChatGPT, in its current state, is not appropriate for automatic integration into our EHR to handle CDSS alerts related to renal dysfunction.

Clinical decision support system supported interventions in hospitalized older patients: a matter of natural course and adequate timing

BackgroundDrug-related problems (DRPs) and potentially inappropriate prescribing (PIP) are associated with adverse patient and health care outcomes. In the setting of hospitalized older patients, Clinical Decision Support Systems (CDSSs) could reduce PIP and therefore improve clinical outcomes. However, prior research showed a low proportion of adherence to CDSS recommendations by clinicians with possible explanatory factors such as little clinical relevance and alert fatigue.ObjectiveTo investigate the use of a CDSS in a real-life setting of hospitalized older patients. We aim to (I) report the natural course and interventions based on the top 20 rule alerts (the 20 most frequently generated alerts per clinical rule) of generated red CDSS alerts (those requiring action) over time from day 1 to 7 of hospitalization; and (II) to explore whether an optimal timing can be defined (in terms of day per rule).MethodsAll hospitalized patients aged ≥ 60 years, admitted to Zuyderland Medical Centre (the Netherlands) were included. The evaluation of the CDSS was investigated using a database used for standard care. Our CDSS was run daily and was evaluated on day 1 to 7 of hospitalization. We collected demographic and clinical data, and moreover the total number of CDSS alerts; the total number of top 20 rule alerts; those that resulted in an action by the pharmacist and the course of outcome of the alerts on days 1 to 7 of hospitalization.ResultsIn total 3574 unique hospitalized patients, mean age 76.7 (SD 8.3) years and 53% female, were included. From these patients, in total 8073 alerts were generated; with the top 20 of rule alerts we covered roughly 90% of the total. For most rules in the top 20 the highest percentage of resolved alerts lies somewhere between day 4 and 5 of hospitalization, after which there is equalization or a decrease. Although for some rules, there is a gradual increase in resolved alerts until day 7. The level of resolved rule alerts varied between the different clinical rules; varying from > 50–70% (potassium levels, anticoagulation, renal function) to less than 25%.ConclusionThis study reports the course of the 20 most frequently generated alerts of a CDSS in a setting of hospitalized older patients. We have shown that for most rules, irrespective of an intervention by the pharmacist, the highest percentage of resolved rules is between day 4 and 5 of hospitalization. The difference in level of resolved alerts between the different rules, could point to more or less clinical relevance and advocates further research to explore ways of optimizing CDSSs by adjustment in timing and number of alerts to prevent alert fatigue.

Novel data visualization method to measure alert efficiency in computerized physician order entry (CPOE) system

ObjectivesTo introduce a novel visualization technique for evaluating the efficacy of clinical decision support system (CDSS) alerts as perceived by physicians and to differentiate between various alert categories for optimization. MethodsWe developed a visualization method, which segments into four distinct zones: Appropriate (+/+), Over-frequent yet Effective (−/+), Potentially Problematic (−/−), and Less Effective but Acceptably Frequent (+/−). Alerts from a 908-bed academic medical center in Northern Taiwan were collected over two years and classified using this technique, along with three perspectives: Safety, Completeness, and Response. ResultsWe collected the viewpoints of 72 clinical physicians on the system's top 20 most frequent alerts. The proposed visualization technique offers a user-centric, adaptable method for assessing CDSS alerts. Roughly five alerts were categorized as Potentially Problematic, whereas another five were deemed Appropriate. Intriguingly, certain alerts, while not beneficial for patient safety, were found to assist physicians in completing clinical workflows. ConclusionsThis approach, emphasizing visual clarity and adaptability, diverges from traditional methods that lean heavily on expert opinions or statistics. It paves the way for diverse assessment perspectives, furnishing healthcare institutions with a valuable tool to improve CDSS alert systems, ensuring a harmonious balance between user efficiency and patient safety.

Facilitators and Barriers to Interacting With Clinical Decision Support in the ICU: A Mixed-Methods Approach.

Clinical decision support systems (CDSSs) are used in various aspects of healthcare to improve clinical decision-making, including in the ICU. However, there is growing evidence that CDSS are not used to their full potential, often resulting in alert fatigue which has been associated with patient harm. Clinicians in the ICU may be more vulnerable to desensitization of alerts than clinicians in less urgent parts of the hospital. We evaluated facilitators and barriers to appropriate CDSS interaction and provide methods to improve currently available CDSS in the ICU. Sequential explanatory mixed-methods study design, using the BEhavior and Acceptance fRamework. International survey study. Clinicians (pharmacists, physicians) identified via survey, with recent experience with clinical decision support. An initial survey was developed to evaluate clinician perspectives on their interactions with CDSS. A subsequent in-depth interview was developed to further evaluate clinician (pharmacist, physician) beliefs and behaviors about CDSS. These interviews were then qualitatively analyzed to determine themes of facilitators and barriers with CDSS interactions. A total of 48 respondents completed the initial survey (estimated response rate 15.5%). The majority believed that responding to CDSS alerts was part of their job (75%) but felt they experienced alert fatigue (56.5%). In the qualitative analysis, a total of five facilitators (patient safety, ease of response, specificity, prioritization, and feedback) and four barriers (excess quantity, work environment, difficulty in response, and irrelevance) were identified from the in-depth interviews. In this mixed-methods survey, we identified areas that institutions should focus on to improve appropriate clinician interactions with CDSS, specific to the ICU. Tailoring of CDSS to the ICU may lead to improvement in CDSS and subsequent improved patient safety outcomes.

Clinical Decision Support Systems in Hospitalized Older Patients: An Exploratory Analysis in a Real-Life Clinical Setting.

Inappropriate prescribing is associated with negative patient outcomes. In hospitalized patients, the use of Clinical Decision Support Systems (CDSSs) may reduce inappropriate prescribing and thereby improve patient-related outcomes. However, recently published large clinical trials (OPERAM and SENATOR) have shown negative results on the use of CDSSs and patient outcomes and strikingly low acceptance of recommendations. The purpose of the present study was to investigate the use of a CDSS in a real-life clinical setting of hospitalized older patients. As such, we report on the real-life pattern of this in-hospital implemented CDSS, including (i) whether generated alerts were resolved; (ii) whether a recorded action by the pharmacist led to an improved number of resolved alerts; and (iii) the natural course of generated alerts, in particular of those in the non-intervention group; as these data are largely lacking in current studies. Hospitalized patients, aged 60 years and older, admitted to Zuyderland Medical Centre, the Netherlands, in 2018 were included. The evaluation of the CDSS was investigated using a database used for standard care. Alongside demographic and clinical data, we also collected the total numbers of CDSS alerts, the number of alerts 'handled' by the pharmacist, those that resulted in an action by the pharmacist, and finally the outcome of the alerts at day 1 and day 3 after the alert was generated. A total of 3574 unique hospitalized patients, mean age 76.7 (SD 8.3) years and 53% female, were included. From these patients, 8073 alerts were generated, of which 7907 (97.9% of total) were handled by the pharmacist (day 1). In 51.6% of the alerts handled by the pharmacist, an action was initiated, resulting in 36.1% of the alerts resolved after day 1, compared with 27.3% if the pharmacist did not perform an action (p<0.001). On day 3, in 52.6% of the alerts an action by the pharmacist was initiated, resulting in 62.4% resolved alerts, compared with 48.0% when no action was performed (p<0.001). In the category renal function, the percentages differed significantly between an action versus no action of the pharmacist at day 1 and at day 3 (16.6% vs 10.6%, p<0.001 [day 1]; 29.8% vs 19.4%, p<0.001 [day 3]). This study demonstrates the pattern and natural course of clinical alerts of an in-hospital implemented CDSS in a real-life clinical setting of hospitalized older patients. Besides the already known beneficial effect of actions by pharmacists, we have also shown that many alerts become resolved without any specific intervention. As such, our study provides an important insight into the spontaneous course of resolved alerts, since these data are currently lacking in the literature.

Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis.

A clinical decision support system (CDSS) informs or generates medical recommendations for healthcare practitioners. An alert is the most common way for a CDSS to interact with practitioners. Research about alerts in CDSS has proliferated over the past ten years. The research trend is ongoing with new emerging terms and focus. Bibliometric analysis is ideal for researchers to understand the research trend and future directions. Influential articles, institutes, countries, authors, and commonly used keywords were analyzed to grasp a comprehensive view on our topic, alerts in CDSS. Articles published between 2011 and 2021 were extracted from the Web of Science database. There were 728 articles included for bibliometric analysis, among which 24 papers were selected for content analysis. Our analysis shows that the research direction has shifted from patient safety to system utility, implying the importance of alert usability to be clinically impactful. Finally, we conclude with future research directions such as the optimization of alert mechanisms and comprehensiveness to enhance alert appropriateness and to reduce alert fatigue.

Nudging within learning health systems: next generation decision support to improve cardiovascular care.

The increasing volume and richness of healthcare data collected during routine clinical practice have not yet translated into significant numbers of actionable insights that have systematically improved patient outcomes. An evidence-practice gap continues to exist in healthcare. We contest that this gap can be reduced by assessing the use of nudge theory as part of clinical decision support systems (CDSS). Deploying nudges to modify clinician behaviour and improve adherence to guideline-directed therapy represents an underused tool in bridging the evidence-practice gap. In conjunction with electronic health records (EHRs) and newer devices including artificial intelligence algorithms that are increasingly integrated within learning health systems, nudges such as CDSS alerts should be iteratively tested for all stakeholders involved in health decision-making: clinicians, researchers, and patients alike. Not only could they improve the implementation of known evidence, but the true value of nudging could lie in areas where traditional randomized controlled trials are lacking, and where clinical equipoise and variation dominate. The opportunity to test CDSS nudge alerts and their ability to standardize behaviour in the face of uncertainty may generate novel insights and improve patient outcomes in areas of clinical practice currently without a robust evidence base.

Medication Errors and Patient Safety: Evaluation of Physicians' Responses to Medication-Related Alert Overrides in Clinical Decision Support Systems.

Background:Clinical decision support systems (CDSS) can enhance patient safety and reduce medication errors by giving physicians alerts while dispensing medications. Physicians inappropriately override these alerts for various reasons, which can possibly lead to medication errors and impact patient safety.Objective:To assess the appropriateness of overridden major medication-related alerts, to investigate the reasons behind inappropriate overriding, and to evaluate if medication errors occur in inappropriately overridden alerts.Methods:A mixed-methods study was conducted.. Quantitative: Retrospective observation to evaluate the appropriateness of major drug-dose related alert overrides. A simple random sample was taken from appropriate and inappropriate overrides and reviewed for medication errors. Qualitative: Semi-Structured Interviews were conducted with ten consultant physicians from various specialties. Interviews were transcribed and coded inductively then analyzed using Thematic Content Analysis.Results:Out of 1087 alert overrides that were evaluated for appropriateness, 738 were inappropriately overridden (67.89%). In a sample of 283 inappropriate and 92 appropriate overrides; the resulted medication errors were 7 and 0, respectively. Qualitative analysis resulted in three emergent themes; Judgement, Experience & Guidelines, CDSS Issues & Limitations, Physician Behavior & Safety.Conclusion:The majority of alerts were found to be inappropriately overridden. This can be attributed to physician reliance on their clinical knowledge and medication databases, having the pharmacists’ checks, and alert fatigue. CDSS alerts can be improved by making them more prominent and suppressing or descaling unnecessary alerts. The drop-down justification list can be enhanced by adding free text options and relating recommended dosing to disease or specialty.

1004. Clinical Decision Support System Alerts for HIV Retention in Care – A Pilot Implementation Research Study

BackgroundClinical decision support system (CDSS) alerts may help retain people living with HIV (PLWH) in care. A system of CDSS alerts utilizing the CHORUS™ portal was developed to identify PLWH at risk of being lost to care. To evaluate feasibility for a larger scale study, a before and after implementation research pilot study was implemented in the OPERA Cohort at three clinic sites in a southeastern US city.MethodsPeriods without intervention (before) or with CDSS alerts (after) were followed by 3 months of follow up. The study population consisted of PLWH with ≥ 1 electronic health record entry in the 2 years prior to, or during, the before or after period (Fig 1). To support clinicians through a discrete implementation strategy, alerts warning of suboptimal patient attendance were generated daily for the eligible PLWH at each site; providers or other clinic staff could respond to the alerts (Fig 2). Alerts, responses, and visits (i.e., meeting with provider or HIV lab measurement) were characterized. The proportion of PLWH with ≥ 1 visit in the before and after periods were compared at each site by Pearson’s Chi-square.Figure 1. Pilot study timeline Figure 2. CDSS alert criteria and response options ResultsA total of 12,230 PLWH were eligible (sites A: 11,271; B: 733; C: 1,344 PLWH), with > 75% in both the before and after periods. The ratio of alerts to responses was 11.9 at site A (2,245 alerts to 189 responses in 309 days; Fig 3A), and comparatively lower at sites B (756 alerts to 334 responses in 352 days, ratio=2.2; Fig 3B) and C (1,305 alerts to 896 responses in 246 days, ratio=1.5; Fig 3C). Responses to alerts were sporadic at sites A and B and consistent at site C. After the intervention, the proportion of PLWH with ≥ 1 visit stayed the same at site A (46% in both periods; p=0.47), decreased at site B (91% to 80%; p< 0.01), and increased at site C (72% to 81%; p< 0.01).Figure 3. Alerts and responses over time in (A) Site A, (B) Site B, and (C) Site C ConclusionThis pilot study was ecological by design: measures of retention in care were compared over two calendar periods, without accounting for changes in study populations, clinic characteristics, and policies in place over time (which could have impacted clinic attendance). Though engagement with the CDSS was suboptimal at some sites, this implementation pilot study has demonstrated the ability to implement a CDSS aimed at identifying at-risk PLWH, while highlighting areas for improvement in future larger scale studies.DisclosuresJoel Wesley Thompson, MHS, PA-C, AAHIVS, DFAAPA, MHS, PA-C, AAHIVS, DFAAPA, Gilead (Shareholder, Speaker’s Bureau)Janssen (Speaker’s Bureau)Theratechnologies (Speaker’s Bureau)ViiV (Speaker’s Bureau) Tammeka Evans, MoP, ViiV Healthcare (Employee)

Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review.

BackgroundThe clinical decision support system (CDSS) has become an indispensable tool for reducing medication errors and adverse drug events. However, numerous studies have reported that CDSS alerts are often overridden. The increase in override rates has raised questions about the appropriateness of CDSS application along with concerns about patient safety and quality of care.ObjectiveThe aim of this study was to conduct a systematic review to examine the override rate, the reasons for the alert override at the time of prescribing, and evaluate the appropriateness of overrides.MethodsWe searched electronic databases, including Google Scholar, PubMed, Embase, Scopus, and Web of Science, without language restrictions between January 1, 2000 and March 31, 2019. Two authors independently extracted data and crosschecked the extraction to avoid errors. The quality of the included studies was examined following Cochrane guidelines.ResultsWe included 23 articles in our systematic review. The range of average override alerts was 46.2%-96.2%. An average of 29.4%-100% of the overrides alerts were classified as appropriate, and the rate of appropriateness varied according to the alert type (drug-allergy interaction 63.4%-100%, drug-drug interaction 0%-95%, dose 43.9%-88.8%, geriatric 14.3%-57%, renal 27%-87.5%). The interrater reliability for the assessment of override alerts appropriateness was excellent (kappa=0.79-0.97). The most common reasons given for the override were “will monitor” and “patients have tolerated before.”ConclusionsThe findings of our study show that alert override rates are high, and certain categories of overrides such as drug-drug interaction, renal, and geriatric were classified as inappropriate. Nevertheless, large proportions of drug duplication, drug-allergy, and formulary alerts were appropriate, suggesting that these groups of alerts can be primary targets to revise and update the system for reducing alert fatigue. Future efforts should also focus on optimizing alert types, providing clear information, and explaining the rationale of the alert so that essential alerts are not inappropriately overridden.

Reducing Inappropriate Drug Use in Older Patients by Use of Clinical Decision Support in Community Pharmacy: A Mixed-Methods Evaluation.

Older people are prone to drug-related harm. Clinical decision support systems (CDSSs) in community pharmacies may improve appropriate prescribing in this population. This study investigated (persistent) drug therapy changes and its determinants to reduce potentially inappropriate medication (PIM) in older patients based on CDSS alerts and to investigate barriers and facilitators for implementation of drug therapy changes based on these CDSS alerts. Five clinical decision rules based on national guidelines for inappropriate drugs in older patients were incorporated in a web-based CDSS in 31 community pharmacies between February and April 2017. The CDSS generated alerts for patients aged > 70years who had prescriptions for one of the following drugs: alprazolam, amitriptyline, barnidipine, duloxetine, fluoxetine, trazodone, quetiapine and olanzapine. The registered alert management data and medication dispensing histories were analysed to find potential determinants of persistent drug therapy changes. Ten pharmacists were interviewed about the barriers and facilitators for implementing drug therapy changes based on CDSS alerts. An inductive thematic analysis of the transcripts was performed. The pharmacists recorded the management of 1810 of the 2589 generated alerts, and 158 (8.7%) alerts were associated with a persistent drug therapy change. A logistic regression analysis found that the drug triggering the alert and the type of prescription [first dispensing vs. repeat; odds ratio 2.1 (95% confidence interval 1.4-3.2)] were significantly associated with persistent drug therapy changes. No association was found between persistent changes and age, sex, number of medicines in use, or recent clinical medication review. Analysis of the interviews revealed nine barriers and facilitators associated with drug therapy change. When community pharmacists implemented CDSS alerts to reduce inappropriate drug use in older patients, they registered a persistent drug therapy change in 8.7% of the cases. Alerts triggered by a first prescription were two times more likely to be associated with a persistent drug therapy change than alerts triggered by repeat prescriptions. This study found that clinical rules can be used to detect inappropriate drug use in older patients and that drug therapy can change based on the alerts. This suggests that CDSS alerts are a useful tool for implementing guidelines on PIM in older patients in daily practice.

Impact of pharmacists’ interventions on physicians’ decision of a knowledge-based renal dosage adjustment system

Background Early interventions with clinical decision support system (CDSS) guidance have ensured appropriate drug dosing for patients with renal impairment. However, the low rates of physician compliance with CDSS alerts have been reported. Objective We investigated whether designated pharmacist interventions were associated with physician' acceptance of the knowledge-based renal dosage adjustment system (K-RDS) for patients with reduced renal function. Setting A retrospective, single-center study was conducted using a healthcare information system at a tertiary teaching hospital. Methods This study compared physicians' acceptance of the K-RDS with and without designated pharmacists. The severity of prescription errors and the impact of service provided by the pharmacist were evaluated using the validated method developed by Overhage and Lukes. From April to June 2017, we enrolled patients who were ≥ 20years of age and admitted with an estimated glomerular filtration rate under 50ml/min on medications that required dose adjustments. Main outcomes measure The number of dosing alerts of the K-RDS and physicians' acceptance rates were compared between a control group guided by the central pharmacy only and a group with assigned designated pharmacists. The factors associated with the physicians' acceptance rate were also analyzed using a multivariate logistic regression method. The impact of service provided by the pharmacist were considered as 'highly significant' (categories: 1-2). Severity of prescription errors were defined as 'serious' if they corresponded to categories 1-2 of the Overhage and Lukes scale for severity, and interventions were relevant if they corresponded to categories 1-3 in the impact of service provided by the pharmacistscale. Results Among 1363 prescription interventions, 491 (36.0%) were performed by designated pharmacists. The K-RDS alert acceptance rate by the physicians was 54.4% in the designated pharmacist group and 47.0% in the control group (p = 0.0233). The statistically significant association was found in the designated pharmacists group in 'highly significant' service provided by the pharmacist (p < 0.001, OR 1.772; 95% CI 1.362-2.305) and 'serious' severity of prescription errors (p = 0.012, OR 1.657; 95% CI 1.116-2.460). The presence of designated pharmacists (OR 1.353, p = 0.0272), patient's gender (OR 0.758, p = 0.0016), department specialty (OR 0.659, p < 0.0001), eGFR (OR 1.538 if < 10ml/min; OR 1.519 if 10-40ml/min, p < 0.0001), and medications (OR 6.058-43.992 depending on the medication category, p < 0.0001) were significant factors affecting physicians' acceptance. Conclusion Pharmacists' interventions effectively improved physicians' acceptance of the K-RDS alerts.

Real Time Applied Therapeutics in the Critical Care Environment: Role of a Data Mining Clinical Decision Support Tool

SESSION TITLE: Critical Care in the ICU SESSION TYPE: Original Investigation Poster Discussion PRESENTED ON: Monday, October 24, 2016 at 12:00 PM - 01:30 PM PURPOSE: Real time drug therapy optimization, with minimal adversities is essential in the critical care environment. Clinical Decision Support System (CDSS), an electronic clinical surveillance system is a data mining tools which feed real-time ADT (Admission, Discharge, and Transfer) laboratory, pharmacy, radiology, and surgery data. The system processes these data, based on pre-defined rules to detect changes in patients’ conditions in real-time, then flags clinical alerts. Real-time critical alerts can be utilized by physicians and pharmacists in optimizing care and minimizing adverse events. METHODS: In the intensive care units, the clinical pharmacy team acknowledged CDSS alerts, consulted with the medical team or used exiting protocols, to modify drug therapies, based on various clinical parameters. Approval for the study was obtained from the IRB. February 2016 alerts, flagged by CDSS, in the Critical Care Units, were evaluated, based on thirty-six intervention categories (Antimicrobial therapy (13), Adverse Drug Event (10), Anticoagulation Dosing Consults (1), Drug Regimen Modification (9), High Cost (1), Intravenous to Oral Therapy Conversion (1), and Renal Dosing Adjustment (1). RESULTS: In February 2016, CDSS fired 7557 (1717 Critical Care Unit) alerts, with 100% acknowledged, by the clinical pharmacy team, at Corpus Christi Medical Center. These alerts resulted in 15057 associated categorized interventions; 2995 in the critical care units. The outcome of these 2995 documentations in the critical care units were as follow: 1685 (appropriate with no action necessary), 791 (drug therapy modifications occurred), 299 (expired / patient discharges), 104 (non-drug interventions), 88 (associated activations), 14 (rejected recommendations), 11 (follow-ups), 2(CDSS rules requiring adjustments), and 1 (consult). Top 10 intervention categories were as follows: Evidence of infection and antimicrobial therapy warranted (634), Adverse drug event - glycemic control (270), Renal dosage adjustment (250), Intravenous to oral therapy conversion (226), Intravenous to oral antimicrobial therapy conversion (208), Renal dosage adjustment - antimicrobial therapy (145), Drug regimen modification - added therapy (136), Adverse drug event - anticoagulation (120), Drug regimen modification - glycemic control (117), Adverse drug event -rescue medication (102). Numbers of interventions for top 10 drugs were: famotidine (173), enoxaparin (149), pantoprazole (106), azithromycin (91), levofloxacin (85), digoxin (62), piperacillin/tazobactam (48), fluconazole (45), vancomycin (42), and cefazolin (41). CONCLUSIONS: CDSS in the critical care environment is being effectively used to optimize therapy, minimize adverse events, and, reduce costs in real time. CLINICAL IMPLICATIONS: CDSS is an effective and valuable tool in applied therapeutics, within the critical care environment. Its employment in this arena can save lives, prevents medication misadventures, and associated morbidities. DISCLOSURE: The following authors have nothing to disclose: George Udeani, Nana Akuffo, John Evans, Salim Surani, Joseph High No Product/Research Disclosure Information

Do false positive alerts in naïve clinical decision support system lead to false adoption by physicians? A randomized controlled trial

ObjectivesFalse positive alerts in patient-safety-related clinical decision support systems (CDSS) are defined as alerts which incorrectly prompt when no-risk patients are encountered. It is an unfavorable condition which may potentially mislead physicians. The aim is to investigate physician responses toward false positive (FP) and true positive (TP) alerts in CDSS for the prevention of contrast-induced nephropathy (CIN). MethodsA two-arm cluster randomized controlled trial was conducted in university hospitals. Eligible physicians were randomized to receive alert intervention or no intervention (groups 1 and 2, respectively). The alert system was embedded with a deliberately non-specific risk detection tool in order to generate TP and FP alerts. The naïve alert system would alert the physician to cancel the order regardless of the patient being at-risk or not at-risk. CIN risk was stratified as at-risk and no-risk according to a patient's pre-existing renal function. Contrast imaging order-cancellation rate was measured as primary outcome. Results3802 contrast-enhanced examination orders from 66 physicians were analyzed. Demographic data and risk distributions of patients were similar and well-balanced between two groups. In the intervention group, a total of 1892 alerts were generated (332 TP alerts and 1560 FP alerts). Order-cancellation rates were 5.1% versus 1.4% in groups 1 and 2 for at-risk patients (relative risk [RR] = 3.69) from TP alerts, and 1.0% versus 1.4% for no-risk patients (RR = 0.71) from FP alerts. Using generalized linear model with generalized estimating equation, the FP alerts had no order-cancellation effect when compared to the control arm (adjusted RR = 0.69; 95%CI, 0.36–1.32). The TP alerts had a larger order-cancellation effect than that of the control arm (adjusted RR = 2.95; 95%CI, 0.94–9.27), which revealed a marginal trend toward significance. However, the effect was not statistically significant (adjusted RR = 1.24; 95%CI, 0.71–2.18) if TP and FP alerts were mixed. ConclusionsPhysicians are not likely to adopt recommendations provided by false positive alerts in patient-safety-related CDSS. If reporting only the adoption rate of CDSS as a whole without differentiating between TP and FP alerts, the effects of TP and FP alerts will be mixed, and thus, will lead to an underestimation of system effectiveness.

Effect of a clinical decision support system on early action on immunological treatment failure in patients with HIV in Kenya: a cluster randomised controlled trial

A clinical decision support system (CDSS) is a computer program that applies a set of rules to data stored in electronic health records to offer actionable recommendations. We aimed to establish whether a CDSS that supports detection of immunological treatment failure among patients with HIV taking antiretroviral therapy (ART) would improve appropriate and timely action. We did this prospective, cluster randomised controlled trial in adults and children (aged ≥18 months) who were eligible for, and receiving, ART at HIV clinics in Siaya County, western Kenya. Health facilities were randomly assigned (1:1), via block randomisation (block size of two) with a computer-generated random number sequence, to use electronic health records either alone (control) or with CDSS (intervention). Facilities were matched by type and by number of patients enrolled in HIV care. The primary outcome measure was the difference between groups in the proportion of patients who experienced immunological treatment failure and had a documented clinical action. We used generalised linear mixed models with random effects to analyse clustered data. This trial is registered with ClinicalTrials.gov, number NCT01634802. Between Sept 1, 2012, and Jan 31, 2014, 13 clinics, comprising 41,062 patients, were randomly assigned to the control group (n=6) or the intervention group (n=7). Data collection at each site took 12 months. Among patients eligible for ART, 10,358 (99%) of 10,478 patients were receiving ART at control sites and 10,991 (99%) of 11,028 patients were receiving ART at intervention sites. Of these patients, 1125 (11%) in the control group and 1342 (12%) in the intervention group had immunological treatment failure, of whom 332 (30%) and 727 (54%), respectively, received appropriate action. The likelihood of clinicians taking appropriate action on treatment failure was higher with CDSS alerts than with no decision support system (adjusted odds ratio 3·18, 95% CI 1·02-9·87). CDSS significantly improved the likelihood of appropriate and timely action on immunological treatment failure. We expect our findings will be generalisable to virological monitoring of patients with HIV receiving ART once countries implement the 2015 WHO recommendation to scale up viral load monitoring. US President's Emergency Plan for AIDS Relief (PEPFAR), through the US Centers for Disease Control and Prevention.

Health IT and clinical decision support systems: human factors and successful adoption.

This special online issue of JAMIA focuses on Health IT (HIT) and Clinical Decision Support Systems (CDSS). It starts with a presentation from an FDA commissioned working group led by Bates et al ( see page e181 ) on a regulatory framework for HIT intended to be simple, effective, and non-duplicative. A significant open question is related to the tradeoff between costs and benefits in HIT and CDSS implementation. Slight et al ( see page e226 ) examine the costs associated with electronic health record (EHR) system implementation in the UK. Calculation of benefits is more difficult because of the variable adoption rates and the relatively small number of articles proving definitive benefits, not to count potential publication biases. Some authors also worried that clinician adoption of CDSS alerts and reminders might vary according to racial differences in the patient population. This variance could exacerbate disparities in health care, but Mishuris and Linder …

Use of a Clinical Decision Support System Alert to Prevent Supratherapeutic Vancomycin Concentrations.

Background: Alerts issued by clinical decision support systems (CDSS) may be useful to identify and prevent the occurrence of acute kidney injury among patients on nephrotoxic drugs, particularly vancomycin. Objective: The purpose of this instructive study was to determine the effectiveness of using a pharmacist-run CDSS alert of early serum creatinine increases in patients receiving intravenous vancomycin to decrease the proportion of severely elevated vancomycin concentrations. Methods: This was a retrospective study of a prospectively reviewed CDSS alert that triggered in patients with an increase in serum creatinine by 25% from baseline within 24 hours. Severely elevated vancomycin concentrations were divided into a control group (before alert implementation) and a study group (after alert implementation) and considered for study inclusion. The proportion of severely elevated vancomycin concentrations (ie, >30 mg/L) were collected in the control and study groups. Results: There were 1290 and 1501 vancomycin concentrations in the control group and the study group, respectively. A total of 696 CDSS alerts triggered during the study period. The proportion of severely elevated vancomycin troughs decreased from 5.3% (n = 68, median = 36.6 mg/L, interquartile range = 33.75-43.2 mg/L) in the control group to 3.7% (n = 55, median = 34.7 mg/L, interquartile range = 31.3-39.3 mg/L) in the study group. This reflects a statistically significant decrease in the proportion of severely elevated vancomycin concentrations (P = .04). Conclusion: Overall, this instructive analysis on a novel use of CDSS software suggests that the implementation of an alert based on early detection of serum creatinine changes led to a significant decrease in the proportion of severely elevated serum vancomycin concentrations.

Evaluation of Real-Time Clinical Decision Support Systems for Platelet and Cryoprecipitate Orders

To evaluate cryoprecipitate and platelet ordering practices after the implementation of real-time clinical decision support systems (CDSSs) in a computerized physician order entry (CPOE) system. Uniform platelet and cryoprecipitate transfusion thresholds were implemented at 11 hospitals in a regional health care system with a common CPOE system. Over 6 months, a variety of information was collected on the ordering physicians and the number of alerts generated by the CDSSs when these products were ordered outside of the institutional guidelines. There were 1,889 orders for platelets and 152 orders for cryoprecipitate placed in 6 months. Of these, 1,102 (58.3%) platelet and 74 (48.7%) cryoprecipitate orders triggered an alert. The proportion of orders canceled after an alert was generated ranged from 13.5% to 17.9% for platelets and 0% to 50.0% for cryoprecipitate orders. CDSS alerts reduce, but do not eliminate, platelet and cryoprecipitate transfusions that do not meet institutional guidelines.

Physicians′ override of computerized alerts for contraindicated medications in patients hospitalized with chronic kidney disease

Objectives: To determine the effectiveness of a clinical decision support system (CDSS) as indicated by a lower proportion of receiving contraindicated medications by patients with severe chronic kidney disease (CKD) compared with patients with less severe CKD. Materials and Methods: This was a retrospective analysis of inpatients with CKD (ICD9-CM 585.xx) admitted to a major tertiary hospital in Saudi Arabia and receiving one of the medications that were documented in the knowledge base of the hospital CDSS to be renally cleared and/or nephrotoxic. Using the Chi square test, the proportion of receiving contraindicated medication was compared between patients with severe CKD and patients with mild/moderate CKD. Multivariate logistic regression was then used to examine the adjusted risk of receiving contraindicated medications among patients with severe CKD despite the presence of guided medication by CDSS. Results: The final analysis was conducted on 346 patients who received prescriptions that were renally cleared and/or nephrotoxic. Of these patients, 17% (n = 58) had severe CKD and 83% (n = 288) had mild/moderate CKD. Among patients with severe CKD, 51.7% (n = 30) received contraindicated medications compared with patients with mild/moderate CKD, 4.9% (n = 14), P Conclusion: Patients with severe CKD continued to receive contraindicated medications despite the availability of medication guidance by the CDSS to prescribing physicians. Improved compliance by physicians to CDSS alerts and better understanding of reasons for non-compliance is still needed, particularly for patients with severe CKD.

Effects of an integrated clinical information system on medication safety in a multi-hospital setting

The implementation of vendor-based integrated clinical information technology was studied, and its effect on medication errors throughout the medication-use process in a health care system was evaluated. The integrated systems selected for implementation included computerized physician order entry, pharmacy and laboratory information systems, clinical decision-support systems (CDSSs), electronic drug dispensing systems (EDDSs), and a bar-code point-of-care medication administration system. The primary endpoint was the reduction in related medication errors. Secondary endpoints included the reductions in medication order turnaround time and EDDS override transactions. Integrated clinical information system technology was implemented in a multihospital health care system with a phased-in approach. A positive effect of this integration on medication errors throughout the medication-use process was demonstrated. Most prescribing errors decreased significantly in the selected categories monitored, specifically drug allergy detection, excessive dosing, and incomplete or unclear orders. Pharmacists were also twice as likely to identify dosages requiring adjustment for renal insufficiency when the integrated technology was in place and more than six times as likely for drug levels outside of the therapeutic range. A positive effect on medication administration safety was also demonstrated: 73 administration-related errors were intercepted through electronic bar-code scanning for every 100,000 doses charted. Integration of clinical information system technology decreased selected types of medication errors throughout the medication-use process in a health care system and improved therapeutic drug monitoring in patients with renal insufficiency and in patients receiving drugs with narrow therapeutic ranges through the use of CDSS alerts.