Abstract
Alerting systems, a type of clinical decision support, are increasingly prevalent in healthcare, yet few studies have concurrently measured the appropriateness of alerts with provider responses to alerts. Recent reports of suboptimal alert system design and implementation highlight the need for better evaluation to inform future designs. The authors present a comprehensive framework for evaluating the clinical appropriateness of synchronous, interruptive medication safety alerts. Through literature review and iterative testing, metrics were developed that describe successes, justifiable overrides, provider non-adherence, and unintended adverse consequences of clinical decision support alerts. The framework was validated by applying it to a medication alerting system for patients with acute kidney injury (AKI). Through expert review, the framework assesses each alert episode for appropriateness of the alert display and the necessity and urgency of a clinical response. Primary outcomes of the framework include the false positive alert rate, alert override rate, provider non-adherence rate, and rate of provider response appropriateness. Application of the framework to evaluate an existing AKI medication alerting system provided a more complete understanding of the process outcomes measured in the AKI medication alerting system. The authors confirmed that previous alerts and provider responses were most often appropriate. The new evaluation model offers a potentially effective method for assessing the clinical appropriateness of synchronous interruptive medication alerts prior to evaluating patient outcomes in a comparative trial. More work can determine the generalizability of the framework for use in other settings and other alert types.
Highlights
Many electronic medical record (EMR) systems and computerized provider order entry (CPOE) systems incorporate clinical decision support (CDS) features that attempt to improve patient safety
Previous studies have examined the usability, rates of responses and overrides, and patient outcomes associated with CDS alerts.2e19 few studies have concurrently examined the appropriateness of CDS alert generation and the corresponding appropriateness of providers’ clinical responses to each alert
The framework complements comparative evaluations of CDS that might determine whether the rate of process and patient outcomes are attributable to the alert
Summary
Many electronic medical record (EMR) systems and computerized provider order entry (CPOE) systems incorporate clinical decision support (CDS) features that attempt to improve patient safety. Overrides occurred after 49e96% of displayed CDS alerts.[2] While several projects have attempted to reduce alert override rates, their methods lacked a generalizable approach to evaluating key trade-offs between avoiding false positive alerts, where clinician time and attention is diverted, and avoiding false negative alerts, which silently leave patients at risk.28e31 Studies examining alert override rates have used chart review or user feedback to conclude that many alert overrides are clinically justifiable Such justifications include the clinical insignificance of an overridden alert, known patient tolerance for a drug or drug dose, and documented clinician intention to monitor the patient.2e12 studies have often not examined the extent to which override rates reflect justifiable non-adherence, such as nonresponse to clinically irrelevant alerts. Evaluations often lacked third-party clinical expert adjudications that could determine whether provider actions subsequent to an alert had potentially beneficial or potentially harmful consequences for the patient
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Journal of the American Medical Informatics Association
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
