Use Of Clinical Decision Support Systems Research Articles

The Effect of Artificial Intelligence on Patient-Physician Trust: Cross-Sectional Vignette Study.

Clinical decision support systems (CDSSs) based on routine care data, using artificial intelligence (AI), are increasingly being developed. Previous studies focused largely on the technical aspects of using AI, but the acceptability of these technologies by patients remains unclear. We aimed to investigate whether patient-physician trust is affected when medical decision-making is supported by a CDSS. We conducted a vignette study among the patient panel (N=860) of the University Medical Center Utrecht, the Netherlands. Patients were randomly assigned into 4 groups-either the intervention or control groups of the high-risk or low-risk cases. In both the high-risk and low-risk case groups, a physician made a treatment decision with (intervention groups) or without (control groups) the support of a CDSS. Using a questionnaire with a 7-point Likert scale, with 1 indicating "strongly disagree" and 7 indicating "strongly agree," we collected data on patient-physician trust in 3 dimensions: competence, integrity, and benevolence. We assessed differences in patient-physician trust between the control and intervention groups per case using Mann-Whitney U tests and potential effect modification by the participant's sex, age, education level, general trust in health care, and general trust in technology using multivariate analyses of (co)variance. In total, 398 patients participated. In the high-risk case, median perceived competence and integrity were lower in the intervention group compared to the control group but not statistically significant (5.8 vs 5.6; P=.16 and 6.3 vs 6.0; P=.06, respectively). However, the effect of a CDSS application on the perceived competence of the physician depended on the participant's sex (P=.03). Although no between-group differences were found in men, in women, the perception of the physician's competence and integrity was significantly lower in the intervention compared to the control group (P=.009 and P=.01, respectively). In the low-risk case, no differences in trust between the groups were found. However, increased trust in technology positively influenced the perceived benevolence and integrity in the low-risk case (P=.009 and P=.04, respectively). We found that, in general, patient-physician trust was high. However, our findings indicate a potentially negative effect of AI applications on the patient-physician relationship, especially among women and in high-risk situations. Trust in technology, in general, might increase the likelihood of embracing the use of CDSSs by treating professionals.

Clinical Needs Assessment of a Machine Learning-Based Asthma Management Tool: User-Centered Design Approach.

Personalized asthma management depends on a clinician's ability to efficiently review patient's data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians' experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. At the "discovery" phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the "define" phase, a synthesis analysis was conducted to converge key results from interviewees' insights into themes, eventually forming critical "how might we" research questions to guide model development and implementation. We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients' high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.

P26 User experience of a clinical decision support system in the form of a mobile app in supporting antimicrobial stewardship

Abstract Background There are advancements in the use of technology in modern healthcare. Healthcare professionals use various digital platforms such as electronic medical records, e- prescriptions, websites and smartphone apps to enable prompt access to clinical information. Healthcare professionals are likely to refer to their smartphones for prompt clinical information or guidance that are easily accessible in contrast to paper guides, handbooks and desktop computers. Some studies have shown that mobile application guidelines within a smart phone app increase adherence to prescribing guidelines by clinicians. Furthermore, benefits of a smart phone app are the ability to update them more quickly than paper-based information and the ability to improve clinical decision-making within the form of an electronic clinical decision support system (CDSS). Despite the increased use of digital information there are limited studies in this area on the use of CDSS within smart phone antimicrobial apps. More evidence is needed to better understand healthcare usage of such apps in the area of antimicrobial stewardship. Microguide V1 (M1) is a smartphone antimicrobial app currently in widespread use in UK hospitals designed to provide access to local prescribing guidelines. However, M1 does not have a support tool to ‘direct’ or map prescribers to a specific treatment plan for their patients based on symptoms or suspected diagnosis. Microguide V2 (M2) is an updated version that includes an additional decision support tool that helps to identify the correct antibiotic to prescribe based on patient’s clinical symptoms. Objectives To evaluate clinician experiences with both versions of the app within Mid and South Essex University Hospitals (MSE) Trust. In MSE, the M1 is currently in use but M2 is being adopted from November 2023 onwards as part of strategies to improve antimicrobial stewardship, and to help with meeting Commissioning for Quality and Innovation (CQUIN) targets for IV to oral switching of antibiotics). Methods A mixed methods approach was devised utilizing an online survey and interviews pre- and post-introduction of M2. Questions were aligned to the COMB-B behaviour model approach influencing change. Topics covered were awareness, experience, barriers, concerns and prescribing behaviour. All prescribers within the Trust working on adult inpatient wards were eligible for participation. Results A pre-implementation survey of M2 was conducted between January and October 2023. A total of 473 people accessed the online survey, with 24 responses, but only a total of 15 participants fully completed the survey. The roles of the participants were 6 pharmacists, 3 nurses and 6 doctors. In terms of awareness of the app, 87% stated yes and 38% agreed that the app increased their awareness of antimicrobial stewardship. Two-thirds (67%) of participants used the app in supporting prescribing decisions, whilst a third had never used it. Nearly 40% of participants agreed that the app encouraged them to challenge inappropriate prescribing and almost half (46%) agreed that that they felt comfortable accessing the app at patient’s bedside. The interview analysis is still ongoing. Conclusions The results so far provide evidence that M1 is being accessed by prescribers in making clinical decisions and they find it useful. The obvious and major limitation of this study was the small number of responses, and this is partially explained by winter pressures and industrial action by nurses and doctors. This led to reduced availability of participants.

Effective decision support systems in clinical practice and prevention: literature review

Clinical decision support systems (CDSS) often outperform human capabilities for processing a large amount of information, dramatically simplifying the work of specialists and avoiding medical errors. The implementation of such systems is a complex task that requires high-tech developments. The annual increase in the development of such systems has a geometric progression. However, it is unclear if most of them will be integrated into clinical practice and recommendations. The use of CDSS to address various disease diagnosis, treatment, and prevention issues is demonstrated, and possible linkages between scientific clinical observations and CDSS are examined. Currently, many data gathering and processing systems use machine learning algorithms and convolutional technologies to create CDSS, resulting in data that exceeds the ability of human thinking to determine the logic of recommended decisions. This study presents the most studied modern CDSS, the possibilities of their application, and the implementation issues.

Expectation of clinical decision support systems: a survey study among nephrologist end-users

BackgroundChronic kidney disease (CKD), a major public health problem with differing disease etiologies, leads to complications, comorbidities, polypharmacy, and mortality. Monitoring disease progression and personalized treatment efforts are crucial for long-term patient outcomes. Physicians need to integrate different data levels, e.g., clinical parameters, biomarkers, and drug information, with medical knowledge. Clinical decision support systems (CDSS) can tackle these issues and improve patient management. Knowledge about the awareness and implementation of CDSS in Germany within the field of nephrology is scarce.PurposeNephrologists’ attitude towards any CDSS and potential CDSS features of interest, like adverse event prediction algorithms, is important for a successful implementation. This survey investigates nephrologists’ experiences with and expectations towards a useful CDSS for daily medical routine in the outpatient setting.MethodsThe 38-item questionnaire survey was conducted either by telephone or as a do-it-yourself online interview amongst nephrologists across all of Germany. Answers were collected and analysed using the Electronic Data Capture System REDCap, as well as Stata SE 15.1, and Excel. The survey consisted of four modules: experiences with CDSS (M1), expectations towards a helpful CDSS (M2), evaluation of adverse event prediction algorithms (M3), and ethical aspects of CDSS (M4). Descriptive statistical analyses of all questions were conducted.ResultsThe study population comprised 54 physicians, with a response rate of about 80–100% per question. Most participants were aged between 51–60 years (45.1%), 64% were male, and most participants had been working in nephrology out-patient clinics for a median of 10.5 years. Overall, CDSS use was poor (81.2%), often due to lack of knowledge about existing CDSS. Most participants (79%) believed CDSS to be helpful in the management of CKD patients with a high willingness to try out a CDSS. Of all adverse event prediction algorithms, prediction of CKD progression (97.8%) and in-silico simulations of disease progression when changing, e. g., lifestyle or medication (97.7%) were rated most important. The spectrum of answers on ethical aspects of CDSS was diverse.ConclusionThis survey provides insights into experience with and expectations of out-patient nephrologists on CDSS. Despite the current lack of knowledge on CDSS, the willingness to integrate CDSS into daily patient care, and the need for adverse event prediction algorithms was high.

Implementation of a clinical decision support system for the optimization of antidiabetic drug orders by pharmacists.

The objective of the study was to describe the impact of a clinical decision support system (CDSS) on antidiabetic drug management by clinical pharmacists for hospitalized patients with T2DM. We performed a retrospective, single-centre study in a teaching hospital, where clinical pharmacists analysed prescriptions and issued pharmacist interventions (PIs) through a computerized physician order entry (CPOE) system. A CDSS was integrated into the pharmacists' workflow in July 2019. We analysed PIs during 2 periods of interest: one before the introduction of the CDSS (from November 2018 to April 2019, PIs issued through the CPOE alone) and one afterwards (from November 2020 to April 2021, PIs issued through the CPOE and/or the CDSS). The study covered nondiabetology wards as endocrinology, diabetes and metabolism departments were not computerized at the time of the study. There were 203 PIs related to antidiabetic drugs in period 1 and 319 in period 2 (a 57.5% increase). Sixty-four of the 319 PIs were generated by the CDSS. Noncompliance/contraindication was the main problem identified by the CDSS (41 PIs, 68.4%), and 57.8% led to discontinuation of the drug. Most of the PIs issued through the CDSS corresponded to orders that had not been flagged up by clinical pharmacists using the CPOE. Conversely, most alerts about indications that were not being treated were detected by the clinical pharmacists using the CPOE and not by the CDSS. Use of CDSS by clinical pharmacists improved antidiabetic drug management for hospitalized patients with T2DM. The CDSS might add value to diabetes care in nondiabetology wards by decreasing the frequency of potentially inappropriate prescriptions and adverse drug reactions.

The Integration of Clinical Decision Support Systems Into Telemedicine for Patients With Multimorbidity in Primary Care Settings: Scoping Review.

Multimorbidity, the presence of more than one condition in a single individual, is a global health issue in primary care. Multimorbid patients tend to have a poor quality of life and suffer from a complicated care process. Clinical decision support systems (CDSSs) and telemedicine are the common information and communication technologies that have been used to reduce the complexity of patient management. However, each element of telemedicine and CDSSs is often examined separately and with great variability. Telemedicine has been used for simple patient education as well as more complex consultations and case management. For CDSSs, there is variability in data inputs, intended users, and outputs. Thus, there are several gaps in knowledge about how to integrate CDSSs into telemedicine and to what extent these integrated technological interventions can help improve patient outcomes for those with multimorbidity. Our aims were to (1) broadly review system designs for CDSSs that have been integrated into each function of telemedicine for multimorbid patients in primary care, (2) summarize the effectiveness of the interventions, and (3) identify gaps in the literature. An online search for literature was conducted up to November 2021 on PubMed, Embase, CINAHL, and Cochrane. Searching from the reference lists was done to find additional potential studies. The eligibility criterion was that the study focused on the use of CDSSs in telemedicine for patients with multimorbidity in primary care. The system design for the CDSS was extracted based on its software and hardware, source of input, input, tasks, output, and users. Each component was grouped by telemedicine functions: telemonitoring, teleconsultation, tele-case management, and tele-education. Seven experimental studies were included in this review: 3 randomized controlled trials (RCTs) and 4 non-RCTs. The interventions were designed to manage patients with diabetes mellitus, hypertension, polypharmacy, and gestational diabetes mellitus. CDSSs can be used for various telemedicine functions: telemonitoring (eg, feedback), teleconsultation (eg, guideline suggestions, advisory material provisions, and responses to simple queries), tele-case management (eg, sharing information across facilities and teams), and tele-education (eg, patient self-management). However, the structure of CDSSs, such as data input, tasks, output, and intended users or decision-makers, varied. With limited studies examining varying clinical outcomes, there was inconsistent evidence of the clinical effectiveness of the interventions. Telemedicine and CDSSs have a role in supporting patients with multimorbidity. CDSSs can likely be integrated into telehealth services to improve the quality and accessibility of care. However, issues surrounding such interventions need to be further explored. These issues include expanding the spectrum of medical conditions examined; examining tasks of CDSSs, particularly for screening and diagnosis of multiple conditions; and exploring the role of the patient as the direct user of the CDSS.

Clinical Decision Support Systems in Hospitalized Older Patients: An Exploratory Analysis in a Real-Life Clinical Setting.

Inappropriate prescribing is associated with negative patient outcomes. In hospitalized patients, the use of Clinical Decision Support Systems (CDSSs) may reduce inappropriate prescribing and thereby improve patient-related outcomes. However, recently published large clinical trials (OPERAM and SENATOR) have shown negative results on the use of CDSSs and patient outcomes and strikingly low acceptance of recommendations. The purpose of the present study was to investigate the use of a CDSS in a real-life clinical setting of hospitalized older patients. As such, we report on the real-life pattern of this in-hospital implemented CDSS, including (i) whether generated alerts were resolved; (ii) whether a recorded action by the pharmacist led to an improved number of resolved alerts; and (iii) the natural course of generated alerts, in particular of those in the non-intervention group; as these data are largely lacking in current studies. Hospitalized patients, aged 60 years and older, admitted to Zuyderland Medical Centre, the Netherlands, in 2018 were included. The evaluation of the CDSS was investigated using a database used for standard care. Alongside demographic and clinical data, we also collected the total numbers of CDSS alerts, the number of alerts 'handled' by the pharmacist, those that resulted in an action by the pharmacist, and finally the outcome of the alerts at day 1 and day 3 after the alert was generated. A total of 3574 unique hospitalized patients, mean age 76.7 (SD 8.3) years and 53% female, were included. From these patients, 8073 alerts were generated, of which 7907 (97.9% of total) were handled by the pharmacist (day 1). In 51.6% of the alerts handled by the pharmacist, an action was initiated, resulting in 36.1% of the alerts resolved after day 1, compared with 27.3% if the pharmacist did not perform an action (p<0.001). On day 3, in 52.6% of the alerts an action by the pharmacist was initiated, resulting in 62.4% resolved alerts, compared with 48.0% when no action was performed (p<0.001). In the category renal function, the percentages differed significantly between an action versus no action of the pharmacist at day 1 and at day 3 (16.6% vs 10.6%, p<0.001 [day 1]; 29.8% vs 19.4%, p<0.001 [day 3]). This study demonstrates the pattern and natural course of clinical alerts of an in-hospital implemented CDSS in a real-life clinical setting of hospitalized older patients. Besides the already known beneficial effect of actions by pharmacists, we have also shown that many alerts become resolved without any specific intervention. As such, our study provides an important insight into the spontaneous course of resolved alerts, since these data are currently lacking in the literature.

Clinical decision support systems for chronic obstructive pulmonary disease (COPD) in hospitals: A systematic review

Objectives To synthesise the literature on clinical decision support (CDS) systems for chronic obstructive pulmonary disease (COPD). We aimed to (1) describe existing COPD CDS systems that have been designed, developed or are being used in practice, (2) describe the impact of COPD CDS systems on outcomes and (3) identify barriers and facilitators to implementation of COPD CDS systems. Methods Five databases were searched to identify relevant studies. All studies in English that described clinician-facing COPD CDS systems designed for, or implemented in, hospitals and hospital-in-the-home settings were included. A qualitative narrative synthesis was undertaken, guided by the RE-AIM framework ( Reach, Effectiveness, Adoption, Implementation and Maintenance). Results Twelve studies reporting the use of CDS in hospital ( n = 7) and hospital-in-the-home ( n = 5) settings were included. Implementation efforts to reach target users were scantly reported, and low-to-medium adoption rates were observed. The reported effectiveness of the CDS systems was mixed. Only one study reported facilitators to the implementation of CDS systems, none reported on barriers to the implementation of CDS systems, and none reported any information on successful strategies to maintain implementation of CDS systems. Conclusion The use of CDS systems in the management of patients with COPD in hospital-related settings is an important emerging field of research. Evidence suggests that the field has largely focused on systems targeted at physicians, often with incomplete descriptions and limited evaluations. Many opportunities to optimise and evaluate the implementation and use of COPD CDS systems in hospital settings remain, including robust evaluation of their impact on patient, clinician and health service outcomes.

Effect of clinical decision support systems on emergency medicine physicians' decision-making: A pilot scenario-based simulation study.

Children with congenital heart disease (CHD) are predisposed to rapid deterioration in the face of common childhood illnesses. When they present to their local emergency departments (ED) with acute illness, rapid and accurate diagnosis and treatment is crucial to recovery and survival. Previous studies have shown that ED physicians are uncomfortable caring for patients with CHD and there is a lack of actionable guidance to aid in their decision making. To support ED physicians' key decision components (sensemaking, anticipation, and managing complexity) when managing CHD patients, a Clinical Decision Support System (CDSS) was previously designed. This pilot study evaluates the effect of this CDSS on ED physicians' decision making compared to usual care without clinical decision support. In a pilot scenario-based simulation study with repeated measures, ED physicians managed mock CHD patients with and without the CDSS. We compared ED physicians' CHD-specific and general decision-making processes (e.g., recognizing sepsis, starting antibiotics, and managing symptoms) with and without the use of CDSS. The frequency of participants' utterances related to each key decision components of sensemaking, anticipation, and managing complexity were coded and statistically analyzed for significance. Across all decision-making components, the CDSS significantly increased ED physicians' frequency of "CHD specific utterances" (Mean = 5.43, 95%CI: 3.7-7.2) compared to the without CDSS condition (Mean = 2.05, 95%CI: 0.3-3.8) whereas there was no significant difference in frequencies of "general utterances" when using CDSS (Mean = 4.62, 95%CI: 3.1-6.1) compared to without CDSS (Mean = 5.14 95%CI: 4.4-5.9). A CDSS that integrates key decision-making components (sensemaking, anticipation, and managing complexity) can trigger and enrich communication between clinicians and enhance the clinical management of CHD patients. For patients with complex and subspecialized diseases such as CHD, a well-designed CDSS can become part of a multifaceted solution that includes knowledge translation, broader communication around interpretation of information, and access to additional expertise to support CHD specific decision-making.

951. Implementation and Rapid Uptake of an Antimicrobial Stewardship Clinical Decision Support System Across 19 UPMC Hospitals

Abstract Background UPMC Antimicrobial stewardship (AS) includes independently functioning AS programs and a central tele antimicrobial stewardship (TASP) (1 ID pharmacist and 1 ID physician) that supports smaller hospitals with few on-site AS resources. We describe implementation of an AS clinical decision support system (CDSS) at 19 UPMC hospitals (12-695 staffed beds). Figure 1 is an example of the CDSS. Figure 1Representative Example of CDSS Patient Overview Page (not real patient data) Methods The CDSS extracts clinical data to provide real-time alerts to AS teams. It also supports asynchronous communications between AS team members, including between a local AS and our TASP. CDSS utilization and intervention data from Oct 21 to Mar 22 are reported. Figure 2 shows hospital locations and the timeline for CDSS implementation. Table 1 shows implementation steps. Figure 2:Implementation Timeline and Geographic Location of HospitalsTable 1:Implementation Steps Results Table 2 summarizes category alerts provided by the CDSS. Category 1 alerts were independently reviewed by local pharmacists. Category 2 alerts were reviewed either by local pharmacists or central TASP team. Figure 3 shows high CDSS usage over time with numerous unique users and frequent logins. Most users were pharmacists. CDSS usage was steady and increased in the final month included here. Communications between users were frequent and increased 46%. On average, 7033 CDSS generated alerts per month (70-798 per hospital) were reviewed from Oct 21 to Mar22. Table 3 groups the alerts into successful, unsuccessful, or nonactionable interventions. 93% of actionable alerts (959 of 1029 per month) produced a successful outcome. Table 2:CDSS Alert CategoriesFigure 3:CDSS Utilization SummaryTable 3:Summary and Alert Statuses and Intervention Outcomes Conclusion After a rapid 5-month deployment across 19 hospitals this CDSS was successfully adopted as evidenced by sustained monthly use, increasing communications, and a high rate of accepted interventions on actionable alerts. A high rate of nonactionable alerts in the first 6 months was a limitation. Although this did not appear to limit CDSS use, sites implementing a CDSS should monitor for nonactionable alerts during early utilization and adjust as needed. CDSS has the potential to improve efficiency and consistency of AS within a health system. After a successful implementation, evaluation of impact on antimicrobial usage and clinical outcomes are important future metrics. Disclosures J Ryan Bariola, MD, Infectious Disease Connect: Salary support|Merck: Grant/Research Support Tina Khadem, PharmD, Infectious Disease Connect: Salary support|Merck: Grant/Research Support Caley Yakemowicz, n/a, Infectious Disease Connect: Employee Courtney Simonick, n/a, Infectious Disease Connect: Stocks/Bonds Erin K. McCreary, PharmD, AbbVie: Advisor/Consultant|Cidara: Advisor/Consultant|Entasis: Advisor/Consultant|Ferring: Advisor/Consultant|Merck: Advisor/Consultant|Shionogi: Advisor/Consultant|Summit: Advisor/Consultant Christina Andrzejewski, n/a, Melinta Therapeutics: employee John W. Mellors, MD, Abound Bio: Multiple|Abound Bio: Ownership Interest|Gilead Sciences: Advisor/Consultant|Infectious Disease Connect: Advisor/Consultant|Infectious Disease Connect: Ownership Interest Rima Abdel-Massih, MD, Infectious Disease Connect: Co founder and Chief Medical Officer|Infectious Disease Connect: Ownership Interest.

Artificial intelligence in healthcare: Threats to the fundamental values of our society

Abstract Summary The article deals with the ethical and legal aspects of the use of artificial intelligence in healthcare. The author analyses key threats to the fundamental values held by the majority of democratic society and the human rights guaranteed by international human rights treaties, such as the European Convention on Human Rights. Emphasis is placed, in particular, on the risk of restricting patients’ autonomy due to an insufficient regulation of the use of clinical decision-support systems, a lack of humanistic care due to excessive use of care robots, a decline in human skills in specific fields of medicine, such as radiology, and the violation of privacy by extensive surveillance of social media. The article provides an overview of the benefits of artificial intelligence for healthcare, but also warns of the downsides, with the aim to encourage a debate that would lead to reasonable legal regulation.

Digitalisation of the drug prescribing process in Swiss hospitals – results of a survey

BackgroundThe state of digitalisation in the healthcare sector in Switzerland is lagging, even as the national electronic health record (EHR) is being gradually implemented. Little is known about the implementation...

The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based Explanations

Explainable artificial intelligence (XAI) has shown benefits in clinical decision support systems (CDSSs); however, it is still unclear to CDSS developers how to select an XAI method to optimize the advice-taking of healthcare practitioners. We performed a user study on healthcare practitioners based on a machine learning-based CDSS for the prediction of gestational diabetes mellitus to explore and compare two XAI methods: explanation by feature contribution and explanation by example. Participants were asked to make estimates for both correctly and incorrectly predicted cases to determine if there were any over-reliance or self-reliance issues. We examined the weight of advice and healthcare practitioners’ preferences. Our results based on statistical tests showed no significant difference between the two XAI methods regarding the advice-taking. The CDSS explained by either method had a substantial impact on the decision-making of healthcare practitioners; however, both methods may lead to over-reliance issues. We identified the inclination towards CDSS use as a key factor in the advice-taking from an explainable CDSS among obstetricians. Additionally, we found that different types of healthcare practitioners had differing preferences for explanations; therefore, we suggest that CDSS developers should select XAI methods according to their target users.

Developing a Clinical Decision Support System for Prediction Postoperative Coronary Artery Bypass Grafting Infection in Diabetic Patients.

Postoperative infection in Coronary Artery Bypass Graft (CABG) is one of the most common complications for diabetic patients, due to an increase in the hospitalization and cost. To address these issues, it is necessary to apply some solutions. The study aimed to the development of a Clinical Decision Support System (CDSS) for predicting the CABG postoperative infection in diabetic patients. This developmental study is conducted on a private hospital in Tehran in 2016. From 1061 CABG surgery medical records, we selected 210 cases randomly. After data gathering, we used statistical tests for selecting related features. Then an Artificial Neural Network (ANN), which was a one-layer perceptron network model and a supervised training algorithm with gradient descent, was constructed using MATLAB software. The software was then developed and tested using the receiver operating characteristic (ROC) diagram and the confusion matrix. Based on the correlation analysis, from 28 variables in the data, 20 variables had a significant relationship with infection after CABG (P<0.05). The results of the confusion matrix showed that the sensitivity of the system was 69%, and the specificity and the accuracy were 97% and 84%, respectively. The Receiver Operating Characteristic (ROC) diagram shows the appropriate performance of the CDSS. The use of CDSS can play an important role in predicting infection after CABG in patients with diabetes. The designed software can be used as a supporting tool for physicians to predict infections caused by CABG in diabetic patients as a susceptible group. However, other factors affecting infection must also be considered for accurate prediction.

Effectiveness of an electronic clinical decision support system in improving the management of childhood illness in primary care in rural Nigeria: an observational study

ObjectivesTo evaluate the impact of ALgorithm for the MANAgement of CHildhood illness (‘ALMANACH’), a digital clinical decision support system (CDSS) based on the Integrated Management of Childhood Illness, on health...

Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence-Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation.

BackgroundDespite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings.ObjectiveWe aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks.MethodsThis mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs.ResultsAll 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings.ConclusionsOur mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS.

Clinical decision support system and hospital readmission reduction: Evidence from U.S. panel data

Using a large-scale panel of U.S. hospitals across health referral regions (HRRs), we empirically examine how a hospital's and its neighboring hospitals' adoption of the Clinical Decision Support System (CDSS) (and their meaningful use of CDSS) affects the hospital's quality of care. We find that CDSS adoption significantly reduces a hospital's readmission rate of heart failure, acute myocardial infarction, and pneumonia patients. We also reveal the regional spillover effects of adoption, where neighboring hospitals' CDSS adoption in the same HRR also reduces a hospital's readmission rates. Such spillover effects become more significant with electronic data interchange, facilitating seamless and errorless communications among hospitals. We also show that the spillover effects may depend on HRRs' different market structures and hospitals' meaningful-use status. Our findings offer theoretical and managerial insights for both healthcare researchers and practitioners.

Impact of a Machine Learning-Based Decision Support System for Urinary Tract Infections: Prospective Observational Study in 36 Primary Care Practices.

BackgroundThere is increasing attention on machine learning (ML)-based clinical decision support systems (CDSS), but their added value and pitfalls are very rarely evaluated in clinical practice. We implemented a CDSS to aid general practitioners (GPs) in treating patients with urinary tract infections (UTIs), which are a significant health burden worldwide.ObjectiveThis study aims to prospectively assess the impact of this CDSS on treatment success and change in antibiotic prescription behavior of the physician. In doing so, we hope to identify drivers and obstacles that positively impact the quality of health care practice with ML.MethodsThe CDSS was developed by Pacmed, Nivel, and Leiden University Medical Center (LUMC). The CDSS presents the expected outcomes of treatments, using interpretable decision trees as ML classifiers. Treatment success was defined as a subsequent period of 28 days during which no new antibiotic treatment for UTI was needed. In this prospective observational study, 36 primary care practices used the software for 4 months. Furthermore, 29 control practices were identified using propensity score-matching. All analyses were performed using electronic health records from the Nivel Primary Care Database. Patients for whom the software was used were identified in the Nivel database by sequential matching using CDSS use data. We compared the proportion of successful treatments before and during the study within the treatment arm. The same analysis was performed for the control practices and the patient subgroup the software was definitely used for. All analyses, including that of physicians’ prescription behavior, were statistically tested using 2-sided z tests with an α level of .05.ResultsIn the treatment practices, 4998 observations were included before and 3422 observations (of 2423 unique patients) were included during the implementation period. In the control practices, 5044 observations were included before and 3360 observations were included during the implementation period. The proportion of successful treatments increased significantly from 75% to 80% in treatment practices (z=5.47, P<.001). No significant difference was detected in control practices (76% before and 76% during the pilot, z=0.02; P=.98). Of the 2423 patients, we identified 734 (30.29%) in the CDSS use database in the Nivel database. For these patients, the proportion of successful treatments during the study was 83%—a statistically significant difference, with 75% of successful treatments before the study in the treatment practices (z=4.95; P<.001).ConclusionsThe introduction of the CDSS as an intervention in the 36 treatment practices was associated with a statistically significant improvement in treatment success. We excluded temporal effects and validated the results with the subgroup analysis in patients for whom we were certain that the software was used. This study shows important strengths and points of attention for the development and implementation of an ML-based CDSS in clinical practice.Trial RegistrationClinicalTrials.gov NCT04408976; https://clinicaltrials.gov/ct2/show/NCT04408976

The Current Landscape and Emerging Applications for Real-World Data in Diagnostics and Clinical Decision Support and its Impact on Regulatory Decision Making.

Real-world data (RWD) and real-world evidence (RWE) are becoming essential tools for informing regulatory decision making in health care and offer an opportunity for all stakeholders in the healthcare ecosystem to evaluate medical products throughout their lifecycle. Although considerable interest has been given to regulatory decisions supported by RWE for treatment authorization, especially in rare diseases, less attention has been given to RWD/RWE related to in vitro diagnostic (IVD) products and clinical decision support systems (CDSS). This review examines current regulatory practices in relation to IVD product development and discusses the use of CDSS in assisting clinicians to retrieve, filter, and analyze patient data in support of complex decisions regarding diagnosis and treatment. The review then explores how utilizing RWD could augment regulatory body understanding of test performance, clinical outcomes, and benefit-risk profiles, and how RWD could be leveraged to augment CDSS and improve safety, quality, and efficiency of healthcare practices. Whereas we present examples of RWD assisting in the regulation of IVDs and CDSS, we also highlight key challenges within the current healthcare system which are impeding the potential of RWE to be fully realized. These challenges include issues such as data availability, reliability, accessibility, harmonization, and interoperability, often for reasons specific to diagnostics. Finally, we review ways that these challenges are actively being addressed and discuss how private-public collaborations and the implementation of standardized language and protocols are working toward producing more robust RWD and RWE to support regulatory decision making.