Design Of Clinical Decision Support System Research Articles

Unlocking Therapeutic Symphonies: Innovations in Clinical Decision Support for Drug-Disease Interactions in Kidney Transplantation

Introduction: Drug-disease interactions (DDSIs) are associated with increasing morbidity, mortality, and healthcare costs. These interactions are preventable if recognized and managed properly. Medication safety is critical in kidney transplant patients due to polypharmacy, co-morbidities, and susceptibility to adverse events. Clinical decision support systems (CDSSs) can play a key role therein. Therefore, this study aims to report on the process of developing an innovative, patient-centered, context-aware CDSS for managing DDSIs in kidney recipients.Material and Methods: Clinically important DDSIs were identified in the medications of patients at a kidney transplant outpatient clinic. Subsequently, rules for their detection and management were extracted based on pharmacology references and clinical expertise. A CDSS was developed and piloted following recommendations on medication CDSS design principles.Results: The knowledge base for this CDSS was developed with clinical context sensitivity. We defined priority levels for alerts, established associated display rules, and determined necessary actions based on the transplantation clinical workflow. The DDSI-CDSS correctly detected 37 DDSIs and displayed nine warnings and 28 cautionary alerts for the medications of 113 study patients (32.7% DDSI rate). The system fired three warnings for diltiazem in bradyarrhythmia, and two for each of the following medications and underlying diseases: aspirin in asthma, erythropoietin alfa in hypertension, and gemfibrozil in gall bladder disease. The potential consequences of the identified DDSIs were GI complications (17%), deterioration of the existing disease/condition (6.1%), and an increased risk of arrhythmias (2.6%), thrombosis (2.6%), and hypertension (1.7%). Complying with system alerts and recommendations would potentially prevent all these DDSIs.Conclusion: This study delineates the process of developing an evidence-based DDSI-CDSS for kidney transplantation, laying the groundwork for future advancements. Our results underscore the clinical significance of these interactions and emphasize the imperative for their accurate and timely detection, particularly in these vulnerable patients.

Evaluating Explanations From AI Algorithms for Clinical Decision-Making: A Social Science-Based Approach.

Explainable Artificial Intelligence (XAI) techniques generate explanations for predictions from AI models. These explanations can be evaluated for (i) faithfulness to the prediction, i.e., its correctness about the reasons for prediction, and (ii) usefulness to the user. While there are metrics to evaluate faithfulness, to our knowledge, there are no automated metrics to evaluate the usefulness of explanations in the clinical context. Our objective is to develop a new metric to evaluate usefulness of AI explanations to clinicians. Usefulness evaluation needs to consider both (a) how humans generally process explanations and (b) clinicians' specific requirements from explanations presented by clinical decision support systems (CDSS). Our new scoring method can evaluate the usefulness of explanations generated by any XAI method that provides importance values for the input features of the prediction model. Our method draws on theories from social science to gauge usefulness, and uses literature-derived biomedical knowledge graphs to quantify support for the explanations from clinical literature. We evaluate our method in a case study on predicting onset of sepsis in intensive care units. Our analysis shows that the scores obtained using our method corroborate with independent evidence from clinical literature and have the required qualities expected from such a metric. Thus, our method can be used to evaluate and select useful explanations from a diverse set of XAI techniques in clinical contexts, making it a fundamental tool for future research in the design of AI-driven CDSS.

Inpatient nurses' preferences and decisions with risk information visualization.

We examined the influence of 4 different risk information formats on inpatient nurses' preferences and decisions with an acute clinical deterioration decision-support system. We conducted a comparative usability evaluation in which participants provided responses to multiple user interface options in a simulated setting. We collected qualitative data using think aloud methods. We collected quantitative data by asking participants which action they would perform after each time point in 3 different patient scenarios. More participants (n = 6) preferred the probability format over relative risk ratios (n = 2), absolute differences (n = 2), and number of persons out of 100 (n = 0). Participants liked average lines, having a trend graph to supplement the risk estimate, and consistent colors between trend graphs and possible actions. Participants did not like too much text information or the presence of confidence intervals. From a decision-making perspective, use of the probability format was associated with greater concordance in actions taken by participants compared to the other 3 risk information formats. By focusing on nurses' preferences and decisions with several risk information display formats and collecting both qualitative and quantitative data, we have provided meaningful insights for the design of clinical decision-support systems containing complex quantitative information. This study adds to our knowledge of presenting risk information to nurses within clinical decision-support systems. We encourage those developing risk-based systems for inpatient nurses to consider expressing risk in a probability format and include a graph (with average line) to display the patient's recent trends.

Complex decision making in an intensive care environment: Perceived practice versus observed reality.

Advancing our understanding of how decisions are made in cognitively, sociallyand technologically complex hospital environments may reveal opportunities to improve healthcare delivery, medical educationand the experience of patients, familiesand clinicians. Explore factors impacting clinician decision making in the Boston Children's Hospital Cardiac Intensive Care Unit. A convergent mixed methods design was used. Quantitative and qualitative data sources consisted of a faculty survey, direct observations of clinical rounds in a specific patient population identified by a clinical decision support system (CDSS)and semistructured interviews (SSIs). Deductive and inductive coding was used for qualitative data. Qualitative data were translated into images using social network analysis which illustrate the frequency and connectivity of the codes in each data set. A total of 25 observations of eightfaculty-led interprofessional teams were performed between 12 Februaryand 31 March 2021. Individual patient characteristics were noted by faculty in SSIs to be the most important factor in their decision making, yet ethnographic observations suggested faculty cognitive traits, team expertiseand value-based decisions were more heavily weighted. The development of expertise was impacted by role modeling. Decisions were perceived to be influenced by the system and environment. Clinician perception of decision making was not congruent with the observed behaviours in a complicated and dynamic system. This study identifies important considerations in clinical curricula as well as the design and implementation of CDSS. Our method of using social network analysis to visualize components of decision making could be adopted to explore other complex environments.

ACO based Clinical Decision Support System for Better Medical Care

In the realm of healthcare, the utilization of clinical decision support systems (CDSSs) has become increasingly prevalent as a means of providing medical professionals with a computer-based tool that grants them access to pertinent data and expertise, thereby aiding in their ability to make informed clinical decisions. The potential applications of a CDSS are numerous, ranging from disease diagnosis and the creation of treatment programs, to patient progress monitoring. A crucial component of a CDSS is its knowledge base, which comprises the data utilized by the system to generate recommendations and provide feedback to healthcare providers. In an effort to enhance the knowledge base of a CDSS for a particular clinical condition, metaheuristic methods such as Ant Colony Optimization (ACO) can be employed to select the most suitable and applicable data. ACO facilitates the identification of the portion of a CDSS's knowledge base that is most likely to result in the optimal clinical decision, from among the vast array of data that it may contain. This study aims to explore the potential benefits of utilizing ACO methods in CDSSs for the betterment of patient care. The paper outlines the design and implementation of an ACO-based CDSS, which can offer tailored treatment plans for patients based on their medical histories and current condition.

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

BackgroundTrauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems. MethodsA user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues. ResultsSeventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems—transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods. ConclusionWe found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

Towards a Generic Description Schema for Clinical Decision Support Systems.

Many clinical decision support systems (CDSS) have shown good performance in the research context, but only a few have been brought into routine care. Analysis of success or failure factors in the design of CDSS may support translation from development to routine care by guiding CDSS design and development along these factors. In this work, we propose a schema to describe CDSS designs in a consistent way. We focus on design criteria with the aim to investigate the observed translation gap in CDSS. Existing description models on different aspects relevant for CDSS are combined to a comprehensive schema that allows description and comparison of CDSS without limitation to the domain or architecture.

Using clinical decision support systems in breast cancer treatment: a critical review

Decision-making in breast cancer treatment is a complex process. There is no certainty of the effects of a treatment regimen in a specific patient and the consequences of erroneous decisions can be serious or even fatal for the patient. Clinical decision support systems (CDSSs) are technology aids that have been progressively introduced in clinical practice to reduce decision errors and improve the quality of clinical decisions.This article presents a critical review of the empirical literature on the use of CDSSs in breast cancer treatment decision-making. Three articles were included and two salient challenges identified in the design and research of CDSSs. First, there is a lack of standardisation in the tumour and patient factors and the health goals used to predict the benefit of different treatment options, as well as in the methods used for the evaluation of these. Second, the design of CDSSs follows a biomedical, disease-focused approach to healthcare, and this reductionist approach also prevails in the related scholarly literature.

A Clinical Decision Support System Design Framework for Nursing Practice

AbstractBackground As nurses increasingly engage in decision-making for patients, a unique opportunity exists to translate research into practice using clinical decision support systems (CDSSs). While research has shown that CDSS has led to improvements in patient outcomes and nursing workflow, the success rate of CDSS implementation in nursing is low. Further, the majority of CDSS for nursing are not designed to support the care of patients with comorbidity.Objectives The aim of the study is to conceptualize an evidence-based CDSS supporting complex patient care for nursing.Methods We conceptualized the CDSS through extracting scientific findings of CDSS design and development. To describe the CDSS, we developed a conceptual framework comprising the key components of the CDSS and the relationships between the components. We instantiated the framework in the context of a hypothetical clinical case.Results We present the conceptualized CDSS with a framework comprising six interrelated components and demonstrate how each component is implemented via a hypothetical clinical case.Conclusion The proposed framework provides a common architecture for CDSS development and bridges CDSS research findings and development. Next research steps include (1) working with clinical nurses to identify their knowledge resources for a particular disease to better articulate the knowledge base needed by a CDSS, (2) develop and deploy a CDSS in practice using the framework, and (3) evaluate the CDSS in the context of nursing care.

Falls and delirium in older inpatients: Work-as-imagined, work-as-done and preferences for clinical decision support systems

BackgroundFalls and delirium are common complications in older inpatients. Implementation of fall and delirium guidelines is complex and may be facilitated by Clinical Decision Support Systems (CDSSs). Our study aimed to understand the differences between guidelines (work-as-imagined) and actual care (work-as-done) and how these can impact the design of a CDSS. MethodsWe used Functional Resonance Analysis Method (FRAM) as structured method to visualize work-as-imagined and work-as-done and to develop an initial CDSS design. As input for work-as-imagined, we used national guidelines. To identify work-as-done and CDSS preferences, we conducted semi-structured interviews in two rounds. We identified differences between guidelines and actual care by creating a FRAM model for work-as-imagined and one for work-as-done. CDSS preferences were coupled to activities in the FRAM models and to the identified barriers and facilitators for following guidelines. ResultsEighteen clinicians participated in 24 individual and small-group interviews. For both falls and delirium, we found substantial differences between work-as-imagined and work-as-done. Several CDSS opportunities to improve fall and delirium care were identified including reminders for screening, an order set to facilitate risk assessment, automatically-generated advice with personalized preventive interventions and support to facilitate medication reviews. The required CDSSs addressed both barriers and facilitators. ConclusionIn our study, work-as-done for falls and delirium differed substantially from work-as-imagined and the preferred CDSSs would address both barriers and facilitators. Furthermore, our study showed that FRAM is a suitable tool to identify differences between actual care and guidelines and to design CDSSs from a user-centred approach.

Clinician involvement in research on machine learning-based predictive clinical decision support for the hospital setting: A scoping review.

The study sought to describe the prevalence and nature of clinical expert involvement in the development, evaluation, and implementation of clinical decision support systems (CDSSs) that utilize machine learning to analyze electronic health record data to assist nurses and physicians in prognostic and treatment decision making (ie, predictive CDSSs) in the hospital. A systematic search of PubMed, CINAHL, and IEEE Xplore and hand-searching of relevant conference proceedings were conducted to identify eligible articles. Empirical studies of predictive CDSSs using electronic health record data for nurses or physicians in the hospital setting published in the last 5 years in peer-reviewed journals or conference proceedings were eligible for synthesis. Data from eligible studies regarding clinician involvement, stage in system design, predictive CDSS intention, and target clinician were charted and summarized. Eighty studies met eligibility criteria. Clinical expert involvement was most prevalent at the beginning and late stages of system design. Most articles (95%) described developing and evaluating machine learning models, 28% of which described involving clinical experts, with nearly half functioning to verify the clinical correctness or relevance of the model (47%). Involvement of clinical experts in predictive CDSS design should be explicitly reported in publications and evaluated for the potential to overcome predictive CDSS adoption challenges. If present, clinical expert involvement is most prevalent when predictive CDSS specifications are made or when system implementations are evaluated. However, clinical experts are less prevalent in developmental stages to verify clinical correctness, select model features, preprocess data, or serve as a gold standard.

XML Data and Knowledge-Encoding Structure for a Web-Based and Mobile Antenatal Clinical Decision Support System: Development Study.

BackgroundDispleasure with the functionality of clinical decision support systems (CDSSs) is considered the primary challenge in CDSS development. A major difficulty in CDSS design is matching the functionality to the desired and actual clinical workflow. Computer-interpretable guidelines (CIGs) are used to formalize medical knowledge in clinical practice guidelines (CPGs) in a computable language. However, existing CIG frameworks require a specific interpreter for each CIG language, hindering the ease of implementation and interoperability.ObjectiveThis paper aims to describe a different approach to the representation of clinical knowledge and data. We intended to change the clinician’s perception of a CDSS with sufficient expressivity of the representation while maintaining a small communication and software footprint for both a web application and a mobile app. This approach was originally intended to create a readable and minimal syntax for a web CDSS and future mobile app for antenatal care guidelines with improved human-computer interaction and enhanced usability by aligning the system behavior with clinical workflow.MethodsWe designed and implemented an architecture design for our CDSS, which uses the model-view-controller (MVC) architecture and a knowledge engine in the MVC architecture based on XML. The knowledge engine design also integrated the requirement of matching clinical care workflow that was desired in the CDSS. For this component of the design task, we used a work ontology analysis of the CPGs for antenatal care in our particular target clinical settings.ResultsIn comparison to other common CIGs used for CDSSs, our XML approach can be used to take advantage of the flexible format of XML to facilitate the electronic sharing of structured data. More importantly, we can take advantage of its flexibility to standardize CIG structure design in a low-level specification language that is ubiquitous, universal, computationally efficient, integrable with web technologies, and human readable.ConclusionsOur knowledge representation framework incorporates fundamental elements of other CIGs used in CDSSs in medicine and proved adequate to encode a number of antenatal health care CPGs and their associated clinical workflows. The framework appears general enough to be used with other CPGs in medicine. XML proved to be a language expressive enough to describe planning problems in a computable form and restrictive and expressive enough to implement in a clinical system. It can also be effective for mobile apps, where intermittent communication requires a small footprint and an autonomous app. This approach can be used to incorporate overlapping capabilities of more specialized CIGs in medicine.

Looking through the eyes of the multidisciplinary team: the design and clinical evaluation of a decision support system for lung cancer care.

BackgroundDecision-making in lung cancer is complex due to a rapidly increasing amount of diagnostic data and treatment options. The need for timely and accurate diagnosis and delivery of care demands high-quality multidisciplinary team (MDT) collaboration and coordination. Clinical decision support systems (CDSSs) can potentially support MDTs in constructing a shared mental model of a patient case. This enables the team to assess the strength and completeness of collected diagnostic data, stratification for the right personalized therapy driven by clinical stage and other treatment-influencing factors, and adapt care management strategies when needed. Current CDSSs often have a suboptimal fit into the decision-making workflow, which hampers their impact in clinical practice.MethodsA CDSS for multidisciplinary decision-making in lung cancer was designed to support the abovementioned goals through presentation of relevant clinical data in line with existing mental model structures of the MDT members. The CDSS was tested in a simulated multidisciplinary tumor board meeting for primary diagnosis and treatment selection, based on de-identified primary lung cancer cases (n=8). Decision course analysis, eye-tracking data and questionnaires were used to assess the impact of the CDSS on constructing shared mental models to improve the decision-making process and outcome.ResultsThe CDSS supported the team in their self-correcting capacity for accurate diagnosis and TNM classification. It enabled cross-validation of diagnostic findings, surfaced discordance between diagnostic tests and facilitated cancer staging according the diagnostic evidence, as well as spotting contra-indications for personalized treatment selection.ConclusionsThis study shows the potential of CDSS on clinical decision making, when these systems are properly designed in line with clinical thinking. The presented setup enables assessment of the impact of CDSS design on clinical decision making and optimization of CDSSs to maximize their effect on decision quality and confidence.

Clinical Decision Support Systems for Opioid Prescribing for Chronic Non-Cancer Pain in Primary Care: A Scoping Review.

Clinical decision support systems (CDSSs) may help clinicians prescribe opioids for chronic noncancer pain (CNCP) more appropriately. This scoping review determined the extent and range of the current evidence on CDSSs for opioid prescribing for CNCP in primary care, and whether investigators followed best evidence and current guidance in designing, implementing and evaluating these complex interventions. We searched 9 electronic databases and other data sources for studies from January 1, 2008 to October 11, 2019. Two reviewers independently screened the citations. One reviewer extracted data and a second verified for accuracy. study of a CDSS for opioid prescribing for CNCP in a primary care clinical setting. We reported quantitative results in tables and qualitative results in narrative form. Our search yielded 5068 records, of which 14 studies met our inclusion criteria. All studies were conducted in the United States. Six studies examined local (eg, health center) CDSSs and 8 examined prescription drug monitoring program CDSSs. Three CDSSs incorporated evidence-based components. Study aims were heterogeneous and study designs included both quantitative and qualitative methodologies. No studies assessed patient health outcomes. Few studies appeared to be following guidance for evaluating complex interventions. Few studies have rigorously assessed the use of CDSSs for opioid prescribing for CNCP in primary care settings. Going forward, investigators should include evidence-based components into the design of CDSSs and follow guidance for the development and evaluation of complex interventions.

A Systematic Review on CDSS Alert Appropriateness.

Clinical decision support systems (CDSSs) provides vital information for managing patients by advising clinicians through an alert or reminders about adverse events and medication errors. Clinicians receive a high number of alerts, resulting in alert override and workflow disruptions. A systematic review was carried out to identify factors affecting CDSS alert appropriateness in supporting clinical workflows using a recently introduced framework. The review findings identified several influencing factors of CDSS alert appropriateness including: technology (usability, alert presentation, workload and data entry), human (training, knowledge and skills, attitude and behavior), organization (rules and regulation, privacy and security) and process (waste, delay, tuning and optimization). The findings can be used to guide the design of CDSS alert and minimise potential safety hazards associated with CDSS use.

Leveraging IoTs and Machine Learning for Patient Diagnosis and Ventilation Management in the Intensive Care Unit.

Future healthcare systems will rely heavily on clinical decision support systems (CDSS) to improve the decision-making processes of clinicians. To explore the design of future CDSS, we developed a research-focused CDSS for the management of patients in the intensive care unit that leverages Internet of Things (IoT) devices capable of collecting streaming physiologic data from ventilators and other medical devices. We then created machine learning (ML) models that could analyze the collected physiologic data to determine if the ventilator was delivering potentially harmful therapy and if a deadly respiratory condition, acute respiratory distress syndrome (ARDS), was present. We also present work to aggregate these models into a mobile application that can provide responsive, real-time alerts of changes in ventilation to providers. As illustrated in the recent COVID-19 pandemic, being able to accurately predict ARDS in newly infected patients can assist in prioritizing care. We show that CDSS may be used to analyze physiologic data for clinical event recognition and automated diagnosis, and we also highlight future research avenues for hospital CDSS.

Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow.

The overwhelming number of medication alerts generated by clinical decision support systems (CDSS) has led to inappropriate alert overrides, which may lead to unintended patient harm. This review highlights the factors affecting the alert appropriateness of CDSS and barriers to the fit of CDSS alert with clinical workflow. A literature review was conducted to identify features and functions pertinent to CDSS alert appropriateness using the five rights of CDSS. Moreover, a process improvement method, namely, Lean, was used as a tool to optimise clinical workflows, and the appropriate design for CDSS alert using a human automation interaction (HAI) model was recommended. Evaluating the appropriateness of CDSS alert and its impact on workflow provided insights into how alerts can be designed and triggered effectively to support clinical workflow. The application of Lean methods and tools to analyse alert efficiencies in supporting workflow in this study provides an in-depth understanding of alert-workflow fit problems and their root cause, which is required for improving CDSS design. The application of the HAI model is recommended in the design of CDSS alerts to support various levels and stages of alert automations, namely, information acquisition and analysis, decision action and action implementation.

Meta-design knowledge for Clinical Decision Support Systems

Knowledge gained from a Decision Support Systems (DSS) design should ideally be reusable by DSS designers and researchers. The majority of existing DSS research has mainly focused on empirical problem solving rather than on developing principles that could inform solution approaches for other user contexts. Design Science Research (DSR) has contributed to effective development of various innovative DSS artefacts and associated knowledge development, but there has been limited progress on new knowledge development from a practical problem context, going beyond product and process descriptions. For DSS applications such as Clinical Decision Support Systems (CDSS) design and development, relevant reusable prescriptive knowledge is of significance not only to understand mutability but also to extend application of theory across domains. In this paper, we develop new design knowledge abstracted from the approach taken in a representative case of innovative CDSS development, specified as an architecture and six design principles. The CDSS design artefact was initially designed for a specific clinical need is shown to be flexible for meeting demands of knowledge production both for diagnosis and treatment. It is argued that the proposed general strategy is applicable to designing CDSS artefacts in similar problem domains representing an important contribution of design knowledge both in DSS and DSR fields.

Clinical Decision-Support Systems for Detection of Systemic Inflammatory Response Syndrome, Sepsis, and Septic Shock in Critically Ill Patients: A Systematic Review.

The design of computerized systems able to support automated detection of threatening conditions in critically ill patients such as systemic inflammatory response syndrome (SIRS) and sepsis has been fostered recently. The increase of research work in this area is due to both the growing digitalization in health care and the increased appreciation of the importance of early sepsis detection and intervention. To be able to understand the variety of systems and their characteristics as well as performances, a systematic literature review is required. Existing reviews on this topic follow a rather restrictive searching methodology or they are outdated. As much progress has been made during the last 5 years, an updated review is needed to be able to keep track of current developments in this area of research. To provide an overview about current approaches for the design of clinical decision-support systems (CDSS) in the context of SIRS, sepsis, and septic shock, and to categorize and compare existing approaches. A systematic literature review was performed in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement. Searches for eligible articles were conducted on five electronic bibliographic databases, including PubMed/MEDLINE, IEEE Xplore, Embase, Scopus, and ScienceDirect. Initial results were screened independently by two reviewers based on clearly defined eligibility criteria. A backward as well as an updated search enriched the initial results. Data were extracted from included articles and presented in a standardized way. Articles were classified into predefined categories according to characteristics extracted previously. The classification was performed according to the following categories: clinical setting including patient population and mono- or multicentric study, support type of the system such as prediction or detection, systems characteristics such as knowledge- or data-driven algorithms used, evaluation of methodology, and results including ground truth definition, sensitivity, and specificity. All results were assessed qualitatively by two reviewers. The search resulted in 2,373 articles out of which 55 results were identified as eligible. Over 80% of the articles describe monocentric studies. More than 50% include adult patients, and only four articles explicitly report the inclusion of pediatric patients. Patient recruitment often is very selective, which can be observed from highly varying inclusion and exclusion criteria. The task of disease detection is covered in 62% of the articles; prediction of upcoming conditions in 33%. Sepsis is covered in 67% of the articles, SIRS as sole entity in only 4%, whereas 27% focus on severe sepsis and/or septic shock. The most common combinations of categories "algorithm used" and "support type" are knowledge-based detection of sepsis and data-driven prediction of sepsis. In evaluations, manual chart review (38%) and diagnosis coding (29%) represent the most frequently used ground truth definitions; most studies present a sample size between 10,001 and 100,000 cases (31%) and performances highly differ with only five articles presenting sensitivities and specificities above 90%; four of them using knowledge-based rather than machine learning algorithms. The presentations of holistic CDSS approaches, including technical implementation details, system interfaces, and data and interoperability aspects enabling the use of CDSS in routine settings are missing in nearly all articles. The review demonstrated the high variety of research in this context successfully. A clear trend is observable toward the use of data-driven algorithms, and a lack of research could be identified in covering the pediatric population as well as acknowledging SIRS as an independent and threatening condition. The quality as well as the significance of the presented evaluations for assessing the performances of the algorithms in clinical routine settings are often not meeting the current standard of scientific work. Our future interest will be concentrated on these realistic settings by implementing and evaluating SIRS detection approaches as well as considering factors to make the CDSS useable in clinical routine from both technical and medical perspectives.

Continuous Improvement of Clinical Decision Support via an Embedded Survey Tool.

Clinical decision support systems (CDSS) are widely used to improve patient care and guide workflow. End users can be valuable contributors to monitoring for CDSS malfunctions. However, they often have little means of providing direct feedback on the design and build of such systems. In this study, we describe an electronic survey tool deployed from within the electronic health record and coupled with a conversation with Clinical Informaticians as a method to manage CDSS design and lifecycle.