Quality Of Decision Support Research Articles

Interactive Versus Static Decision Support Tools for COVID-19: Randomized Controlled Trial

BackgroundDuring the COVID-19 pandemic, medical laypersons with symptoms indicative of a COVID-19 infection commonly sought guidance on whether and where to find medical care. Numerous web-based decision support tools (DSTs) have been developed, both by public and commercial stakeholders, to assist their decision making. Though most of the DSTs’ underlying algorithms are similar and simple decision trees, their mode of presentation differs: some DSTs present a static flowchart, while others are designed as a conversational agent, guiding the user through the decision tree’s nodes step-by-step in an interactive manner.ObjectiveThis study aims to investigate whether interactive DSTs provide greater decision support than noninteractive (ie, static) flowcharts.MethodsWe developed mock interfaces for 2 DSTs (1 static, 1 interactive), mimicking patient-facing, freely available DSTs for COVID-19-related self-assessment. Their underlying algorithm was identical and based on the Centers for Disease Control and Prevention’s guidelines. We recruited adult US residents online in November 2020. Participants appraised the appropriate social and care-seeking behavior for 7 fictitious descriptions of patients (case vignettes). Participants in the experimental groups received either the static or the interactive mock DST as support, while the control group appraised the case vignettes unsupported. We determined participants’ accuracy, decision certainty (after deciding), and mental effort to measure the quality of decision support. Participants’ ratings of the DSTs’ usefulness, ease of use, trust, and future intention to use the tools served as measures to analyze differences in participants’ perception of the tools. We used ANOVAs and t tests to assess statistical significance.ResultsOur survey yielded 196 responses. The mean number of correct assessments was higher in the intervention groups (interactive DST group: mean 11.71, SD 2.37; static DST group: mean 11.45, SD 2.48) than in the control group (mean 10.17, SD 2.00). Decisional certainty was significantly higher in the experimental groups (interactive DST group: mean 80.7%, SD 14.1%; static DST group: mean 80.5%, SD 15.8%) compared to the control group (mean 65.8%, SD 20.8%). The differences in these measures proved statistically significant in t tests comparing each intervention group with the control group (P<.001 for all 4 t tests). ANOVA detected no significant differences regarding mental effort between the 3 study groups. Differences between the 2 intervention groups were of small effect sizes and nonsignificant for all 3 measures of the quality of decision support and most measures of participants’ perception of the DSTs.ConclusionsWhen the decision space is limited, as is the case in common COVID-19 self-assessment DSTs, static flowcharts might prove as beneficial in enhancing decision quality as interactive tools. Given that static flowcharts reveal the underlying decision algorithm more transparently and require less effort to develop, they might prove more efficient in providing guidance to the public. Further research should validate our findings on different use cases, elaborate on the trade-off between transparency and convenience in DSTs, and investigate whether subgroups of users benefit more with 1 type of user interface than the other.Trial RegistrationDeutsches Register Klinischer Studien DRKS00028136; https://tinyurl.com/4bcfausx (retrospectively registered)

Quality of Decision Support in Computerized Provider Order Entry: Systematic Literature Review.

BackgroundComputerized decision support systems have raised a lot of hopes and expectations in the field of order entry. Although there are numerous studies reporting positive impacts, concerns are increasingly high about alert fatigue and effective impacts of these systems. One of the root causes of fatigue alert reported is the low clinical relevance of these alerts.ObjectiveThe objective of this systematic review was to assess the reported positive predictive value (PPV), as a proxy to clinical relevance, of decision support systems in computerized provider order entry (CPOE).MethodsA systematic search of the scientific literature published between February 2009 and March 2015 on CPOE, clinical decision support systems, and the predictive value associated with alert fatigue was conducted using PubMed database. Inclusion criteria were as follows: English language, full text available (free or pay for access), assessed medication, direct or indirect level of predictive value, sensitivity, or specificity. When possible with the information provided, PPV was calculated or evaluated.ResultsAdditive queries on PubMed retrieved 928 candidate papers. Of these, 376 were eligible based on abstract. Finally, 26 studies qualified for a full-text review, and 17 provided enough information for the study objectives. An additional 4 papers were added from the references of the reviewed papers. The results demonstrate massive variations in PPVs ranging from 8% to 83% according to the object of the decision support, with most results between 20% and 40%. The best results were observed when patients’ characteristics, such as comorbidity or laboratory test results, were taken into account. There was also an important variation in sensitivity, ranging from 38% to 91%.ConclusionsThere is increasing reporting of alerts override in CPOE decision support. Several causes are discussed in the literature, the most important one being the clinical relevance of alerts. In this paper, we tried to assess formally the clinical relevance of alerts, using a near-strong proxy, which is the PPV of alerts, or any way to express it such as the rate of true and false positive alerts. In doing this literature review, three inferences were drawn. First, very few papers report direct or enough indirect elements that support the use or the computation of PPV, which is a gold standard for all diagnostic tools in medicine and should be systematically reported for decision support. Second, the PPV varies a lot according to the typology of decision support, so that overall rates are not useful, but must be reported by the type of alert. Finally, in general, the PPVs are below or near 50%, which can be considered as very low.

Quality of online information on type 2 diabetes: a cross-sectional study

Evidence-based health information is a prerequisite for patients with type 2 diabetes to engage in self-management and to make informed medical decisions. The Internet is an important source of health information. In the present study, we systematically assessed formal quality, quality of decision support and usability of German and English language websites on type 2 diabetes. The search term 'type 2 diabetes' was entered in the two most popular search engines. Descriptive data on website quality are presented. Additionally, associations between website quality and affiliation (commercial vs. non-commercial), presence of the HON code quality seal and website traffic were explored. Forty-six websites were included. Most websites provided basic information necessary for decision-making, while only one website also provided decision support. Websites with a HON code had significantly better formal quality than websites without HON code. We found a highly significant correlation between usability and website traffic and a significant correlation between formal quality and website traffic. Most websites do not provide sufficient information to support patients in medical decision-making. Our finding that usability and website traffic are tightly associated is consistent with previous research indicating that design is the most important cue for users assessing website credibility.

Understanding Organisational Decision Support Maturity

Forty years after Gorry and Scott Morton’s seminal paper on DSS, supporting decisions in organisation is still a critical objective. Given the elapsed time since DSSs were first introduced, it is important to gauge the scope and quality of decision support provided to managers. Using Executive MBA students as informants about decision making in their organisations, the authors carried out 10 case studies of Irish organisations to assess their maturity in terms of decision support usage. The findings indicate that, in the vast majority of firms, decision support is still not available to help manage in situations involving high levels of abstraction. As was the case at the beginning of the history of DSS, the operational level is still where DSSs are used most consistently across firms. Furthermore, this study illustrates that engaging with managers on the topic of decision making is difficult, given the possibility of bias and misrepresentation inherent in the reality of decision making.