Automated Decision Aids Research Articles

Evaluation of an Automated Decision Aid for the Further Referral of Emergency Room Patients—A Prospective Cohort Study.

Instruments for the initial evaluation of emergency room patients as an aid to their onward referral to ambulatory care structures are a matter of current interest. In this study, we assess the safety of the software application SmED-Kontakt+ for this purpose. SmED is an abbreviation for Strukuturierte Medizinische Ersteinschätzung Deutschland, "structured initial medical evaluation in Germany." In this prospective cohort study, we compared the recommendations of SmED-Kontakt+ concerning the time and place of further care with those of the treating physicians. The subjects were adult patients who were able to walk and had presented themselves to the emergency room. Whenever SmED-Kontakt+ assessed the situation less critically than the physicians, and in 5% of the remaining cases, the potential endangerment of patient safety was assessed by an expert panel (expected value <1%). In a total of 1840 cases, SmED-Kontakt+ agreed with the physicians' assessment in 353 (19%). The assessment of SmED-Kontakt+ was more critical in 1221 cases (66%) and less critical in 266 (15%; potential undertriage). There was potential endangerment in 49 cases (2.7%; 95% confidence interval [2.0; 3.5]). Potential endangerment was less common among patients in the more urgent recommendation categories [0.1; 2.6] and more common in the categories of a recommended outpatient physician visit or telephone consultation within or beyond 24 hours [2.4; 17.2]. SmED-Kontakt+ generally assessed these patients' needs for further care as more urgent than the treating physicians did. Nevertheless, the percentage of potentially endangered patients was higher than expected. We conclude that further care should be provided in timely fashion with an obligatory appointment. The sites of further ambulatory care should be immediately accessible and properly equipped. Patients should not be referred for further care at later times on the basis of an SmED-Kontakt+ assessment.

Evaluating the Influence of Incorrect Reassurances on Trust in Imperfect Automated Decision Aid

Prior research on trust in automation has primarily focused on instances where decision aids form predictions based on the same raw input information as humans. This study extends the paradigms into the human-processed input framework, where machine predictions are based on human-processed data, instead of the raw input. The human-processed input framework introduces a unique error pattern, “incorrect reassurance,” where faulty automation prediction erroneously validates users’ initial errors. This study examined the impact of “incorrect reassurance” on human trust in automation in the human-processed input framework. Thirty-five participants completed a Mental Rotation Task (MRT), using an imperfect automated decision aid. Results show that incorrect reassurance patterns led to larger trust decrement, lower final performance, and quicker final reaction time compared to other performance patterns. The findings emphasize the importance of understanding trust in automation in the context of the human-processed input framework.

Using anthropomorphism, transparency, and feedback to improve phishing detection aids

Phishing is a common cybersecurity threat to email users. An automated phishing decision-support aid can help users identify suspicious emails. The aid’s success depends on both the aid’s capability and the user’s trust in and usage of the aid. In this study, 465 participants were asked to judge phishing emails with an automated decision aid. We measured how users’ trust and decision making were affected by the type of aid (human, AI, text, no aid), gender (male, female), decision transparency (reasoning for aid’s decision present, absent), as well as feedback (present, absent). We found that an aid was helpful regardless of its anthropomorphic appearance (human vs. AI) and gender. Transparency was helpful with the human aid, but not the AI aid. Feedback effectively improved trust in all the aids, although it helped aid retention only for the text aid. Participants had overall positive comments about the aid and found it helpful. The findings can be applied to automated aid design to understand potential avenues for improving users’ trust, performance, and aid use.

Automated decision aids: When are they advisors and when do they take control of human decision making?

We applied a computational model to examine the extent to which participants used an automated decision aid as an advisor, as compared to a more autonomous trigger of responding, at varying levels of decision aid reliability. In an air traffic control conflict detection task, we found higher accuracy when the decision aid was correct, and more errors when the decision aid was incorrect, as compared to a manual condition (no decision aid). Responses that were correct despite incorrect automated advice were slower than matched manual responses. Decision aids set at lower reliability (75%) had smaller effects on choices and response times, and were subjectively trusted less, than decision aids set at higher reliability (95%). We fitted an evidence accumulation model to choices and response times to measure how information processing was affected by decision aid inputs. Participants primarily treated low-reliability decision aids as an advisor rather than directly accumulating evidence based on its advice. Participants directly accumulated evidence based upon the advice of high-reliability decision aids, consistent with granting decision aids more autonomous influence over decisions. Individual differences in the level of direct accumulation correlated with subjective trust, suggesting a cognitive mechanism by which trust impacts human decisions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

How Do People Process Information from Automated Decision Aids: an Application of Systems Factorial Technology

How Do People Process Information from Automated Decision Aids: an Application of Systems Factorial Technology

Can increased automation transparency mitigate the effects of time pressure on automation use?

A factor that can potentially negatively impact the accuracy of automated decision aid use, and increase perceived workload, is time pressure. Increased automation transparency can increase the accuracy of automation use. We examined the extent to which increased transparency can mitigate the negative effects of time pressure on the accuracy of automation use and perceived workload. Participants completed an uninhabited vehicle (UV) management task where they assigned the best UV to complete missions by either accepting or rejecting automated advice. Participants made a decision after either 25s (low time pressure) or 12s (high time pressure). The accuracy of automation use decreased, and perceived workload increased, when under higher time pressure. Higher transparency benefited the accuracy of automation use and increased perceived trust and usability. However, high transparency did not mitigate the negative impacts of high time pressure, indicating that increased time pressure can influence the processing of highly transparent information.

Transparent Automated Advice to Mitigate the Impact of Variation in Automation Reliability.

To examine the extent to which increased automation transparency can mitigate the potential negative effects of low and high automation reliability on disuse and misuse of automated advice, and perceived trust in automation. Automated decision aids that vary in the reliability of their advice are increasingly used in workplaces. Low-reliability automation can increase disuse of automated advice, while high-reliability automation can increase misuse. These effects could be reduced if the rationale underlying automated advice is made more transparent. Participants selected the optimal UV to complete missions. The Recommender (automated decision aid) assisted participants by providing advice; however, it was not always reliable. Participants determined whether the Recommender provided accurate information and whether to accept or reject advice. The level of automation transparency (medium, high) and reliability (low: 65%, high: 90%) were manipulated between-subjects. With high- compared to low-reliability automation, participants made more accurate (correctly accepted advice and identified whether information was accurate/inaccurate) and faster decisions, and reported increased trust in automation. Increased transparency led to more accurate and faster decisions, lower subjective workload, and higher usability ratings. It also eliminated the increased automation disuse associated with low-reliability automation. However, transparency did not mitigate the misuse associated with high-reliability automation. Transparency protected against low-reliability automation disuse, but not against the increased misuse potentially associated with the reduced monitoring and verification of high-reliability automation. These outcomes can inform the design of transparent automation to improve human-automation teaming under conditions of varied automation reliability.

Simple manipulations of anthropomorphism fail to induce perceptions of humanness or improve trust in an automated agent

Although automation is employed as an aid to human performance, operators often interact with automated decision aids inefficiently. The current study investigated whether anthropomorphic automation would engender higher trust and use, subsequently improving human-automation team performance. Participants performed a multi-element probabilistic signal detection task in which they diagnosed a hypothetical nuclear reactor as in a state of safety or danger. The task was completed unassisted and assisted by a 93%-reliable agent varying in anthropomorphism. Results gave no evidence that participants’ perceptions of anthropomorphism differed between conditions. Further, anthropomorphic automation failed to bolster trust and automation-aided performance. Findings suggest that the benefits of anthropomorphism may be limited in some contexts.

Pathologist pupil dilation reflects experience level and difficulty in diagnosing medical images.

Purpose: Digital whole slide imaging allows pathologists to view slides on a computer screen instead of under a microscope. Digital viewing allows for real-time monitoring of pathologists' search behavior and neurophysiological responses during the diagnostic process. One particular neurophysiological measure, pupil diameter, could provide a basis for evaluating clinical competence during training or developing tools that support the diagnostic process. Prior research shows that pupil diameter is sensitive to cognitive load and arousal, and it switches between exploration and exploitation of a visual image. Different categories of lesions in pathology pose different levels of challenge, as indicated by diagnostic disagreement among pathologists. If pupil diameter is sensitive to the perceived difficulty in diagnosing biopsies, eye-tracking could potentially be used to identify biopsies that may benefit from a second opinion. Approach: We measured case onset baseline-corrected (phasic) and uncorrected (tonic) pupil diameter in 90 pathologists who each viewed and diagnosed 14 digital breast biopsy cases that cover the diagnostic spectrum from benign to invasive breast cancer. Pupil data were extracted from the beginning of viewing and interpreting of each individual case. After removing 122 trials ( ) with poor eye-tracking quality, 1138 trials remained. We used multiple linear regression with robust standard error estimates to account for dependent observations within pathologists. Results: We found a positive association between the magnitude of phasic dilation and subject-centered difficulty ratings and between the magnitude of tonic dilation and untransformed difficulty ratings. When controlling for case diagnostic category, only the tonic-difficulty relationship persisted. Conclusions: Results suggest that tonic pupil dilation may indicate overall arousal differences between pathologists as they interpret biopsy cases and could signal a need for additional training, experience, or automated decision aids. Phasic dilation is sensitive to characteristics of biopsies that tend to elicit higher difficulty ratings and could indicate a need for a second opinion.

Do concurrent task demands impact the benefit of automation transparency?

Automated decision aids typically improve decision-making, but incorrect advice risks automation misuse or disuse. We examined the novel question of whether increased automation transparency improves the accuracy of automation use under conditions with/without concurrent (non-automated assisted) task demands. Participants completed an uninhabited vehicle (UV) management task whereby they assigned the best UV to complete missions. Automation advised the best UV but was not always correct. Concurrent non-automated task demands decreased the accuracy of automation use, and increased decision time and perceived workload. With no concurrent task demands, increased transparency which provided more information on how the automation made decisions, improved the accuracy of automation use. With concurrent task demands, increased transparency led to higher trust ratings, faster decisions, and a bias towards agreeing with automation. These outcomes indicate increased reliance on highly transparent automation under conditions with concurrent task demands and have potential implications for human-automation teaming design.

Supplemental Material for Automated Decision Aids: When Are They Advisors and When Do They Take Control of Human Decision Making?

Supplemental Material for Automated Decision Aids: When Are They Advisors and When Do They Take Control of Human Decision Making?

When Transparency Fails: Compliance with Decision Support in Nautical Collision Avoidance

To attempt to counter uncalibrated compliance with an automated decision aid, the current study sought to introduce some level of transparency into the system’s recommendations. An experiment was conducted using a simple, simulated maritime collision avoidance task featuring an imperfect, but highly reliable, de-cision aid. Specifically, the aid’s recommendations were sometimes paired with a single confidence meas-ure associated with the actual level of difficulty of the trial. Compliance with the aid produced safer out-comes in terms of miss distance at closest point of approach. However, transparency in the form of confi-dence measures did not increase overall compliance with the aid, nor affect the safety of maneuvers. Im-plications for transparency and compliance are discussed.

A personalized decision aid for prostate cancer shared decision making

BackgroundA shared decision-making model is preferred for engaging prostate cancer patients in treatment decisions. However, the process of assessing an individual’s preferences and values is challenging and not formalized. The purpose of this study is to develop an automated decision aid for patient-centric treatment decision-making using decision analysis, preference thresholds and value elicitations to maximize the compatibility between a patient’s treatment expectations and outcome.MethodsA template for patient-centric medical decision-making was constructed. The inputs included prostate cancer risk group, pre-treatment health state, treatment alternatives (primarily focused on radiation in this model), side effects (erectile dysfunction, urinary incontinence, nocturia and bowel incontinence), and treatment success (5-year freedom from biochemical failure). A linear additive value function was used to combine the values for each attribute (side effects, success and the alternatives) into a value for all prospects. The patient-reported toxicity probabilities were derived from phase II and III trials. The probabilities are conditioned on the starting state for each of the side effects. Toxicity matrices for erectile dysfunction, urinary incontinence, nocturia and bowel incontinence were created for the treatment alternatives. Toxicity probability thresholds were obtained by identifying the patient’s maximum acceptable threshold for each of the side effects. Results are represented as a visual. R and Rstudio were used to perform analyses, and R Shiny for application creation.ResultsWe developed a web-based decision aid. Based on preliminary use of the application, every treatment alternative could be the best choice for a decision maker with a particular set of preferences. This result implies that no treatment has determinist dominance over the remaining treatments and that a preference-based approach can help patients through their decision-making process, potentially affecting compliance with treatment, tolerance of side effects and satisfaction with the decision.ConclusionsWe present a unique patient-centric prostate cancer treatment decision aid that systematically assesses and incorporates a patient’s preferences and values to rank treatment options by likelihood of achieving the preferred outcome. This application enables the practice and study of personalized medicine. This model can be expanded to include additional inputs, such as genomics, as well as competing, concurrent or sequential therapies.

Inhibitory Cognitive Control Allows Automated Advice to Improve Accuracy While Minimizing Misuse

Humans increasingly use automated decision aids. However, environmental uncertainty means that automated advice can be incorrect, creating the potential for humans to act on incorrect advice or to disregard correct advice. We present a quantitative model of the cognitive process by which humans use automation when deciding whether aircraft would violate requirements for minimum separation. The model closely fitted the performance of 24 participants, who each made 2,400 conflict-detection decisions (conflict vs. nonconflict), either manually (with no assistance) or with the assistance of 90% reliable automation. When the decision aid was correct, conflict-detection accuracy improved, but when the decision aid was incorrect, accuracy and response time were impaired. The model indicated that participants integrated advice into their decision process by inhibiting evidence accumulation toward the task response that was incongruent with that advice, thereby ensuring that decisions could not be made solely on automated advice without first sampling information from the task environment.

Strategy Use in Automation-Aided Decision Making

When human operators make signal detection judgments with assistance from an automated decision aid, they perform better than they could unaided but fail to reach optimal sensitivity. We investigated the decision strategies that produce this suboptimal performance. Participants ( N = 130) performed a two-response classification task that required them to mentally estimate the mean of a set of randomly sampled values each trial. The task was performed with and without assistance from a 93% reliable decision aid. Psychometric functions were fit to the classification data, and data were fit with two cognitive models of automation use. The first model assumed that participants made automation-aided judgments using a contingent criterion strategy, adjusting their response cutoff for yes vs. no responses following a cue from the aid. The second strategy, a discrete state model, assumed that participants made aided judgments by simply deferring to the aid on some proportion of trials. A measure of model fit favored the discrete-state process model, with parameter estimates indicating large individual differences in deferral rate between participants (range = 2% and 95%).

Toward Quantifying Trust Dynamics: How People Adjust Their Trust After Moment-to-Moment Interaction With Automation.

We examine how human operators adjust their trust in automation as a result of their moment-to-moment interaction with automation. Most existing studies measured trust by administering questionnaires at the end of an experiment. Only a limited number of studies viewed trust as a dynamic variable that can strengthen or decay over time. Seventy-five participants took part in an aided memory recognition task. In the task, participants viewed a series of images and later on performed 40 trials of the recognition task to identify a target image when it was presented with a distractor. In each trial, participants performed the initial recognition by themselves, received a recommendation from an automated decision aid, and performed the final recognition. After each trial, participants reported their trust on a visual analog scale. Outcome bias and contrast effect significantly influence human operators' trust adjustments. An automation failure leads to a larger trust decrement if the final outcome is undesirable, and a marginally larger trust decrement if the human operator succeeds the task by him/herself. An automation success engenders a greater trust increment if the human operator fails the task. Additionally, automation failures have a larger effect on trust adjustment than automation successes. Human operators adjust their trust in automation as a result of their moment-to-moment interaction with automation. Their trust adjustments are significantly influenced by decision-making heuristics/biases. Understanding the trust adjustment process enables accurate prediction of the operators' moment-to-moment trust in automation and informs the design of trust-aware adaptive automation.

Measuring the Efficiency of Automation-Aided Performance in a Simulated Baggage Screening Task.

The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making. Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal. Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid's judgment relative to the raw stimuli, target certainty, and target prevalence. Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid's diagnosis with a probability of 50%. Performance was similar across experiments. Results suggest that human operators' performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.

Human Performance Consequences of Automated Decision Aids: The Impact of Time Pressure.

The study addresses the impact of time pressure on human interactions with automated decision support systems (DSSs) and related performance consequences. When humans interact with DSSs, this often results in worse performance than could be expected from the automation alone. Previous research has suggested that time pressure might make a difference by leading humans to rely more on a DSS. In two laboratory experiments, participants performed a luggage screening task either manually, supported by a highly reliable DSS, or by a low reliable DSS. Time provided for inspecting the X-rays was 4.5 s versus 9 s varied within-subjects as the time pressure manipulation. Participants in the automation conditions were either shown the automation's advice prior (Experiment 1) or following (Experiment 2) their own inspection, before they made their final decision. In Experiment 1, time pressure compromised performance independent of whether the task was performed manually or with automation support. In Experiment 2, the negative impact of time pressure was only found in the manual but not in the two automation conditions. However, neither experiment revealed any positive impact of time pressure on overall performance, and the joint performance of human and automation was mostly worse than the performance of the automation alone. Time pressure compromises the quality of decision-making. Providing a DSS can reduce this effect, but only if the automation's advice follows the assessment of the human. The study provides suggestions for the effective implementation of DSSs in addition to supporting concerns that highly reliable DSSs are not used optimally by human operators.

Effect of automation transparency in the management of multiple unmanned vehicles

Automated decision aids can undoubtedly benefit system performance, but have the potential to provide incorrect advice, creating the possibility for automation disuse or misuse. This paper examined the extent to which increased automation transparency could improve the accuracy of automation use in a simulation of unmanned vehicle control. Participants were required to assign the best unmanned vehicle to complete missions. An automated recommender system provided advice but was not always reliable. Three levels of automation transparency were manipulated between-participants. Providing transparency regarding the reasoning underlying automated recommendations improved the accuracy of automation use, with no cost to decision time or subjective workload. Higher transparency in the form of projected decision outcome visualizations led to faster decisions, but less accurate automation use and an automation bias. Implications for the design of transparent interfaces to improve human-autonomy teaming outcomes in time-pressured environments are discussed.

CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?

CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?