Automation Bias Research Articles

Automation Bias and the “Goldilocks Effect” in Auditing Blockchain

ABSTRACT Blockchain technology appears to have unique characteristics such as inherent trust, immutability, transparency, and tighter security that might facilitate the audit process. Using a survey of AICPA members, we examine responses to the evaluation of audit evidence related to management assertions in a blockchain-enabled environment. We posit three possible effects pertaining to reliance on audit evidence in blockchain-enabled environments: (1) Goldilocks Effect (no automation bias) or the ideal scenario where auditors exhibit “just right” levels of professional skepticism and due care resulting in an optimal level of auditing; (2) Halo Effect or positive automation bias, which potentially results in over-reliance on evidence and, in turn, underauditing and; (3) Hype Effect or negative automation bias, which potentially results in under-reliance on evidence and, in turn, overauditing. Our results suggest that respondents to our survey are predominantly influenced by the Halo Effect, indicating a propensity for positive automation bias across management assertions.

AN INTER-DISCIPLINARY APPROACH TO AUTOMATION TECHNOLOGY IN FINANCE - WHAT CAN HISTORY, LAW AND DATA SCIENCE TEACH US?

The year 2008 is etched in human history as the year of the ‘Global Financial Crises’. Post the crises, Historians and financial commentators alike rushed to impute blame. Some blamed securitizations, some the banks and some Lehman Brothers and AIG. However, in the midst of all of this humbug, a key epicentre of the crises escaped academic scrutiny; ‘Automation Technology’. The paper therefore aims to present an alternative view of financial history; one which impleads ‘automation technology in finance’ i.e., Risk Modelling Algorithms and RegTech. However, the underlying aim of this paper is to make a case against systemic automation bias in finance and to achieve that end, the paper employs an inter-disciplinary approach and uses history, law and data science to show case the multifarious perils of using automation technology blindfold in finance whilst also proposing possible solutions such as the incorporating of design thinking and systems theory in finance. Expired data sets, human assumptions, turning code in law, and a lack of standardized financial semantics as but some of these ‘perils’. On the law front; it presents a twofold challenge under constitutional and anti-trust law and aims to reconcile law and technology. Lastly the paper aims to guide regulators by categorizing multiple stages of technological complexity and recommends application of different regulatory approaches to regulating automation. Therefore, the paper shall maintain a ‘solution’ oriented approach throughout.

Exploring the Impact of Automation Bias and Complacency on Individual Criminal Responsibility for War Crimes

Abstract With advancing technology, complex decision-making in warfare, including targeting, is increasingly assisted by machines. Although involving humans in decision-making is often seen as a safeguard against machine errors, it does not always prevent them. Machines can make incorrect determinations or delay them when time is critical. In these cases, human operators, influenced by automation bias (excessive trust in machines’ determinations, despite the availability of contradicting or different information from other sources) or complacency (excessive trust in machines’ determinations, leading to reduced vigilance), may fail to recognize machine errors, potentially resulting in conduct amounting to a war crime. Considering the role of automation bias and complacency in the determination of the criminal responsibility of systems’ operators is crucial, especially, for understanding the accountability framework for war crimes involving autonomous weapon systems (AWS). By exploring how automation bias and complacency affect the determination of criminal responsibility for humans who operate AWS, this article offers insights for lawmakers at the national and international levels to understand complexities and effectively shape legislative responses with respect to criminal responsibility. This article also examines automation bias and complacency in psychology, their relevance in military operations employing AWS, and their potential to exonerate human operators from criminal responsibility. It concludes by advocating for legislative, organizational, and technical measures to counteract automation bias and complacency.

Hooked on artificial agents: a systems thinking perspective

Following recent technological developments in the artificial intelligence space, artificial agents are increasingly taking over organizational tasks typically reserved for humans. Studies have shown that humans respond differently to this, with some being appreciative of their advice (algorithm appreciation), others being averse toward them (algorithm aversion), and others still fully relinquishing control to artificial agents without adequate oversight (automation bias). Using systems thinking, we analyze the existing literature on these phenomena and develop a conceptual model that provides an underlying structural explanation for their emergence. In doing so, we create a powerful visual tool that can be used to ground discussions about the impact artificial agents have on organizations and humans within them.

Decision-supporting algorithms in the public sector: The influence on caseworkers’ discretionary assessment and reasoning

Caseworkers in the public administration must make decisions based on accurate and objective information and be able to explain their reasoning. They are increasingly relying on decision-supporting algorithms that prioritize output over reasoning. While these algorithms can streamline and improve workflows, they also challenge fundamental principles of legal certainty. Utilizing data from the ASTA report on caseworkers’ experiences with algorithms in risk assessments of newly unemployed individuals as a case, we examine how automation bias and the black box paradox can affect caseworkers’ reasoning and justification of decisions. While the decision-making process may become more efficient in handling large datasets, fundamental principles of the rule of law may be challenged.

Human factors in the clinical implementation of deep learning-based automated contouring of pelvic organs at risk for MRI-guided radiotherapy.

Deep neural nets have revolutionized the science of auto-segmentation and present great promise for treatment planning automation. However, little data exists regarding clinical implementation and human factors. We evaluated the performance and clinical implementation of a novel deep learning-based auto-contouring workflow for 0.35T magnetic resonance imaging (MRI)-guided pelvic radiotherapy, focusing on automation bias and objective measures of workflow savings. An auto-contouring model was developed using a UNet-derived architecture for the femoral heads, bladder, and rectum in 0.35T MR images. Training data was taken from 75 patients treated with MRI-guided radiotherapy at our institution. The model was tested against 20 retrospective cases outside the training set, and subsequently was clinically implemented. Usability was evaluated on the first 30 clinical cases by computing Dice coefficient (DSC), Hausdorff distance (HD), and the fraction of slices that were used un-modified by planners. Final contours were retrospectively reviewed by an experienced planner and clinical significance of deviations was graded as negligible, low, moderate, and high probability of leading to actionable dosimetric variations. In order to assess whether the use of auto-contouring led to final contours more or less in agreement with an objective standard, 10 pre-treatment and 10 post-treatment blinded cases were re-contoured from scratch by three expert planners to get expert consensus contours (EC). EC was compared to clinically used (CU) contours using DSC. Student's t-test and Levene's statistic were used to test statistical significance of differences in mean and standard deviation, respectively. Finally, the dosimetric significance of the contour differences were assessed by comparing the difference in bladder and rectum maximum point doses between EC and CU before and after the introduction of automation. Median (interquartile range) DSC for the retrospective test data were 0.92(0.02), 0.92(0.06), 0.93(0.06), 0.87(0.04) for the post-processed contours for the right and left femoral heads, bladder, and rectum, respectively. Post-implementation median DSC were 1.0(0.0), 1.0(0.0), 0.98(0.04), and 0.98(0.06), respectively. For each organ, 96.2, 95.4, 59.5, and 68.21 percent of slices were used unmodified by the planner. DSC between EC and pre-implementation CU contours were 0.91(0.05*), 0.91*(0.05*), 0.95(0.04), and 0.88(0.04) for right and left femoral heads, bladder, and rectum, respectively. The corresponding DSC for post-implementation CU contours were 0.93(0.02*), 0.93*(0.01*), 0.96(0.01), and 0.85(0.02) (asterisks indicate statistically significant difference). In a retrospective review of contours used for planning, a total of four deviating slices in two patients were graded as low potential clinical significance. No deviations were graded as moderate or high. Mean differences between EC and CU rectum max-doses were 0.1±2.6Gy and -0.9±2.5Gy for pre- and post-implementation, respectively. Mean differences between EC and CU bladder/bladder wall max-doses were -0.9±4.1Gy and 0.0±0.6Gy for pre- and post-implementation, respectively. These differences were not statistically significant according to Student's t-test. We have presented an analysis of the clinical implementation of a novel auto-contouring workflow. Substantial workflow savings were obtained. The introduction of auto-contouring into the clinical workflow changed the contouring behavior of planners. Automation bias was observed, but it had little deleterious effect on treatment planning.

Norwegian radiologists’ expectations of artificial intelligence in mammographic screening – A cross-sectional survey

PurposeTo explore Norwegian breast radiologists’ expectations of adding artificial intelligence (AI) in the interpretation procedure of screening mammograms. MethodsAll breast radiologists involved in interpretation of screening mammograms in BreastScreen Norway during 2021 and 2022 (n = 98) were invited to take part in this anonymous cross-sectional survey about use of AI in mammographic screening. The questionnaire included background information of the respondents, their expectations, considerations of biases, and ethical and social implications of implementing AI in screen reading. Data was collected digitally and analyzed using descriptive statistics. ResultsThe response rate was 61% (60/98), and 67% (40/60) of the respondents were women. Sixty percent (36/60) reported ≥10 years’ experience in screen reading, while 82% (49/60) reported no or limited experience with AI in health care. Eighty-two percent of the respondents were positive to explore AI in the interpretation procedure in mammographic screening. When used as decision support, 68% (41/60) expected AI to increase the radiologists’ sensitivity for cancer detection. As potential challenges, 55% (33/60) reported lack of trust in the AI system and 45% (27/60) reported discrepancy between radiologists and AI systems as possible challenges. The risk of automation bias was considered high among 47% (28/60). Reduced time spent reading mammograms was rated as a potential benefit by 70% (42/60). ConclusionThe radiologists reported positive expectations of AI in the interpretation procedure of screening mammograms. Efforts to minimize the risk of automation bias and increase trust in the AI systems are important before and during future implementation of the tool.

The possibility of AI-induced medical manslaughter: Unexplainable decisions, epistemic vices, and a new dimension of moral luck

The use of artificial intelligence (AI) systems in healthcare provides a compelling case for a re-examination of ‘gross negligence’ as the basis for criminal liability. AI is a smart agency, often using self-learning architectures, with the capacity to make autonomous decisions. Healthcare practitioners (HCPs) will remain responsible for validating AI recommendations but will have to contend with challenges such as automation bias, the unexplainable nature of AI decisions, and an epistemic dilemma when clinicians and systems disagree. AI decisions are the result of long chains of sociotechnical complexity with the capacity for undetectable errors to be baked into systems, which introduces a new dimension of moral luck. The ‘advisory’ nature of AI decisions constructs a legal fiction, which may leave HCPs unjustly exposed to the legal and moral consequences when systems fail. On balance, these novel challenges point towards a legal test of subjective recklessness as the better option: it is practically necessary; falls within the historic range of the offence; and offers clarity, coherence, and a welcome reconnection with ethics.

Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias.

Incorporating artificial intelligence (AI) into clinics brings the risk of automation bias, which potentially misleads the clinician's decision-making. The purpose of this study was to propose a potential strategy to mitigate automation bias. This was a laboratory study with a randomized cross-over design. The diagnosis of anterior cruciate ligament (ACL) rupture, a common injury, on magnetic resonance imaging (MRI) was used as an example. Forty clinicians were invited to diagnose 200 ACLs with and without AI assistance. The AI's correcting and misleading (automation bias) effects on the clinicians' decision-making processes were analyzed. An ordinal logistic regression model was employed to predict the correcting and misleading probabilities of the AI. We further proposed an AI suppression strategy that retracted AI diagnoses with a higher misleading probability and provided AI diagnoses with a higher correcting probability. The AI significantly increased clinicians' accuracy from 87.2%±13.1% to 96.4%±1.9% (P < .001). However, the clinicians' errors in the AI-assisted round were associated with automation bias, accounting for 45.5% of the total mistakes. The automation bias was found to affect clinicians of all levels of expertise. Using a logistic regression model, we identified an AI output zone with higher probability to generate misleading diagnoses. The proposed AI suppression strategy was estimated to decrease clinicians' automation bias by 41.7%. Although AI improved clinicians' diagnostic performance, automation bias was a serious problem that should be addressed in clinical practice. The proposed AI suppression strategy is a practical method for decreasing automation bias.

Role of Artificial Intelligence in Global Surgery: A Review of Opportunities and Challenges.

Global surgery broadly refers to a rapidly expanding multidisciplinary field concerned with providing better and equitable surgical care across international health systems. Global surgery initiatives primarily focus on capacity building, advocacy, education, research, and policy development in low- and middle-income countries (LMICs). The inadequate surgical, anesthetic, and obstetric care currently contributes to 18 million preventable deaths each year. Hence, there is a growing interest in the rapid growth of artificial intelligence (AI) that provides a distinctive opportunity to enhance surgical services in LMICs. AI modalities have been used for personalizing surgical education, automating administrative tasks, and developing realistic and cost-effective simulation-training programs with provisions for people with special needs. Furthermore, AI may assist with providing insights for governance, infrastructure development, and monitoring/predicting stock take or logistics failure that can help in strengthening global surgery pillars. Numerous AI-assisted telemedicine-based platforms have allowed healthcare professionals to virtually assist in complex surgeries that may help to improve surgical accessibility across LMICs. Challenges in implementing AI technology include the misrepresentation of minority populations in the datasets leading to discriminatory bias. Human hesitancy, employment uncertainty, automation bias, and role of confounding factors need to be further studied for equitable utilization of AI. With a focused and evidence-based approach, AI could help several LMICs overcome bureaucratic inefficiency and develop more efficient surgical systems.

Are the Pilots Onboard? Equipping Radiologists for Clinical Implementation of AI.

The incorporation of artificial intelligence into radiological clinical workflow is on the verge of being realized. To ensure that these tools are effective, measures must be taken to educate radiologists on tool performance and failure modes. Additionally, radiology systems should be designed to avoid automation bias and the potential decline in radiologist performance. Designed solutions should cater to every level of expertise so that patient care can be enhanced and risks reduced. Ultimately, the radiology community must provide education so that radiologists can learn about algorithms, their inputs and outputs, and potential ways they may fail. This manuscript will present suggestions on how to train radiologists to use these new digital systems, how to detect AI errors, and how to maintain underlying diagnostic competency when the algorithm fails.

Assessing the Utility, Impact, and Adoption Challenges of an Artificial Intelligence-Enabled Prescription Advisory Tool for Type 2 Diabetes Management: Qualitative Study.

The clinical management of type 2 diabetes mellitus (T2DM) presents a significant challenge due to the constantly evolving clinical practice guidelines and growing array of drug classes available. Evidence suggests that artificial intelligence (AI)-enabled clinical decision support systems (CDSSs) have proven to be effective in assisting clinicians with informed decision-making. Despite the merits of AI-driven CDSSs, a significant research gap exists concerning the early-stage implementation and adoption of AI-enabled CDSSs in T2DM management. This study aimed to explore the perspectives of clinicians on the use and impact of the AI-enabled Prescription Advisory (APA) tool, developed using a multi-institution diabetes registry and implemented in specialist endocrinology clinics, and the challenges to its adoption and application. We conducted focus group discussions using a semistructured interview guide with purposively selected endocrinologists from a tertiary hospital. The focus group discussions were audio-recorded and transcribed verbatim. Data were thematically analyzed. A total of 13 clinicians participated in 4 focus group discussions. Our findings suggest that the APA tool offered several useful features to assist clinicians in effectively managing T2DM. Specifically, clinicians viewed the AI-generated medication alterations as a good knowledge resource in supporting the clinician's decision-making on drug modifications at the point of care, particularly for patients with comorbidities. The complication risk prediction was seen as positively impacting patient care by facilitating early doctor-patient communication and initiating prompt clinical responses. However, the interpretability of the risk scores, concerns about overreliance and automation bias, and issues surrounding accountability and liability hindered the adoption of the APA tool in clinical practice. Although the APA tool holds great potential as a valuable resource for improving patient care, further efforts are required to address clinicians' concerns and improve the tool's acceptance and applicability in relevant contexts.

Accelerating voxelwise annotation of cross-sectional imaging through AI collaborative labeling with quality assurance and bias mitigation.

precision-medicine quantitative tools for cross-sectional imaging require painstaking labeling of targets that vary considerably in volume, prohibiting scaling of data annotation efforts and supervised training to large datasets for robust and generalizable clinical performance. A straight-forward time-saving strategy involves manual editing of AI-generated labels, which we call AI-collaborative labeling (AICL). Factors affecting the efficacy and utility of such an approach are unknown. Reduction in time effort is not well documented. Further, edited AI labels may be prone to automation bias. In this pilot, using a cohort of CTs with intracavitary hemorrhage, we evaluate both time savings and AICL label quality and propose criteria that must be met for using AICL annotations as a high-throughput, high-quality ground truth. 57 CT scans of patients with traumatic intracavitary hemorrhage were included. No participant recruited for this study had previously interpreted the scans. nnU-net models trained on small existing datasets for each feature (hemothorax/hemoperitoneum/pelvic hematoma; n = 77-253) were used in inference. Two common scenarios served as baseline comparison- de novo expert manual labeling, and expert edits of trained staff labels. Parameters included time effort and image quality graded by a blinded independent expert using a 9-point scale. The observer also attempted to discriminate AICL and expert labels in a random subset (n = 18). Data were compared with ANOVA and post-hoc paired signed rank tests with Bonferroni correction. AICL reduced time effort 2.8-fold compared to staff label editing, and 8.7-fold compared to expert labeling (corrected p < 0.0006). Mean Likert grades for AICL (8.4, SD:0.6) were significantly higher than for expert labels (7.8, SD:0.9) and edited staff labels (7.7, SD:0.8) (corrected p < 0.0006). The independent observer failed to correctly discriminate AI and human labels. For our use case and annotators, AICL facilitates rapid large-scale curation of high-quality ground truth. The proposed quality control regime can be employed by other investigators prior to embarking on AICL for segmentation tasks in large datasets.

The effects of explanations on automation bias

In this paper we explore the effect of explanations on reducing errors in the human decision making process caused by placing excessive reliance on automated decision support systems. We develop and implement different forms of explanations based on cognitive principles and evaluate their effect over two different domains: our new version of the Coloured Trails game, and over a simulated radiological task. We found that explanations did not reduce this aspect of automation bias and sometimes increased it. However, they reduced completion time and often increased user decision accuracy, despite not altering the perceived task load. Overall, explanations were beneficial though the benefits were highly context dependent. This work contributes to the complex interplay between automation bias, performance and explanations.

Automation complacency on the road

Given that automation complacency, a hitherto controversial concept, is already used to blame and punish human drivers in current accident investigations and courts, it is essential to map complacency research in driving automation and determine whether current research can support its legitimate usage in these practical fields. Here, we reviewed its status quo in the domain and conducted a thematic analysis. We then discussed five fundamental challenges that might undermine its scientific legitimation: conceptual confusion exists in whether it is an individual versus systems problem; uncertainties exist in current evidence of complacency; valid measures specific to complacency are lacking; short-term laboratory experiments cannot address the long-term nature of complacency and thus their findings may lack external validity; and no effective interventions directly target complacency prevention. The Human Factors/Ergonomics community has a responsibility to minimise its usage and defend human drivers who rely on automation that is far from perfect. Practitioner summary: Human drivers are accused of complacency and overreliance on driving automation in accident investigations and courts. Our review work shows that current academic research in the driving automation domain cannot support its legitimate usage in these practical fields. Its misuse will create a new form of consumer harms.

Mirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies

Predictive algorithmic scores can significantly impact the lives of assessed individuals by shaping decisions of organizations and institutions that affect them, for example, influencing the hiring prospects of job applicants or the release of defendants on bail. To better protect people and provide them the opportunity to appeal their algorithmic assessments, data privacy advocates and regulators increasingly push for disclosing the scores and their use in decision-making processes to scored individuals. Although inherently important, the response of scored individuals to such algorithmic transparency is understudied. Inspired by psychological and economic theories of information processing, we aim to fill this gap. We conducted a comprehensive empirical study to explore how and why disclosing the use of algorithmic scoring processes to (involuntarily) scored individuals affects their behaviors. Our results provide strong evidence that the disclosure of fundamentally erroneous algorithmic scores evokes self-fulfilling prophecies that endogenously steer the behavior of scored individuals toward their assessment, enabling algorithms to help produce the world they predict. Our results emphasize that isolated transparency measures can have considerable side effects with noticeable implications for the development of automation bias, the occurrence of feedback loops, and the design of transparency regulations.

Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance.

Background Automation bias (the propensity for humans to favor suggestions from automated decision-making systems) is a known source of error in human-machine interactions, but its implications regarding artificial intelligence (AI)-aided mammography reading are unknown. Purpose To determine how automation bias can affect inexperienced, moderately experienced, and very experienced radiologists when reading mammograms with the aid of an artificial intelligence (AI) system. Materials and Methods In this prospective experiment, 27 radiologists read 50 mammograms and provided their Breast Imaging Reporting and Data System (BI-RADS) assessment assisted by a purported AI system. Mammograms were obtained between January 2017 and December 2019 and were presented in two randomized sets. The first was a training set of 10 mammograms, with the correct BI-RADS category suggested by the AI system. The second was a set of 40 mammograms in which an incorrect BI-RADS category was suggested for 12 mammograms. Reader performance, degree of bias in BI-RADS scoring, perceived accuracy of the AI system, and reader confidence in their own BI-RADS ratings were assessed using analysis of variance (ANOVA) and repeated-measures ANOVA followed by post hoc tests and Kruskal-Wallis tests followed by the Dunn post hoc test. Results The percentage of correctly rated mammograms by inexperienced (mean, 79.7% ± 11.7 [SD] vs 19.8% ± 14.0; P < .001; r = 0.93), moderately experienced (mean, 81.3% ± 10.1 vs 24.8% ± 11.6; P < .001; r = 0.96), and very experienced (mean, 82.3% ± 4.2 vs 45.5% ± 9.1; P = .003; r = 0.97) radiologists was significantly impacted by the correctness of the AI prediction of BI-RADS category. Inexperienced radiologists were significantly more likely to follow the suggestions of the purported AI when it incorrectly suggested a higher BI-RADS category than the actual ground truth compared with both moderately (mean degree of bias, 4.0 ± 1.8 vs 2.4 ± 1.5; P = .044; r = 0.46) and very (mean degree of bias, 4.0 ± 1.8 vs 1.2 ± 0.8; P = .009; r = 0.65) experienced readers. Conclusion The results show that inexperienced, moderately experienced, and very experienced radiologists reading mammograms are prone to automation bias when being supported by an AI-based system. This and other effects of human and machine interaction must be considered to ensure safe deployment and accurate diagnostic performance when combining human readers and AI. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Baltzer in this issue.

Bias Present in Computerized Algorithms

Abstract: Automation biases stemming from deep-rooted computer algorithms are blamed for the false results and misinterpretations it presents; However, algorithmic biases in machine learning models are based on user interpretation and input, and therefore the machine's output may not correspond with the input's attitude. This stems from the variety of information that is fed in machine learning models and possible faulty glitches within computers themselve

Argument by Numbers: The Normative Impact of Statistical Legal Tech

The introduction of statistical ‘legal tech’ raises questions about the future of law and legal practice. While technologies have always mediated the concept, practice, and texture of law, a qualitative and quantitative shift is taking place: statistical legal tech is being integrated into mainstream legal practice, and particularly that of litigators. These applications – particularly in search and document generation – mediate how practicing lawyers interact with the legal system. By shaping how law is ‘done’, the applications ultimately come to shape what law is. Where such applications impact on the creative elements of the litigator’s practice, for example via automation bias, they affect their professional and ethical duty to respond appropriately to the unique circumstances of their client’s case – a duty that is central to the Rule of Law. The statistical mediation of legal resources by machine learning applications must therefore be introduced with great care, if we are to avoid the subtle, inadvertent, but ultimately fundamental undermining of the Rule of Law. In this contribution we describe the normative effects of legal tech application design, how they are potentially (in)compatible with law and the Rule of Law as normative orders, particularly with respect to legal texts which we frame as the proper source of ‘lossless law’, uncompressed by statistical framing. We conclude that reliance on the vigilance of individual lawyers is insufficient to guard against the potentially harmful effects of such systems, given their inscrutability, and suggest that the onus is on the providers of legal technologies to demonstrate the legitimacy of their systems according to the normative standards inherent in the legal system.

Check the box! How to deal with automation bias in AI-based personnel selection.

Artificial Intelligence (AI) as decision support for personnel preselection, e.g., in the form of a dashboard, promises a more effective and fairer selection process. However, AI-based decision support systems might prompt decision makers to thoughtlessly accept the system's recommendation. As this so-called automation bias contradicts ethical and legal requirements of human oversight for the use of AI-based recommendations in personnel preselection, the present study investigates strategies to reduce automation bias and increase decision quality. Based on the Elaboration Likelihood Model, we assume that instructing decision makers about the possibility of system errors and their responsibility for the decision, as well as providing an appropriate level of data aggregation should encourage decision makers to process information systematically instead of heuristically. We conducted a 3 (general information, information about system errors, information about responsibility) x 2 (low vs. high aggregated data) experiment to investigate which strategy can reduce automation bias and enhance decision quality. We found that less automation bias in terms of higher scores on verification intensity indicators correlated with higher objective decision quality, i.e., more suitable applicants selected. Decision makers who received information about system errors scored higher on verification intensity indicators and rated subjective decision quality higher, but decision makers who were informed about their responsibility, unexpectedly, did not. Regarding aggregation level of data, decision makers of the highly aggregated data group spent less time on the level of the dashboard where highly aggregated data were presented. Our results show that it is important to inform decision makers who interact with AI-based decision-support systems about potential system errors and provide them with less aggregated data to reduce automation bias and enhance decision quality.