Artificial Intelligence Decision Support Research Articles

Automatized self-supervised learning for skin lesion screening

Melanoma, the deadliest form of skin cancer, has seen a steady increase in incidence rates worldwide, posing a significant challenge to dermatologists. Early detection is crucial for improving patient survival rates. However, performing total body screening (TBS), i.e., identifying suspicious lesions or ugly ducklings (UDs) by visual inspection, can be challenging and often requires sound expertise in pigmented lesions. To assist users of varying expertise levels, an artificial intelligence (AI) decision support tool was developed. Our solution identifies and characterizes UDs from real-world wide-field patient images. It employs a state-of-the-art object detection algorithm to locate and isolate all skin lesions present in a patient’s total body images. These lesions are then sorted based on their level of suspiciousness using a self-supervised AI approach, tailored to the specific context of the patient under examination. A clinical validation study was conducted to evaluate the tool’s performance. The results demonstrated an average sensitivity of 95% for the top-10 AI-identified UDs on skin lesions selected by the majority of experts in pigmented skin lesions. The study also found that the tool increased dermatologists’ confidence when formulating a diagnosis, and the average majority agreement with the top-10 AI-identified UDs reached 100% when assisted by our tool. With the development of this AI-based decision support tool, we aim to address the shortage of specialists, enable faster consultation times for patients, and demonstrate the impact and usability of AI-assisted screening. Future developments will include expanding the dataset to include histologically confirmed melanoma and validating the tool for additional body regions.

Impact of artificial intelligence (AI) decision support on clinical trial participation: A before-after implementation study on a nationwide molecular tumor board.

1557 Background: Molecular Tumor Boards (MTB) have become the standard for deriving consensus therapy (Rx) recommendations (rec) for patients (pts) with advanced cancer undergoing comprehensive genomic profiling (CGP). However, finding matched Rx and trial options remains laborious, needing informatics strategies. Methods: We examined the outcomes before and after implementation of a decision support system (DSS) at the MTB of the Australian Molecular Screening & Therapeutics program (MoST, ACTRN12616000908437), enrolling pts with refractory solid tumors and linking CGP results to Rx and trials through rec on clinical reports. The DSS comprises a variant interpretation pipeline, a symbolic AI system integrating the TOPOGRAPH database and an annotated trials registry, and a web platform used by the MTB for producing reports shortlisting matching Rx and therapeutic trials. The primary endpoint is the cumulative participation rate (CPR) in genomic matched trials (excluding MoST-related trials) in pts who received Rx after CGP. The secondary endpoints are the CPR in any trial, and overall survival (OS) from MTB. The Fine-Gray model was used to estimate the subdistribution hazard ratios (sHR) for between-group differences in participation. Kaplan-Meier and Cox regression were used for OS analysis. The time-to-event analyses were adjusted for age, ECOG, cancer type, lines of prior rx, and lead-time bias (OS only). Results: In 2203 of 5186 pts (42.5%) received ≥ 1 Rx after CGP, 971 pts were reviewed at MTB before and 1272 after DSS implementation in Sep 2020. At 3, 6 and 12 months (m) after enrolment, CPR in matched trials were 3.2%, 5.0%, and 6.8% before implementation and 3.6%, 5.8%, and 8.9% afterwards (sHR 1.19, 95% CI 0.90–1.57, p=0.23). Pts who received an MTB rec were more likely to participate in a matched trial (sHR 4.75, 3.13–7.23, p<0.001). Notably, DSS specifically increased the likelihood of trial participation following MTB rec (interaction sHR 2.69, 1.02–7.07, p=0.045), with 12m CPR increasing from 10.0% to 12.2% after implementation of the DSS in pts with a MTB rec, whilst for pts without a MTB rec, the CPR fell from 3.1% to 1.6% after implementation. No association was seen between participation in any trials and MTB recs (sHR 1.12, 0.95–1.32) or implementation (sHR 0.95, 0.81–1.11; CPR pre 21.6% v post 23.7% at 12m). No OS difference was seen comparing before with after implementation of the DSS (median 16.7 v 15.3m, adjusted HR 0.97, 0.87–1.08). However, pts receiving genomic matched Rx had a longer OS (median 20.1 v 15.0m unmatched, HR 0.76, p=0.002). Conclusions: The observed interaction between the implementation of DSS and MTB rec suggests there is effect modification in participation rates of pts in genomic matched trials, warranting exploration into how AI might enhance trial participation among pts with incurable cancers undergoing CGP.

Predicting left atrial appendage thrombus in patients with persistent atrial fibrillation or flutter and no oral anticoagulation using machine-learning models

Abstract Background Atrial fibrillation and flutter (AF/AFL) increase the risk of thromboembolic events by promoting the formation of a left atrial appendage thrombus (LAT). We previously developed artificial intelligence (AI) models to predict LAT based on data from the multicentre, prospective LATTEE registry (Left Atrial Thrombus on Transoesophageal Echocardiography, n= 2489) establishing the utility of AI for identifying the presence of LAT. While our model is now available as an online calculator to date there is no data regarding the utility of our AI in guiding cardioversion in haemodynamically stable patients with persistent arrhythmia and no prior oral anticoagulation (OAC). Purpose To assess the performance of AI decision support to perform TOE or readmit the patient after at least 3 weeks of OAC. Methods In this study, we applied the LAT-AI model (based on 39 features) and the LAT-AI reduced model (based on 8 features) to patients with no or insufficient prior OAC, who underwent TOE before direct current cardioversion or catheter ablation (n=311). We excluded patients with paroxysmal arrhythmia, resulting in a final cohort of 150 patients originating from four sites. We investigated the sensitivity and specificity of the respective models to predict LAT based on the score thresholds established in previous studies (0.37 for the LAT-AI and 0.34 for the LAT-AI reduced). Results The median age in the external cohort was 66 (interquartile range 56-73), 59% were male, and 7.3% had AFL. LAT was found in 17 patients (11%). The LAT-AI model achieved 100% sensitivity (confidence interval [CI]: 40-100) and 12% specificity (95% CI: 7-18). The LAT-AI reduced model achieved 100% sensitivity (95% CI: 87-100) and 14% specificity (95% CI: 8-21). By applying the score threshold which has been derived in our prior studies for the the LAT-AI reduced model, the TOE-first approach would be chosen in 18% and no LAT would be found in this group, while the remaining 82% would be recommended for 3 weeks of OAC before TOE. Such approach could be used to identify patients at low risk of LAT to undergo TOE despite the lack of prior OAC, while the remaining patients at higher risk of LAT could potentially avoid repeated TOE thanks to the 3-week course of OAC before the first TOE. Conclusion AI scores allow high sensitivity for prediction of the presence of LAT in patients with AF/AFL irrespective of OAC treatment and have the potential to optimise the management of patients with indications for elective cardioversion and no prior or ineffective OAC.Simulated AI-supported decisionScreenshot of the online app

Understanding Physician's Perspectives on AI in Health Care: Protocol for a Sequential Multiple Assignment Randomized Vignette Study.

As the availability and performance of artificial intelligence (AI)-based clinical decision support (CDS) systems improve, physicians and other care providers poised to be on the front lines will be increasingly tasked with using these tools in patient care and incorporating their outputs into clinical decision-making processes. Vignette studies provide a means to explore emerging hypotheses regarding how context-specific factors, such as clinical risk, the amount of information provided about the AI, and the AI result, may impact physician acceptance and use of AI-based CDS tools. To best anticipate how such factors influence the decision-making of frontline physicians in clinical scenarios involving AI decision-support tools, hypothesis-driven research is needed that enables scenario testing before the implementation and deployment of these tools. This study's objectives are to (1) design an original, web-based vignette-based survey that features hypothetical scenarios based on emerging or real-world applications of AI-based CDS systems that will vary systematically by features related to clinical risk, the amount of information provided about the AI, and the AI result; and (2) test and determine causal effects of specific factors on the judgments and perceptions salient to physicians' clinical decision-making. US-based physicians with specialties in family or internal medicine will be recruited through email and mail (target n=420). Through a web-based survey, participants will be randomized to a 3-part "sequential multiple assignment randomization trial (SMART) vignette" detailing a hypothetical clinical scenario involving an AI decision support tool. The SMART vignette design is similar to the SMART design but adapted to a survey design. Each respondent will be randomly assigned to 1 of the possible vignette variations of the factors we are testing at each stage, which include the level of clinical risk, the amount of information provided about the AI, and the certainty of the AI output. Respondents will be given questions regarding their hypothetical decision-making in response to the hypothetical scenarios. The study is currently in progress and data collection is anticipated to be completed in 2024. The web-based vignette study will provide information on how contextual factors such as clinical risk, the amount of information provided about an AI tool, and the AI result influence physicians' reactions to hypothetical scenarios that are based on emerging applications of AI in frontline health care settings. Our newly proposed "SMART vignette" design offers several benefits not afforded by the extensively used traditional vignette design, due to the 2 aforementioned features. These advantages are (1) increased validity of analyses targeted at understanding the impact of a factor on the decision outcome, given previous outcomes and other contextual factors; and (2) balanced sample sizes across groups. This study will generate a better understanding of physician decision-making within this context. DERR1-10.2196/54787.

Cultural-Centric Online English Teaching with Artificial Intelligence Technology and Decision Support System

Cultural-Centric Online English Teaching with Artificial Intelligence Technology and Decision Support System

Artificial intelligence-based decision support software to improve the efficacy of acute stroke pathway in the NHS: an observational study.

In a drip-and-ship model for endovascular thrombectomy (EVT), early identification of large vessel occlusion (LVO) and timely referral to a comprehensive center (CSC) are crucial when patients are admitted to an acute stroke center (ASC). Several artificial intelligence (AI) decision-aid tools are increasingly being used to facilitate the rapid identification of LVO. This retrospective cohort study aimed to evaluate the impact of deploying e-Stroke AI decision support software in the hyperacute stroke pathway on process metrics and patient outcomes at an ASC in the United Kingdom. Except for the deployment of e-Stroke on 01 March 2020, there were no significant changes made to the stroke pathway at the ASC. The data were obtained from a prospective stroke registry between 01 January 2019 and 31 March 2021. The outcomes were compared between the 14 months before and 12 months after the deployment of AI (pre-e-Stroke cohort vs. post-e-Stroke cohort) on 01 March 2020. Time window analyses were performed using Welch's t-test. Cochran-Mantel-Haenszel test was used to compare changes in disability at 3 months assessed by modified Rankin Score (mRS) ordinal shift analysis, and Fisher's exact test was used for dichotomised mRS analysis. In the pre-e-Stroke cohort, 19 of 22 patients referred received EVT. In the post-e-Stroke cohort, 21 of the 25 patients referred were treated. The mean door-in-door-out (DIDO) and door-to-referral times in pre-e-Stroke vs. post-e-Stroke cohorts were 141 vs. 79 min (difference 62 min, 95% CI 96.9-26.8 min, p < 0.001) and 71 vs. 44 min (difference 27 min, 95% CI 47.4-5.4 min, p = 0.01), respectively. The adjusted odds ratio (age and NIHSS) for mRS ordinal shift analysis at 3 months was 3.14 (95% CI 0.99-10.51, p = 0.06) and the dichotomized mRS 0-2 at 3 months was 16% vs. 48% (p = 0.04) in the pre- vs. post-e-Stroke cohorts, respectively. In this single-center study in the United Kingdom, the DIDO time significantly decreased since the introduction of e-Stroke decision support software into an ASC hyperacute stroke pathway. The reduction in door-in to referral time indicates faster image interpretation and referral for EVT. There was an indication of an increased proportion of patients regaining independent function after EVT. However, this should be interpreted with caution given the small sample size. Larger, prospective studies and further systematic real-world evaluation are needed to demonstrate the widespread generalisability of these findings.

Navigating Now and Next: Recent Advances and Future Horizons in Robotic Radical Prostatectomy.

Robotic-assisted radical prostatectomy (RARP) has become the leading approach for radical prostatectomy driven by innovations aimed at improving functional and oncological outcomes. The initial advancement in this field was transperitoneal multiport robotics, which has since undergone numerous technical modifications. These enhancements include the development of extraperitoneal, transperineal, and transvesical approaches to radical prostatectomy, greatly facilitated by the advent of the Single Port (SP) robot. This review offers a comprehensive analysis of these evolving techniques and their impact on RARP. Additionally, we explore the transformative role of artificial intelligence (AI) in digitizing robotic prostatectomy. AI advancements, particularly in automated surgical video analysis using computer vision technology, are unprecedented in their scope. These developments hold the potential to revolutionize surgeon feedback and assessment and transform surgical documentation, and they could lay the groundwork for real-time AI decision support during surgical procedures in the future. Furthermore, we discuss future robotic platforms and their potential to further enhance the field of RARP. Overall, the field of minimally invasive radical prostatectomy for prostate cancer has been an incubator of innovation over the last two decades. This review focuses on some recent developments in robotic prostatectomy, provides an overview of the next frontier in AI innovation during prostate cancer surgery, and highlights novel robotic platforms that may play an increasing role in prostate cancer surgery in the future.

Prospective deployment of an automated implementation solution for artificial intelligence translation to clinical radiation oncology.

Artificial intelligence (AI)-based technologies embody countless solutions in radiation oncology, yet translation of AI-assisted software tools to actual clinical environments remains unrealized. We present the Deep Learning On-Demand Assistant (DL-ODA), a fully automated, end-to-end clinical platform that enables AI interventions for any disease site featuring an automated model-training pipeline, auto-segmentations, and QA reporting. We developed, tested, and prospectively deployed the DL-ODA system at a large university affiliated hospital center. Medical professionals activate the DL-ODA via two pathways (1): On-Demand, used for immediate AI decision support for a patient-specific treatment plan, and (2) Ambient, in which QA is provided for all daily radiotherapy (RT) plans by comparing DL segmentations with manual delineations and calculating the dosimetric impact. To demonstrate the implementation of a new anatomy segmentation, we used the model-training pipeline to generate a breast segmentation model based on a large clinical dataset. Additionally, the contour QA functionality of existing models was assessed using a retrospective cohort of 3,399 lung and 885 spine RT cases. Ambient QA was performed for various disease sites including spine RT and heart for dosimetric sparing. Successful training of the breast model was completed in less than a day and resulted in clinically viable whole breast contours. For the retrospective analysis, we evaluated manual-versus-AI similarity for the ten most common structures. The DL-ODA detected high similarities in heart, lung, liver, and kidney delineations but lower for esophagus, trachea, stomach, and small bowel due largely to incomplete manual contouring. The deployed Ambient QAs for heart and spine sites have prospectively processed over 2,500 cases and 230 cases over 9 months and 5 months, respectively, automatically alerting the RT personnel. The DL-ODA capabilities in providing universal AI interventions were demonstrated for On-Demand contour QA, DL segmentations, and automated model training, and confirmed successful integration of the system into a large academic radiotherapy department. The novelty of deploying the DL-ODA as a multi-modal, fully automated end-to-end AI clinical implementation solution marks a significant step towards a generalizable framework that leverages AI to improve the efficiency and reliability of RT systems.

Urban community governance and machine learning: practice and prospects for intelligent decision making

Abstract Urban community governance faces unprecedented challenges, but machine learning provides new ideas for it. The objective of this study is to examine the use of machine learning technology in urban community governance to enhance governance efficiency and decision-making quality. The study constructs an artificial neural network intelligent decision support model based on genetic algorithm optimization, which is based on the operational requirements of a smart decision support system. A weighted fuzzy inference network is further developed by the fusion of fuzzy logic and neural networks to enhance the system’s ability to deal with uncertainty and ambiguity. Next, prediction tests were conducted on the development levels of six communities to validate the model’s effectiveness. After 1686 training steps, the error squared SSE drops below 0.2%, according to the results. The composite index’s prediction error was 5.12%, while the minimum error was −1.79%. The predicted rankings of the communities did not change from the actual ones, which was in line with the normal trend, and the algorithmic model achieved better prediction results. This study not only provides a new intelligent decision support tool for community governance but also sets a theoretical and practical foundation for the intelligent development of urban community governance in the future.

Trusting under risk – comparing human to AI decision support agents

The growing number of safety-critical technologized workplaces leads to enhanced support of complex human decision-making by artificial intelligence (AI), increasing the relevance of risk in the joint decision process. This online study examined participants' trust, attitude and behavior during a visual estimation task supported by either a human or an AI decision support agent. Throughout the online studyrisk levels were manipulated through different scenarios. Contrary to recent literature, no main effects were found in participants' trust attitude or trust behavior between support agent conditions or risk levels. However, participants using AI support exhibited increased trust behavior under higher risk, while participants with human support agents did not display behavioral differences. Self-confidence vs. trust and an increased feeling of responsibility may be possible reasons. Furthermore, participants reported the human support agent to be more responsible for possible negative outcomes of the joint task than the AI support agent. Hereby, risk did not influence perceived responsibility. However, the study's findings concerning trust behavior underscore the crucial importance of investigating the impact of risk in workplaces, shedding light on the under-researched effect of risk on trust attitude and behavior in AI-supported human decision-making.

Artificial Intelligence Decision Support Tools for End-of-Life Care Planning Conversations

Why Is This an Issue?&#x0D; &#x0D; End-of-life care provides support for patients and their families during the last stage of life. End-of-life conversations aim to help people better understand their disease prognosis and expected survival, enabling them to make informed decisions regarding end-of-life care.&#x0D; Palliative care focuses on relieving symptoms and improving quality of life for patients with serious or life-threatening diseases. Approximately 89% of patients with life-limiting diseases, such as cancer, can benefit from palliative care. However, not all patients receive it in a timely manner.&#x0D; Due in part to prognostic uncertainty and optimism bias, end-of-life planning conversations and palliative care decisions do not occur early enough to have maximum benefit. Interventions that aim to prompt or help identify those patients who can benefit from palliative and/or end-of-life planning could improve the quality of care.&#x0D; &#x0D; What Is the Technology?&#x0D; &#x0D; An artificial intelligence (AI)–based “nudge” is a decision-making support tool that uses prompts and alerts to aid clinicians in deciding whether and when to discuss end-of-life planning with patients.&#x0D; The nudge sends alerts and/or reminders to clinicians to prompt end-of-life conversations with patients who are at high risk of short-term mortality. These patients are identified by machine learning mortality prediction algorithms incorporated in the electronic health record (EHR) system.&#x0D; Two AI-based nudges designed for patients with cancer were identified. Both tools were developed and internally validated in the US.&#x0D; &#x0D; What Is the Potential Impact?&#x0D; &#x0D; AI-based nudges have the potential to increase the number of end-of-life planning conversations between clinicians and patients as well as the number of referrals to end-of life services.&#x0D; Implementing the nudges into clinical workflows could also help clinicians more easily identify patients with palliative care needs.&#x0D; &#x0D; What Else Do We Need to Know?&#x0D; &#x0D; No AI-based nudges have been approved for use in Canada at the time of this writing nor have there been validation studies using Canadian data.&#x0D; As with many AI algorithms, there is uncertainty about the validity and generalizability of the mortality predictive algorithms used in the nudges.&#x0D; The acceptance of AI-based nudges by clinicians is unclear due to varying clinician attitudes and experiences with nudges and because we did not identify any studies that reported the experience of AI-based nudges from the patient perspective.&#x0D;

Validating racial and ethnic non-bias of artificial intelligence decision support for diagnostic breast ultrasound evaluation

Validating racial and ethnic non-bias of artificial intelligence decision support for diagnostic breast ultrasound evaluation

Exploring the feasibility of an artificial intelligence based clinical decision support system for cutaneous melanoma detection in primary care – a mixed method study

Objective: Skin examination to detect cutaneous melanomas is commonly performed in primary care. In recent years, clinical decision support systems (CDSS) based on artificial intelligence (AI) have been introduced within several diagnostic fields. Setting: This study employs a variety of qualitative and quantitative methodologies to investigate the feasibility of an AI-based CDSS to detect cutaneous melanoma in primary care. Subjects and Design: Fifteen primary care physicians (PCPs) underwent near-live simulations using the CDSS on a simulated patient, and subsequent individual semi-structured interviews were explored with a hybrid thematic analysis approach. Additionally, twenty-five PCPs performed a reader study (diagnostic assessment on the basis of image interpretation) of 18 dermoscopic images, both with and without help from AI, investigating the value of adding AI support to a PCPs decision. Perceived instrument usability was rated on the System Usability Scale (SUS). Results: From the interviews, the importance of trust in the CDSS emerged as a central concern. Scientific evidence supporting sufficient diagnostic accuracy of the CDSS was expressed as an important factor that could increase trust. Access to AI decision support when evaluating dermoscopic images proved valuable as it formally increased the physician’s diagnostic accuracy. A mean SUS score of 84.8, corresponding to ‘good’ usability, was measured. Conclusion: AI-based CDSS might play an important future role in cutaneous melanoma diagnostics, provided sufficient evidence of diagnostic accuracy and usability supporting its trustworthiness among the users.

Artificial Intelligence Decision Support for Triple-Negative Breast Cancers on Ultrasound.

To assess performance of an artificial intelligence (AI) decision support software in assessing and recommending biopsy of triple-negative breast cancers (TNBCs) on US. Retrospective institutional review board-approved review identified patients diagnosed with TNBC after US-guided biopsy between 2009 and 2019. Artificial intelligence output for TNBCs on diagnostic US included lesion features (shape, orientation) and likelihood of malignancy category (benign, probably benign, suspicious, and probably malignant). Artificial intelligence true positive was defined as suspicious or probably malignant and AI false negative (FN) as benign or probably benign. Artificial intelligence and radiologist lesion feature agreement, AI and radiologist sensitivity and FN rate (FNR), and features associated with AI FNs were determined using Wilcoxon rank-sum test, Fisher's exact test, chi-square test of independence, and kappa statistics. The study included 332 patients with 345 TNBCs. Artificial intelligence and radiologists demonstrated moderate agreement for lesion shape and orientation (k = 0.48 and k = 0.47, each P <.001). On the set of examinations using 6 earlier diagnostic US, radiologists recommended biopsy of 339/345 lesions (sensitivity 98.3%, FNR 1.7%), and AI recommended biopsy of 333/345 lesions (sensitivity 96.5%, FNR 3.5%), including 6/6 radiologist FNs. On the set of examinations using immediate prebiopsy diagnostic US, AI recommended biopsy of 331/345 lesions (sensitivity 95.9%, FNR 4.1%). Artificial intelligence FNs were more frequently oval (q < 0.001), parallel (q < 0.001), circumscribed (q = 0.04), and complex cystic and solid (q = 0.006). Artificial intelligence accurately recommended biopsies for 96% to 97% of TNBCs on US and may assist radiologists in classifying these lesions, which often demonstrate benign sonographic features.

Improving Trust and Accountability in AI Systems through Technological Era Advancement for Decision Support in Indonesian Manufacturing Companies

This study explores how technological developments in Artificial Intelligence (AI) decision support systems within Indonesian manufacturing organizations interact with the intricate dynamics of trust, accountability, and technology. The study employed a cross-sectional quantitative research approach to gather responses from a representative sample of professionals spanning different organizational levels, age groups, and functions. The results show that there is a high degree of trust in AI systems, which is largely impacted by dependability and transparency. Strong perceived accountability frameworks encourage prudent decision-making. Technological developments have a big impact on trust and responsibility, especially in Explainable AI and bias prevention. A nuanced interpretation is ensured by the study's demographic analysis, which provides practitioners and policymakers with practical insights to support ethical AI integration in Indonesia's industrial sector.

Prospective Application of an Artificial Intelligence Decision Support Tool for Applicator and Needle Selection in High-Dose-Rate Brachytherapy for Cervical Cancer Patients

To assess the performance of an artificial intelligence (AI)-based decision support tool under differing thresholds to determine optimal applicator selection in a prospective clinical setting. Cervical cancer patients scheduled for high-dose-rate brachytherapy implants with intracavitary tandem and ring (IC) or interstitial tandem and ring (IC/IS) implants in a single tertiary cancer center were eligible for enrollment. Prior to the first brachytherapy implant, a diagnostic MRI was acquired. The clinical target volume and the expected inserted position of the intrauterine tandem and ring were identified and contoured on the T2-MRI. An in-house artificial intelligence-based predicted the need for an IC or IC/IS implant, based on target volume geometric features. For IC/IS implants, an optimal needle arrangement for target coverage was also predicted. Blinded to the AI outcome, a clinical determination was made by the clinician reviewing the MR image. AI algorithm prediction provided a confidence level associated with each decision. The algorithm performance for different confidence thresholds using the IC applicator of 50%, 60%, and 80% was investigated. Performance metrics of the initial clinical determination and the AI prediction were calculated based on the consensus optimal applicator determined from an assessment of planning dosimetry in the first fraction and clinical use for the final brachytherapy implants and fractions. The performance metrics were accuracy, precision, and recall. A total of 10 eligible patients were accrued between December 2020 and October 2022. Table 1 shows the initial clinical determination and consensus applicator AI predictions that were made with different confidence thresholds. The optimal confidence threshold (60%) yielded performance scores of 80%, 83.3%, and 83.3% for the accuracy, precision, and recall, respectively. The performance metrics were equivalent for the optimal confidence threshold and the initial clinical decision. The AI-based decision support tool shows strong predictive results for a clinical brachytherapy application that is important to patient outcome. This prospective study demonstrates that the algorithm's utility is a critical step in using AI-based tools clinically. Further work to determine the optimal brachytherapy applicator, based on treatment planning dosimetry, is required.

Combining uncertainty information with AI recommendations supports calibration with domain knowledge

The use of Artificial Intelligence (AI) decision support is increasing in high-stakes contexts, such as healthcare, defense, and finance. Uncertainty information may help users better leverage AI predictions, especially when combined with their domain knowledge. We conducted a human-subject experiment with an online sample to examine the effects of presenting uncertainty information with AI recommendations. The experimental stimuli and task, which included identifying plant and animal images, are from an existing image recognition deep learning model, a popular approach to AI. The uncertainty information was predicted probabilities for whether each label was the true label. This information was presented numerically and visually. In the study, we tested the effect of AI recommendations in a within-subject comparison and uncertainty information in a between-subject comparison. The results suggest that AI recommendations increased both participants’ accuracy and confidence. Further, providing uncertainty information significantly increased accuracy but not confidence, suggesting that it may be effective for reducing overconfidence. In this task, participants tended to have higher domain knowledge for animals than plants based on a self-reported measure of domain knowledge. Participants with more domain knowledge were appropriately less confident when uncertainty information was provided. This suggests that people use AI and uncertainty information differently, such as an expert versus second opinion, depending on their level of domain knowledge. These results suggest that if presented appropriately, uncertainty information can potentially decrease overconfidence that is induced by using AI recommendations.

When Do Humans Heed AI Agents' Advice? When Should They?

We manipulate the presence, skill, and display of artificial intelligence (AI) recommendations in a strategy game to measure their effect on users' performance. Many applications of AI require humans and AI agents to make decisions collaboratively. Success depends on how appropriately humans rely on the AI agent. We demonstrate an evaluation method for a platform that uses neural network agents of varying skill levels for the simple strategic game of Connect Four. We report results from a 2 × 3 between-subjects factorial experiment that varies the format of AI recommendations (categorical or probabilistic) and the AI agent's amount of training (low, medium, or high). On each round of 10 games, participants proposed a move, saw the AI agent's recommendations, and then moved. Participants' performance improved with a highly skilled agent, but quickly plateaued, as they relied uncritically on the agent. Participants relied too little on lower skilled agents. The display format had no effect on users' skill or choices. The value of these AI agents depended on their skill level and users' ability to extract lessons from their advice. Organizations employing AI decision support systems must consider behavioral aspects of the human-agent team. We demonstrate an approach to evaluating competing designs and assessing their performance.

Retracted: Research on the Difficulties and Countermeasures of the Practical Teaching of Ideological and Political Theory Courses in Colleges and Universities Based on Wireless Communication and Artificial Intelligence Decision Support

Retracted: Research on the Difficulties and Countermeasures of the Practical Teaching of Ideological and Political Theory Courses in Colleges and Universities Based on Wireless Communication and Artificial Intelligence Decision Support

Spirometry services in England post-pandemic and the potential role of AI support software: a qualitative study of challenges and opportunities.

Spirometry services to diagnose and monitor lung disease in primary care were identified as a priority in the NHS Long Term Plan, and are restarting post-COVID-19 pandemic in England; however, evidence regarding best practice is limited. To explore perspectives on spirometry provision in primary care, and the potential for artificial intelligence (AI) decision support software to aid quality and interpretation. Semi-structured interviews with stakeholders in spirometry services across England. Participants were recruited by snowball sampling. Interviews explored the pre- pandemic delivery of spirometry, restarting of services, and perceptions of the role of AI. Transcripts were analysed thematically. In total, 28 participants (mean years' clinical experience = 21.6 [standard deviation 9.4, range 3-40]) were interviewed between April and June 2022. Participants included clinicians (n = 25) and commissioners (n = 3); eight held regional and/or national respiratory network advisory roles. Four themes were identified: 1) historical challenges in provision of spirometry services; 2) inequity in post- pandemic spirometry provision and challenges to restarting spirometry in primary care; 3) future delivery closer to patients' homes by appropriately trained staff; and 4) the potential for AI to have supportive roles in spirometry. Stakeholders highlighted historic challenges and the damaging effects of the pandemic contributing to inequity in provision of spirometry, which must be addressed. Overall, stakeholders were positive about the potential of AI to support clinicians in quality assessment and interpretation of spirometry. However, it was evident that validation of the software must be sufficiently robust for clinicians and healthcare commissioners to have trust in the process.