Risk Of AIS Research Articles

Potential benefits and risks of advanced AI in healthcare

Potential benefits and risks of advanced AI in healthcare

Resistance to medical artificial intelligence: Integrating AI awareness, AI risks, and displacement of responsibility

Resistance to medical artificial intelligence: Integrating AI awareness, AI risks, and displacement of responsibility

Atlas of AI Risks: Enhancing Public Understanding of AI Risks

The prevailing methodologies for visualizing AI risks have focused on technical issues such as data biases and model inaccuracies, often overlooking broader societal risks like job loss and surveillance. Moreover, these visualizations are typically designed for tech-savvy individuals, neglecting those with limited technical skills. To address these challenges, we propose the Atlas of AI Risks—a narrative-style tool designed to map the broad risks associated with various AI technologies in a way that is understandable to non-technical individuals as well. To both develop and evaluate this tool, we conducted two crowdsourcing studies. The first, involving 40 participants, identified the design requirements for visualizing AI risks for decision-making and guided the development of the Atlas. The second study, with 140 participants reflecting the US population in terms of age, sex, and ethnicity, assessed the usability and aesthetics of the Atlas to ensure it met those requirements. Using facial recognition technology as a case study, we found that the Atlas is more user-friendly than a baseline visualization, with a more classic and expressive aesthetic, and is more effective in presenting a balanced assessment of the risks and benefits of facial recognition. Finally, we discuss how our design choices make the Atlas adaptable for broader use, allowing it to generalize across the diverse range of technology applications represented in a database that reports various AI incidents.

Exploring the impact of generative AI technologies on education: Academic expert perspectives, trends, and implications for sustainable development goals

This study provides empirical data on the impact of generative AI in education, with special emphasis on sustainable development goals (SDGs). By conducting a thorough analysis of the relationship between generative AI technologies and educational outcomes, this research fills a critical gap in the literature. The insights offered are valuable for policymakers seeking to leverage new educational technologies to support sustainable development. Using Smart-PLS4, five hypotheses derived from the research questions were tested based on data collected from an E-Questionnaire distributed to academic faculty members and education managers. Of the 311 valid responses, the measurement model assessment confirmed the validity and reliability of the data, while the structural model assessment validated the hypotheses. The study’s findings reveal that New Approaches to Learning Outcome Assessment (NALOA) significantly contribute to achieving SDGs, with a path coefficient of 0.477 (p < 0.001). Similarly, the Use of Generative AI Technologies (UGAIT) has a notable positive impact on SDGs, with a value of 0.221 (p < 0.001). A Paradigm Shift in Education and Educational Process Organization (PSEPQ) also demonstrates a significant, though smaller, effect on SDGs with a coefficient of 0.142 (p = 0.008). However, the Opportunities and Risks of Generative AI in Education (ORGIE) study did not find statistically significant evidence of an impact on SDGs (p = 0.390). These findings highlight the potential opportunities and challenges of using generative AI technologies in education and underscore their key role in advancing sustainable development goals. The study also offers a strategic roadmap for educational institutions, particularly in Oman to harness AI technology in support of sustainable development objectives.

The need for a risk management framework for data science projects: a systematic literature review

Many data science endeavors encounter failure, surfacing at any project phase. Even after successful deployments, data science projects grapple with ethical dilemmas, such as bias and discrimination. Current project management methodologies prioritize efficiency and cost savings over risk management. The methodologies largely overlook the diverse risks of sociotechnical systems and risk articulation inherent in data science lifecycles. Conversely, while the established risk management framework (RMF) by NIST and McKinsey aims to manage AI risks, there is a heavy reliance on normative definitions of risk, neglecting the multifaceted subjectivities of data science project failures. This paper reports on a systematic literature review that identifies three main themes: Big Data Execution Issues, Demand for a Risk Management Framework tailored for Large-Scale Data Science Projects, and the need for a General Risk Management Framework for all Data Science Endeavors. Another overarching focus is on how risk is articulated by the institution and the practitioners. The paper discusses a novel and adaptive data science risk management framework – “DS EthiCo RMF” – which merges project management, ethics, and risk management for diverse data science projects into one holistic framework. This agile risk management framework DS EthiCo RMF can bridge the current divide between normative risk standards and the multitude of data science requirements, offering a human-centric method to navigate the intertwined sociotechnical risks of failure in data science projects.

CardioRiskNet: A Hybrid AI-Based Model for Explainable Risk Prediction and Prognosis in Cardiovascular Disease.

The global prevalence of cardiovascular diseases (CVDs) as a leading cause of death highlights the imperative need for refined risk assessment and prognostication methods. The traditional approaches, including the Framingham Risk Score, blood tests, imaging techniques, and clinical assessments, although widely utilized, are hindered by limitations such as a lack of precision, the reliance on static risk variables, and the inability to adapt to new patient data, thereby necessitating the exploration of alternative strategies. In response, this study introduces CardioRiskNet, a hybrid AI-based model designed to transcend these limitations. The proposed CardioRiskNet consists of seven parts: data preprocessing, feature selection and encoding, eXplainable AI (XAI) integration, active learning, attention mechanisms, risk prediction and prognosis, evaluation and validation, and deployment and integration. At first, the patient data are preprocessed by cleaning the data, handling the missing values, applying a normalization process, and extracting the features. Next, the most informative features are selected and the categorical variables are converted into a numerical form. Distinctively, CardioRiskNet employs active learning to iteratively select informative samples, enhancing its learning efficacy, while its attention mechanism dynamically focuses on the relevant features for precise risk prediction. Additionally, the integration of XAI facilitates interpretability and transparency in the decision-making processes. According to the experimental results, CardioRiskNet demonstrates superior performance in terms of accuracy, sensitivity, specificity, and F1-Score, with values of 98.7%, 98.7%, 99%, and 98.7%, respectively. These findings show that CardioRiskNet can accurately assess and prognosticate the CVD risk, demonstrating the power of active learning and AI to surpass the conventional methods. Thus, CardioRiskNet's novel approach and high performance advance the management of CVDs and provide healthcare professionals a powerful tool for patient care.

Data Scientists Discuss AI Risks and Opportunities

Data Scientists Discuss AI Risks and Opportunities

AI Effectiveness and Risk Assessment of Investments in High-Risk Start-Ups

Which business idea or project would be successful? Is it worth investing in it and to what extent? What is the risk level and how can it be mitigated to achieve success? All these questions are relevant to both entrepreneurs and investors, especially when it comes to high-risk start-ups. The purpose of this paper is to present a methodology for intelligent effectiveness and risk assessment (IERA) of investments in high-risk start-ups, with a focus on space industries. Such an assessment is challenging for several reasons, including the lack of unified statistics on space industries, evaluation of benefits other than economic ones, long-term development challenges, etc. The developed IERA methodology integrates a combination of various known methods used in the space industries for both investment appraisal and risk assessment. The application of various AI-based tools for methodology criteria, weights, and scores contributes to obtaining realistic values and ensures the success of the analysis results, thereby benefiting the project itself. Thus, the applied IERA methodology can be implemented in real-time, based on a broad knowledge base, with high accuracy, and requiring significantly fewer financial resources. The main advantage of the developed methodology is the assurance it provides to both entrepreneurs and investors, offering them sufficient certainty and confidence.

Public Relations and Issues of Technological Progress: Communicating the Benefits and Risks of AI and IoT to the Public

Public Relations and Issues of Technological Progress: Communicating the Benefits and Risks of AI and IoT to the Public

Navigating the Risks of Shadow AI

Abstract Dr T. W. Kwan FBCS explores the challenges associated with shadow AI and identifies the strategies that, when implemented, will help business leaders prepare for an increase in shadow usage.

Managing extreme AI risks amid rapid progress.

Preparation requires technical research and development, as well as adaptive, proactive governance.

The Necessity of Machine Learning Theory in Mitigating AI Risk

Highlights SUMMARY In the last years we have witnessed rapidly accelerating progress in Neural Network-based Artificial Intelligence. Yet our fundamental understanding of these methods has lagged far behind. Never before had a technology been developed so quickly and deployed so widely with so little understanding of its basic principles. In this document we argue that deep learning systems with their human-like complexity cannot be controlled and guided in socially acceptable ways or countered in adversarial situations, without an underlying theory. Theory in this context refers to identifying precise measurable quantities and mathematically describing their patterns, the way it is done in physics and engineering, rather than necessarily proving rigorous theorems. Given the societal impact of rapidly proliferating AI, developing such a theory and applying its analyses to real systems is a compelling and urgent need.

Challenges and efforts in managing AI trustworthiness risks: a state of knowledge.

This paper addresses the critical gaps in existing AI risk management frameworks, emphasizing the neglect of human factors and the absence of metrics for socially related or human threats. Drawing from insights provided by NIST AI RFM and ENISA, the research underscores the need for understanding the limitations of human-AI interaction and the development of ethical and social measurements. The paper explores various dimensions of trustworthiness, covering legislation, AI cyber threat intelligence, and characteristics of AI adversaries. It delves into technical threats and vulnerabilities, including data access, poisoning, and backdoors, highlighting the importance of collaboration between cybersecurity engineers, AI experts, and social-psychology-behavior-ethics professionals. Furthermore, the socio-psychological threats associated with AI integration into society are examined, addressing issues such as bias, misinformation, and privacy erosion. The manuscript proposes a comprehensive approach to AI trustworthiness, combining technical and social mitigation measures, standards, and ongoing research initiatives. Additionally, it introduces innovative defense strategies, such as cyber-social exercises, digital clones, and conversational agents, to enhance understanding of adversary profiles and fortify AI security. The paper concludes with a call for interdisciplinary collaboration, awareness campaigns, and continuous research efforts to create a robust and resilient AI ecosystem aligned with ethical standards and societal expectations.

Abstract PS10-08: Validation of a clinical image-based AI-risk model for individualized breast cancer screening in a multi-national setting

Abstract Background: Image-derived AI risk models for breast cancer have shown high discriminatory performances compared with clinical risk models based on family history and lifestyle factors. However, little is known about their generalizability across different screening settings and clinical feasibility. Purpose: To investigate the predictive performance of a clinically used image-derived AI-based breast cancer risk model in multiple European screening populations. Methods: Four European mammographic screening populations in three countries screened between 2009-2020 for women aged 45-69 was used to perform a nested case-control study. In total, 739 women with incident breast cancers were included together with 7,812 controls matched to cases on year of study-entry. Mammographic features (density, microcalcifications, masses, left-right breast asymmetries of these features) for risk assessment were extracted using AI from full-field digital mammograms. Breast cancer occurrence was assessed after two years of follow-up. Absolute risks of breast cancer were predicted using the risk model from negative mammograms at study-entry. Adjusted Area Under the receiver operating characteristic Curves (aAUC) estimated discriminatory performance and, adjusted risk-ratios estimated the stratification performance of women at high/general risk per the clinical guidelines. Results: The overall aAUC of the AI risk model was 0.72 (95%CI 0.70-0.75), range 0.71 (95%CI 0.67-0.75) to 0.74 (95%CI 0.69-0.78) for breast cancers developed in four screening populations. In the 4.6% of women classified at high risk using the NICE guidelines thresholds, cancers were more likely diagnosed after 2 years follow-up, risk-ratio (RR) 6.7 (95%CI 5.6-8.0), compared with the 71% of women classified at general risk by the model. Similar risk-ratios were observed across tertiles of mammographic density. In the high-risk group, 22% of the 2-year future cancers were diagnosed, and 29% of stage 2 and higher cancers, p< 0.01. Conclusions: The AI risk model showed generalizable discriminatory performances across European populations and, captured ~30% of clinically relevant stage 2 and higher breast cancers in ~5% of high-risk women who were sent home with a negative mammogram. Similar results were seen in fatty and dense breasts. An image-derived AI model is feasible for personalized screening to improve screening outcomes. Citation Format: Mikael Eriksson, Marta Román, Axel Gräwingholt, Sylvia Heywang-Köbrunner, Paolo Rossi, Per Hall. Validation of a clinical image-based AI-risk model for individualized breast cancer screening in a multi-national setting [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PS10-08.

Abstract PO4-07-03: Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study

Abstract Background and Aim Some studies have reported the use of mammogram AI deep learning algorithms to accurately predict the risk of future breast cancer development in women. In these studies, random and unrelated mammograms were independently used for training of the AI model. We propose a novel deep learning system using sequential past mammograms instead of random studies and aimed to determine if it can further improve risk prediction in a pilot study. Methods Pre-training of the backbone of a Swin-transformer machine learning model was performed by applying region-of-interest lesional masks of benign and malignant on a training set of 1644 mammograms derived from the open-access Curated Breast Imaging Subset of Digital Database for Screening Mammography dataset. To further develop the model, deep learning training and validation were performed on the Cohort of Screen-age Women (CSAW) open-access screening mammogram set which was derived from the long term data of 873 women who developed breast cancer for the first time and 7850 non-cancer women. We then compared the diagnostic accuracy of a model using sequential past mammograms for training against the standard, non-sequential model. For the sequential model, imaging segmentation of the potential cancer site on mammography was performed on all past, pre-cancer mammograms obtained within 60 to 2555 days of cancer diagnosis in the pre-processing stage for breast cancer cases. AI training was then carried out by combining the pre-cancer mammograms belonging to the same woman in a temporal sequence using a transformer encoder. The learning rate started from 1e-6 and decayed 10 times every 10 epochs. For the non-sequential model, we removed the temporal encoder and used random, unrelated mammogram studies for training. Validation was performed on a dataset consisting of 75 pre-cancer mammograms from breast cancer cases and 69 mammograms from non-cancer cases. Results Sensitivity, specificity and overall accuracy on the validation set were 77.3% (58/75), 68.1% (47/69) and (105/144) 72.9% respectively for the sequential model compared to 69.3% (52/75), 69.6% (48/69) and 69.4% (100/144) respectively for the non-sequential model. Conclusion The study showed that deep learning breast cancer risk prediction can be further improved by using sequential past mammograms instead of random mammograms for AI model training. This can also potentially enhance other AI risk prediction models that employ combined mammogram and traditional breast cancer clinical risk factors. A larger validation study will be useful. Contact information: email Lester Leong at lester.leong.c.h@singhealth.com.sg ROC curve for breast cancer risk prediction using AI deep learning training with sequential past mammograms. Citation Format: Lester Leong, Mingjie Xu, Weimin Huang, Erli Zhang, Engracia Loh, Sze Yiun Teo, Geok Hoon Lim, Veronique Kiak Mien Tan, Yirong Sim, Fredrik Strand, Ryan Tan. Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO4-07-03.

Towards algorithm auditing: managing legal, ethical and technological risks of AI, ML and associated algorithms.

-Business reliance on algorithms is becoming ubiquitous, and companies are increasingly concerned about their algorithms causing major financial or reputational damage. High-profile cases include Google's AI algorithm for photo classification mistakenly labelling a black couple as gorillas in 2015 (Gebru 2020 In The Oxford handbook of ethics of AI, pp. 251-269), Microsoft's AI chatbot Tay that spread racist, sexist and antisemitic speech on Twitter (now X) (Wolf et al. 2017 ACM Sigcas Comput. Soc. 47, 54-64 (doi:10.1145/3144592.3144598)), and Amazon's AI recruiting tool being scrapped after showing bias against women. In response, governments are legislating and imposing bans, regulators fining companies and the judiciary discussing potentially making algorithms artificial 'persons' in law. As with financial audits, governments, business and society will require algorithm audits; formal assurance that algorithms are legal, ethical and safe. A new industry is envisaged: Auditing and Assurance of Algorithms (cf. data privacy), with the remit to professionalize and industrialize AI, ML and associated algorithms. The stakeholders range from those working on policy/regulation to industry practitioners and developers. We also anticipate the nature and scope of the auditing levels and framework presented will inform those interested in systems of governance and compliance with regulation/standards. Our goal in this article is to survey the key areas necessary to perform auditing and assurance and instigate the debate in this novel area of research and practice.

Assessment of AI Risk Scores on Screening Mammograms Preceding Breast Cancer Diagnosis.

Assessment of AI Risk Scores on Screening Mammograms Preceding Breast Cancer Diagnosis.

Government Interventions to Avert Future Catastrophic AI Risks

Government Interventions to Avert Future Catastrophic AI Risks

Sticks and stones may break my bones, but words will never hurt me!—Navigating the cybersecurity risks of generative AI

Sticks and stones may break my bones, but words will never hurt me!—Navigating the cybersecurity risks of generative AI

Evaluating approaches for reducing catastrophic risks from AI

AbstractAccording to a growing number of researchers, AI may pose catastrophic – or even existential – risks to humanity. Catastrophic risks may be taken to be risks of 100 million human deaths, or a similarly bad outcome. I argue that such risks – while contested – are sufficiently likely to demand rigorous discussion of potential societal responses. Subsequently, I propose four desiderata for approaches to the reduction of catastrophic risks from AI. The quality of such approaches can be assessed by their chance of success, degree of beneficence, degree of non-maleficence, and beneficent side effects. Then, I employ these desiderata to evaluate the promises, limitations and risks of alignment research, timelines research, policy research, halting or slowing down AI research, and compute governance for tackling catastrophic AI risks. While more research is needed, this investigation shows that several approaches for dealing with catastrophic AI risks are available, and where their respective strengths and weaknesses lie. It turns out that many approaches are complementary and that the approaches have a nuanced relationship to approaches to present AI harms. While some approaches are similarly useful for addressing catastrophic risks and present harms, this is not always the case.