Transparency In AI Research Articles

Ensuring AI explainability in healthcare: problems and possible policy solutions

ABSTRACT AI promises to address health services’ quality and cost challenges, however, errors and bias in medical devices decisions pose threats to human health and life. This has also led to the lack of trust in AI medical devices among clinicians and patients. The goal of this article is to assess whether AI explainability principle established in numerous ethical AI frameworks can help address these and other challenges posed by AI medical devices. We first define the AI explainability principle, delineate it from the AI transparency principle, and examine which stakeholders in healthcare sector would need AI to be explainable and for what purpose. Second, we analyze whether explainable AI in healthcare is capable of achieving its intended goals. Finally, we examine robust regulatory approval framework as an alternative – and a more suitable – way in addressing challenges caused by black-box AI.

AI Transparency in a Real-World Context: What we can learn from past examples of algorithmic and statistical decision-making

Public discussion about transparency for AI-enabled decisions tends to focus on the challenge of AI explainability. However, there are additional real-world factors which can hamper individuals seeking to understand or challenge decisions impacting them, even when the AI or algorithm is entirely explainable. Although AI enabled decision tools are relatively new, algorithmic and statistical decision tools are not. This paper examines past efforts by individuals to access algorithms, statistical models, and data used in making decisions which impacted them. The results of those attempts are considered in light of public expectations for transparency of AIenabled decision tools, as well as current and developing guidance. Legal changes will be needed if governments wish to meet citizen expectations for real-world transparency of AI-enabled decision systems. In the meantime, there are opportunities for AI experts and others to protect the potential for greater transparency through open data, open source licensing, and engagement in policy development

A study on the adoption challenges and solutions for transforming healthcare with generative AI

The use of generative AI in healthcare has the ability to completely transform the way patients are diagnosed, treated, and receive care. Concerning data protection, cooperation, ethical issues, user interface design, and training requirements, this research analyses the primary obstacles to the implementation of generative AI in healthcare. The findings provide important insights into healthcare workers' opinions and preparedness for AI integration through a mixed-methods approach that includes quantitative surveys measuring user experiences and qualitative interviews with healthcare professionals. Results indicate high confidence in data security measures but highlight the necessity for enhanced interdisciplinary collaboration and transparency in AI decision-making processes. Additionally, while user satisfaction with AI interfaces is generally positive, there remains a need for improved training programs to empower healthcare professionals effectively. This study highlights the need of tackling these obstacles to guarantee the effective deployment of generative AI technologies, which seek to improve diagnostic precision, simplify processes, and heighten patient involvement in their treatment.

Public sector AI transparency standard: UK Government seeks to lead by example

In releasing the Algorithmic Transparency Standard, the UK government has reiterated its commitment to greater algorithmic transparency in the public sector. The Standard signals that the UK government is both pushing forward with the AI standards agenda and ensuring that those standards benefit from empirical practitioner-led experience, enabling coherent, widespread adoption. The two-tier approach of the Algorithmic Transparency Standard encourages transparency inclusivity across distinct audiences, facilitating trust across algorithm stakeholders. Moreover, it can be understood that implementation of the Standard within the UK’s public sector will inform standards more widely, influencing best practice in the private sector. This article provides a summary and commentary of the text.

Causation in Civil Law and the Problems of Transparency in AI

Although a number of European Union legal documents require AI to be transparent, this is frequently not the case. Such opacity can be attributed to a diverse range of causes, ranging from so called ‘black box’ algorithms, trade secrets and patents, to the cognitive barriers of non-professionals. Transparency, in contrast, is, in many cases, indispensable for establishing generic or specific causation in deciding civil law cases, especially within the scope of contractual or tort liability. Therefore, the concept of causation applied in various parts of civil law should be rethought, and some remedies for this problem should be adopted.

Enhancing AI Transparency: Innovative Methods to Explain Complex AI Decisions to Non-Experts

Enhancing AI Transparency: Innovative Methods to Explain Complex AI Decisions to Non-Experts

Propagation of representation bias in machine learning

Transfer learning is an emerging paradigm in machine learning that involves reusing existing pretrained models to develop new machine learning applications. As machine learning grows in importance, keeping new models and applications clear of biases is of paramount importance. While previous works focus on the development of bias free models, they fail to address the mechanism of bias propagation in transfer learning. Using facial recognition as a use-case scenario, we attempt to identify sources of bias in a model developed using transfer learning. To achieve this task, we developed a model based on a pre-trained facial recognition model, and scrutinized the accuracy of the model’s image classification against factors such as age, gender, and race to observe whether or not the model performed better on some demographic groups than others. By identifying the bias and finding potential sources of bias, his work contributes a unique technical perspective from the view of a small scale developer to emerging discussions of accountability and transparency in AI.

AI transparency: a matter of reconciling design with critique

In the late 2010s, various international committees, expert groups, and national strategy boards have voiced the demand to ‘open’ the algorithmic black box, to audit, expound, and demystify artificial intelligence. The opening of the algorithmic black box, however, cannot be seen only as an engineering challenge. In this article, I argue that only the sort of transparency that arises from critique—a method of theoretical examination that, by revealing pre-existing power structures, aims to challenge them—can help us produce technological systems that are less deceptive and more just. I relate the question of AI transparency to the broader challenge of responsible making, contending that future action must aim to systematically reconcile design—as a way of concealing—with critique—as a manner of revealing.

Artificial Intelligence and Transparency: A Blueprint for Improving the Regulation of AI Applications in the EU

The adoption of Artificial Intelligence is steadily increasing, but the underlying algorithms have become so complex that they are no longer transparent. The EU has introduced some modest AI transparency requirements as part of its General Data Protection Regulation. However, two years after their introduction, the effectiveness of these rules remains questionable. Our aim is to contribute towards the further development of a governance framework for AI. We begin by explaining how the algorithms that enable the speed and data processing power of AI also obscure its transparency. We review how major guidelines on AI ethics operationalize algorithmic transparency, following which we assess whether these principles are adequately covered by the GDPR. We then present the results of semi-structured interviews of a heterogeneous sample of stakeholders of consumer information online (N=75). Our data provide evidence that the current implementation of the EU’s informed consumer paradigm fails to establish a satisfactory level of consumer protection and information online. If simple technological applications such as cookies remain non-transparent to consumers, the current approaches are entirely incapable of addressing the problem of complex AI applications. We conclude by formulating a policy proposal as to how the transparency of AI applications could be improved from the perspective of end users. artificial intelligence, explainable AI, automated individual decision-making, profiling, consumer protection, data protection, GDPR, information disclosures, transparency, AI ethics

Patterns and Antipatterns, Principles, and Pitfalls: Accountability and Transparency in Artificial Intelligence

This article discusses a set of principles for accountability and transparency in AI as well as a set of antipatterns or harmful trends too often seen in deployed systems. It provides concrete suggestions for what can be done to shift the balance away from these antipatterns and toward more positive ones.

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.

Instinct: A biologically inspired reactive planner for intelligent embedded systems

The Instinct Planner is a new biologically inspired reactive planner, based on an established behaviour based robotics methodology and its reactive planner component — the POSH planner implementation. It includes several significant enhancements that facilitate plan design and runtime debugging. It has been specifically designed for low power processors and has a tiny memory footprint. Written in C++, it runs efficiently on both Arduino (Atmel AVR) and Microsoft VC++ environments and has been deployed within a low cost maker robot to study AI Transparency. Plans may be authored using a variety of tools including a new visual design language, currently implemented using the Dia drawing package.

Designing and implementing transparency for real time inspection of autonomous robots

ABSTRACTThe EPSRC's Principles of Robotics advises the implementation of transparency in robotic systems, however research related to AI transparency is in its infancy. This paper introduces the reader of the importance of having transparent inspection of intelligent agents and provides guidance for good practice when developing such agents. By considering and expanding upon other prominent definitions found in literature, we provide a robust definition of transparency as a mechanism to expose the decision-making of a robot. The paper continues by addressing potential design decisions developers need to consider when designing and developing transparent systems. Finally, we describe our new interactive intelligence editor, designed to visualise, develop and debug real-time intelligence.