Epistemic Opacity Research Articles

Is Epistemic Autonomy Technologically Possible Within Social Media? A Socio-Epistemological Investigation of the Epistemic Opacity of Social Media Platforms

AbstractThis article aims to provide a coherent and comprehensive theoretical framework of the main socio-epistemic features of social media. The first part consists of a concise discussion of the main epistemic consequences of personalised information filtering, with a focus on echo chambers and their many different implications. The middle section instead hosts an analytical investigation of the cognitive and epistemic environments of these platforms aimed at establishing whether, and to what extent, they allow their users to be epistemically vigilant with respect to their sources and the content recommended to them. Finally, in the last part, of a more exquisitely normative nature, some strategies are proposed and discussed that, by reducing the epistemic opacity of social media, could contribute to greater epistemic justice within social media and, concurrently, to augmenting the epistemic autonomy of users.

Find the Gap: AI, Responsible Agency and Vulnerability

The responsibility gap, commonly described as a core challenge for the effective governance of, and trust in, AI and autonomous systems (AI/AS), is traditionally associated with a failure of the epistemic and/or the control condition of moral responsibility: the ability to know what we are doing and exercise competent control over this doing. Yet these two conditions are a red herring when it comes to understanding the responsibility challenges presented by AI/AS, since evidence from the cognitive sciences shows that individual humans face very similar responsibility challenges with regard to these two conditions. While the problems of epistemic opacity and attenuated behaviour control are not unique to AI/AS technologies (though they can be exacerbated by them), we show that we can learn important lessons for AI/AS development and governance from how philosophers have recently revised the traditional concept of moral responsibility in response to these challenges to responsible human agency from the cognitive sciences. The resulting instrumentalist views of responsibility, which emphasize the forward-looking and flexible role of agency cultivation, hold considerable promise for integrating AI/AS into a healthy moral ecology. We note that there nevertheless is a gap in AI/AS responsibility that has yet to be extensively studied and addressed, one grounded in a relational asymmetry of vulnerability between human agents and sociotechnical systems like AI/AS. In the conclusion of this paper we note that attention to this vulnerability gap must inform and enable future attempts to construct trustworthy AI/AS systems and preserve the conditions for responsible human agency.

Scientific understanding through big data: From ignorance to insights to understanding

Here I argue that scientists can achieve some understanding of both the products of big data implementation as well as of the target phenomenon to which they are expected to refer—even when these products were obtained through essentially epistemically opaque processes. The general aim of the paper is to provide a road map for how this is done; going from the use of big data to epistemic opacity (Sec. 2), from epistemic opacity to ignorance (Sec. 3), from ignorance to insights (Sec. 4), and finally, from insights to understanding (Sec. 5, 6)

Levels of explicability for medical artificial intelligence: What do we normatively need and what can we technically reach?

Definition of the problemThe umbrella term “explicability” refers to the reduction of opacity of artificial intelligence (AI) systems. These efforts are challenging for medical AI applications because higher accuracy often comes at the cost of increased opacity. This entails ethical tensions because physicians and patients desire to trace how results are produced without compromising the performance of AI systems. The centrality of explicability within the informed consent process for medical AI systems compels an ethical reflection on the trade-offs. Which levels of explicability are needed to obtain informed consent when utilizing medical AI?ArgumentsWe proceed in five steps: First, we map the terms commonly associated with explicability as described in the ethics and computer science literature, i.e., disclosure, intelligibility, interpretability, and explainability. Second, we conduct a conceptual analysis of the ethical requirements for explicability when it comes to informed consent. Third, we distinguish hurdles for explicability in terms of epistemic and explanatory opacity. Fourth, this then allows to conclude the level of explicability physicians must reach and what patients can expect. In a final step, we show how the identified levels of explicability can technically be met from the perspective of computer science. Throughout our work, we take diagnostic AI systems in radiology as an example.ConclusionWe determined four levels of explicability that need to be distinguished for ethically defensible informed consent processes and showed how developers of medical AI can technically meet these requirements.

Deep Learning Opacity in Scientific Discovery

AbstractWhile philosophers have focused on epistemological and ethical challenges of using artificial intelligence (AI) in science, scientists have focused largely on opportunities. I argue that this disconnect between philosophical pessimism and scientific optimism is driven by failures to critically examine the practice of AI-infused science. To appreciate the epistemic justification for AI-powered breakthroughs, philosophers must analyze the role of AI as part of a wider process of discovery. I demonstrate the importance of this with two cases from the scientific literature, and show that epistemic opacity need not diminish AI’s capacity to lead scientists to significant and justifiable breakthroughs.

Misplaced Trust and Distrust: How Not to Engage with Medical Artificial Intelligence.

Artificial intelligence (AI) plays a rapidly increasing role in clinical care. Many of these systems, for instance, deep learning-based applications using multilayered Artificial Neural Nets, exhibit epistemic opacity in the sense that they preclude comprehensive human understanding. In consequence, voices from industry, policymakers, and research have suggested trust as an attitude for engaging with clinical AI systems. Yet, in the philosophical and ethical literature on medical AI, the notion of trust remains fiercely debated. Trust skeptics hold that talking about trust in nonhuman agents constitutes a category error and worry about the concept being misused for ethics washing. Proponents of trust have responded to these worries from various angles, disentangling different concepts and aspects of trust in AI, potentially organized in layers or dimensions. Given the substantial disagreements across these accounts of trust and the important worries about ethics washing, we embrace a diverging strategy here. Instead of aiming for a positive definition of the elements and nature of trust in AI, we proceed ex negativo, that is we look at cases where trust or distrust are misplaced. Comparing these instances with trust expedited in doctor-patient relationships, we systematize these instances and propose a taxonomy of both misplaced trust and distrust. By inverting the perspective and focusing on negative examples, we develop an account that provides useful ethical constraints for decisions in clinical as well as regulatory contexts and that highlights how we should not engage with medical AI.

Against explainability requirements for ethical artificial intelligence in health care

It is widely accepted that explainability is a requirement for the ethical use of artificial intelligence (AI) in health care. I challenge this Explainability Imperative (EI) by considering the following question: does the use of epistemically opaque medical AI systems violate existing legal standards for informed consent? If yes, and if the failure to meet such standards can be attributed to epistemic opacity, then explainability is a requirement for AI in healthcare. If not, then based on at least one metric of ethical medical practice (informed consent), explainability is not required for the ethical use of AI in healthcare. First, I show that the use of epistemically opaque AI applications is compatible with meeting accepted legal criteria for informed consent. Second, I argue that human experts are also black boxes with respect to the criteria by which they arrive at a diagnosis. Human experts can nonetheless meet established requirements for informed consent. I conclude that the use of black-box AI systems does not violate patients’ rights to informed consent, and thus, with respect to informed consent, explainability is not required for medical AI.

Analogue Models and Universal Machines. Paradigms of Epistemic Transparency in Artificial Intelligence

The problem of epistemic opacity in Artificial Intelligence (AI) is often characterised as a problem of intransparent algorithms that give rise to intransparent models. However, the degrees of transparency of an AI model should not be taken as an absolute measure of the properties of its algorithms but of the model’s degree of intelligibility to human users. Its epistemically relevant elements are to be specified on various levels above and beyond the computational one. In order to elucidate this claim, I first contrast computer models and their claims to algorithm-based universality with cybernetics-style analogue models and their claims to structural isomorphism between elements of model and target system (in: Black, Models and metaphors, 1962). While analogue models aim at perceptually or conceptually accessible model-target relations, computer models give rise to a specific kind of underdetermination in these relations that needs to be addressed in specific ways. I then undertake a comparison between two contemporary AI approaches that, although related, distinctly align with the above modelling paradigms and represent distinct strategies towards model intelligibility: Deep Neural Networks and Predictive Processing. I conclude that their respective degrees of epistemic transparency primarily depend on the underlying purposes of modelling, not on their computational properties.

Simulation models and probabilities: a Bayesian defense of the value-free ideal

Some philosophers of science have recently argued that the epistemic assessment of complex simulation models, such as climate models, cannot be free of the influence of social values. In their view, the assignment of probabilities to the different hypotheses or predictions that result from simulations presupposes some methodological decisions that rest on value judgments. In this article, I criticize this claim and put forward a Bayesian response to the arguments from inductive risk according to which the influence of social values on the calculation of probabilities is negligible. I conclude that the epistemic opacity of complex simulations, such as climate models, does not preclude the application of Bayesian methods.

Explorations about the Family’s Role in the German Transplantation System: Epistemic Opacity and Discursive Exclusion

ABSTRACT With regard to organ donation, Germany is an ‘opt-in’ country, which requires explicit consent from donors. The relatives are either asked to decide on behalf of the donors’ preferences, if these are unknown or if the potential donor has explicitly transferred the decision to them. At the core of this policy lies the sociocultural and moral premise of a rational, autonomous individual, whose rights require legal protection in order to guarantee a voluntary decision. In concrete transplantation practices, the family plays an even more important role. Potential donors and their families decide while being embedded in relations, a point which does still not gain full recognition. This particular discrepancy between policy and practice creates conflicts, which remain taboos of academic inquiry and public discourse. Our analysis shows a plurality of the family’s role in the transplantation process, which reveals an inner tension of the organ donation system. This tension provokes epistemic opacity on the one hand and different collective strategies as responses to discursive exclusion on the other. In future deliberations about organ donation, it is important to create spaces for open discussion, but also practices of communicative engagement, which take care of the needs and emotions attached to taboos.

Discrimination in the age of artificial intelligence

In this paper, I examine whether the use of artificial intelligence (AI) and automated decision-making (ADM) aggravates issues of discrimination as has been argued by several authors. For this purpose, I first take up the lively philosophical debate on discrimination and present my own definition of the concept. Equipped with this account, I subsequently review some of the recent literature on the use AI/ADM and discrimination. I explain how my account of discrimination helps to understand that the general claim in view of the aggravation of discrimination is unwarranted. Finally, I argue that the use of AI/ADM can, in fact, increase issues of discrimination, but in a different way than most critics assume: it is due to its epistemic opacity that AI/ADM threatens to undermine our moral deliberation which is essential for reaching a common understanding of what should count as discrimination. As a consequence, it turns out that algorithms may actually help to detect hidden forms of discrimination.

Concepts as decision functions. The issue of epistemic opacity of conceptual representations in artificial computing systems

The treatment of concepts as decision functions characteristic for computer science and Artificial Intelligence has its roots in the logical tradition of Gottlob Frege. In its modern incarnation, decision functions, embedded in concrete knowledge representations, describe either crisp or vague assignments of categorization decisions to objects. In order to allow for effective communication between humans and computer systems, these decisions should be either effectively explained by the system or explainable with respect to the system, which in turn depends on the degree of epistemic transparency of the computer system and the conceptual representations it uses. In this, paper we distinguish between two basic types of representation: logic-based and nature-based. With respect to the latter, we identify five contributing factors to their relative epistemic opacity: A. structural complexity, B. learning procedures, C. operating on uncertain data, D. lack of translation procedures for low-level rules and representations, and E. lack of complete knowledge about natural computing systems. We consider the fourth factor (D.) to be the most important source of opacity on the level of computer systems and the fifth (E.) the main contributing factor on the level of human knowledge.

Validating the Universe in a Box

Computer simulations of the formation and evolution of large-scale structure in the universe are integral to the enterprise of modern cosmology. Establishing the reliability of these simulations has been extremely challenging, primarily because of epistemic opacity. In this setting, robustness analysis defined by requiring converging outputs from a diverse ensemble of simulations is insufficient to determine simulation validity. We propose an alternative path of structured code validation that applies eliminative reasoning to isolate and reduce possible sources of error, a potential path that is already being explored by some cosmologists.

Models, Parameterization, and Software: Epistemic Opacity in Computational Chemistry

Computational chemistry grew in a new era of “desktop modeling,” which coincided with a growing demand for modeling software, especially from the pharmaceutical industry. Parameterization of models in computational chemistry is an arduous enterprise, and we argue that this activity leads, in this specific context, to tensions among scientists regarding the epistemic opacity transparency of parameterized methods and the software implementing them. We relate one flame war from the Computational Chemistry mailing List in order to assess in detail the relationships between modeling methods, parameterization, software and the various forms of their enclosure or disclosure. Our claim is that parameterization issues are an important and often neglected source of epistemic opacity and that this opacity is entangled in methods and software alike. Models and software must be addressed together to understand the epistemological tensions at stake.

The epistemic opacity of autonomous systems and the ethical consequences

This paper takes stock of all the various factors that cause the design-time opacity of autonomous systems behaviour. The factors include embodiment effects, design-time knowledge gap, human factors, emergent behaviour and tacit knowledge. This situation is contrasted with the usual representation of moral dilemmas that assume perfect information. Since perfect information is not achievable, the traditional moral dilemma representations are not valid and the whole problem of ethical autonomous systems design proves to be way more empirical than previously understood.

Chess, Artificial Intelligence, and Epistemic Opacity

In 2017 AlphaZero, a neural network-based chess engine shook the chess world by convincingly beating Stockfish, the highest-rated chess engine. In this paper, I describe the technical differences between the two chess engines and based on that, I discuss the impact of the modeling choices on the respective epistemic opacities. I argue that the success of AlphaZero’s approach with neural networks and reinforcement learning is counterbalanced by an increase in the epistemic opacity of the resulting model.

A Puzzle concerning Compositionality in Machines

This paper attempts to describe and address a specific puzzle related to compositionality in artificial networks such as Deep Neural Networks and machine learning in general. The puzzle identified here touches on a larger debate in Artificial Intelligence related to epistemic opacity but specifically focuses on computational applications of human level linguistic abilities or properties and a special difficulty with relation to these. Thus, the resulting issue is both general and unique. A partial solution is suggested.

Your evidence? Machine learning algorithms for medical diagnosis and prediction.

Computer systems for medical diagnosis based on machine learning are not mere science fiction. Despite undisputed potential benefits, such systems may also raise problems. Two (interconnected) issues are particularly significant from an ethical point of view: The first issue is that epistemic opacity is at odds with a common desire for understanding and potentially undermines information rights. The second (related) issue concerns the assignment of responsibility in cases of failure. The core of the two issues seems to be that understanding and responsibility are concepts that are intrinsically tied to the discursive practice of giving and asking for reasons. The challenge is to find ways to make the outcomes of machine learning algorithms compatible with our discursive practice. This comes down to the claim that we should try to integrate discursive elements into machine learning algorithms. Under the title of “explainable AI” initiatives heading in this direction are already under way. Extensive research in this field is needed for finding adequate solutions.

Managing Algorithmic Accountability: Balancing Reputational Concerns, Engagement Strategies, and the Potential of Rational Discourse

While organizations today make extensive use of complex algorithms, the notion of algorithmic accountability remains an elusive ideal due to the opacity and fluidity of algorithms. In this article, we develop a framework for managing algorithmic accountability that highlights three interrelated dimensions: reputational concerns, engagement strategies, and discourse principles. The framework clarifies (a) that accountability processes for algorithms are driven by reputational concerns about the epistemic setup, opacity, and outcomes of algorithms; (b) that the way in which organizations practically engage with emergent expectations about algorithms may be manipulative, adaptive, or moral; and (c) that when accountability relationships are heavily burdened by the opacity and fluidity of complex algorithmic systems, the emphasis of engagement should shift to a rational communication process through which a continuous and tentative assessment of the development, workings, and consequences of algorithms can be achieved over time. The degree to which such engagement is, in fact, rational can be assessed based on four discourse-ethical principles of participation, comprehension, multivocality, and responsiveness. We conclude that the framework may help organizations and their environments to jointly work toward greater accountability for complex algorithms. It may further help organizations in reputational positioning surrounding accountability issues. The discourse-ethical principles introduced in this article are meant to elevate these positioning contests to extend beyond mere adaption or compliance and help guide organizations to find moral and forward-looking solutions to accountability issues.

Structural Injustice, Epistemic Opacity, and the Responsibilities of the Oppressed

Structural Injustice, Epistemic Opacity, and the Responsibilities of the Oppressed