Lethal Autonomous Weapons Research Articles

The “Trolley Problem” in Fully Automated AI-Driven Media: A Challenge Beyond Autonomous Driving

ABSTRACT The rapid progress of artificial intelligence (AI) has resulted in its integration into various stages of the media process, including information gathering, processing, and distribution. This integration has raised the possibility of AI dominating the media industry, leading to an era of “autonomous driving” within AI-driven media systems. Similar to the ethical dilemma known as the “trolley problem” (TP) in autonomous driving, a comparable problem arises in AI automated media. This study examines the emergence of the new TP in fully automated AI-driven media (FAAIM), recognizing the complexity of this problem given the nature of media content and its societal impact. To address this complexity, we propose the adoption of the theory of “meaningful human control,” originally developed to address responsibility in the governance of lethal autonomous weapons systems, as a framework for governing FAAIM. By ensuring that humans can be held responsible for the operations of FAAIM, this paper aims to proactively confront the ethical challenges arising from FAAIM, identify potential solutions, and guide future research in media ethics.

Operating itself safely: merging the concepts of ‘safe to operate’ and ‘operate safely’ for lethal autonomous weapons systems containing artificial intelligence

ABSTRACT The Ministry of Defence, specifically the Royal Navy, uses the ‘Duty Holder Structure’ to manage how it complies with deviations to maritime laws and health and safety regulations where military necessity requires it. The output statements ensuring compliance are ‘safe to operate’ certification for all platforms and equipment, and the ‘operate safely’ declaration for people who are suitably trained within the organisation. Together this forms the Safety Case. Consider a handgun; the weapon has calibration, design and maintenance certification to prove it is safe to operate, and the soldier is trained to be qualified competent to make predictable judgement calls on how and when to pull the trigger (operate safely). Picture those statements as separate circles drawn on a Venn diagram. As levels of autonomy and complexity are dialled up the two circles converge. Should autonomy increase to the point that the decision to fire be under the control of an Artificial Intelligence within the weapon’s software then the two circles will overlap. This paper details research conclusions within the overlap, and proposes a new methodology able to certify that an AI based autonomous weapons system is “safe to operate itself safely” when in an autonomous state.

Lethal Autonomous Weapon Systems: Ethical Dilemmas and Legal Compliance in the Era of Military Disruptive Technologies

Lethal Autonomous Weapon Systems (LAWS) have emerged as one of the most significant advancements in military technology, leveraging artificial intelligence (AI) and machine learning to execute missions without direct human control. As these systems become central to modern warfare, they raise critical questions about their compliance with International Humanitarian Law (IHL) and International Human Rights Law (IHRL). This paper delves into the legal and ethical debates surrounding LAWS with particular attention to the discussions within the Group of Governmental Experts on LAWS (GGE on LAWS). We analyze whether these technologies can adhere to fundamental human rights while maintaining their operational efficacy. Through the application of the Autonomy Spectrum Framework to real-world scenarios, the study highlights both the strategic advantages of LAWS and the risks of dehumanizing warfare. The need for robust legal frameworks to ensure accountability and human oversight remains paramount.

Lethal Autonomous Weapon Systems and Automation Bias

Autonomy in weapon systems is already a genuine concern. States try to come up with their own definitions of these systems and pay utmost effort to impose their own understanding of these systems upon other states. For a fairly high number of states barring a total ban on such weapons would be the ideal solution; however, such states that are anxious about the increase in autonomy in war-making capabilities, as adopts a second-best scenario to contain risks created by the deployment of such systems. To this end, placing them under meaningful human control emerges as an important political and legal objective. The author believes that placing autonomous weapons under human supervision, despite its initial promise, will yield negative results. This is due to the fact that humans tend rather to be too willing to follow the solutions generated by autonomous systems. First observed in other industries of civilian nature like aviation or health, automation bias has the potential to negate most if not all of supervision measures expected to ensure proper implementation of international humanitarian law.

Putting algorithmic bias on top of the agenda in the discussions on autonomous weapons systems

Biases in artificial intelligence have been flagged in academic and policy literature for years. Autonomous weapons systems—defined as weapons that use sensors and algorithms to select, track, target, and engage targets without human intervention—have the potential to mirror systems of societal inequality which reproduce algorithmic bias. This article argues that the problem of engrained algorithmic bias poses a greater challenge to autonomous weapons systems developers than most other risks discussed in the Group of Governmental Experts on Lethal Autonomous Weapons Systems (GGE on LAWS), which should be reflected in the outcome documents of these discussions. This is mainly because it takes longer to rectify a discriminatory algorithm than it does to issue an apology for a mistake that occurs occasionally. Highly militarised states have controlled both the discussions and their outcomes, which have focused on issues that are pertinent to them while ignoring what is existential for the rest of the world. Various calls from civil society, researchers, and smaller states for a legally binding instrument to regulate the development and use of autonomous weapons systems have always included the call for recognising algorithmic bias in autonomous weapons, which has not been reflected in discussion outcomes. This paper argues that any ethical framework developed for the regulation of autonomous weapons systems should, in detail, ensure that the development and use of autonomous weapons systems do not prejudice against vulnerable sections of (global) society.

HUMAN DIGNITY, MORAL DAMAGE, AND KILLINGS BY AUTONOMOUS WEAPONS SYSTEMS

The present article aims to discuss if the fact that Lethal Autonomous Weapons Systems (LAWS) killed or injured a person generates moral damage, and if there is a presumption of moral damage, being unnecessary for the plaintiff to prove the moral suffering to be entitled to reparation from the State. First, the article defines LAWS and provides a brief perspective on moral damage in international law based on the International Law Commission´s Draft Articles on State Responsibility and scholarship. Next, it discusses if LAWS killings or injuries violate the principle of human dignity. Then it highlights international case-law from the international human rights courts on moral damage in re ipsa. The article concludes that LAWS's killings or injuries generate a relative presumption of moral damage.

Digital Targeting: Artificial Intelligence, Data, and Military Intelligence

Abstract It is widely believed that we are on the brink of another military revolution. Today, states are actively seeking to harness the power of AI for military advantage. The question of AI is therefore of profound concern to security studies scholars concerned with global issues. Up to now, the literature has tended to concentrate on AI-enabled lethal autonomous weapons; scholars have been fascinated by the possible appearance of autonomous drone swarms and their implications for security, conflict, and war. This article takes an alternative view. It argues that AI has already begun to play a significant role in military operations and is likely to be more important in the future. However, the attention to lethal autonomous weapons is exaggerated. The armed forces have principally employed AI, not to automate weapons but to help process data. AI has been used to augment military intelligence. Above all, the armed forces have harnessed AI to accelerate and improve military targeting. The article explores two recent cases where the armed forces have used data and AI to target: COVID testing in Liverpool in 2020 and the US’s Security Assistance Group-Ukraine in the Ukraine War in 2022.

Robot wars: Autonomous drone swarms and the battlefield of the future

ABSTRACT We seem to be on the cusp of an AI-driven revolution in military affairs. Scholars have explored many aspects of this revolution but one of the most vibrant debates has addressed the question of lethal autonomous weapons. Some scholars believe that autonomous weapons, and especially autonomous drone swarms, are about to colonise the battlefield. This paper assesses this argument. It identifies three common mistakes in discussions of lethal autonomy. Scholars overestimate the capability of autonomous drone swarms. They underestimate their dependence on other weapon systems. Finally, they presume that autonomous weapons will favour the offence. This paper rejects all three claims.

Front and Back Covers, Volume 40, Number 1. February 2024

Front and back cover caption, volume 40 issue 1EYE IN THE SKY: ANTHROPOLOGY IN THE DRONE ERAA civilian contractor launches a V‐BAT surveillance and reconnaissance drone from the USS Rushmore in the Philippine Sea. Manufactured by the California‐based startup Shield AI, the V‐BAT takes off vertically and operates autonomously without operator guidance or GPS (global positioning system). Its ‘swarming’ capability allows for potential combat use.Drones have had a profound impact in theatres of war for decades, causing lethal outcomes for combatants and non‐combatants alike. The US‐led ‘war on terror’ in regions like Afghanistan, Pakistan and Yemen has seen drone strikes causing tens of thousands of deaths, including many civilians. Beyond the direct, destructive impact, American drone pilots report significant mental health issues like anxiety, depression and PTSD due to the nature of remote‐controlled warfare.Recent conflicts in Ukraine and Gaza have seen an expansion in the types of drones used, ranging from inexpensive commercial drones with explosives to advanced armed drones. While traditionally operated remotely, there is an increasing shift towards autonomous functioning.This escalation in drone warfare technology, particularly the development of lethal autonomous weapon systems, has led to warnings of a global AI arms race involving powers like the US, China and Russia. The concept of ‘sensor‐to‐shooter’ machine learning infrastructures by military planners underscores the ethical, moral and sociopolitical dilemmas in a world increasingly leaning towards algorithmic warfare.In this issue, Roberto González argues that anthropologists need to illuminate how drone warfare impacts societies, the effects on both operators and targeted populations and the broader sociopolitical implications of autonomous weapons. This critical, publicly oriented anthropological perspective is essential to navigating the complex interplay of technology, ethics and human values in modern warfare, offering a deeper understanding of the societal impacts of these rapidly evolving military technologies.UNRAVELLING ‘REAL’ KINSHIPThis thought‐provoking Illustration shows the contrast between the idealized concept of unilineal descent and the intricate realities of kinship in everyday life. The left side of the image displays a simplified, stylized, unilineal family tree. With its clear, vertical lines, this depiction symbolizes kinship as a straightforward, linear progression. It embodies an idealized perspective on lineage, where relationships are traced through a single ancestral line, mirroring classical kinship theory.Conversely, the right side of the image presents a more complex and interwoven family tree. This part challenges the simplistic notions of unilineal descent, revealing kinship's rich, multifaceted nature. Here, the interconnected lines represent a spectrum of relationships extending beyond mere descent, encompassing marriage, adoption, communal ties and other social bonds. These aspects are frequently marginalized in conventional kinship models but, as anthropologists know, they are essential to understanding the full scope of human connections.Sabina Cveček's insightful analysis in this issue shows how archaeologists’ interpretation of unilineal kinship systems tend to be incomplete. Her work underscores the disparity between theoretical models and the nuanced realities of kinship, prompting a call for a more dynamic interpretation of kinship as a socially constructed and evolving network. This illustration invites introspection on the diversity and complexity of kinship and advocates a more inclusive and holistic approach to studying human relationships.Cveček argues that the dichotomy between idealized models and actual kinship dynamics is a recurring theme in anthropological literature. Anthropology remains crucial in understanding the depths and dimensions of kinship systems, even in today's era of scientific advancements like ancient DNA analysis.

Emergent Normativity: Communities of Practice, Technology, and Lethal Autonomous Weapon Systems

Abstract Lethal autonomous weapon systems (LAWS) are the subject of considerable international debate turning around the extent to which humans remain in control over using force. But what is precisely at stake is less clear as stakeholders have different perspectives on the technologies that animate LAWS. Such differences matter because they shape the substance of the debate, which regulatory options are put on the table, and also normativity on LAWS in the sense of understandings of appropriateness. To understand this process, I draw on practice theories, science and technology studies (STS), and critical norm research. I argue that a constellation of communities of practice (CoPs) shapes the public debate about LAWS and focus on three of these CoPs: diplomats, weapon manufacturers, and journalists. Actors in these CoPs discursively perform practices of boundary-work, in the STS sense, to shape understandings of technologies at the heart of LAWS: automation, autonomy, and AI. I analyze these dynamics empirically in two steps: first, by offering a general-level analysis of practices of boundary-work performed by diplomats at the Group of Governmental Experts on LAWS from 2017 to 2022; and second, through examining such practices performed by weapon manufacturers and journalists in relation to the use of loitering munitions, a particular type of LAWS, in the Second Libyan Civil War (2014–2020).

Supporting Ethical Decision-Making for Lethal Autonomous Weapons

ABSTRACT This article describes a new and innovative methodology for calibrating trust in ethical actions by Lethal Autonomous Weapon Systems (LAWS). For the foreseeable future, LAWS will require human operators for mission planning, decision-making, and supervisory control; yet humans lack the cognitive bandwidth and processing speed to make prompt, real-time ethical decisions. As a result, trustworthy Artificial Intelligence (AI) will be required to support ethical decision-making. We use a Bayesian ethical decision model for: (1) human setting of ethical preferences and thresholds in accordance with Laws of War and tactical criteria; and (2) highlighting the factors that contribute to strike/no-strike recommendations for human evaluation. The model can perform an ethical analysis, provide a quantitative ethical strike/no-strike score, and recommend actions to reduce decision uncertainty. In this article, we describe successful initial evaluation trials of the Bayesian model and of a human interface for interaction with the model. Our Bayesian ethical decision model has an immediate application in wargames; the model can also be used to train operators in understanding the principles and key factors relevant to ethical decision-making; and it may eventually be used in actual military operations employing LAWS.

The regulatory, ethical, and technological progress of Lethal Autonomous Weapons

ltiples distopías, siendo su aplicación en el ámbito de la Seguridad y la Defensa uno de sus aspectos más controvertidos. A pesar de las numerosas llamadas de atención de diferentes actores gubernamentales, no gubernamentales e interestatales, los Estados afrontan el desarrollo de los Sistemas Autónomos Letales (SALAS) de forma diferente en función de sus intereses, su cultura y sus valores éticos, frecuentemente, ante la ausencia de una clara legalidad internacional. Es en este entorno geopolítico, donde los principios y valores occidentales que antaño fueron guía en la esfera mundial y que en la actualidad se encuentran cuestionados y eclipsados por una realidad propia de un mundo diferente, en el que las Naciones Unidas tratan de alcanzar acuerdos que regulen el desarrollo de los SALAS y limiten el riesgo de que las máquinas pudieran escapar al control de los seres humanos.

POTENTIAL IMPACT OF LETHAL AUTONOMOUS WEAPON SYSTEMS ON STRATEGIC STABILITY AND NUCLEAR DETERRENCE IN SOUTH ASIA

Several trends show that the efforts to alter South Asian strategic landscape have increased, which primarily includes unprecedented proliferation of technology to India. This has exacerbated Pakistan’s security dilemma and is a matter of significant concern for South Asian security environment. Militaries worldwide are preparing for a new warfare trend in emerging technologies that include artificial intelligence and machine learning-based autonomous weapon systems, robotics, hypersonic weapons, and quantum technologies. These technologies are expected to change the character of war besides affecting geopolitical competitions and rivalries. It is pertinent to analyse the impact of these new technologies on regional rivalries to identify and implement effective solutions, thus mitigating potential risks and preventing the occurrence of a catastrophe. As several states are developing AI-based Lethal Autonomous Weapon Systems, capable of targeting without human supervision, possible exploitation of these technologies by nuclear-armed states may increase the risk of war leading to more aggressive nuclear postures. Such a risk would be higher in South Asia as the ballistic missile flight time would be less than ten minutes, and conventional military interactions of rival states could occur near a long border. The integration of autonomy into conventional weapons and nuclear systems has the potential to undermine strategic stability and will be a quick recipe for accidents and miscalculations. This study explores the region’s threat perceptions and analyses the potential use of LAWS and how it can impact nuclear deterrence and strategic stability of South Asia. This study hypothesizes that LAWS will lower the nuclear threshold in the region and undermine deterrence stability. Bibliography Entry Masood, Maryyum and Muhammad Ali Baig. 2023. "Potential Impact of Lethal Autonomous Weapon Systems on Strategic Stability and Nuclear Deterrence in South Asia." Margalla Papers 27 (2): 27-43.

Killer Robots and Inauthenticity: A Heideggerian Response to the Ethical Challenge Posed by Lethal Autonomous Weapons Systems

This paper addresses the ethical challenges raised by the use of lethal autonomous weapons systems. Using aspects of the philosophy of Martin Heidegger, the paper demonstrates that lethal autonomous weapons systems create ethical problems because of the lack of moral agency in an autonomous system, and the inauthentic nature of the deaths caused by such a system. The paper considers potential solutions for these issues before arguing that from a Heideggerian standpoint they cannot be overcome, and thus the development and use of lethal autonomous weapons systems should be resisted and prohibited.

Theory of Primary Intelligence (TPI) and Lethal Autonomous Weapon Systems (LAWS)

Umjetna inteligencija (UI) neizbježan je proizvod razvoja tehnologija 21. stoljeća i stoga predstavlja kontinuitet dosadašnjeg tehnološkog napretka. Međutim, UI je ujedno i transformativna tehnologija s neograničenim potencijalom daljnjeg razvoja u različitim područjima znanosti, pa time i u svakodnevnom životu. Upotreba UI-ja u vojne svrhe poput LAWS-a otvara nove dileme, pa se postavljaju ključna pitanja: možemo li proizvesti takvu umjetnu inteligenciju koja s vremenom može postati ne samo replika nas samih kao ljudskih bića već i nešto više, te što to znači za našu sigurnost? Ako je UI u stanju reproducirati kognitivna svojstva čovjeka, nije li samo pitanje bliže budućnosti kada će strojevi donositi odluke umjesto čovjeka? U nekim područjima to i nije novost. Pametna računala i njihove razvijene programske podrške (softveri) već donose odluke, ali još uvijek uz pomoć čovjeka koji ih opskrbljuje neophodnim podatcima, pa govorimo o UI-u 20. stoljeća (UI-20). UI 21. stoljeća (UI-21) treba sve manje ljudskog „uplitanja“. U radu se definira teorija primarne inteligencije (TPI) kao izvorni i do sada neobjavljeni znanstveni koncept o primarnoj poziciji čovjeka u odnosu na umjetnu inteligenciju 21. stoljeća. Teorija polazi od nezaustavljivog tehnološkog razvoja i etičke dimenzije problema upotrebe UI-21.

Regulatory Choices at the Advent of Gig Warfare

Abstract Regulation of military ai may take place in three ways. First, existing rules and principles in ihl is already or could be extended via reinterpretation to apply to military ai; second, new ai regulation may appear via “add-ons” to existing rules, finally, regulation of military ai may appear as a completely new framework, either through new state behavior that results in customary international law or through a new legal act or treaty. By introducing this typology, one may identify possible manners of regulation that are presently under-researched and/or ignored, for example how Rules of Engagement (roe) may be a way to control the use of military ai. Expanding on existing scholarship, the articles discusses how military ai may operate under different forms of military command and control systems, how regulation of military ai is not only a question of “means” but also “methods” of warfare and how the doctrine of supervisory responsibility may go beyond the doctrine of command responsibility. In the case that fully-automated Lethal Autonomous Weapons Systems (laws) are available and considered for use, it is suggested that their use should be prohibited in densely populated areas following the same logic as incendiary weapons. Further, one could introduce certain export restrictions on fully-automated laws to prevent proliferation to non-state actors and rogue states.

Examining Supply Chain Risks in Autonomous Weapon Systems and Artificial Intelligence

The development of increasingly AI-enabled autonomous systems and other military applications of Artificial Intelligence (AI) have been recognised as emergent major military innovations. In the absence of an effective and enforceable ban on their development and/or usage arising from the Group of Governmental Experts on Lethal Autonomous Weapon Systems (LAWS), it is likely that such systems will continue to be development. Amongst the legal, ethical, practical, and strategic concerns raised by the emergence of such systems, it is important not to lose sight of the risks involved in relying on a high-manufactured system in place of a human. This places additional strains and importance on securing diverse, com�plex, and over cross-jurisdictional supply chains. This article focuses on the vulnerability of and the risks to the integrity and security of the supply chains responsible for producing AI-enabled autonomous military systems.

Laws on LAWS (Lethal Autonomous Weapon Systems): The Work of the United Nations and the Swedish Position

Laws on LAWS (Lethal Autonomous Weapon Systems): The Work of the United Nations and the Swedish Position

The Evolution of Air Threats in Future Conflicts

Currently, there is an observable trend towards continuous and comprehensive development in both fixed-wing and rotary-wing aircraft design. Furthermore, a dynamic progression has been also witnessed in the evolution of unmanned aerial vehicles, loitering munitions, lethal autonomous weapons systems, ballistic missiles, cruise missiles, hypersonic cruise missiles, lighter-than-air sensor platforms, high altitude pseudo satellites and any others. The development of modern technologies and the acquisition of new materials create opportunities for the modernization of existing air threats as well as the development of novel types. Additionally, the potential of artificial intelligence in unmanned systems is a significant factor that should not be overlooked. The aforementioned circumstances have prompted the author of the article to conduct a thorough examination of contemporary military air threats and to make a forecast regarding the evolution of air threats in future conflicts.

Machine guardians: The Terminator, AI narratives and US regulatory discourse on lethal autonomous weapons systems

References to the Terminator films are central to Western imaginaries of Lethal Autonomous Weapons Systems (LAWS). The puzzle of whether references to the Terminator franchise have featured in the United States’ international regulatory discourse on these technologies nevertheless remains underexplored. Bringing the growing study of AI narratives into a greater dialogue with the International Relations literature on popular culture and world politics, this article unpacks the repository of different stories told about intelligent machines in the first two Terminator films. Through an interpretivist analysis of this material, we examine whether these AI narratives have featured in the US written contributions to the international regulatory debates on LAWS at the United Nations Convention on Certain Conventional Weapons in the period between 2014 and 2022. Our analysis highlights how hopeful stories about what we coin ‘machine guardians’ have been mirrored in these statements: LAWS development has been presented as a means of protecting humans from physical harm, enacting the commands of human decision makers and using force with superhuman levels of accuracy. This suggests that, contrary to existing interpretations, the various stories told about intelligent machines in the Terminator franchise can be mobilised to both support and oppose the possible regulation of these technologies.