Automation Decision-making Research Articles

Automated decision-making, discretion and public values: a case study of two municipalities and their case management of social assistance

ABSTRACT Our aim is to increase knowledge about discretion and automated decision-making (ADM) in social work based on an approach that brings forward the role of humans and technologies and the resulting public values. This approach is applied to a case study of two Swedish municipalities and their social assistance case management. Our research questions are twofold: (1) How do humans and technologies appear in decisions about social assistance? and (2) How do they influence digital discretion and the resulting public values? A case management process with human actors (caseworkers and clients) and non-human actors (case management systems, e-applications, robotic process automation and algorithms) is used, influencing the digital discretion of caseworkers and the resulting ethical, democratic and professional values. Digital discretion is not dichotomous but, rather, related to a repertoire of technologies, their design and their use in a routine that ADM is a part of. The interaction with clients may be considered an important rationale for ADM itself. There is a necessary difference in weight between public values related to more straightforward (an ‘improved’ process enabling professional values) and complicating factors (distrust between caseworkers and the unwanted use of information technology).

Decision-Making Practices in Transition

The following contribution presents a standardized, multi-layered framework to analyse technical and societal aspects of automated decision-making (ADM) among private entities. It relies on the premise that in the context of technology, it is useful to shift the legal perspective towards sociotechnical practices. The taxonomy starts with the basic structure of the decision, its subject and the parties involved, then touches upon the relationship between the parties involved. Next, the taxonomy focuses on sociotechnical aspects, the optimization promises embodied in ADM and the application parameters of automated decision systems. Finally, it contains an analytical vocabulary for dysfunctionalities in ADM. As an example, the taxonomy is used to analyse and discuss the practice of robot recruiting.

Imaginaries of better administration: Renegotiating the relationship between citizens and digital public power

This article investigates future visions of digital public administration as they appear within a particular regulatory process that aims to enable automated decision-making (ADM) in public administration in Finland. By drawing on science and technology studies, public administration studies, and socio-legal studies we analyze law in the making and identify four imaginaries of public digital administration: understandable administration, self-monitoring administration, adaptive administration, and responsible administration. We argue that digital administration is seen from the perspective of public authorities serving their current needs of legitimizing existing automation practices. While technology is pictured as unproblematic, the citizen perspective is missing. We conclude that the absence of an in-depth understanding of the diverse needs of citizens raises the question whether the relationship between public power and citizens is becoming a one-way street despite of the public administration ideals that express values of citizen engagement.

Automated Decision-Making (ADM) in EU Public Law

Automated Decision-Making (ADM) in EU Public Law

Disclosures about algorithmic decision making in the corporate reports of Western European companies

Over the last decade, the use of different artificial intelligence (AI) tools has increased. To shed some light on the emerging trend of AI disclosure, the aim of this paper is to analyse the current practices of major Western European companies regarding the automated decision-making (ADM) disclosure in their annual or sustainability reports. This paper proposes a methodology based on bigrams that enables the automatic extraction of the information on ADM that companies disclose. The sample consisted of 962 annual/sustainability reports, published in 2018 and 2019, of 337 companies listed on 13 Western European countries’ stock markets. Our findings show that ADM disclosure is still at an early stage and that the first adopters are mostly companies operating in the financial sector.

Automation Is Coming! Exploring Future(s)-Oriented Methods in Education

Emerging automated-decision making (ADM) technologies invite scholars to engage with future points in time and contexts that have not yet arisen. This particular state of not knowing yet implies the methodological challenge of examining images of the future and how such images will materialize in practice. In this respect, we ask the following: what are appropriate research methods for studying emerging ADM technologies in education? How do researchers explore sociotechnical practices that are in the making? Guided by these questions, we investigate the increasing adoption of ADM in teachers’ assessment practices. This constitutes a case in point for reflecting on the research methods applied to address the future of assessment in education. In this context, we distinguish between representational methods oriented to recounting past experiences and future(s) methods oriented to making futures. Studying the literature on speculative methods in digital education, we illustrate four categories of future(s)-oriented methods and reflect on their characteristics through a backcasting workshop conducted with teachers. We conclude by discussing the need to reconsider the methodological choices made for studying emerging technologies in critical assessment practices and generate new knowledge on methods able to contribute to alternative imaginaries of automation in education.

Humans versus machines: Who is perceived to decide fairer? Experimental evidence on attitudes toward automated decision-making

SummaryHuman perceptions of fairness in (semi-)automated decision-making (ADM) constitute a crucial building block toward developing human-centered ADM solutions. However, measuring fairness perceptions is challenging because various context and design characteristics of ADM systems need to be disentangled. Particularly, ADM applications need to use the right degree of automation and granularity of data input to achieve efficiency and public acceptance. We present results from a large-scale vignette experiment that assessed fairness perceptions and the acceptability of ADM systems. The experiment varied context and design dimensions, with an emphasis on who makes the final decision. We show that automated recommendations in combination with a final human decider are perceived as fair as decisions made by a dominant human decider and as fairer than decisions made only by an algorithm. Our results shed light on the context dependence of fairness assessments and show that semi-automation of decision-making processes is often desirable.

Citizens’ attitudes towards automated decision-making

Public organisations are starting to show an interest in automated decision-making (ADM). So far, existing research focuses on the governmental perspective on this phenomenon. Less attention is paid to citizens’ views on ADM. The aim of this study is to provide empirical insights into citizen awareness of and beliefs about ADM in public-sector services. To this end, we participated in an annual national survey in Sweden carried out by the SOM Institute at Gothenburg University concluding that a minority of the citizens know about the use of ADM in public-sector services. Furthermore, when computers instead of civil servants make decisions in the public-sector, citizens expect decisions by computers to become less legally secure but more impartial. They also expect ADM to take personal circumstances into account to a lesser degree and become less transparent. Finally, we found that citizens with that awareness expect decisions by computers to become more reliable and impartial. Based on our empirical findings in relation to previous research, we suggest four hypotheses on citizen’s awareness and beliefs about public-sector ADM.

Artificial intelligence and automated decision making: The new frontier of privacy challenges and opportunities

This paper addresses the privacy component of broader artificial intelligence (AI) ethical considerations. We begin with an overview of the regulatory landscape, or lack thereof, and then call out the specific provisions of EU data protection law applicable to AI while focusing on examples of country-specific approaches, including some recent regulatory action. This regulatory action is particularly insightful since it identifies the key challenges that companies face, or will eventually face, when adopting AI-based solutions. These challenges include how to anticipate and prevent bias in automated decision making (ADM) and how to provide transparency to data subjects, despite the complexity of machine learning processes, while protecting business secrets and know-how.

From Making Automated Decision Making Visible to Mapping the Unknowable Human: Counter-Mapping Automated Decision Making in Social Services in Australia

Automated decision-making (ADM) technologies, like artificial intelligence and machine learning, are increasingly being used by governments. Researchers have attempted to map the deployment of these technologies. However, mapping is an inherently political act, reinforcing dominant discourses and imaginings of technological futures. In this article, I engage with critical cartography to outline the potential of counter-mapping for researching automation in decision making, with the purpose of mapping, to quote from Hodgson and Schroeder in 2002, “against dominant power structures, to further seemingly progressive goals.” Drawing on the case of ADM in Australian social services, I reflexively account for how counter-mapping can provide a method for moving beyond dominant discourses of efficiency, cost cutting, and industriousness, to allow the alternative voices of service users’ experiences of ADM to be heard. I argue that future ADM mapping needs to focus on making visible those who are subject to the decisions of automated systems, but are usually made unknowable by the over-confident calculability of dominant ADM discourses.

Integrating Behavioral, Economic, and Technical Insights to Understand and Address Algorithmic Bias: A Human-Centric Perspective

Many important decisions are increasingly being made with the help of information systems that use artificial intelligence and machine learning models. These computational models are designed to discover useful patterns from large amounts of data, which augment human capabilities to make decisions in various application domains. However, there are growing concerns regarding the ethics challenges faced by these automated decision-making (ADM) models, most notably on the issue of algorithmic bias , in which the models systematically produce less favorable (i.e., unfair) decisions for certain groups of people. In this commentary, we argue that algorithmic bias is not just a technical (e.g., computational or statistical) problem, and its successful resolution requires deep insights into individual and organizational behavior, economic incentives, as well as complex dynamics of the sociotechnical systems in which the ADM models are embedded. We discuss a human-centric, fairness-aware ADM framework that highlights the holistic involvement of human decision makers in each step of ADM. We review the emerging literature on fairness-aware machine learning and then discuss various strategic decisions that humans need to make, such as formulating proper fairness objectives, recognizing fairness-induced trade-offs and implications, utilizing machine learning model outputs, and managing/governing the decisions of ADM models. We further illustrate how these strategic decisions are jointly informed by behavioral, economic, and design sciences. Our discussions reveal a number of future research opportunities uniquely suitable for Management Information Systems (MIS) researchers to pursue.

The Legal Regulation of Artificial Intelligence and Edge Computing Automation Decision-Making Risk in Wireless Network Communication

This article is aimed at studying the legal regulation of artificial intelligence and edge computing automated decision-making risks in wireless network communications. The data under artificial intelligence is full of flexibility and vitality, which has changed the way of data existence in the whole society. Its core is various algorithm programs, which determine the existence of artificial intelligence. In this environment, society develops rapidly with unstoppable momentum. However, from a legal perspective, artificial intelligence has algorithmic discrimination, such as gender discrimination, clothing discrimination, and racial discrimination. It does not possess openness, objectivity, and accountability. The consequences are sometimes serious enough to endanger the public interest of the entire society, leading to market disorder, etc. Therefore, the problem of artificial intelligence algorithm discrimination remains to be solved. This article uses algorithms to adjust algorithm discrimination to reduce the harm caused by artificial intelligence algorithm discrimination to a certain extent. First of all, this article introduces a regulatory-based edge cloud computing architecture model. It is mentioned that distributed cloud computing can use subsystems to calculate various resources and storage resources and can make automated decisions when calculating certain data. In order to reduce the impact of algorithm discrimination and trigger data diversification to reduce the probability of discrimination, an edge computing network data capture system is designed. And this article mentions the BP neural network model. The BP neural network model is divided into input layer, output layer, and hidden layer. The training samples are passed from the input layer to the output layer through the hidden layer. If the output information does not meet expectations, the error will be back-propagated, and the connection weight will be adjusted continuously. This paper proposes a deep learning system model in real-time artificial intelligence driven by edge computing. When this model is applied to legal regulations, it can cooperate with edge computing and artificial intelligence algorithms to provide high-precision automated decision-making. Finally, this paper designs an artificial intelligence-assisted automated decision-making experiment based on the theory of legal computing. This paper proposes a Bayesian algorithm that uses edge algorithms to merge into artificial intelligence and verifies the feasibility of this hypothesis through experiments. The experimental results show that it has a certain ability to regulate algorithmic discrimination caused by artificial intelligence in legal regulations. It can improve the regulatory effects of laws and regulations to a certain extent, and the improved artificial intelligence Bayesian algorithm clustering effect of edge computing is increased by about 7.2%.

Automated decision-making, digital inclusion and intersectional disabilities

Disability is a long-standing area of digital inclusion finally emerging out of the shadows. In this article, we argue that a critical understanding of digital media from the perspectives of disability and intersectionality will offer generative insights for framing the terms and agenda of digital inclusion in the next decade. With a focus on the area of automated decision-making (ADM) in social and welfare services, we reflect upon the controversial 2015–2020 Australian government programme widely known as ‘Robodebt’ that recovers putative debts from support recipients – and we discuss implications for Indigenous Australians with disabilities in particular. We contrast the ‘Robodebt’ programme with explicit digital inclusion policy on disability in Australia, noting that such digital inclusion policy does not specifically acknowledge yet alone address ADM or other aspects of automation. Here, there is a major opportunity for overdue acknowledgement of disability and intersectionality to spur and shape an affirmative and just agenda on people with disabilities’ digital inclusion, ADM and other associated areas of automated technologies.

Selection of traceability-based, automated decision-making methods in global production networks

Automating traceability-based decision-making can shorten the reaction time to supply chain disruptions. This paper develops a framework for choosing automated decision-making (ADM) methods based on traceability data. It contains a toolbox comprising methods suitable for ADM, respective selection criteria and a new process to select a suitable ADM method based on companies’ requirements. This process is based on an evaluation matrix matching methods and criteria. As a result, the ADM framework suggests the most suitable method to automate a specifically chosen decision. The developed framework is validated in the supply chain of a globally operating truck manufacturer.

Suspect Development Systems: Databasing Marginality and Enforcing Discipline

Algorithmic accountability law—focused on the regulation of data-driven systems like artificial intelligence (AI) or automated decision-making (ADM) tools—is the subject of lively policy debates, heated advocacy, and mainstream media attention. Concerns have moved beyond data protection and individual due process to encompass a broader range of group-level harms such as discrimination and modes of democratic participation. While a welcome and long overdue shift, the current discourse ignores systems like databases, which are viewed as technically “rudimentary” and often siloed from regulatory scrutiny and public attention. Additionally, burgeoning regulatory proposals like algorithmic impact assessments are not structured to surface important –yet often overlooked –social, organizational, and political economy contexts that are critical to evaluating the practical functions and outcomes of technological systems. This Article presents a new categorical lens and analytical framework that aims to address and overcome these limitations. “Suspect Development Systems” (SDS) refers to: (1) information technologies used by government and private actors, (2) to manage vague or often immeasurable social risk based on presumed or real social conditions (e.g. violence, corruption, substance abuse), (3) that subject targeted individuals or groups to greater suspicion, differential treatment, and more punitive and exclusionary outcomes. This framework includes some of the most recent and egregious examples of data-driven tools (such as predictive policing or risk assessments), but critically, it is also inclusive of a broader range of database systems that are currently at the margins of technology policy discourse. By examining the use of various criminal intelligence databases in India, the United Kingdom, and the United States, we developed a framework of five categories of features (technical, legal, political economy, organizational, and social) that together and separately influence how these technologies function in practice, the ways they are used, and the outcomes they produce. We then apply this analytical framework to welfare system databases, universal or ID number databases, and citizenship databases to demonstrate the value of this framework in both identifying and evaluating emergent or under-examined technologies in other sensitive social domains. Suspect Development Systems is an intervention in legal scholarship and practice, as it provides a much-needed definitional and analytical framework for understanding an ever-evolving ecosystem of technologies embedded and employed in modern governance. Our analysis also helps redirect attention toward important yet often under-examined contexts, conditions, and consequences that are pertinent to the development of meaningful legislative or regulatory interventions in the field of algorithmic accountability. The cross-jurisdictional evidence put forth across this Article illuminates the value of examining commonalities between the Global North and South to inform our understanding of how seemingly disparate technologies and contexts are in fact coaxial, which is the basis for building more global solidarity.

Legal contestation of artificial intelligence-related decision-making in the United Kingdom: reflections for policy

ABSTRACT This paper considers legal contestation in the UK as a source of useful reflections for AI policy. The government has published a ‘National AI Strategy’, but it is unclear how effective this will be given doubts about levels of public trust. One key concern is the UK’s apparent ‘side-lining’ of the law. A series of events were convened to investigate critical legal perspectives on the issues, culminating in an expert workshop addressing five sectors. Participants discussed AI in the context of wider trends towards automated decision-making (ADM). A recent proliferation in legal actions is expected to continue. The discussions illuminated the various ways in which individual examples connect systematically to developments in governance and broader ‘AI-related decision-making’, particularly due to chronic problems with transparency and awareness. This provides a fresh and current insight into the perspectives of key groups advancing criticisms relevant to policy in this area. Policymakers’ neglect of the law and legal processes is contributing to quality issues with recent practical ADM implementation in the UK. Strong signals are now required to switch back from the vicious cycle of increasing mistrust to an approach capable of generating public trust. Suggestions are summarised for consideration by policymakers.

Explicability of humanitarian AI: a matter of principles

In the debate on how to improve efficiencies in the humanitarian sector and better meet people’s needs, the argument for the use of artificial intelligence (AI) and automated decision-making (ADMs) systems has gained significant traction and ignited controversy for its ethical and human rights-related implications.Setting aside the implications of introducing unmanned and automated systems in warfare, we focus instead on the impact of the adoption of AI-based ADMs in humanitarian response. In order to maintain the status and protection conferred by the humanitarian mandate, aid organizations are called to abide by a broad set of rules condensed in the humanitarian principles and notably the principles of humanity, neutrality, impartiality, and independence. But how do these principles operate when decision-making is automated?This article opens with an overview of AI and ADMs in the humanitarian sector, with special attention to the concept of algorithmic opacity. It then explores the transformative potential of these systems on the complex power dynamics between humanitarians, principled assistance, and affected communities during acute crises. Our research confirms that the existing flaws in accountability and epistemic processes can be also found in the mathematical and statistical formulas and in the algorithms used for automation, artificial intelligence, predictive analytics, and other efficiency-gaining-related processes.In doing so, our analysis highlights the potential harm to people resulting from algorithmic opacity, either through removal or obfuscation of the causal connection between triggering events and humanitarian services through the so-called black box effect (algorithms are often described as black boxes, as their complexity and technical opacity hide and obfuscate their inner workings (Diakopoulos, Tow Center for Digital Journ, 2017). Recognizing the need for a humanitarian ethics dimension in the analysis of automation, AI, and ADMs used in humanitarian action, we endorse the concept of “explicability” as developed within the ethical framework of machine learning and human-computer interaction, together with a set of proxy metrics.Finally, we stress the need for developing auditable standards, as well as transparent guidelines and frameworks to rein in the risks of what has been defined as humanitarian experimentation (Sandvik, Jacobsen, and McDonald, Int. Rev. Red Cross 99(904), 319–344, 2017). This article concludes that accountability mechanisms for AI-based systems and ADMs used to respond to the needs of populations in situation of vulnerability should be an essential feature by default, in order to preserve the respect of the do no harm principle even in the digital dimension of aid.In conclusion, while we confirm existing concerns related to the adoption of AI-based systems and ADMs in humanitarian action, we also advocate for a roadmap towards humanitarian AI for the sector and introduce a tentative ethics framework as basis for future research.

A data quality approach to the identification of discrimination risk in automated decision making systems

Automated decision-making (ADM) systems may affect multiple aspects of our lives. In particular, they can result in systematic discrimination of specific population groups, in violation of the EU Charter of Fundamental Rights. One of the potential causes of discriminative behavior, i.e., unfairness, lies in the quality of the data used to train such ADM systems.Using a data quality measurement approach combined with risk management, both defined in ISO standards, we focus on balance characteristics and we aim to understand how balance indexes (Gini, Simpson, Shannon, Imbalance Ratio) identify discrimination risk in six large datasets containing the classification output of ADM systems.The best result is achieved using the Imbalance Ratio index. Gini and Shannon indexes tend to assume high values and for this reason they have modest results in both aspects: further experimentation with different thresholds is needed.In terms of policies, the risk-based approach is a core element of the EU approach to regulate algorithmic systems: in this context, balance measures can be easily assumed as risk indicators of propagation – or even amplification – of bias in the input data of ADM systems.

Personalization and the Smart Home: questioning techno-hedonist imaginaries

Smart home futures predicted and promoted in the discourses and practices of industry increasingly figure forms of automated decision making (ADM) into existing narratives of technological solutionism. This article interrogates how these narratives are constituted, the futures they imagine, predict and promote, and how people and households are presented within these futures. Our discussion is based on a content analysis of technology industry reports. The reports emphasize that the key to business success in the future is to use ADM to instantly satisfy individual preferences and to tailor products and services to the “market of one.” While the role of AI and algorithms in the creation of personalized media—particularly social media feeds—has garnered attention for its role in increasing political divisiveness around the world, less attention has been paid to the consequences of this individualized logic when applied and recreated in the design and installation of automated smart home technology. The technology sector’s emphasis on personalization treats the household as an individual self-motivated decision-making entity, despite the home being a complex socio-material assemblage shaped by constantly negotiated power differentials, such as by age, gender, and species. Thus, the discourse of “personalization” underpinning the imagination of emerging technology for the home and the individualized conception of human behavior which underpins it, lead to significant oversights when situated in the complexity of domestic life. Further we argue that this risks embedding an alienated understanding of human behavior—techno-hedonism—into ADM systems. We discuss how a persona of the techno-hedonist underlies visions of the personalized smart home, and its potential consequences through three interrelated concepts: convenience, control, and choice.

Automated Decision-Making and Environmental Impact Assessments: Decisions, Data Analysis and Predictions

This article critically examines the opportunities and challenges that automated decision-making (ADM) poses for environmental impact assessments (EIAs) as a crucial aspect of environmental law. It argues that while fully or partially automating discretionary EIA decisions is legally and technically problematic, there is significant potential for data-driven decision-making tools to provide superior analysis and predictions to better inform EIA processes. Discretionary decision-making is desirable for EIA decisions given the inherent complexity associated with environmental regulation and the prediction of future impacts. This article demonstrates that current ADM tools cannot adequately replicate human discretionary processes for EIAs—even if there is human oversight and review of automated outputs. Instead of fully or partially automating EIA decisions, data-driven decision-making can be more appropriately deployed to enhance data analysis and predictions to optimise EIA decision-making processes. This latter type of ADM can augment decision-making processes without displacing the critical role of human discretion in weighing the complex environmental, social and economic considerations inherent in EIA determinations.