AI Regulation Research Articles

Towards an Ecosystem for Consumer Protection in the Context of AI-based Credit Scoring

The Big Data phenomenon first, and AI most recently, have significantly changed the way in which credit scoring takes place and creditworthiness is evaluated by lenders. Beside traditional credit data, the creation of consumers’ credit-scores now involves also non-traditional data and is based on the predictions that lenders can make on the basis of those data. The use of AI, coupled with the availability of extensive amounts of ‘alternative’ data, poses several questions as to the level of protection granted to consumers in relation to the discriminatory effects that an ungoverned used of such technology can generate. The article addresses the suitability of the Proposal for a new Directive on Consumer Credit to protect consumers that enter into credit agreements where access to credit is determined by AI-based credit scoring systems. In doing so, it also takes into consideration other rules within the EU legal framework that can provide, albeit indirectly, protection to consumers, such as antidiscrimination law and Article 22 of the GDPR. It concludes that in a technologically complicated scenario such as the one of AI credit scoring, consumers can be effectively protected only by introducing an ecosystem of rules that while empowering consumers also regulates the use of AI on the business side of the credit agreement. Credit scoring, Artificial Intelligence, Big Data, Alternative Data, Directive on Consumer Credit, AI Regulation.

Enactment and Characteristics of the European Union AI Norms

지난 2020년 2월 19일 유럽집행위원회는 인공지능백서(White Paper on Artificial Intelligence)를 발표하였다. 또한 2021년 4월 21일에는 인공지능명령(Regulation on AI)을 제안하였다. 유럽집행위원회가 제안한 인공지능명령은 인공지능에 관한 최초의 규범이라고 볼 수 있다. 그러나 이러한 인공지능명령안이 제안되기 전에도 유럽집행위원회는 유럽연합의 미래 디지털전략을 제시하고 인공지능백서에서 그러한 정책을 포함하고 있다. 인공지능백서는 인공지능에 대한 유럽연합의 미래 청사진이다. 특히 인공지능백서에서 ‘신뢰 생태계’를 강조하고 있다는 점이 주목된다. 인공지능의 불확실성과 인공지능이 초래할 기본권 침해나 민주주의에 대한 위협 등을 우려하고 있으며, 신뢰 생태계의 구축은 이에 대한 대응이다. 유럽연합은 인공지능의 불확실성을 전제로 신뢰 생태계의 구축을 통해 이러한 문제를 해결하려고 노력하고 있다. 이처럼 유럽연합의 인공지능에 대한 정책은 인간중심적 인공지능을 구현하는 것을 목적으로 한다. 그러나 인공지능백서에는 구체적이고 구속력이 있는 규제방안이 포함되어 있지 않다. 이러한 상황에서 2021. 4. 21. 유럽집행위원회가 제안한 ‘인공지능명령’은 실천적 의미가 있다. 인공지능명령안에는 인공지능백서에서 제시된 중요한 골격이 대체로 유지되고 있으며, 이러한 내용은 인공지능명령의 중심적 요소를 이루고 있다고 볼 수 있다. 인공지능명령(안)이 제정되면, 회원국에 대해 직접 구속력을 가진 최초의 규율체계가 될 것이다. 인공지능규범은 인공지능에 대한 혁신과 투자를 위한 최소한의 안전장치이다. 리스크에 기초한 규제체계를 확립한 점이 주목된다. 특히 높은 리스크를 가진 인공지능에 대한 요건과 의무부과는 인상적인데, 인공지능에 대한 규제체계를 확립할 때 참고할 만한 사항이다. 이러한 내용은 종전의 인공지능백서에서 제시한 리스크 기반의 인공지능 규제정책이 그대로 투영된 것이다. 이러한 규제가 유럽연합 회원국에 국한되지 않고 제3국의 제공자나 사용자에게 미칠 수 있다는 점에서 인공지능명령의 제정과정과 동향을 주목하고 이에 효과적인 대응방안을 마련해야 한다.On February 19, 2020, the European Commission published the White Paper on Artificial Intelligence. In addition, on April 21, 2021, an Regulation on AI was proposed. The AI regulation proposed by the European Commission can be seen as the first norm on artificial intelligence. However, even before such an AI regulation was proposed, the European Commission presented the EU’s future digital strategy and included such a policy in the White Paper on AI. The AI White Paper is the European Union’s future blueprint for AI. In particular, it is noteworthy that the artificial intelligence white paper emphasizes the ‘ecosystem of trust’. There are concerns about the uncertainty of artificial intelligence and the violation of fundamental rights or threats to democracy that artificial intelligence will cause, and the establishment of an ecosystem of trust is a response to this. The European Union is trying to solve this problem by building a trust ecosystem on the premise of the uncertainty of artificial intelligence. As such, the EU policy on artificial intelligence aims to implement human-centered artificial intelligence. However, the AI white paper does not contain specific and binding regulatory measures. Under these circumstances, the Regulation on Artificial Intelligence proposed by the European Commission on April 21, 2021 has practical meaning. In the AI Regulation, the important framework presented in the AI White Paper is largely maintained, and it can be seen that these contents form a central element of the AI Regulation. If the AI Regulation is enacted, it will be the first regulatory system with direct binding force on member states. The AI norms are the minimum safeguards for innovation and investment in AI. It is noteworthy that the regulatory system based on risk was established. In particular, the requirements and obligations for high-risk AI are impressive, and it is worth referring to establishing a regulatory system for artificial intelligence in Korea. These contents are a reflection of the risk-based AI regulatory policy presented in the previous White Paper on AI. Considering that these regulations are not limited to EU member states and can affect providers or users in third countries, it is necessary to pay attention to the process and trends in the enactment of artificial intelligence commands and to prepare effective countermeasures.

The Good, the Bad, and the Invisible with Its Opportunity Costs: Introduction to the ‘J’ Special Issue on “the Impact of Artificial Intelligence on Law”

Scholars and institutions have been increasingly debating the moral and legal challenges of AI, together with the models of governance that should strike the balance between the opportunities and threats brought forth by AI, its ‘good’ and ‘bad’ facets. There are more than a hundred declarations on the ethics of AI and recent proposals for AI regulation, such as the European Commission’s AI Act, have further multiplied the debate. Still, a normative challenge of AI is mostly overlooked, and regards the underuse, rather than the misuse or overuse, of AI from a legal viewpoint. From health care to environmental protection, from agriculture to transportation, there are many instances of how the whole set of benefits and promises of AI can be missed or exploited far below its full potential, and for the wrong reasons: business disincentives and greed among data keepers, bureaucracy and professional reluctance, or public distrust in the era of no-vax conspiracies theories. The opportunity costs that follow this technological underuse is almost terra incognita due to the ‘invisibility’ of the phenomenon, which includes the ‘shadow prices’ of economy. This introduction provides metrics for such assessment and relates this work to the development of new standards for the field. We must quantify how much it costs not to use AI systems for the wrong reasons.

Regulating AI. A label to complete the proposed Act on Artificial Intelligence

AI regulation is emerging in the EU. The European authorities, NGOs and academics have already issued a series of proposals to accommodate the ‘development and uptake of AI’ with an ‘appropriate ethical and legal framework’ and promote what the European Commission has called an ‘ecosystem of trust’. In the spring of 2020, the European Commission submitted a legislative proposal for public consultation including four options ranging from “soft law only” to a broad scope of mandatory requirements and combinations thereof, for addressing the risks linked to the development and use of certain AI applications. One year later, the Commission unveiled on 21 April 2021 the EU Act on Artificial Intelligence.11Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts, Brussels, 21.4.2021 COM (2021) 206 final. The proposal primarily focusses on regulating ’high-risk’ systems through mandatory requirements and prohibition measures. This approach leaves a wide range of AI-systems, with potentially serious impact on fundamental rights, merely unregulated as regards specifically AI related risks. This paper explores the boundaries of the impact of the Act for primarily non-high-risk systems and discuss the options for introducing a voluntary labelling scheme for enhancing protection against the risks of medium and low risk AI systems.

Insight: AI

Abstract What is next for AI regulation? Senior Technologist Adam Leon Smith CITP FBCS explores the implications of the EU AI Act.

The EU's Capacity to Lead the Transatlantic Alliance in AI Regulation

In the face of Chinese advances in AI in terms of technological prowess and influence, there has been a call for collaboration between the EU and the US to create a foundation for AI governance based on shared democratic beliefs. This paper maps out the EU, US, and Chinese approaches to AI development and regulation as we analyze the capacity of the US and EU to establish international standards for AI regulation through channels such as the TTC. As the EU rolled out a proportionate and risk-based approach to ensure stricter regulation for high-risk AI technologies, it laid the foundation for international rule-shaping in the AI domain. The important question is whether the EU can effectively collaborate with the US in response to China’s aggressive AI initiative and, more importantly, lead the transatlantic effort to become the “world-leading region on developing and deploying cutting-edge, ethical and secure AI.”

The AI Regulation and Higher Education: Preliminary Observations and Critical Perspectives

The AI Regulation and Higher Education: Preliminary Observations and Critical Perspectives

Dilemmas in AI Regulation: An Exposition of the Regulatory Trade-Offs Between Responsibility and Innovation

Dilemmas in AI Regulation: An Exposition of the Regulatory Trade-Offs Between Responsibility and Innovation

Reconciling Conflicting Interests in Data through Data Governance. An Analytical Framework (and a Brief Discussion of the Data Governance Act Draft, the Data Act Draft, the AI Regulation Draft, as well as the GDPR)

Reconciling Conflicting Interests in Data through Data Governance. An Analytical Framework (and a Brief Discussion of the Data Governance Act Draft, the Data Act Draft, the AI Regulation Draft, as well as the GDPR)

European AI Regulation: Brussels Effect versus Human Dignity?

European AI Regulation: Brussels Effect versus Human Dignity?

High-risk AI systems in the EU Commission’s Proposal for an AI Act

On April 21, 2021, the EU Commission published a comprehensive proposal for an AI Act, which is intended to be the world s first set of regulations to create the legal framework for the development, use and sale of AI systems. The focus of the proposal is on the regulation of so-called high-risk AI systems. The article elaborates the obligations of providers, importers, traders and users of high-risk AI systems. These are part of IT and product safety law and - even if not explicitly addressed in the proposal - have an impact on contract law and tort law, which the article also illuminates. It is consequent that the Commission holds both providers and users responsible for subjecting high-risk AI systems to monitoring and security obligations throughout their entire life cycle. The requirements and obligations of the Proposed AI Act can ensure an increased level of legal certainty with regard to the use of AI systems in civil law. Nevertheless, the Proposed AI Act also reveals considerable weaknesses. First of all, the fuzzy definition of AI systems, which also includes software that does not pose any AI-specific risks, should be emphasized. Furthermore, the security requirements of the Proposed AI Act are partly too vague or difficult to implement in practice, which is particularly clear from the requirement for complete and error-free data sets and the retailer s inspection obligations. The relief for small and medium-sized companies is still too vague. In its current form, the Proposed AI Act could therefore ensure that Europe as an AI location is extremely cautious in developing and using AI. With a view to the wide range of applications of AI, it can be assumed that the AI Regulation will have just as much of an impact on the European and global market as the GDPR.

Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation

The proposed European Artificial Intelligence Act (AIA) is the first attempt to elaborate a general legal framework for AI carried out by any major global economy. As such, the AIA is likely to become a point of reference in the larger discourse on how AI systems can (and should) be regulated. In this article, we describe and discuss the two primary enforcement mechanisms proposed in the AIA: the conformity assessments that providers of high-risk AI systems are expected to conduct, and the post-market monitoring plans that providers must establish to document the performance of high-risk AI systems throughout their lifetimes. We argue that the AIA can be interpreted as a proposal to establish a Europe-wide ecosystem for conducting AI auditing, albeit in other words. Our analysis offers two main contributions. First, by describing the enforcement mechanisms included in the AIA in terminology borrowed from existing literature on AI auditing, we help providers of AI systems understand how they can prove adherence to the requirements set out in the AIA in practice. Second, by examining the AIA from an auditing perspective, we seek to provide transferable lessons from previous research about how to refine further the regulatory approach outlined in the AIA. We conclude by highlighting seven aspects of the AIA where amendments (or simply clarifications) would be helpful. These include, above all, the need to translate vague concepts into verifiable criteria and to strengthen the institutional safeguards concerning conformity assessments based on internal checks.

Policing faces: the present and future of intelligent facial surveillance

ABSTRACT In this paper, we discuss the present and future uses of intelligent facial surveillance (IFS) in law enforcement. We present an empirical and legally focused case study of live automated facial recognition technologies (LFR) in British policing. In Part I, we analyse insights from 26 frontline police officers exploring their concerns and current scepticism about LFR. We analyse recent UK case law on LFR use by police which raises concerns around human rights, data protection and anti-discrimination laws. In Part II, we consider frontline officers’ optimism around future uses of LFR and explore emerging forms of IFS, namely emotional AI (EAI) technologies. A key novelty of the paper is our analysis on how the proposed EU AI Regulation (AIR) will shape future uses of IFS in policing. AIR makes LFR a prohibited form of AI and EAI use by law enforcement will be regulated as high-risk AI that has to comply with new rules and design requirements. Part III presents a series of 10 practical lessons, drawn from our reflections on the legal and empirical perspectives. These aim to inform any future law enforcement use of IFS in the UK and beyond.

8.G. Skills building seminar: AI for public health dummies – overview and a benchmark algorithm: Random Forests

Abstract Added value The development of digital health is crucial to the improvement of health care services and systems, and AI / Machine learning (ML) is undoubtedly a significant part of it. AI has brought tremendous improvements in multiple domains, particularly in medical imaging or biosignal detection, and is currently under development for decision-making aid and early diagnostic prediction (precision medicine). However, it is often seen as an overly complicated set of methods that few people understand and do not offer many advantages over traditional methods for addressing real-life problems and may even pose risks to health system users. In this regard, on April 21, 2021, the European Commission adopted a proposal for a regulation (the AI Regulation) on ‘artificial intelligence systems,' which it describes as ‘the first-ever legal framework on AI.' According to the AI Regulation, an AI system is a ‘software that is developed with one or more of [certain] approaches and techniques ... and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.' For public health professionals, the challenges are in understanding the key concepts and technical aspects of AI and identifying its most relevant applications in terms of impact on population health and health systems. This skill-building seminar aims to exhibit and discuss the usefulness of AI methods in public health and introduce its main concepts and frameworks as well as some important methods and algorithms. In order to transfer practical skills to the participants, we will introduce a particular algorithm, Random Forests, focusing on its main theoretical ideas and how to use it in practice for modeling (with Python and R languages). We will also present practical cases of random forest use in public health and, finally, discuss their many advantages and potential limitations compared to more traditional statistical methods. Coherence between the presentations & workshop topic We will discuss the importance of AI and machine learning within the digital health domain and its potential benefits, particularly concerning predictive medicine and health care services and systems' improvement. We will present real-life case studies using the capabilities of random forests for a variety of modeling tasks. Finally, we will seek to improve the skills of public health professionals in AI, particularly in machine learning, through the transfer of theoretical and practical (coding) knowledge. Format of the workshop The seminar will be composed of two presentations, each lasting 20-25 minutes. We will interact with the participants about the practical and valuable aspects of the methods presented during the presentations. After/Before the presentation, we will foster a discussion about the usefulness of these methods in the public health field. Key messages The use of AI could be very beneficial to digital public health for decision-making and prediction purposes – as long as efforts are made to grasp its concepts and abilities. Random Forests is a powerful, easy to use, and volatile method, offering valuable aid to traditional statistical methods for various complex tasks and difficult data handling situations.

Guest Note on the proposed EU AI Regulation

Guest Note on the proposed EU AI Regulation

Philosophical Foundations of AI Regulation

Philosophical Foundations of AI Regulation

Law, Code and Exploitation: How Corporations Regulate the Working Conditions of the Digital Proletariat

Contrary to what orthodox Marxism claims, the article defends that the legal field has been a fundamental aspect of the capitalist social ordering, and an unavoidable feature to understand how dominated subjectivities are produced and exploited. Expanding Lessig’s concept of ‘code as law’ with Marxist scholarly insight, the article argues that digital capitalists are reorganising work and the labour force through a form of algorithmic regulation. The article states that algorithms – that is, digital machines – have become not only part of the means of production of the era of automation, but also the code by which capitalists are writing the conditions of existence and exploitation of the digital proletariat. The article bridges recent contributions on labour law, AI and algorithmic regulation with the latest Marxist sociological contributions analysing the relation of work and digital exploitation, opening with it new ways to understand how sociotechnical systems owned by corporations regulate the behaviour not only of the working class but of the wider citizenry.

Towards legal regulation of artificial intelligence

This paper addresses the current state of AI regulation and discusses the feasibility and need for regulation through legally binding instruments, beyond the field of ethics. History shows what can happen if the State withdraws and allows private companies to set their own exclusive regulatory standards. We have a unique opportunity to create laws and principles governing AI on a common basis and with an indispensable public-private partnership that should preferably be international in scope with the leadership of the UN or, failing that, at European level led by EU institutions, as they had already achieved in the areas of privacy and data protection.

The political AI

This article adopts a political theoretical perspective to address the problem of AI regulation. By disregarding the political problem of enforceability, it is argued that the applied ethics approach dominant in the discussions on AI regulation is incomplete. Applying realist political theory, the article demonstrates how prescriptive accounts of the development, use, and functioning of AI are necessarily political. First, the political nature of the problem is investigated by focusing on the use of AI in politics on the one hand and the political nature of the AI regulation problem on the other. Second, the article claims that by revisiting some of the oldest political and theoretical questions, the discourse on guidelines and regulation can be enriched through the adoption of AGI and superintelligence as tools for political theoretical inquiry.

AI and the resurrection of Technological Determinism

This paper elaborates on the connection between the AI regulation fever and the generic concept of Social Control of Technology. According to this analysis, the amplitude of the regulatory efforts may reflect the lock-in potential of the technology in question. Technological lock-in refers to the ability of a limited set of actors to force subsequent generations onto a certain technological trajectory, hence evoking a new interpretation of Technological Determinism. The nature of digital machines amplifies their lock-in potential as the multiplication and reuse of such technology is typically almost cost-free. I sketch out how AI takes this to a new level because it can be software and an autonomous agent simultaneously.