Judges-in-the-loop? Judicial involvement in human oversight of high-risk decision support systems under the EU AI Act

Technological advancements have transformed contemporary societies, allowing for the proliferation of Artificial Intelligence. Nevertheless, artificial intelligence has raised several concerns regarding the impact of artificial intelligence on fundamental rights. The European Union (EU) has addressed the problems arising from the adoption of Artificial Intelligence in regulation, policy, and legal challenges. This article examines the multifaceted relationship between Artificial Intelligence (AI) and fundamental rights within the European Union (EU), highlighting the legal, ethical, and governance challenges posed by rapidly advancing AI technologies. It outlines the EU’s foundational rights framework - rooted in the Charter of Fundamental Rights, international human rights instruments, and the European Convention on Human Rights - and analyses how AI systems intersect with key rights, including privacy and data protection, equality and non-discrimination, freedom of expression and information, human dignity, autonomy, and procedural safeguards. The paper reviews major regulatory instruments such as the General Data Protection Regulation (GDPR) and the proposed AI Act, emphasizing their risk-based approaches, obligations for high-risk systems, and implications for accountability and human oversight.

Social Rights in Refugee Law and Human Rights Law: The Non-discrimination Principle as a Harmonization Tool

It is rather trendy to talk about the ‘horizontal effect’ of fundamental rights in a society where the dividing line between public and private power is increasingly blurred. However, as soon as one tries to define the exact meaning of horizontal effect and its conditions, several issues swiftly arise. This article explores the past, the present, and the future of the horizontal direct effect in the European Union (EU) legal order, by focusing on the application of this doctrine to the fundamental labour rights today protected in the Charter of fundamental rights. Most notably, with the approaching of the fifth birthday of Bauer and Max-Planck, by looking at the case of the fundamental right to paid annual leave under Article 31(2) of the Charter, this article aims at unravelling the actual meaning of the conditions for the horizontal direct effect of the EU fundamental rights and the actual perimeter of this direct effect. Although the effectiveness of minimum labour rights mandatorily enhanced by this case law should be welcomed, it is argued that the doctrine of the horizontal direct effect of EU fundamental rights seems still stuck between being a proper constitutional doctrine and a mere type of direct effect. It is also claimed, first, that horizontal direct effect is not a unitary phenomenon in EU law. In spite of the continuity in the ECJ’s technical reasoning, there is indeed a striking contrast between the language of ‘perfect equalisation’ of the Charter to the treaties and its actual approach to the assessment of the horizontal direct effect of fundamental (labour) rights. The ECJ’s case law analysis on the fundamental right to paid annual leave will prove this point. Secondly, it is suggested that horizontal direct effect can be defined as a sui generis, on-call, subsidiary, corollary, and ‘amputee’ doctrine, and we will offer some considerations on each of these features by reference to the Court’s practice under investigation. Thirdly, we suggest that the horizontal direct effect of fundamental (labour) rights relies more on the history and trajectory of the EU regulation of that specific right in the EU legal order rather than on the wording of the Charter’s provision or other technicalities.

As AI systems are increasingly adopted in high-stakes domains such as healthcare, autonomous driving, and criminal justice, their failures may threaten human safety and rights. Human oversight of AI systems is therefore critically important, as a potential safeguard to prevent harmful consequences in high-risk AI applications. Although regulations like the European AI Act mandate human oversight for high-risk AI, we lack methodologies and conceptual clarity to implement it effectively. Independent of policy and regulation, poorly designed oversight can create dangerous illusions of safety while obscuring accountability. This interdisciplinary workshop aims to bring together researchers from various disciplines, including AI, HCI, psychology, law, and policy, to address this critical gap. We will explore the following questions — How can we design AI systems that enable meaningful human oversight? What methods effectively communicate system states and risks to human overseers? How do we ensure scalable and effective interventions? Through papers, talks, and interactive group discussions, participants will identify oversight challenges, examine stakeholder roles, discuss supporting tools, methods, regulatory frameworks, and establish a collaborative research agenda. Our central goal is to further a roadmap that enables effective human oversight for the responsible deployment of AI in society.

As technologies acquire self-operating capabilities – whether automated or autonomous – their safe use demands human oversight. Yet these capabilities create challenges for the existing oversight regime in safety rules. This paper identifies regulatory challenges for the human oversight of civil drones as found in the European Union (EU) rules on the operational (Commission Implementing Regulation (EU) 2019/947), design (Commission Delegated Regulation (EU) 2019/945), and traffic (Commission Implementing Regulation (EU) 2021/664) safety of civil drones. Accordingly, the oversight regime is challenged by self-operating systems in three ways: (a) due to a problematic definition of the term ‘remote pilot’; (b) because (fully) autonomous operations are treated paradoxically; and (c) due to inadequacies of design rules. Collectively, these challenges have implications for the human-centrism to which the EU remains committed. Such challenges can be addressed by revisiting different human roles as well as the ‘operation-centric’ regulatory approach of EU drone law.

The aim of this Article is to show that the enforcement of Article 2 of the Treaty on European Union (‘TEU’) values vis-à-vis Member States could benefit from the application of the EU Charter of Fundamental Rights (‘CFR’) also in instances where the current interpretation of Article 51(1) CFR prevents this. This would be the case if the CFR were also applicable to purely domestic cases, eg—but not only—with regard to fundamental rights-relevant violations related to the values enshrined in Article 2 TEU. In this case, the European Court of Justice, which has already partly taken this path recently, could prevent the violation of core EU values. The most important historical challenge to those values in Europe today is the systematic dismantling of the rule of law and democracy in certain Member States. It is the very purpose of fundamental rights to provide answers to such dangers. When one speaks of the rule of law and democracy, one necessarily also means fundamental rights. This Article thus advocates an EU which perceives itself as a complete fundamental rights union. While the traditional interpretation of Article 51(1) CFR had a balanced division of competence between the EU and its Member States in mind, the disregard of Article 2 TEU values triggers a clausula rebus sic stantibus: the neat federal balance can only be upheld if both ends stick to the original promise made. It demonstrates two ways of completing the European fundamental rights union: treaty revision, on the one hand, and reinterpretation of Article 51 CFR, on the other hand. Both ways have in common that only a complete fundamental rights union can establish a system of fundamental rights protection that is uniform and thus equality-preserving in cases where national fundamental rights fail to provide sufficient protection.

Protection of constitutional identity in light of the jurisprudence of the Constitutional Court of the Republic of Poland – a comparative study

The article examines the legal framework for the use of artificial intelligence in administrative proceedings, emphasizing the significance of risk classification for effective regulation. The EU AI Act distinguishes between prohibited, high-risk, transparency-risk, and minimal-risk AI systems. AI tools designed to support judicial decision-making are generally classified as high-risk due to their potential effects on fundamental rights, procedural fairness, and legal certainty. The article analyzes the principal legal requirements for such systems, including risk management, data governance, technical documentation, transparency, human oversight, and cybersecurity. The issue of human oversight in the judicial context receives particular attention. Although judges may formally participate in human oversight mechanisms, they should not hold primary or exclusive responsibility for monitoring high-risk AI systems. Article 14(4) of the EU AI Act requires a degree of technical competence and operational control that cannot reasonably be expected from judges. Consequently, a more balanced model is proposed in which judges retain responsibility for legal assessment and final decisions, while technical experts and court administration assume responsibility for system-level supervision and operational control. The article further explores the potential integration of AI into Ukraine's administrative proceedings via the Unified Judicial Information and Telecommunication System. Should Ukraine enact legislation similar to the EU AI Act, the legal status of the State Judicial Administration of Ukraine will depend on the development and implementation of relevant AI modules, potentially qualifying it as either a provider or a deployer. The article concludes that any future Ukrainian model for AI use in administrative proceedings should ensure clear role allocation, risk-based differentiated regulation, and a realistic assessment of the limits of judicial involvement in technical oversight.

This article is an examination of human oversight in EU policy for controlling algorithmic systems in automated legal decision making. Despite the shortcomings of human control over complex technical systems, human oversight is advocated as a solution against the risks of increasing reliance on algorithmic tools. For law, human oversight provides an attractive, easily implementable and observable procedural safeguard. However, without awareness of its inherent limitations, human oversight is in danger of becoming a value in itself, an empty procedural shell used as a stand-in justification for algorithmization but failing to provide protection for fundamental rights. By complementing socio-legal analysis with Science and Technology Studies, critical algorithm studies, organization studies and human-computer interaction research, the author explores the importance of keeping the human in the loop and asks what the human element at the core of legal decision making is. Through algorithmization it is made visible how law conceptualises decision making through human actors, personalises legal decision making through the decision-maker’s discretionary power that provides proportionality and common sense, prevents gross miscarriages of justice and establishes the human encounter deemed essential for the feeling of being heard. The analysis demonstrates the necessary human element embedded in legal decision making, against which the meaningfulness of human oversight needs to be examined.

The decision of the British people to leave the European Union (EU) raises foundational questions for many legal fields. The effects are especially likely to be felt within domestic employment law, which now has a strong basis in EU law. Of particular concern is the removal of the nascent EU fundamental employment rights influence over domestic legislation. Employment lawyers have long relied on fundamental rights as a means of preserving the autonomy of their subject from general private law. One manifestation of this turn to fundamental rights concepts has been the ‘constitutionalisation’ of employment rights. EU law, notably the Charter of Fundamental Rights, has become a key underpinning of this constitutionalisation process. This article considers the effects of the constitutionalisation in the United Kingdom employment sphere of some of the rights found in the Charter’s Solidarity Title, through its role in the emergence of a hierarchy of sources or ‘norms’ in the employment field. In order to address the question of the Charter’s influence on the hierarchy of sources in the employment context, three interrelated processes are examined. The article begins by exploring the ‘constitutionalisation’ process, by setting out the nature of the Charter and the effects of its employment rights on the hierarchy of sources. This is followed by a consideration of the ‘deconstitutionalisation’ process brought about by Brexit, before finally examining whether a potential ‘reconstitutionalisation’ process might be underway by looking at key terms of the EU (Withdrawal) Act 2018 and the potential to replicate the Charter in domestic law.

Aida Torres Perez� Conflicts of Rights in the European Union. A Theory of Supranational Adjudication 1 is a comprehensive monograph dealing with one of the most striking normative challenges in the European Union (EU): the relationship between the European Court of Justice (ECJ) and Member State courts in adjudicating fundamental rights. Torres Perez presents the existing spheres of fundamental rights protection in the EU and provides a thorough analysis of the conflicts that emerge where these different spheres overlap. Her volume covers a number of different approaches and provides suggestions on how to deal with these conflicts and eventually proposes a normative model for ECJ adjudication through judicial dialogue based on comparative constitutional reasoning. The book is well structured in three parts. The first part gives a brief but thorough overview of the different systems of fundamental rights protection open to EU citizens. The author describes these different systems as the multilevel protection of rights in Europe and distinguishes between human rights protection through national constitutions (constitutional rights), through the ECJ (EU fundamental rights) and through the European Convention on Human Rights (convention rights). She outlines the conflicts that arise when these different systems of fundamental rights protection overlap. In general, such conflicts may arise when different rights are considered to be fundamental (at 10) and where community members disagree regarding fundamental rights interpretation (at 11), especially concerning sensitive issues like abortion or affirmative action (at 16). A potential for conflict exists whenever states act within a field of application of EU law which includes two types of situations: (i) state acts implementing EU law, and (ii) state acts derogating from the EU basic freedoms of movement (at 16). An example of a rights conflict between German courts and the ECJ is the �banana saga�, where the courts disagreed on �

INTRODUCTION One of the principal constitutional functions of the European Court of Justice (ECJ) is to secure legitimacy of the European Union (EU) system of governance by protecting the fundamental values on which this system is said to rest. This role of the ECJ – as a guardian of democracy, fundamental rights and the rule of law – is complicated by the multi-layered nature of the EU's sources of legal authority: the EU institutions and the law-making bodies of Member States (MSs), and by the heavy reliance on the co-operation of National Courts (NCs) in building up and maintaining the EU's legal system. I focus on providing an overview of selected problems that are directly related to the ECJ's legitimising role, and I will do so from the perspective of the ECJ's constitutional politics. First, I will consider whether the alleged prioritising – by the ECJ – of the principles of integrity of the EU legal system and economic regulations over constitutional values – such as fundamental rights and democracy – amounts to an ideological project that undermines the legitimacy of the EU law. I will focus on instances where the ECJ failed to recognise concerns of the NCs related to the standard of protection of these values in the EU law for the sake of protecting its own authority and the uniformity of the EU law. Historically, these concerns were most famously identified in the so-called ‘supremacy’ challenges, but there was also a string of other cases where the protection of market freedoms clashed with fundamental rights and other constitutional principles. Th is period of the ECJ jurisprudence took a more rights-friendly turn in cases such as Omega and Schmidberger . The more recent NCs challenges yet again put the ECJ under pressure, this time because of its reluctance to recognise the higher standards of fundamental rights protection related to criminal trials in national constitutions over those existing in the EU law. The string of the European Arrest Warrant (EAW) cases seems to suggest that the ECJ yet again entered a path of prioritising the supremacy of the EU law over substantive constitutional objectives.

The impact of the COVID-19 outbreak is being endured throughout the world, and the European Union (EU) is no exception. The rapid spreading of the virus effected, among other things, restriction on the freedom of movement. The EU member states introduced national response measures to contain the pandemic and protect public health. While broadly similar, the measures differ with regard to strictness and the manner of introduction, reflecting the political legitimacy of the respective country. With the ‘Guidelines concerning the exercise of the free movement of workers during COVID-19 outbreak’ – its first COVID-19-related Communication – the European Commission (EC) attempted to curb differing practices of the EU member states and ensure a coordinated approach. Ultimately, this action was aimed at upholding of fundamental rights as guaranteed to EU citizens, one such being the freedom of movement. Thus, from the very start of the pandemic, the coordinated actions of EU institutions sought to contain the spread of COVID-19 infections with the support and cooperation of EU member states. This is confirmed by the most recent Council of the EU (Council) recommendation on a coordinated approach to restrictions to freedom of movement within the EU of October 2020. While they did prevent the spread of infection and save countless lives, the movement restriction measures and the resulting uncertainty have greatly affected the people, the society, and the economy, thereby demonstrating that they cannot remain in force for an extended period. This paper examines the measures introduced by EU member states and analyses the legal basis for introducing therewith limitations on human rights and market freedoms. To what extent are the EU and member states authorized to introduce restrictions on the freedom of movement in the interest of public health? Have the EU and member states breached their obligations regarding market freedoms and fundamental rights under the Treaty? And most importantly: have they endangered the fundamental rights of the citizens of the EU?

This chapter compares which circumstances make Charter and the fundamental rights attached to European Union (EU) citizenship applicable at national level. The comparison shows that the application of Union citizenship rights at national level, as developed in the case law of the Court of Justice, is not exclusively triggered in situations where an individual has exercised his or her rights attached to Union citizenship. Ever since the Court in 1969 essentially ruled that identifying a beneficiary for cheaper butter on a social welfare coupon is disproportionate, the Court has protected fundamental rights enshrined as general principles of EU law. The chapter shows how the link with EU law is traditionally created to make rights attached to EU citizenship and the Charter applicable at Member States' level. It has been argued that the Charter should apply also in situations at national level where EU law was not concretely implemented. Keywords: Court of Justice; EU law; European Union (EU) citizenship; fundamental rights; Member States

The aim of this chapter is to explore whether the European Charter of Fundamental Rights (ECFR) can improve the protection of trainees by broadening the scope of national labour laws or the labour directives of the European Union (EU), or by creating new labour rights directly applicable to trainees. I argue that, despite the fundamental nature of many important labour rights, the limitations to the application of the ECFR, as well as the restrictive interpretation of its provisions, reduce the ability of the charter to improve the rights of trainees. However, the ECFR is a useful tool for the Court of Justice of the EU to broaden the personal scope of secondary EU legislation, and to contest national limitations to the application of fundamental labour rights. If EU and national labour law provisions were consciously interpreted in the light of fundamental rights, the rights of trainees would improve slightly.