Humans in the GDPR and AIA governance of automated and algorithmic systems. Essential pre-requisites against abdicating responsibilities

Abstract

The GDPR mandates humans to intervene in different ways in automated decision-making (ADM). Similar human intervention mechanisms can be found amongst the human oversight requirements in the future regulation of AI in the EU. However, Article 22 GDPR has become an unenforceable second-class right, following the fate of its direct precedent -Article 15 of the 1995 Data Protection Directive-. Then, why should European policymakers rely on mandatory human intervention as a governance mechanism for ADM systems? Our approach aims to move away from a view of human intervention as an individual right towards a procedural right that is part of the culture of accountability in the GDPR. The core idea to make humans meaningfully intervene in ADM is to help controllers comply with regulation and to demonstrate compliance. Yet, human intervention alone is not sufficient to achieve appropriate human oversight for these systems. Human intervention will not work without human governance. This is why DPIAs should play a key role before introducing it and throughout the life-cycle of the system. This approach fits better with the governance model proposed in the Artificial Intelligence Act. Human intervention is not a panacea, but we claim that it should be better understood and integrated into the regulatory ecosystem to achieve appropriate oversight over ADM systems.