Pandemic-Sanctioned AI Surveillance: Human Rights under the Threat of Algorithmic Injustice in the EU

Abstract

Attention to algorithmic injustice has long characterised the perspective of European Union (EU) institutions toward artificial intelligence (AI), given the potential threats to citizens and democracies. From a global perspective, the EU has likewise championed in the pandemic context thanks to higher attention to concerns such as privacy in the deployment of technological solutions to help control the outbreak. Nevertheless, as digital tools became more and more pervasive, their proliferation far exceeded official contact tracing apps to include a multitude of public and private surveillance solutions. Our work considers the current European regulatory framework and it highlights how problematic pandemic surveillance digital tools in terms of privacy and data protection, digital accessibility, non-discrimination and social exclusion may fall through the cracks, especially within the private sector. The legal analysis complemented by empirical examples of COVID-19 related apps assesses how the pandemic offers a breeding ground for algorithmic injustice. Similarly, we evaluate the extent to which, in its current form, the European Commission Proposal for an AI Regulation (the AI Act) may fail to fully mitigate in practice such threats to human rights. Specifically, the contribution of the paper is to highlight how - even in a context such as the EU where notable attention is given to citizens' rights and their balancing against the need of protecting public health - COVID-19 and its algorithmic response poses a substantial risk to human rights. More broadly, the analysis offers a cautionary tale for post-pandemic societies in which AI surveillance is bound to remain a ubiquitous feature, for which current regulatory efforts may not prove sufficient guarantees.