Fairness by awareness? On the inclusion of protected features in algorithmic decisions

Abstract

AI decisions are increasingly determining our everyday lives. At present, European anti-discrimination law is process-oriented; it prohibits the inclusion of sensitive data that is particularly protected. However, especially in the context of AI decisions, constellations can be identified in which the inclusion of sensitive characteristics will lead to better and sometimes even less discriminatory result. A result-oriented approach, therefore, might be a more fitting strategy for algorithmic decision making.In this paper we examine the legal framework for including sensitive features in a Support Vector Machine for a fictitious scenario and discuss the resulting challenges in practical application. It turns out that generally ignoring sensitive features - as has been the practice up to now - does not seem to be a fitting strategy for algorithmic decision making. A process-oriented procedure only supposedly comes closer to individual case justice: If one assumes that fewer errors occur when protected characteristics are included, individuals will ultimately also be assessed incorrectly less often, especially when one protected group is more prone to errors than the other.This paper aims to support the current debate about legal regulation of algorithmic decision making systems by discussing an often neglected perspective.