Accuracy and Fairness in Pupil Detection Algorithm

Abstract

Despite being highly accurate, widely used multimedia analysis algorithms can suffer from bias, affecting users’ trust in them. For instance, algorithms for face recognition, pedestrian detection, and image search have recently been reported to be biased. In this paper, we move the discussion on algorithmic fairness to a new domain, namely, pupil detection. We audit a widely used OpenCV algorithm for pupil detection from the perspective of fairness and accuracy. The algorithm is audited using two different datasets: a single-person image dataset (CelebA), and a group image dataset (Images of Groups). In both datasets, we found the OpenCV pupil detection algorithm to provide reasonably high accuracy but also yield statistically significant bias with respect to gender. The results provide the first empirical evidence for the existence of gender bias in pupil detection algorithms in both single person as well as group image settings. These results could have a deleterious impact on downstream multimedia applications such as identity authentication, attention assessment, and e-learning. Finally, we discuss the process of appropriately choosing the pupil detection algorithm parameters that may help reduce bias while maintaining high accuracy in various multimedia applications.