Uncertainties of facial emotion recognition technologies and the automation of emotional labour

Abstract

ABSTRACTAutomated detection and recognition of faces have been implemented in a broad range of media environments. Following that development, what concerns us in this article is the analysis of emotions from facial expressions using computer-based systems, in relation to which we critically investigate the use of theories of basic emotions. We explore in depth the company Affectiva’s attempts to translate, represent and schematize human emotions, as they raise a variety of problems and issues of uncertainty. We analyse the uncertainties concerning the processing of the human face ‘as image’ due to issues concerning temporality and static images as well as polyphony and modulations of the spectrum of expressions. One of our key arguments concerns the temporal character of human emotions, and we address how algorithmically regulated protocols of discretization may be said to prompt specific patterns of emotional responses and expressions based on an ideal of eliminating uncertainty. Through discussions via art pieces by Lauren McCarthy and Kyle McDonald, we question what happens when the protocols of computer systems start to perform aspects of emotional labour for us, making judgments by predicting adequate emotional responses based on the use of the strict metrics criticized in the article.