Automating social media content moderation: implications for governance and labour discretion

Abstract

Content moderation is key to platform operations. Given the largely outsourced character of content moderation work and the dynamic character of social media platforms, technology firms have to address the accompanying high degrees of uncertainty and labour indeterminacy. Central to their managerial strategies is the use of automated technology that allows them to organise work by incorporating the social media user activities within the production processes, and control workers for ensuring the accuracy of content moderation decisions. The labour process analysis is informed by two workshops with ten participants at a Berlin-based IT-services firm providing content moderation services to a lead firm based in the USA. The research design combines together the design thinking method and the focus group interview method to examine the worker–machine interaction. The research findings indicate that technical control results in continuous standardising of content moderation work through routinisation of tasks and codification of time. Its combination with bureaucratic control through the supply-side managerial functions aims to ensure the quality service delivery and points to the continued significance of human supervision. Correspondingly, there are two main contributions of our study: first, regarding the governance in content moderation value chains and second, regarding the worker experiences of technical-driven control. On account of the limited resistance observed in the labour process, we conclude that instead of seeing it as the totalisation of technical control, our findings point towards the structural conditions in Germany that restrict migrant workers’ agency.