CRITICAL SIMULATION: INVESTIGATING THE WORK OF MACHINE VISION IN VISUAL SOCIAL MEDIA CULTURE

Abstract

Social media are in the midst of an emphatic visual turn (Highfield & Leaver 2016), characterised by the convergence of everyday visual expression with professional creative practice and advertising. The advertising-driven business models of social media platforms increasingly depend on automation. Platforms’ use of machine vision is a key frontier in the algorithmic classification of culture. Machine vision algorithms automatically classify and misclassify faces, expressions, objects, and brand logos in the images users create and share. Images shared by platform users form vast databases used to train these same algorithms. Despite widespread use by social platforms, machine vision is poorly understood and accounted for in the study of everyday visual cultures. In this paper we detail a critical response to the use of automation in visual social media, called critical simulation. We outline a critical simulation framework, the ‘Image Machine’, focussed firstly on Instagram. The Image Machine comprises an Instagram data harvester, and open-access machine vision toolbox that allows digital humanities researchers to interrogate the inner workings of these algorithms and analyse their visual (mis)classifications. In this paper we showcase results from the Image Machine applied to images emanating from a major Australian music festival, Splendour in the Grass. This case examines not only how machine vision classifies and operates on culture, but also how these techniques are being operationalised within the advertising model and promotional culture of platforms like Instagram. We argue that the commercial application of machine vision is interdependent with the participatory culture of platforms like Instagram.