Beyond the Command

Abstract

Machines, from artificially intelligent digital assistants to embodied robots, are becoming more pervasive in everyday life. Drawing on feminist science and technology studies (STS) perspectives, we demonstrate how machine designers are not just crafting neutral objects, but relationships between machines and humans that are entangled in human social issues such as gender and power dynamics. Thus, in order to create a more ethical and just future, the dominant assumptions currently underpinning the design of these human-machine relations must be challenged and reoriented toward relations of justice and inclusivity. This paper contributes the "social machine" as a model for technology designers who seek to recognize the importance, diversity and complexity of the social in their work, and to engage with the agential power of machines. In our model, the social machine is imagined as a potentially equitable relationship partner that has agency and as an "other" that is distinct from, yet related to, humans, objects, and animals. We critically examine and contrast our model with tendencies in robotics that consider robots as tools, human companions, animals or creatures, and/or slaves. In doing so, we demonstrate ingrained dominant assumptions about human-machine relations and reveal the challenges of radical thinking in the social machine design space. Finally, we present two design challenges based on non-anthropomorphic figuration and mutuality, and call for experimentation, unlearning dominant tendencies, and reimagining of sociotechnical futures.