Conclusion: Balancing Data-Driven Personalisation and Law as Social Systems

Abstract

The contributions in this volume have examined the wide-ranging implications of data-driven personalisation. This conclusion will aspire to consolidate them through the use of systems theory. Systems theory identifies some entities as capable of sustaining themselves through adaptive reproduction of their own subcomponents and internal organisational schemas. These self-ordering systems are deemed autopoietic (Greek for ‘self-producing’). While systems theory has its origins in the biological sciences, it has become a prominent means of understanding social systems as well. For social systems, ‘the basic element…is communication’. The defining feature of an autopoietic social system is that it autonomously reproduces itself through internal communication. One way of imagining data-driven personalisation is as one of the subcomponents of society as a whole. Yet even more intriguing is the conceptualisation of data-driven personalisation as its own autopoietic system. The idea is an elegant fit: data-driven personalisation consists of a recursive feedback loop between profiled persons who provide a continuously evolving mass data set, and an algorithm that, working only with such data, provides continuously updated recommendations to these persons. A data-driven personalisation system is thus iteratively self-replicating through communication between its two main subcomponents: profiled persons and a corresponding data set. Imagining data-driven personalisation as a self-contained autopoietic system as well as part of the broader autopoietic system of society illuminates central themes in the volume and connects its diverse contributions. Firstly, such a conceptualisation unifies the inability of consent to adequately give persons control over data-driven personalisation. Because the substance of data-driven personalisation is continually redefined through interaction and adaptation, a static grant of permission is woefully inadequate to enable persons to control their information. Furthermore, insofar as data-driven personalisation redefines the persons themselves, they are not appropriately positioned to monitor and control their role in personalisation as if they were an observer. Secondly, recognising that data-driven personalisation is autopoietic explains why it is so difficult to purge the system of its power inequities. Systems theorists recognise that systems have moments of genesis and operate in environments that provide continuous external stimulation. But once an autopoietic system has internalised the prejudices of its creation, these tendencies cannot be purged by deliberate intervention. By its very self-sustaining nature, the system cannot be ‘reprogrammed’ from the outside – a characteristic that explains many of the biases that plague data-driven personalisation. This conclusion offers a final thought as to how law can be leveraged to address these challenges of personalisation, by considering both as subcomponents of society more broadly. Because any data-driven personalisation system is dynamic and self-contained, attempts to tame it through a single legal concept (equality, privacy, non-discrimination) will likely face parallel problems to consent: the system will simply internalise the efforts as environmental ‘shocks’ in its process of recursive self-perpetuation. However, system theorists have also redefined another subcomponent of society, the legal system, as autopoietic. By adaptively counterbalancing the invasive and hierarchical potential of personalisation, law may not be able to ‘purify’ personalisation of its pathologies, but it can soften its harmful impacts on society and individuals.