Distributed data analytics

Abstract

The advertising and associated online industries in particular have fuelled a rapid rise in the deployment of personal data collection and analytics tools. Distributed data analytics, where code and models for training and inference are distributed to the places where data is collected, has been boosted by two recent and ongoing developments: (i) increased processing power and memory capacity available in user devices at the edge of the network such as smartphones and home assistants and (ii) increased sensitivity to the highly intrusive nature of many of these devices and services and the attendant demands for improved privacy. Indeed, the potential for increased privacy is not the only benefit of distributing data analytics to the edges of the network: reducing the movement of large volumes of data can also improve energy efficiency, helping to ameliorate the ever-increasing carbon footprint of our digital infrastructure, and enable much lower latency for service interactions than is possible when services are cloud-hosted. We begin by discussing the motivations for distributing analytics and outlining the different approaches that have been taken. We then expand on ways in which analytics can be distributed to the very edges of the network, before presenting the Databox, a platform for supporting distributed analytics. We continue by discussing personalising and scaling learning on such a platform, before concluding.