SplitStream: Distributed and workload-adaptive video analytics at the edge

Abstract

Deep learning-based video analytics is computation-intensive. Manufacturers such as Nvidia have launched many embedded deep learning accelerators and are rapidly gaining market share. However, the computing resources of such accelerators are still limited and heterogeneous. Although existing systems aim at optimizing video query tasks from a variety of perspectives, they rarely consider the general cooperation between heterogeneous edge devices and the dynamic workload of video content. In this work, we present SplitStream, a distributed system for accelerating video query tasks across heterogeneous edge devices, which is able to fully utilize the resources on each device and adapt to the workload dynamics. The key to achieving this is the data parallelism brought by the multi-instance mechanism and the dynamic workload adaptability brought by the two-level workload balancing mechanism. Evaluation results show SplitStream reduces the result retrieval time by up to 19% and improves the resource utilization by up to 234%.