Scalable Deployments for Real-Time AI Video Stream Processing

Abstract

Real-time stream processing is becoming more prevalent today due to huge chunks of data needing to be processed upon arrival. In video streaming the need for real-time management is both important and challenging because video frames come at high frequency. AI advances have made it possible to understand video feeds at a high level in real-time, making them a valuable source of information in regards to human behavior, trends, surveillance and much more. As a consequence of these reasons, there is a need for a highly performant and deployable system in terms of latency, scalability, accuracy of results and computing power, which leverages the cloud as a service. A design for a low-latency and highly scalable system is much needed as video stream processing or video analytics has uses in areas such as surveillance, real-time video analytics, criminality and autonomous vehicles, which require fast and accurate analysis of data. Such a system should be able to employ not only streaming but also different processing actions including machine learning models which can be applied to frames resulting in data analysis or further data pipelines. We developed a scalable real-time video processing system which consumes video frames, processes them using deep learning models, renders them and stores the resulting semantic information in a database for further downstream processing. We show that our proposed processing architectures are a suitable solution for modern video analytics systems, which can be scaled both vertically and horizontally, and achieves real-time latency within maximum of 1 second for frame rates ranging from 10 to 60 fps.