Digital retina: revolutionizing camera systems for the smart city

Abstract

The primary viewpoints presented in this article are as follows: (1) The method to real-time gather and aggregate all kinds of urban big data, especially image and video data from video surveillance networks, and subsequently analyze and mine the value of these big data in the city brain to effectively support the urban operation and management is a key problem in the development of smart cities. (2) Recently, some city brains are established to mine the large visual data source to obtain valuable insights about the activities in the city (e.g., the urban traffic status). However, it is recognized that compression will inevitably affect visual feature extraction, and consequently degrading the subsequent analysis and retrieval performance. More importantly, it is impractical to aggregate all video streams from hundreds of thousands of cameras distributed across the city into a city brain for big data analysis and retrieval. These issues and challenges are rooted in the camera framework currently in use. (3) To address these challenges, a new camera framework should be developed from the fact that retina can encode both pixels and features. Such a retina-like camera, or directly referred to as digital retina, is typically equipped with a globally unified timer and an accurate positioner, and can output two streams simultaneously, including a compressed video stream for online/offline viewing and data storage, and a compact feature stream extracted from the original image/video signals for visual analysis and search. By real-time feeding only the feature streams into the city brain, these digital cameras form a compound-eye camera system for the smart city. (4) To promote the wide application of digital retinas in the smart city, the relevant works should be addressed in the near future, including standardization, hardware implementation, open-source software development, and the deployment of large-scale testbeds.