HODET: Hybrid object detection and tracking using mmWave radar and visual sensors

Abstract

Image sensors have been explored heavily in automotive applications for collision avoidance and varying levels of autonomy. It requires a degree of brightness, therefore, the use of an image sensor in nighttime operation or dark conditions can be problematic along with challenging weather such as fog. Radar sensors have been employed to help cover the various environmental challenges with visible spectrum cameras. Edge computing technology has the potential to address a number of issues such as real-time processing requirements, off-loading of processing from congested servers, and size, weight, power, and cost (SWaP-C) constraints. This paper proposes a novel Hybrid Object DEtection and Tracking (HODET) using mmWave Radar and Visual Sensors at the edge. The HODET is a computing application of low SWaP-C electronics performing object detection, tracking and identification algorithms with the simultaneous use of image and radar sensors. While the machine vision camera alone could estimate the distance of an object, the radar sensor will provide an accurate distance and vector of movement. This additional data accuracy can be leveraged to further discriminate a detected object to protect against spoofing attacks. A real-world smart community public safety monitoring scenario is selected to verify the effectiveness of HODET, which detects, tracks objects of interests and identify suspicious activities. The experimental results demonstrate the feasibility of the approach.