Extending Reliability of mmWave Radar Tracking and Detection via Fusion With Camera

Renyuan Zhang,Siyang Cao

doi:10.1109/access.2019.2942382

Renyuan Zhang, Siyang Cao

Open Access

https://doi.org/10.1109/access.2019.2942382

Copy DOI

Abstract

In this paper, a new radar-camera fusion system is presented. The fusion system takes into consideration the error bounds of the two different coordinate systems from the heterogeneous sensors, and further a new fusion-extended Kalman filter is utilized to adapt to the heterogeneous sensors. Real-world application considerations such as asynchronous sensors, multi-target tracking and association are also studied and illustrated in this paper. Experimental results demonstrated that the proposed fusion system can realize a range accuracy of 0.29m with an angular accuracy of 0.013rad in real-time. Therefore, the proposed fusion system is effective, reliable and computationally efficient for real-time kinematic fusion applications.

Highlights

Radar, working at W-band, is becoming an important sensor in advanced driver assistance system (ADAS) and autonomous driving fields [1]–[5]
The latest millimeter wave radars at W-band are surging for automobile applications, e.g., adaptive cruise control, pedestrian detection, collision avoidance, lane changing monitoring and emergency braking. mmWave radar research has gained immense popularity with the recent increasing demand of automotive radars in ADAS and autonomous driving industry, for instance, the automotive target shape estimation using relaxation algorithm [6]; the super-resolution automotive radars [7]; and the automotive radar sensor fusion with other sensors to improve target tracking and classification [4]
We present a new fusion-extended Kalman filter, which is designed to fuse data from heterogeneous sensors such as mmWave radar and monocamera with real-time fusion algorithm running for tracking

Summary

Introduction

Radar, working at W-band, is becoming an important sensor in advanced driver assistance system (ADAS) and autonomous driving fields [1]–[5]. Ongoing developments in research and industry primarily focus on safety, reliability, compact and low-cost sensor systems. ADAS systems comprise of multiple sensors, such as radar, camera and Lidar, for specific applications commensurate with the sensor’s offered advantage. MmWave radar research has gained immense popularity with the recent increasing demand of automotive radars in ADAS and autonomous driving industry, for instance, the automotive target shape (height and width) estimation using relaxation algorithm [6]; the super-resolution automotive radars [7]; and the automotive radar sensor fusion with other sensors to improve target tracking and classification [4]. Sensors’ reliability, especially for avoiding false detection, missing detection, blocking, blind spot, adverse weather and failure, is essential in ADAS and autonomous driving applications.

Objectives

Methods

Results

Conclusion