MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance.

Jianxiao Zhu,Zhengliang Sun,Xiang Song,Qimin Xu,Xu Li,Peng Jin

doi:10.3390/s21010027

Jianxiao Zhu, Zhengliang Sun + Show 4 more

Open Access

https://doi.org/10.3390/s21010027

Copy DOI

Abstract

As an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolutional network model, called multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO), with consideration of hybrid realistic scene of scales, illumination, and occlusion. MME-YOLO consists of two tightly coupled structures, i.e., the enhanced inference head and the LiDAR-Image composite module. More specifically, the enhanced inference head preliminarily equips the network with stronger inference abilities for redundant visual cues by attention-guided feature selection blocks and anchor-based/anchor-free ensemble head. Furthermore, the LiDAR-Image composite module cascades the multi-level feature maps from the LiDAR subnet to the image subnet, which strengthens the generalization of the detector in complex scenarios. Compared with YOLOv3, the enhanced inference head achieves a 5.83% and 4.88% mAP improvement on visual dataset LVSH and UA-DETRAC, respectively. Integrated with the composite module, the overall architecture gains 91.63% mAP in the collected Road-side Dataset. Experiments show that even under the abnormal lightings and the inconsistent scales at evening rush hours, the proposed MME-YOLO maintains reliable recognition accuracy and robust detection performance.

Highlights

Received: 17 October 2020 Accepted: 21 December 2020 Published: 23 December 2020Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.Vehicle detection is a well-known question in traffic scenes, especially for intelligent vehicles
We first briefly review the related works of traditional vehicle detectors in the traffic surveillance system, we introduce the deep-learning-based vehicle detectors from the vehicle perspective, we compare several multi-sensor fusion architectures from on-board views, and we introduce the attention skills used in visual detection networks for multi-scale objects
We propose a multi-sensor multi-level composite fusion network for robust multi-scale vehicle detection under variable lighting conditions

Summary

Introduction

Received: 17 October 2020 Accepted: 21 December 2020 Published: 23 December 2020Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.Vehicle detection is a well-known question in traffic scenes, especially for intelligent vehicles. Received: 17 October 2020 Accepted: 21 December 2020 Published: 23 December 2020. Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Vehicle detection is a well-known question in traffic scenes, especially for intelligent vehicles. With such algorithms, smart cars can identify and predict dynamic targets in the surrounding environment, thereby reducing accidents such as collisions. Through the vehicle detection and behavior prediction algorithm, the cooperative roadside system can give early warning signals to vehicles that may have accidents in the near future.

Methods

Results

Conclusion