Robust Object Detection Based on Deep Neural Network and Saliency Features from Visible and Thermal Images

Abstract

Robust and accurate object detection is crucial for autonomous automotive devices. Recently, object detection has made considerable progress with deep learning. However, object detection approaches fail in challenging illumination conditions such as fog, low light environment, and darkness. To address these challenges, in this paper, we propose a novel approach based on deep neural network and saliency features for fusing RGB visible images with Thermal images and use the complementarity of the two modalities for a robust and accurate multi-category object detection under challenging conditions. Specifically, we introduce a thermal saliency map to augment the RGB Image for improving detection in various conditions. We also propose a new fusing strategy that integrates salient features as guidance in the decision stage for improving detection accuracy. Extensive experiments are conducted on UTokyo multispectral object detection dataset [1]. Experimental results show the effectiveness of the proposed approach.