Augmenting Object Detection in Adverse Conditions with Polarization-Encoded Images

Abstract

AbstractThe performance of object detection has improved significantly in the past decade with the development of deep learning. However, robust and high precision object detection in complex conditions remains a challenge, and a promising solution is to combine new sensing technology with deep learning. In this paper, we have extracted multiple types of sensing information from polarization images, and have explored using the different combinations of polarization informations to augment object recognition capability. In detail, we have built a new dataset of synchronized polarization and RGB images taken at different seasons and illumination conditions. With the obtained dataset, different types of polarization information, including raw intensity at different polarized orientations, Stokes vector, degree of polarization, and angle of polarization. In addition, we have also utilized the high dynamic range (HDR) method to preprocess the extracted polarization information. To explore the object capability with polarization-encoded images, different combinations of polarization information have been combined to train and test the deep network; to make fair comparisons, the same training and test process have been implemented with the corresponding RGB images. Results have shown that polarization images can efficiently detect objects in the scene, especially in presence of the metal object. For vehicle detection, polarization images have better robustness on sense which is not in training set. What’s more, compared to the Stokes vector, the degree of polarization and the angle of polarization also provide more features for network. HDR processing on polarization images also is effective.KeywordsObject detectionPolarization imagingDeep learning