Multimodal Polarimetric And Color Fusion For Road Scene Analysis In Adverse Weather Conditions

Abstract

Road scene analysis is a fundamental task for both autonomous vehicles and ADAS systems. Nowadays, most of autonomous vehicles are able to properly detect objects in good weather conditions; however, some improvements still need to be done when the visibility is altered. Polarimetric imaging is a rich modality that enables to describe an object by its physical information and has recently shown great performances in enhancing road scenes analysis under adverse weather conditions, especially under fog. Besides, it has been shown in a previous work that this modality could be complementary to classical color images especially for car detection. In this work, four different multimodal fusion schemes as well as different color and polarimetric features combinations are explored to achieve a robust scene analysis. The combination of both modalities could be a great asset to describe road scenes when the visibility is altered. Experimental results have shown that, using a well chosen fusion scheme with an adapted features combination, the detection of objects in road scenes under fog was reinforced. The different detection tasks show a significant improvement when using the adapted fusion scheme and features combination. Thanks to the architecture of the fusion scheme and to the properties of the selected features, these results could be extended to other adverse weather conditions.