CROSE: Low-light enhancement by CROss-SEnsor interaction for nighttime driving scenes

Abstract

An increasing number of image perception models are being utilized in the field of autonomous driving. During nighttime driving, the visual perception capabilities of a single-modality RGB sensor become compromised. To address this issue, image fusion methods that utilize multi-modality data to provide a more comprehensive visual representation for nighttime scenes. Nevertheless, the existing image fusion methods are limited by poor generalization abilities and lighting conditions, resulting in inadequate handling of the contrast and spectral corruption. In this paper, we propose a cross-sensor low-light enhancement algorithm that provides more accurate visual perceptions. Our approach employs multiple sensors as preceptors for nighttime driving scenes. Compared with typical fusion algorithms that use a simple one-stage workflow for cross-sensor data, our proposed method adopts an advanced content-enhancement strategy by recursively and interactively scaling up informative pixels. More specifically, our approach employs information measurement to describe the information and then utilizes a cross-sensor content enhancement module to dynamically enhance the mutual information between the infrared spectrum and the RGB streams. Our experiments show that CROSE effectively preserves texture details, resulting in clearer and more solid fusion results for nighttime driving scenes.