Abstract
Object detection is essential for an autonomous driving sensing system. Since the light condition is changed in unconstrained scenarios, the detection accuracy based on visible images can be greatly degraded. Although the detection accuracy can be improved by fusing visible and infrared images, existing multispectral object detection (MOD) algorithms suffer from inadequate inter-modal interaction and a lack of global dependence in the fusion approach. Thus, we propose a multispectral object detection framework called YOLO-MS by designing a Feature Interaction and Self-Attention Fusion Network (FISAFN) as the backbone network. Within the FISAFN, correlations between two modalities are extracted by the Feature Interaction Module (FIM) for reconstructing the information components of each modality and enhancing capability of information exchange. To filter redundant features and enhance complementary features, long-range information dependence between two modalities are established by using a Self-Attention Feature Fusion Module (SAFFM). Thus, a better information richness of the fused features can be achieved. Experimental results on the FLIR-aligned dataset and the MFD dataset demonstrate that the proposed YOLO-MS performs favorably against state-of-the-art approaches, including feature-level fusion and pixel-level fusion. And further, the proposed YOLO-MS possesses good detection performance under diverse scene conditions.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE Transactions on Cognitive and Developmental Systems
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.