MOD-YOLO: Multispectral object detection based on transformer dual-stream YOLO

Abstract

Multispectral object detection can effectively improve the precision of object detection in low-visibility scenes, which increases the reliability and stability of the object detection application in the open environment. Cross-Modality Fusion Transformer (CFT) can effectively fuse different spectral information, but this method relies on large models and expensive computing resources. In this paper, we propose multispectral object detection dual-stream YOLO (MOD-YOLO), based on Cross Stage Partial CFT (CSP-CFT), to address the issue that prior studies need heavy inference calculations from the recurrent fusing of multispectral features. This network can divide the fused feature map into two parts, respectively for cross stage output and combined with the next stage feature, to achieve the correct speed/memory/precision balance. To further improve the accuracy, SIoU was selected as the loss function. Ultimately, extensive experiments on multiple publicly available datasets demonstrate that our model, which achieves the smallest model size and excellent performance, produces better tradeoffs between accuracy and model size than other popular models.