Siamese network object tracking based on fusion of visible and event cameras

Abstract

Object tracking is widely used in daily life. The existing visible camera-based tracking methods are difficult to track objects accurately in the face of degraded scenes such as fast movement, high contrast, and low illumination, which will cause the loss of tracking performance. An event camera is a biologically driven sensor with higher dynamic range, smaller time delay, and higher light sensitivity than traditional visible cameras. We propose a siamese network object tracking method that fuses visible and event cameras to realize more reliable tracking. We design a feature fusion method combining visible and event camera features based on attention mechanism. It realizes the cooperation of both sensors and improves the accuracy of tracking. Experiments on the VisEvent dataset reveal that our network outperforms single-modality trackers by 5.8% and outperforms other fusion-based methods by 2.1%.