A 64.1mW Accurate Real-Time Visual Object Tracking Processor With Spatial Early Stopping on Siamese Network

Abstract

A low power real-time visual object tracking (VOT) processor using the siamese network (SiamNet) is proposed for mobile devices. Two key features enable a real-time VOT with low power consumption on mobile devices. First, correlation-based spatial early stopping (CSES) is proposed to reduce the computational workload. CSES reduces ~56.8% of the overall computation of the SiamNet by gradually eliminating the background. Second, the dual mode reuse core (DMRC) is proposed for supporting both the convolution layer and the cross-correlation layer with high core utilization. Finally, the proposed VOT processor is implemented in 28 nm CMOS technology and occupies 0.42 mm2. The proposed processor achieves 0.587 for the success rate and 0.778 for the precision in the OTB-100 dataset with SiamRPN++-AlexNet. Compared to previous VOT processors, the proposed processor shows state-of-the-art performance while showing lower power consumption. The proposed processor achieves 64.1 mW peak power and 58.2 mW tracking power consumption at 32.1 frame-per-second (fps) real-time VOT on mobile devices.