A 49.5 mW Multi-Scale Linear Quantized Online Learning Processor for Real-Time Adaptive Object Detection

Abstract

Online training is essential to maintain a high object detection (OD) in various environments. However, additional computation workload, EMA, and high bit precision is the problem of conventional online learning scheme on mobile devices. Therefore, a low power real-time online learning OD processor is proposed with three key features. First, multi-scale linear quantization (MSLQ) and MSLQ-aware PE structure are proposed for low-bit computation. Second, channel-wise gradient skipping is proposed to reduce computation and EMA based on temporal correlation. These schemes reduce ~56% of computation burden and ~30% of EMA, and also improve detection accuracy. Lastly, gradient norm clipping with norm estimation achieves 3.8 mAP improvement at YouTube-Objects dataset by fast adaptation with under 1% of the additional computation. Finally, the proposed online learning OD processor is implemented in 28 nm CMOS technology and occupies 1.2 mm². The proposed processor achieves 78 mAP of detection accuracy at the YouTube-Objects dataset. Compared to the previous OD processor, this brief shows state-of-the-art performance by achieving 49.5 mW power consumption and 34.4 frame-per-second real-time online learning OD on mobile devices.