Acceleration of DNN-Based Video Object Detection Using Temporal Dependency of the Object Size

Abstract

Many studies have been proposed to improve the accuracy of Deep Neural Network based object detection. Some of them bring an increase in computation, which is a problem in tasks such as autonomous driving that requires high accuracy and low latency. Feature Pyramid Network (FPN) is a structure commonly used in improving the accuracy of object detection. However, it slows down the inference speed because of the high computation. To accelerate the inference while maintaining the accuracy of FPN, this paper proposes dynamic acceleration of object detection with FPN using temporal dependency of object sizes in a video. We modify FPN to have faster inference speed when targeting certain object sizes. By using the previous object sizes, the target object size is determined. The modified FPN is used in a dynamic manner, which speeds up the inference. In this method, we achieve 20.9% faster inference at the cost of a 0.06 mAP drop on the ImageNet VID validation dataset.