Towards efficient quantized inference for convolutional neural network on edge system

Abstract

Convolutional Neural Networks (CNN) have made splendid achievements in various object detection tasks. To extend the applications of CNN detection models, the implementation of model inference on edge platforms, such as ASIC, FPGA and other embedded systems, has been intensively investigated in recent years. However, the huge model size and its enormous overhead constrain the deployment of detection model on edge platform which always has limited computational capability. Quantized inference of CNN model is one of the efficient approach to running model on edge platform. In this paper, we develop a hardware-friendly quantized inference scheme of detection model that is used for efficient inference on embedded FPGA systems. The proposed method contains several techniques that are able to optimize the quantized inference of detection model on FPGA device. The experimental results demonstrate that not only make the detection model quantized inference more efficient but also maintain the accuracy of object detection.