Quantizing Neural Networks for Low-Power Computer Vision

Abstract

Over the last years, Neural Networks (NNs) have been widely adapted in Computer Vision (CV) applications. While for many tasks they outperform traditional CV algorithms they often come at a high compute cost. Even mobile friendly architectures such as MobileNet still require hundreds of million floating point operations. To further reduce the energy efficiency and latency of NNs, quantization can be used to replace the original floating-point operations with low bit fixed-point operations. In this chapter we introduce NN quantization for low-power computer vision. Afterward we highlight recent advances in post-training quantization, a class of algorithms that can be applied to pretrained NNs and do not require any expert knowledge. In the last part we will focus on quantization-aware training, a technique that trains NNs with simulated quantization operations. Take-aways Introduces neural network quantization Serves as a practical guide to quantization simulation with HW considerations Introduces state-of-the-art post-training quantization (PTQ) techniques that are easy to use. Introduces state-of-the-art quantization-aware training (QAT) approaches that result in best performance. Defines standard PTQ and QAT pipeline and evaluates them on several computer vision models and tasks.