Binarized and Full-Precision 3D-CNN in Action Recognition

Abstract

As a part of the video classification task, action recognition is also known as a task with heavy computational load, with models mostly trained on devices with multiple GPUs. The pre-trained models suffer from large size and take a long time to infer the test data, especially on devices with low specifications and mobile devices. The recent development of neural networks introduces the Binarized Neural Network (BNN), which offers a solution to these problems. BNNs are trained with binary activations and weights, which reduces the computation from 32-bits to 1-bit. Theoretically, this feature can perform using 32x less memory and hardware resource compared to the conventional, full-precision neural networks. Theoretically, the conversion from full-precision CNN to BNN should result in a smaller model size and faster inference time. In this research, a binarized 3D CNN model is built using the principles of BNN and tested against the full-precision CNN. The BNN model is able to reach 78.7% train accuracy, 76.3% validation accuracy, and 79.6% inference accuracy, which means that the model is working according to the standards defined in this research.