Real Time Human Activity Recognition Using Convolutional Neural Network and Deep Gated Recurrent Unit

Abstract

Activity recognition on video is a challenging problem. To be used in the real application, the recognition approach should have good performance both in accuracy and speed. This research proposes a video-based human activity recognition method using a convolutional neural network (CNN) and a deep gated recurrent unit (GRU) network. First, selecting images in multiples of six on a sequence of frames is carried out to reduce redundant information and decrease computational complexity. Features extraction is then performed on the selected images using CNN with the MobileNetV2 architecture that has been trained in the ImageNet dataset. A collection of features extracted from a sequential frame in an activity video is fed to the GRU to analyze spatiotemporal features. Experiment on the YouTube 11 Actions dataset gives a promising result with an F1 score of 92.01% and 65.43 FPS speeds.