Deep Optical Flow Feature Fusion Based on 3D Convolutional Networks for Video Action Recognition

Abstract

In the real world, due to the existence of complex scenes and different perspectives, different types of behavior are very different in appearance and behavior models. The traditional 3D convolutional networks have greatly improved the extraction of time series information, but at the same time it also loses some behavioral characteristics, which lead to the recognition rate not high. And the optical flow information can represent the motion information of human behavior well, so extract more effective optical flow information before extracting RGB depth features by the traditional 3D convolutional networks is necessary. Extracting the deep RGB feature and the deep optical flow feature respectively by the feature extractor with 3D Convolutional Networks, Then the cascade fusion of two deep features make the feature stronger classification ability. The experimental results show that the improved recognition method has a better performance for the video behavior recognition than the traditional 3D convolutional networks method.