Human Action Recognition in Video via Fused Optical Flow and Moment Features – Towards a Hierarchical Approach to Complex Scenario Recognition

Abstract

This paper explores using motion features for human action recognition in video, as the first step towards hierarchical complex event detection for surveillance and security. We compensate for the low resolution and noise, characteristic of many CCTV modalities, by generating optical flow feature descriptors which view motion vectors as a global representation of the scene as opposed to a set of pixel-wise measurements. Specifically, we combine existing optical flow features with a set of moment-based features which not only capture the orientation of motion within each video scene, but incorporate spatial information regarding the relative locations of directed optical flow magnitudes. Our evaluation, using a benchmark dataset, considers their diagnostic capability when recognizing human actions under varying feature set parameterizations and signal-to-noise ratios. The results show that human actions can be recognized with mean accuracy across all actions of 93.3%. Furthermore, we illustrate that precision degrades less in low signal-to -noise images when our moments-based features are utilized.