Motion Recognition Algorithm in VR Video Based on Dual Feature Fusion and Adaptive Promotion

Kunni Han

doi:10.1109/access.2020.3023755

Abstract

VR video recognition in complex environment, a motion recognition algorithm based on two-feature fusion and adaptive enhancement is proposed to solve the problems of inaccurate target position, target drift and recognition error caused by the vulnerability to light change, target rotation and occlusion. First, based on the spatio-temporal context (STC) mechanism, image sequence features are extracted through spatio-temporal context relationship and visual system characteristics to reduce the influence of light changes and occlusion on behaviors. Secondly, reliable feature point tracks are obtained through image feature point tracking and background track cutting, and a rich set of action descriptors (AD) are calculated from which local motion information, shape and static appearance information of the track are retained. After that, the principal component analysis operator is introduced to define the double feature fusion rules, and the STC feature and AD feature are combined to form a more accurate and complete feature representation. Finally, adaptive boosting algorithm (ABA) is used to train the classification through the new features obtained and complete the decision judgment of behavior and action. The experimental results show that the proposed algorithm has higher recognition accuracy and robustness compared with the current commonly used motion recognition methods, and can better adapt to complex background and motion changes.

Highlights

In recent years, behavior recognition in video has become an important task in the field of computer vision, and it is of great significance to video surveillance, video information retrieval, human-computer interaction and other work [1]–[4]
In order to obtain more accurate and comprehensive action features, this paper introduces a principal component analysis (PCA)-based feature fusion algorithm, which combines the extracted spatio-temporal context (STC) features with action descriptors (AD) features to form a more effective feature representation
In this paper, the STC features of image sequences are extracted through spatiotemporal context and visual system characteristics, which can reduce the influence of light changes and occlusion on behaviors and actions

Summary

INTRODUCTION

Behavior recognition in video has become an important task in the field of computer vision, and it is of great significance to video surveillance, video information retrieval, human-computer interaction and other work [1]–[4]. If the window is too large, it will cause multiple actions in a window, the system needs too long waiting time, and the accuracy and effectiveness of the system are reduced; if the window is too small, each window cannot completely contain a motion cycle, and the extracted features cannot reflect the human behavior, which affects the recognition accuracy. D. FEATURE EXTRACTION AND SELECTION The pre-processed data cannot characterize human behavior and cannot be directly input into the classification model. MOTION RECOGNITION ALGORITHM BASED ON DUAL FEATURE FUSION AND ADAPTIVE LIFTING Extracting the STC features of an image sequence through the temporal and spatial context and the characteristics of the visual system can reduce the impact of changes in illumination and occlusion on behavior.

STC FEATURES

CONCLUSION