Multifeature fusion action recognition based on key frames

Abstract

SummaryAs an important technology in computer vision, video‐based human action recognition has a great commercial value, which has attracted extensive attention in the field of computer vision and pattern recognition in both academia and industry. To date, there are a wide variety of applications of human action recognition, such as surveillance, robotics, health care, video searching, and human–computer interaction. However, there are many challenges involved in human action recognition in videos, such as cluttered backgrounds, occlusions, viewpoint variation, execution rate, and camera motion. However, data redundancy and single feature were largely limited the accuracy of human action recognition. In this article, adopting the key frame extraction and multifeature fusion techniques, a novel action recognition method was proposed, which can improve the recognition accuracy. The main works are as follows: 1) in order to solve the problem of data redundancy, a key frame extraction method based on node contribution weighting is proposed to extract video key frames; 2) different kinds of information flows are extracted from the obtained key frame sequences, and different convolutional neural networks are used to obtain corresponding classification results and merge, so as to better complement the information in different flows. Lastly, the experimental results show that our method improves the accuracy of action recognition.