Understanding dance semantics using spatio-temporal features coupled GRU networks

Abstract

The efforts are taken for cultural heritage preservation lead to many digitization initiatives. Indian Classical Dance and its tradition has proven historical importance. Computer-aided archiving and preservation of Indian classical dance resources open enormous opportunities for computational analysis. With the help of recent computational advancements in the field of Computer Vision, these archives can be transformed into intelligent information retrieval systems. In this work, we propose a novel method for understanding the dance semantics by making use of the Spatio-temporal variations of dance features. A video archive is created as a part of this work, from live recordings of different trained dancers and clippings from the internet. The Deep Pose Estimator coupled GRU Model deals with the spatial aspects with a deep learning pose estimator and handles the temporal perspective with GRU Network. The efficiency of the proposed method was compared with benchmark methods such as A 3D-Convolutional Neural Network-based Model, Time Distributed CNN-LSTM Model, and Hybrid Transfer Learning - LSTM Model and the results show the proposed method outperformed others even with different video resolutions.