Capsule Networks for Classifying Conflicting Double-Handed Classical Dance Gestures

Abstract

A challenging application of computer vision is the classification of gestures in classical dance. Classical dance is one of the most accepted forms of arts in India. Due to the complexity of various gestures used to convey unique meaning to the audiance, computational analysis of the dance gestures is a complex task. Hence, it requires efficient machine learning techniques to solve this problem. In this paper, we introduce a classification model for identifying a subclass of hastas called Samyuta hastas (double-handed gestures which preserve hierarchy) where the mudras or hastas are unique hand gestures that imply some actions and ideologies. A classification technique that preserves the object location within an image is necessary for this problem. Hence, we use capsule networks, which maintain equivariance in an image and can analyze the hierarchy and order of objects within the image. A dataset comprising of around 2400 images distributed evenly among six classes of Asamyutha hastas is used for the work. The classification was performed by the deep learning pipelines using typical convolutional neural networks (CNN), transfer learning, and the proposed capsule network (hasta CapsNet). The results show that since CNN loses the location information of an object due to its pooling layers that extract only the vital features, it tends to wrongly classify the double-handed gestures, whereas the capsule networks have achieved better accuracy on the dataset