Detection and Classification of Indian Classical Bharathanatyam Mudras Using Enhanced Deep Learning Technique

Abstract

Indian Classical dance forms like bharathanatyam are composed of advanced hand gestures, facial expressions moreover as body moments. Because of its complexity, identifying each mudra in bharathanatyam is extremely difficult. This paper demonstrates a massive Convolutional Neural Network (CNN) that was trained on Google Colaboratory using a single step model, You Only Look Once version 3 (YOLOv3), to analyze images in the dataset and detect and classify the mudras. Open datasets of mudras are not presently available. So Bharatanatyam mudra dataset of single hand gesture images of 28 classes was created. This proposed system is, as far as we know, the first attempt in this subject. YOLOv3 was never used to detect mudras. YOLOv3 divides the image into sectors, predicts bounding boxes, and calculates probability for each. These bounding boxes are then weighted according to the projected probability, and the model is then able to detect the object based on the final weights. The neural network was able to correctly generate test data after being trained, with a mean average precision (mAP) of 73%.