Supervised Contrastive Multi-tasking Learning Based Hierarchical Yoga Pose Classification Using CNNs

Abstract

In this paper, we propose a technique for hierarchical yoga pose classification in a multi-tasking framework. Novelty lies in the proposed supervised contrastive combined loss function. We propose the usage of linear combination of three loss functions: cross entropy, self-supervised contrastive loss and supervised contrastive loss in a multi-tasking manner. We introduce radial and cosine margin into the formulation of self-supervised and supervised contrastive loss to pull feature embeddings of same classes closer together compared to feature embeddings of different classes. We use a two stage transfer learning based end to end training methodology trained over novel supervised contrastive multi-tasking combined loss function in the first stage and later fine tune over cross entropy multi-tasking loss in the second stage. We apply our methodology on the publicly available Yoga-82 large-scale dataset. We report peak Top-1 yoga pose classification accuracy of 94.86% over 6 pose classes (Yoga-6), 91.9% over 20 pose classes (Yoga-20) and 87.17% over 82 pose classes (Yoga-82). Our proposed method achieves 5% improvement over Top-1 classification accuracy in Yoga-6, 7.3% improvement in Yoga-20 and 8.1% improvement in Yoga-82 in comparison with state of the art (SOTA) methodology. We achieve SOTA accuracies in all three hierarchies.