Tracking using Human Pose Matching with Deep Association Metric

Abstract

This paper proposes a novel approach to track multiple people utilizing skeletal information combined with visual appearance features to improve the accuracy of tracking people across different frames of a video. We extracted the appearance feature vectors and skeletal feature vectors for each detected person in every frame. Each individual was tracked by considering the cosine distance between the skeletal feature vectors along with the euclidean distance between the appearance feature vectors across different frames of a video. This reduces the dependency of the tracker over appearances of people thus making it more consistent, especially in videos with people having similar appearances such as sports videos with players wearing similar jerseys. The stance of an individual in continuing frames is expected to be similar considering the high frame rate of modern camera devices. Therefore it is befitting to consider skeletal features along with appearance features for tracking. Our paper is an incremental paper demonstrating improvement over SORT with a deep association metric approach. Our approach utilizing skeletal information combined with visual appearance information returns better MOT results on the MOT17 dataset using the yolov3 detector.