Deep key frame extraction for sport training

Abstract

Abstract For some professional sports, it is highly required to supervise and analyze the athletics pose in training of athletes. Key frame extraction from training videos plays a key role to facilitate the browse of sport training videos. In this paper, we propose a deep key frame extraction method for analyzing weightlifting sport training videos. To alleviate the bias from complex background, Fully Convolutional Networks (FCN) is employed firstly to extract the region of interest (ROI) which contains mainly the athlete and barbell for a more precise pose estimation of frames. Then over the extracted ROI, Convolutional Neural Networks (CNN) are leveraged to estimate the pose probability of each frame. Finally, a variation aware key frame extraction is constructed to extract the key frames considering neighboring probability difference of frames. The experimental results demonstrate that the proposed method achieves good performance in key frame extraction of sport videos, and significantly outperforms the comparisons.