Video Shot Boundary Detection Using Principal Component Analysis (PCA) and Deep Learning

Abstract

After COVID-19 pandemic, with the tremendously increasing reliance on digital media platform all over the world, advanced video surveillance, video content analysis, and video retrieval tasks have become an essential field which makes traditional method for shot boundary detection (SBD) less efficient, especially for gradual transition type shot boundary. SBD is an important preliminary step in video processing which helps to separate video sequences into shots. Large videos (news, national ceremony, sports, movies, etc.) contain several types of shot transitions (gradual, cut), and to detect them with higher accuracy an efficient SBD method is needed. Previous researchers have proposed methods for SBD by using Supervised machine learning approaches, however, data training is a limitation for them. Some previous papers have proposed unsupervised approaches, however detection for gradual transition shot boundaries has not achieved good accuracy. This paper proposes a distance calculating algorithms for SBD by using principal component analysis (PCA) features and then deep learning method to improve the detection precision of gradual transition shot boundary frames and normal frames. Firstly, PCA is used for feature extraction of the video frames and the most significant Eigenvectors and Eigenvalues are analyzed. Secondly, a distance calculating algorithm is used on the adjacent video frames depending on their Eigenvectors and Eigenvalues to determine the shot boundaries. Finally, to improve detection accuracy, false detection boundaries are reviewed and classified using CNN. Experimental results show the efficiency of the proposed method.