Shot boundary detection with effective prediction of transitions' positions and spans by use of classifiers and adaptive thresholds

Abstract

Video shot boundary detection (SBD), is the process of segmenting a video sequence into smaller temporal units, dubbed shots. SBD is the primary step for any further video analyses. In this article we propose a new and effective SBD method for detecting cut transitions (CTs) and gradual transitions (GTs) of a video sequence, in a unified scheme. As the first step in our scheme, we introduce a significantly effective candidate segment selection method by use of frame histogram, and a locally-defined adaptive threshold. This step avoids the processing of unnecessary video segments in the subsequent stages and accelerate the whole SBD process. Next, we extract CTs from candidate segments by use of a robust multi-stage thresholding. Then, we pass the remained segments through a novel candidate segment adjustment stage, which is realized by use of K-means classifiers, and its purpose is to give a better initial prediction of the location of GTs. Afterwards, we use some measures of difference along with singular value decomposition (SVD) on histograms of frames in each candidate GT segment, to obtain discriminating feature vectors with desired dimensions. Features are then fed into linear support vector machine (SVM) classifiers to separate segments with GTs from the rest. Finally, we determine the boundary of GTs by evaluating the gradient of a metric associated to each GT segment. By use of gradient for determining the range of GTs, we avoid the iterative procedures that are usually used in this purpose. Experimental results on standard videos, show the capability and supremacy of our method in comparison with state-of-the-art SBD procedures.