An effective post-refinement method for shot boundary detection

Abstract

In content-based video analysis, shot boundary detection (SBD) is a common first step which segments video data into elementary shots, each comprising a sequence of consecutive frames recording a video event or scene continuous in time and space. Many SBD methods have been proposed in the literature, and experimental results show that the existing methods work reasonably well for abrupt shot boundaries, but less effectively for gradual shot boundaries. In this paper, we propose an effective post-refinement method for identifying actual shot boundaries from the results obtained by existing SBD methods. The proposed method formulates the SBD problem as sequential detection of changes in the underlying feature distributions whose parameters are estimated from existing video shots. Specifically, the proposed post-refinement method enhances the performance of SBD by identifying as many false positives (false detections) and false negatives (miss detections) as possible. Experiments conducted on a large set of test videos, whose initial shot boundaries are obtained by four existing SBD methods, show that the proposed post-refinement method can improve markedly the detection recall and precision and is rather insensitive to the thresholds used by the existing methods in detecting the initial shot boundaries.