A Gradual Shot Change Detection Using Combination of Luminance and Motion Features for Frame Rate Up Conversion

Abstract

Frame rate up conversion (FRUC) is a method that automatically generates and inserts virtual frames between input frames for smoother video playback. When a simple FRUC is applied to a video clip, the image quality around shot boundaries is severely deteriorated. Therefore, shot boundary detection (SBD) is essential for ensuring the quality of video clips. Detection of gradual shot transition still remains a challenging issue. In this paper, we propose a gradual shot transition detection method that uses both an SBD based on motion-detection using multiple frames and an SBD based on luminance. Luminance-based SBD is robust to dynamic motions and sensitive to light effect, in contrast, motion-based SBD is sensitive to dynamic motions and robust to luminance effects. Both SBD methods are performed independently at first. In multi-frame luminance-based SBD, each algorithm is performed and the results are validated to detect shot boundaries by using distribution of luminance change. Threshold is adaptively changed depending on the borders of frame shots and gradual shot change is identified with multi-frame motion-based gradual shot transition detection. Our proposed method of shot transition boundary detection showed 97.36% recall and 92.61% precision on average in our experiment.