A Novel Framework for Robust Annotation and Retrieval in Video Sequences

Abstract

AbstractThis paper describes a method for automatic video annotation and scene retrieval based on local region descriptors. A novel framework is proposed for combined video segmentation, content extraction and retrieval. A similarity measure, previously proposed by the authors based on local region features, is used for video segmentation. The local regions are tracked throughout a shot and stable features are extracted. The conventional key frame method is replaced with these stable local features to characterise different shots. Compared to previous video annotation approaches, the proposed method is highly robust to camera and object motions and can withstand severe illumination changes and spatial editing. We apply the proposed framework to shot cut detection and scene retrieval applications and demonstrate superior performance compared to existing methods. Furthermore as segmentation and content extraction are performed within the same step, the overall computational complexity of the system is considerably reduced.KeywordsVideo SequenceVideo SegmentationVideo AnnotationMaximally Stable Extremal RegionRegion DescriptorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.