An automatic video annotation framework based on two level keyframe extraction mechanism

Abstract

Large increase in audio, video and digital data in the internet signifies the importance of video annotation techniques. This paper mainly deals with the development of a hybrid algorithm for automatic Video Annotation (VA). Another aim in developing the algorithm is to improve the performance and precision as well as to reduce the amount of time required to obtain the annotations. The overall process leads to the development of efficient techniques for shot detection followed by two level key frame extractions and saliency based residual approach for feature extraction. For all the stages in VA like shot detection, keyframe extraction and feature extraction, factors relating to improve the performance are addressed here. The combination of color histogram difference (CBD) and Edge change ratio (ECR) is used here; as these two are the most promising techniques in shot detection. The new idea is proposed to fine tune the keyframe extraction, which extracts keyframe in two levels. At first level, the first frame in the shot is considered as a keyframe. But to remove redundancy, it enters into second level and finds the optimal set of keyframes by using fuzzy c-means clustering technique. Colour and texture features are used for feature extraction. Here the Video annotation process is divided into two sections, training and testing. The weight vector is found in training stage. Based on this feature vector, the similarity array is calculated in testing phase which further finds corrected annotations. The proposed method is compared with OMG-SSL and MMT-MGO and results are found better on Trechvid dataset. The significance of using weight vector is also experimentally shown here.