Web video classification with visual and contextual semantics

Abstract

SummaryOn the social Web, the amount of video content either originated from wireless devices or previously received from media servers has increased enormously in the recent years. The astounding growth of Web videos has stimulated researchers to propose new strategies to organize them into their respective categories. Because of complex ontology and large variation in content and quality of Web videos, it is difficult to get sufficient, precisely labeled training data, which causes hindrance in automatic video classification. In this paper, we propose a novel content‐ and context‐based Web video classification framework by rendering external support through category discriminative terms (CDTs) and semantic relatedness measure (SRM). Mainly, a three‐step framework is proposed. Firstly, content‐based video classification is proposed, where twofold use of high‐level concept detectors is leveraged to classify Web videos. Initially, category classifiers induced from VIREO‐374 detectors are trained to classify Web videos, and then concept detectors with high confidence for each video are mapped to CDT through SRM‐assisted semantic content fusion function to further boost the category classifiers, which intuitively provide a more robust measure for Web video classification. Secondly, a context‐based video classification is proposed, where twofold use of contextual information is also harnessed. Initially, cosine similarity and then semantic similarity are measured between text features of each video and CDT through vector space model (VSM)‐ and SRM‐assisted semantic context fusion function, respectively. Finally, classification results from content and context are fused to compensate for the shortcomings of each other, which enhance the video classification performance. Experiments on large‐scale video dataset validate the effectiveness of the proposed solution.