Semantic video classification and feature subset selection under context and concept uncertainty

Abstract

As large collections of videos become one key component of digital libraries, there is an urgent need of semantic video classification and feature subset selection to enable more effective video database organization and retrieval. However, most existing techniques for classifier training require a large number of labeled samples to learn correctly and suffer from the problems of context and concept uncertainty when only a limited number of labeled samples are available. To address the problems of context and concept uncertainty, we have proposed a novel framework to achieve incremental classifier training by integrating a limited number of labeled samples with a large number of unlabeled samples. Specifically, the contributions of this paper include: (a) Using the salient objects to achieve a middle-level understanding of video contents and enhance the quality of features on discriminating among different semantic video concepts; (b) Modeling the semantic video concepts by using the finite mixture models to approximate the class distributions of the relevant salient objects; (c) Developing an adaptive EM algorithm to integrate the unlabeled samples to enable incremental classifier training and address the problem of context uncertainty; (d) Proposing a cost-sensitive video classification technique to address the problem of concept uncertainty over time; (e) Supporting automatic video annotation via semantic classification. Our experimental results in a certain domain of medical education videos have also been provided a convincing proof of our conclusions.