Semantic video classification by integrating flexible mixture model with adaptive EM algorithm

Abstract

Digital video now plays an important role in medical education and healthcare, but our ability to automatic video indexing at semantic level is currently primitive. In this paper, we propose a novel framework to enable more effective semantic video classification and indexing in a specific surgery education video domain. Specifically, this framework includes: (a) A novel semantic-sensitive video content characterization and representation framework by using principal video shotsand theirperceptual multimodal features. (b) A novel semantic medical concept interpretation technique by using flexible mixture model. (c) A semantic video classifier by using an adaptive Expectation-Maximization (EM) algorithm for automatic parameter estimation and model selection (i.e., selecting the optimal number of mixture Gaussian components). Since more effective video content characterization framework has been integrated with an adaptive EM algorithm for video classification, our semantic video classifier has improved the classification accuracy significantly. For skin classification, its accuracy is close to 95.5%. For semantic surgical video classification, it achieves overall ≈ 84.6% accuracy.