Learning a General Assignment Model for Video Analytics

Abstract

Video analytics, also known as video content analysis, involves a variety of pivotal tasks, such as video segmentation and recognition. In essence, performing most of these tasks can be viewed as learning an assignment model. Here, assignment model refers to what each element in a target set is assigned to the element in an opposite source set under some kind of constraint. Existing assignment models generally suffer from some limitations. For instance, imperfect results can be obtained when the source set is artificially provided with less representativeness, or part of the target set has expected assignments, thus significantly limiting the application of assignment model. To alleviate these issues, we develop, in this paper, a general assignment model to dynamically learn a source set by minimizing structural dissimilarity in a low-dimensional space. Furthermore, by associating the expected assignment solution with empirical assignment indicator via a consistency boosting strategy, the proposed assignment model is provided with a potential powerful generalization ability to deal flexibly with the unsupervised, semi-supervised, and fully supervised scenarios in assignment model learning. Considering the separability of both objective and constraints to be solved, an alternating direction method of multipliers solver is presented with rigorous theoretical analysis on its convergence. Experimental results on video analysis, including motion segmentation, activities recognition, and scene categorization, demonstrate that the proposed general assignment model is considerable superior to the state-of-the-art methods.