Relevance feedback for human motion retrieval using a boosting approach

Abstract

Content-based human motion retrieval (CBMR) has been more and more important with the rapid growth of motion capture data, but the gap between high-level semantic concepts and low-level features hinders further performance improvement. Relevance feedback is an effective tool to narrow the semantic gap and enhance the retrieval performance. However, as a type of variable-length multivariate time series (VLMTS), motion capture data has its own characteristics including high-dimensionality, demand for elastic matching, and difficulty representing different movements in a uniform feature space, which make it much more challenging to design an effective relevance feedback approach. This paper presents a novel boosting approach for CBMR and the main contributions include three aspects. First, to fit in with the characteristics of VLMTS data and meet the real-time requirement of relevance feedback, the ensemble learning framework RankBoost is introduced and k-nearest neighbors combining with dynamic time warping (KNN-DTW) is employed as its weak ranker. Second, the set of extended Boolean geometry features containing much richer geometry elements and measures is used to represent motion content, and it provides a comparatively complete feature set for designing the weak ranker of RankBoost. Third, to solve the over-fitting problem caused by the small-sample training of relevance feedback, a novel learning objective composed of minimizing empirical ranking loss and minimizing the maximum generalization loss is proposed for RankBoost ensemble learning. Experimental results on CMU database and its extended database verify the effectiveness of the proposed approach.