Structurally enhanced latent semantic analysis for video object retrieval

Abstract

The work presented in this paper aims at reducing the semantic gap between low level video features and semantic video contents. The proposed method for finding associations between segmented frame region characteristics relies on the strength of latent semantic analysis (LSA). Previous work, using colour histograms and Gabor features, have rapidly shown the potential of this approach but also uncovered some of its limitations. The use of structural information is necessary, yet rarely employed for such a task. This paper addresses two important issues: the first is to verify that using structural information does indeed improve information retrieval performances, while the second concerns the manner in which this additional information is integrated within the framework. Here, two methods are proposed using the structural information contained in object parts' topological arrangement. The first adds structural constraints indirectly to the LSA during the preprocessing of the video, while the other includes the structure directly within the LSA. Finally, retrieval results demonstrate that when the structure is added directly to the LSA the performance gain of combining visual (low level) and structural information is convincing.