Semantic consistency cross-modal dictionary learning with rank constraint

Abstract

Cross-modal retrieval develops rapidly due to the growth and widespread applications of multimodal data. How to reduce the heterogeneous gap and impose effective constraints on different modalities are two basic problems. In this paper, we propose a novel Semantic Consistency cross-modal Dictionary learning algorithm with rank Constraint (SCDC) to solve these aforementioned problems. An orthogonal space learned by spectral regression is introduced, in which different modalities can be measured directly. Specifically, images and texts are encoded by their dictionaries to obtain corresponding reconstruction coefficients. A l21-norm term is imposed on these coefficients in order to select discriminative features and avoid over-fitting simultaneously. In the meantime, a rank constraint is imposed on the transformed features so as to improve the correlation of different modalities. Experimental results on three popular datasets demonstrate that SCDC is significantly superior to several state-of-the-art methods.