Multi-modal discrete tensor decomposition hashing for efficient multimedia retrieval

Abstract

In the research field of multimedia retrieval, unsupervised multi-modal hashing has received widespread attention because of its high retrieval efficiency, low storage cost and semantic label independence. However, there are still several problems that need to be resolved: 1) All existing methods are based on matrix factorizations, which has limited capability on fusing heterogeneous multi-modal features under the unsupervised learning paradigm. 2) Most methods adopt two-step learning strategy to separately learn the hash codes and functions, which may generate sub-optimal hash functions. 3) The optimization strategies used by most methods produce significant quantization loss and computation cost. In this paper, an efficient unsupervised Multi-modal Discrete Tensor Decomposition Hashing (MDTDH) approach is proposed to solve the above problems. Specifically, all multi-modal features are first stacked together into a three-dimensional tensor after nonlinear mapping, and then it is decomposed further into a core tensor and two factor matrices by Tucker decomposition. A series of hash functions are learned simultaneously by mapping the nonlinear features of the training instances to their corresponding hash codes. To reduce the quantization loss and computation cost, a fast discrete optimization strategy is proposed to generate hash codes directly without relaxing quantization loss. Extensive experimental results on three benchmarks demonstrate that our proposed MDTDH obtains superior performance than state-of-the-art unsupervised baselines, and even defeats several supervised baselines. Our source codes and testing datasets are available athttps://github.com/XizeWu/MDTDH.