Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning

Abstract

To explore the rich information contained in multi-modal data and take into account efficiency, deep cross-modal hash retrieval (DCMHR) is a wise solution. But currently, most DCMHR methods have two key limitations, one is that the recommended classification of DCMHR models is conditioned only on the objects in different regions, respectively. Another flaw is that these methods either do not learn the unified hash codes in training or cannot design an efficient training process. To solve these two problems, this paper designs Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning (HUMOR). For the proposed related labels classified by ImgNet, HUMOR uses Multiple Instance Learning (MIL) to reason the correlation of these labels. When regional correlation reasoning is low, these labels will be through “reduce-add” to rectification from max-to-min (global precedence) or min-to-max (regional precedence). Then, HUMOR conducts unified learning on hash loss and classification loss, adopts the four-step iterative algorithm to optimize the unified hash codes, and reduces bias in the model. Experiments on two baseline datasets show that the average performance of this method is higher than most of the DCMHR methods. The results demonstrate the effectiveness and innovation of our method.