Existing Hashing Methods Research Articles

Pyramid hybrid pooling quantization for efficient fine-grained image retrieval

Deep hashing approaches, including deep quantization and deep binary hashing, have become a common solution to large-scale image retrieval due to their high computation and storage efficiency. Most existing hashing methods cannot produce satisfactory results for fine-grained retrieval, because they usually adopt the outputs of the last CNN layer to generate binary codes. Since deeper layers tend to summarize visual clues, e.g., texture, into abstract semantics, e.g., dogs and cats, the feature produced by the last CNN layer is less effective in capturing subtle but discriminative visual details that mostly exist in shallow layers. To improve fine-grained image hashing, we propose Pyramid Hybrid Pooling Quantization (PHPQ). Specifically, we propose a Pyramid Hybrid Pooling (PHP) module to capture and preserve fine-grained semantic information from multi-level features, which emphasizes the subtle discrimination of different sub-categories. Besides, we propose a learnable quantization module with a partial codebook attention mechanism, which helps to optimize the most relevant codewords and improves the quantization. Comprehensive experiments on two widely-used public benchmarks, i.e., CUB-200-2011 and Stanford Dogs, demonstrate that PHPQ outperforms state-of-the-art methods.

Deep global semantic structure-preserving hashing via corrective triplet loss for remote sensing image retrieval

With the explosive increase of remote sensing data, how to search for remote sensing data quickly and accurately in a vast dataset is an incredibly critical matter for research subjects. The deep hashing method has become the dominant method for remote sensing image retrieval because of its low-cost storage and high-speed retrieval. However, for the reason of the limitation of fixed convolutional kernels, deep hashing frameworks based on Convolutional Neural Networks (CNNs) fail to obtain the global semantic information well, which leads to the generation of suboptimal solutions. Furthermore, existing hashing methods commonly employ the random sampling strategy or hardest sample mining to build training batches, resulting in bad local minima. To remedy these problems, a novel Deep Global Semantic Structure-preserving Hashing framework via corrective triplet loss (DGSSH) is proposed for remote sensing image retrieval to learn a discriminative and stable embedding space, achieving intra-class confusion and inter-class diversity. Specifically speaking, the feature extraction module based on Swim Transformer architecture is developed to capture global semantic information and multiscale features from remote sensing images. Based on a distribution matching constraint, the corrective triplet loss for deep hashing schemes is designed to reduce the distribution shift caused by the random selection or hardest sample mining. Meanwhile, to reduce the time overhead of the model, the asymmetric learning strategy is employed to perform effective compact representation learning. Numerous experiments have been carried out on three publicly available benchmarks, which indicates that the proposed DGSSH framework could achieve optimal performance for remote sensing image retrieval applications. The source code of our DGSSH framework is hosted at https://github.com/QinLab-WFU/DGSSH.git.

RICH: A rapid method for image-text cross-modal hash retrieval

Deep cross-modal hash retrieval (DCMHR) methods can effectively analyze the correlation of multimodal data while maintaining efficiency. However, to pursue better accuracy, most existing hash methods forget that the original purpose of introducing hash technology is to reduce training consumption, and overtraining also leads to overfitting. This paper proposes a rapid method for image-text cross-modal hash retrieval (RICH) based on DenseNet and multi-head attention (MHA) BOW (Bag of Words), which makes full use of unlabeled samples and uses EarlyStop in training. To fully extract image case features, we propose multiple dense feature sampling for cross-modal retrieval. It is worth noting that this method applies DenseNet and EarlyStop to unsupervised cross-modal retrieval for the first time and greatly reduces training costs while keeping good results. Then it is discussed that the MHA be carried in the TxtNet, which can extract neglected features. Furthermore, to alleviate the heterogeneous gap between different modalities, we also use the auxiliary similarity metrics. Experiments on three datasets show that the average performance of this method is higher than most of the DCMHR methods. In addition, compared to most of the SoTA unsupervised DCMHR methods, the training cost and stability of RICH are more excellent, which proves the effectiveness and superiority of this method.

Attention-Oriented Deep Multi-Task Hash Learning

Hashing has wide applications in image retrieval at large scales due to being an efficient approach to approximate nearest neighbor calculation. It can squeeze complex high-dimensional arrays via binarization while maintaining the semantic properties of the original samples. Currently, most existing hashing methods always predetermine the stable length of hash code before training the model. It is inevitable for these methods to increase the computing time, as the code length converts, caused by the task requirements changing. A single hash code fails to reflect the semantic relevance. Toward solving these issues, we put forward an attention-oriented deep multi-task hash learning (ADMTH) method, in which multiple hash codes of varying length can be simultaneously learned. Compared with the existing methods, ADMTH is one of the first attempts to apply multi-task learning theory to the deep hashing framework to generate and explore multi-length hash codes. Meanwhile, it embeds the attention mechanism in the backbone network to further extract discriminative information. We utilize two common available large-scale datasets, proving its effectiveness. The proposed method substantially improves retrieval efficiency and assures the image characterizing quality.

Supervised Discrete Multiple-Length Hashing for Image Retrieval

Hashing can facilitate efficient retrieval and storage for large-scale images due to the binary representation. In the real applications, the trade-off between retrieval accuracy and speed is essential for designing a hashing framework, which is reflected by variable hash code lengths. In light of this, the existing hashing methods need to train different models for different lengths of hash codes, leading to considerable training time cost and hashing flexibility reduction. Given that a sample can be represented by various hash codes with different lengths, there are some helpful relationships that can boost the performance of hashing methods. However, the existing hashing methods do not fully utilize these relationships. To address the aforementioned issues, we propose a new model, known as supervised discrete multiple-length hashing (SDMLH), to simultaneously learn hash codes with multiple lengths. In this proposed SDMLH method, three types of information are respectively derived, from the hash codes with different lengths. The original features of the samples, and the label, are applied for hash learning. Unlike the existing hashing methods, SDMLH can fully employ the assistance among hash codes with different lengths and learn them in one step. Furthermore, given a hash length meeting the demand of users, we propose a hash fusion strategy to obtain the hash code with this desirable length by fusing the multiple-length hash codes. This obtained hash code outperforms the one learned directly. In addition, SDMLH can generate the hash code of any length that is shorter than the sum length of given multiple hash codes with the fusion strategy. To the best of our knowledge, SDMLH is one of the first attempts for learning multiple-length hash codes simultaneously. We conduct extensive experiments based on three benchmark datasets, demonstrating the superiority of this proposed method.

ViT2CMH: Vision Transformer Cross-Modal Hashing for Fine-Grained Vision-Text Retrieval

In recent years, the development of deep learning has further improved hash retrieval technology. Most of the existing hashing methods currently use Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to process image and text information, respectively. This makes images or texts subject to local constraints, and inherent label matching cannot capture fine-grained information, often leading to suboptimal results. Driven by the development of the transformer model, we propose a framework called ViT2CMH mainly based on the Vision Transformer to handle deep Cross-modal Hashing tasks rather than CNNs or RNNs. Specifically, we use a BERT network to extract text features and use the vision transformer as the image network of the model. Finally, the features are transformed into hash codes for efficient and fast retrieval. We conduct extensive experiments on Microsoft COCO (MS-COCO) and Flickr30K, comparing with baselines of some hashing methods and image-text matching methods, showing that our method has better performance.

Deep Feature Pyramid Hashing for Efficient Image Retrieval

Thanks to the success of deep learning, deep hashing has recently evolved as a leading method for large-scale image retrieval. Most existing hashing methods use the last layer to extract semantic information from the input image. However, these methods have deficiencies because semantic features extracted from the last layer lack local information, which might impact the global system’s performance. To this end, a Deep Feature Pyramid Hashing DFPH is proposed in this study, which can fully utilize images’ multi-level visual and semantic information. Our architecture applies a new feature pyramid network designed for deep hashing to the VGG-19 model, so the model becomes able to learn the hash codes from various feature scales and then fuse them to create final binary hash codes. The experimental results performed on two widely used image retrieval datasets demonstrate the superiority of our method.

Label Guided Discrete Hashing for Cross-Modal Retrieval

Due to their low storage capacity and fast retrieval speed, hashing techniques have received much attention in cross-modal retrieval. However, there are some issues that need to be further explored. First, some existing hashing methods use the labels to construct the semantic similarity matrix between pairwise data, ignoring the potential manifold structure between heterogeneous data. Second, some existing methods underestimate the importance of multi-label and the gaps between different class labels, making the learned hash codes less discriminative. Third, few of them embed both manifold and balanced structures within the same model, and the relaxation of discrete constraints will lead to an increasing quantization error. To mitigate these problems, this paper proposes a novel supervised hashing method, termed Label Guided Discrete Hashing (LGDH), which simultaneously preserves the comprehensive manifold structure and discriminative balanced codes that are both constructed by label information into Hamming space. We develop a local category distribution of the nearest neighbors, to excavate the underlying manifold structure of heterogeneous data. To maximize the gaps of different categories, a balanced matrix is constructed by labels to generate hash codes with balanced bits. For multi-label data, we also design a novel multi-label manifold and balanced structure matrix to adapt the real-world scenarios. An effective discrete optimization method is used to optimize our proposed objective function instead of the relaxation one. Extensive experiments on three benchmark datasets verify the effectiveness of LGDH. The comparison results demonstrat that LGDH achieves about 2% and 3% improved to different cross-modal tasks on average.

Domain Adaptation Preconceived Hashing for Unconstrained Visual Retrieval.

Learning to hash has been widely applied for image retrieval due to the low storage and high retrieval efficiency. Existing hashing methods assume that the distributions of the retrieval pool (i.e., the data sets being retrieved) and the query data are similar, which, however, cannot truly reflect the real-world condition due to the unconstrained visual cues, such as illumination, pose, background, and so on. Due to the large distribution gap between the retrieval pool and the query set, the performances of traditional hashing methods are seriously degraded. Therefore, we first propose a new efficient but transferable hashing model for unconstrained cross-domain visual retrieval, in which the retrieval pool and the query sample are drawn from different but semantic relevant domains. Specifically, we propose a simple yet effective unsupervised hashing method, domain adaptation preconceived hashing (DAPH), toward learning domain-invariant hashing representation. Three merits of DAPH are observed: 1) to the best of our knowledge, we first propose unconstrained visual retrieval by introducing DA into hashing for learning transferable hashing codes; 2) a domain-invariant feature transformation with marginal discrepancy distance minimization and feature reconstruction constraint is learned, such that the hashing code is not only domain adaptive but content preserved; and 3) a DA preconceived quantization loss is proposed, which further guarantees the discrimination of the learned hashing code for sample retrieval. Extensive experiments on various benchmark data sets verify that our DAPH outperforms many state-of-the-art hashing methods toward unconstrained (unrestricted) instance retrieval in both single- and cross-domain scenarios.

Structure-Adaptive Neighborhood Preserving Hashing for Scalable Video Search

In this paper, we propose a Structure-adaptive Neighborhood Preserving Hashing (SNPH) method for unsupervised scalable video search. Unlike most existing hashing methods which equally encode an entire video into a binary feature vector, we propose a neighborhood attention mechanism which encodes the neighborhood-relevant content of a video to better preserve the neighborhood relationships among videos. Motivated by the fact that a video usually contains multiple shots and each shot depicts a different activity, we further develop a structure-adaptive encoder to model the hierarchical structure of the video. Specifically, the encoder adaptively divides each video into multiple segments via detecting temporal boundaries across frames and encodes these segments as a compact binary vector to capture rich structural information. We integrate the neighborhood attention mechanism into the structure-adaptive encoder to learn hash functions that jointly preserve the neighborhood relationships among videos and exploit the hierarchical structure in a video. Experimental results on three widely used benchmark datasets show that our proposed method consistently outperforms state-of-the-art unsupervised video hashing methods.

Multiview Graph Convolutional Hashing for Multisource Remote Sensing Image Retrieval

Recently, hashing has been successfully applied for large-scale remote sensing image retrieval (LSRSIR) due to its advantage in terms of computation and storage. In LSRSIR, existing hashing methods mainly focus on single-source remotely sensed data. They cannot effectively fuse multisource remotely sensed data, which has a large potential for LSRSIR. To fulfill this gap, this letter proposes a novel deep hashing method, dubbed Multiview Graph Convolutional Hashing (MGCH) that can successfully fuse multisource remote sensing image. Since graph convolutional network (GCN) has been applied as an effective means that expresses and integrates relationships into features, MGCH applies a GCN to explore inherent structural similarity among multiview data, which will help to generate discriminative hash codes. An asymmetric scheme is developed that optimizes the proposed deep model in an end-to-end manner to improve training efficiency. We evaluate the proposed method by fusing two different kinds of RS images, i.e., multispectral (MUL) image and panchromatic (PAN) image. The experimental results on the dual-source RS image data set (DSRSID) show that the proposed MGCH outperforms state-of-the-art multiview hashing methods.

Scalable Distributed Hashing for Approximate Nearest Neighbor Search.

Hashing has been widely applied to the large-scale approximate nearest neighbor search problem owing to its high efficiency and low storage requirement. Most investigations concentrate on learning hashing methods in a centralized setting. However, in existing big data systems, data is often stored across different nodes. In some situations, data is even collected in a distributed manner. A straightforward way to solve this problem is to aggregate all the data into the fusion center to obtain the search result (aggregating method). However, this strategy is not feasible because of the prohibitive communication cost. Although a few distributed hashing methods have been proposed to reduce this cost, they only focus on designing a distributed algorithm for a specific global optimization objective without considering scalability. Moreover, existing distributed hashing methods aim at finding a distributed solution to hashing, meanwhile avoiding accuracy loss, rather than improving accuracy. To address these challenges, we propose a Scalable Distributed Hashing (SDisH) model in which most existing hashing methods can be extended to process distributed data with no changes. Furthermore, to improve accuracy, we utilize the search radius as a global variable across different nodes to achieve a global optimum search result for every iteration. In addition, a voting algorithm is presented based on the results produced by multiple iterations to further reduce search errors. Theoretical analyses of communication, computation, and accuracy demonstrate the superiority of the proposed model. Numerical simulations on three large-scale and two relatively small benchmark datasets also show that the SDisH model achieves up to 44.75% and 10.23% accuracy gains compared to the aggregating method and state-of-the-art distributed hashing methods, respectively.

Beyond ITQ: Efficient binary multi-view subspace learning for instance retrieval

The existing hashing methods mainly handle either the feature based nearest-neighbor search or the category-level image retrieval, whereas a few efforts are devoted to instance retrieval problem. In this paper, we propose a binary multi-view fusion framework for directly recovering a latent Hamming subspace from the multi-view features for instance retrieval. More specifically, the multi-view subspace reconstruction and the binary quantization are integrated in a unified framework so as to minimize the discrepancy between the original multi-view high-dimensional Euclidean space and the resulting compact Hamming subspace. Besides, our method is essentially an unsupervised learning scheme without any labeled data involved, and thus can be used in the cases when the supervised information is unavailable or insufficient. Experiments on public benchmark and large-scale datasets reveal that our method achieves competitive retrieval performance comparable to the state-of-the-arts and has excellent scalability in large-scale scenario.

DHLBT: Efficient Cross-Modal Hashing Retrieval Method Based on Deep Learning Using Large Batch Training

Cross-modal hashing has attracted considerable attention as it can implement rapid cross-modal retrieval through mapping data of different modalities into a common Hamming space. With the development of deep learning, more and more cross-modal hashing methods based on deep learning are proposed. However, most of these methods use a small batch to train a model. The large batch training can get better gradients and can improve training efficiency. In this paper, we propose the DHLBT method, which uses the large batch training and introduces orthogonal regularization to improve the generalization ability of the DHLBT model. Moreover, we consider the discreteness of hash codes and add the distance between hash codes and features to the objective function. Extensive experiments on three benchmarks show that our method achieves better performance than several existing hashing methods.

Probability Ordinal-Preserving Semantic Hashing for Large-Scale Image Retrieval

Semantic hashing enables computation and memory-efficient image retrieval through learning similarity-preserving binary representations. Most existing hashing methods mainly focus on preserving the piecewise class information or pairwise correlations of samples into the learned binary codes while failing to capture the mutual triplet-level ordinal structure in similarity preservation. In this article, we propose a novel Probability Ordinal-preserving Semantic Hashing (POSH) framework, which for the first time defines the ordinal-preserving hashing concept under a non-parametric Bayesian theory. Specifically, we derive the whole learning framework of the ordinal similarity-preserving hashing based on the maximum posteriori estimation, where the probabilistic ordinal similarity preservation, probabilistic quantization function, and probabilistic semantic-preserving function are jointly considered into one unified learning framework. In particular, the proposed triplet-ordering correlation preservation scheme can effectively improve the interpretation of the learned hash codes under an economical anchor-induced asymmetric graph learning model. Moreover, the sparsity-guided selective quantization function is designed to minimize the loss of space transformation, and the regressive semantic function is explored to promote the flexibility of the formulated semantics in hash code learning. The final joint learning objective is formulated to concurrently preserve the ordinal locality of original data and explore potentials of semantics for producing discriminative hash codes. Importantly, an efficient alternating optimization algorithm with the strictly proof convergence guarantee is developed to solve the resulting objective problem. Extensive experiments on several large-scale datasets validate the superiority of the proposed method against state-of-the-art hashing-based retrieval methods.

Relaxed Asymmetric Deep Hashing Learning: Point-to-Angle Matching.

Due to the powerful capability of the data representation, deep learning has achieved a remarkable performance in supervised hash function learning. However, most of the existing hashing methods focus on point-to-point matching that is too strict and unnecessary. In this article, we propose a novel deep supervised hashing method by relaxing the matching between each pair of instances to a point-to-angle way. Specifically, an inner product is introduced to asymmetrically measure the similarity and dissimilarity between the real-valued output and the binary code. Different from existing methods that strictly enforce each element in the real-valued output to be either +1 or -1, we only encourage the output to be close to its corresponding semantic-related binary code under the cross-angle. This asymmetric product not only projects both the real-valued output and the binary code into the same Hamming space but also relaxes the output with wider choices. To further exploit the semantic affinity, we propose a novel Hamming-distance-based triplet loss, efficiently making a ranking for the positive and negative pairs. An algorithm is then designed to alternatively achieve optimal deep features and binary codes. Experiments on four real-world data sets demonstrate the effectiveness and superiority of our approach to the state of the art.

Scalable deep asymmetric hashing via unequal-dimensional embeddings for image similarity search

In recent years, Hashing has become a popular technique used to support large-scale image retrieval, due to its significantly reduced storage, high search speed and capability of mapping high dimensional original features into compact similarity-preserving binary codes. Although effectiveness achieved, most existing hashing methods are still some limitations, including: (1) Many supervised hashing methods only transform the label information into pairwise similarities to guide the hash code learning process, which will lead to the loss of rich semantic information between image pairs. (2) Some pioneer hashing methods use a relaxation-based strategy to solve discrete problems, resulting in a large quantization error. (3) Some supervised hashing methods handle the hashing learning procedure based on an asymmetric learning manner, and although this partially solves the problems of low efficiency and accuracy of symmetric learning strategy, they are all based on the embeddings of equal dimension, which leads to the reduction in the models representation ability and an increase in potential noise. To overcome the above limitations, in this paper, we propose a novel yet simple but effective hashing method, named Scalable Deep Asymmetric Hashing (SDAH). Specifically, SDAH is an end-to-end deep hashing method based on a fast iterative optimization strategy, which utilizes two real-valued embeddings of unequal dimensions, i.e., real-valued embeddings of images and labels, to flexibly perform asymmetric similarity computation. It can circumvent the use of the large pairwise similarity matrix by introducing an intermediate label matrix term which results in a remarkable reduction in the memory space cost. By doing this, the learned hash codes are more semantically informative for image retrieval tasks. Experimental results on several benchmark datasets highlight the superiority of SDAH in comparison with many state-of-the-art hashing methods.

Multiple-Instance ranking based deep hashing for multi-Label image retrieval

Hashing methods have been widely applied to approximate nearest neighbor search in large-scale image retrieval due to its fast search speed and efficient storage space. In practice, most images are with multiple category-aware objects, i.e., multi-label images. This paper focuses on hash code learning for multi-label image retrieval. Most existing hashing methods directly extract one patch such as a downsized crop from each image as a training example, which ignores the multi-label characteristic of images and leads to suboptimal representations for multi-label images. Some researches have proved that each multi-label image follows a multi-instance assumption, where each image is represented as a bag of category-aware proposals (instances). However, existing multiple-instance learning methods use predefined statistical functions with limited learning capability to construct bag features, and they are designed for classification or pairwise-similarity preserving. Thus, directly applying existing multiple-instance learning methods into deep hashing framework still leads to suboptimal hash codes for retrieval. In this paper, we pose hashing learning for multi-label image retrieval as a problem of multiple-instance ranking learning. To solve this problem, we present an end-to-end deep hashing framework, referred to as Deep Multiple-Instance Ranking based Hashing (DMIRH). In DMIRH, we design a category-aware bag feature construction module, which jointly assigns the learned instances into categories and aggregates the selected instance features into a bag feature representation that can capture the multi-label information of each image. In addition, we propose a novel learning objective, which consists of an Inner Product distance based quantization loss to control the hash quality and a listwise ranking loss to preserve the ranking relationships. Experimental results on public benchmarks show the superiority of DMIRH over several state-of-the-art hashing methods.

Learning Multifunctional Binary Codes for Personalized Image Retrieval

Due to the highly complex semantic information of images, even with the same query image, the expected content-based image retrieval results could be very different and personalized in different scenarios. However, most existing hashing methods only preserve one single type of semantic similarity, making them incapable of addressing such realistic retrieval tasks. To deal with this problem, we propose a unified hashing framework to encode multiple types of information into the binary codes by exploiting convolutional networks (CNNs). Specifically, we assume that typical retrieval tasks are generally defined in two aspects, i.e. high-level semantics (e.g. object categories) and visual attributes (e.g. object shape and color). To this end, our Dual Purpose Hashing model is trained to jointly preserve two kinds of similarities characterizing the two aspects respectively. Moreover, since images with both category and attribute labels are scarce, our model is carefully designed to leverage the abundant partially labelled data as training inputs to alleviate the risk of overfitting. With such a framework, the binary codes of new-coming images can be readily obtained by quantizing the outputs of a specific CNN layer, and different retrieval tasks can be achieved by using the binary codes in different ways. Experiments on two large-scale datasets show that our method achieves comparable or even better performance than those state-of-the-art methods specifically designed for each individual retrieval task while being more compact than the compared methods.

Deep semantic similarity adversarial hashing for cross-modal retrieval

Cross-modal retrieval has attracted considerate attention due to the rapid development of Internet and social media, and cross-modal hashing has been widely and successfully used in this domain. However, most existing hashing methods consider much little the semantic similarity levels between instances, whereas simply classified the semantic relationship as either similar or dissimilar. Besides, the issue of preservation of semantic similarity of original data between the extracted features is less explored from the existing methods. Due to the heterogeneity between different modalities, similarity of different modality features cannot be calculated directly. Therefore, in this paper, we propose a deep semantic similarity adversarial hashing (DSSAH) for cross-modal retrieval. We first calculate semantic similarity by using both label and feature information to provide a more accurate value for similarity between instances. And then an adversarial modality discriminator is introduced to establish a common feature space where similarity of each modality features can be calculated. Finally, two loss functions referring to inter-modal loss and intra-modal loss are designed to generate high quality hash codes. Experiments on three common datasets for cross-modal retrieval show that DSSAH outperforms state-of-the-art cross-modal hashing methods in cross-modal retrieval applications.