Compact Hash Codes Research Articles

Deep Progressive Asymmetric Quantization Based on Causal Intervention for Fine-Grained Image Retrieval

In the field of computer vision, fine-grained image retrieval is an extremely challenging task due to the inherently subtle intra-class object variations. In addition, the high-dimensional real-valued features extracted from large-scale fine-grained image datasets slow the retrieval speed and increase the storage cost. To solve above issues, existing fine-grained image retrieval methods mainly focus on finding more discriminative local regions for generating discriminative and compact hash codes, which achieve limited fine-grained image retrieval performance due to the large quantization errors and the confounding granularities and context of discriminative parts, i.e., the correct recognition of fine-grained objects mainly attribute to the discriminative parts and their context. To learn robust causal features and reduce the quantization errors, we propose a deep progressive asymmetric quantization (DPAQ) method based on causal intervention to learn compact and robust descriptions for fine-grained image retrieval task. Specifically, we introduce a structural causal model to learn robust casual features via causal intervention for fine-grained visual recognition. Subsequently, we design a progressive asymmetric quantization layer in the feature embedding space, which can preserve the semantic information and reduce the quantization errors sufficiently. Finally, we incorporate both the fine-grained image classification and retrieval tasks into an end-to-end deep learning architecture for generating robust and compact descriptions. Experimental results on several fine-grained image retrieval datasets demonstrate that the proposed DPAQ method performs the best for fine-grained image retrieval task and surpasses the state-of-the art fine-grained hashing methods by a large margin.

Look into Gradients: Learning Compact Hash Codes for Out-of-distribution Retrieval

Look into Gradients: Learning Compact Hash Codes for Out-of-distribution Retrieval

Unsupervised Hashing Retrieval via Efficient Correlation Distillation

Deep hashing has been widely used in multimedia retrieval systems due to its storage and computation efficiency. Unsupervised hashing has received a lot of attention in recent years because it does not rely on label information. However, existing deep unsupervised hashing methods usually use rough pairwise relations to constrain the similarity between hash codes locally, which is insufficient and inefficient to reconstruct accurate correlations across samples. To address this issue, we propose a generic distillation framework for the preservation of the similarity relationship. Specifically, we design a distillation loss to reconstruct the batchwise similarity distribution between feature space and hash code space, allowing us to capture the global correlation knowledge contained in features and propagate it into hash codes efficiently. This framework can apply to both intra-modal and inter-modal scenarios. Furthermore, we design a new quantization method that quantizes the continuous values to a clipping value instead of ±1 to reduce the inconsistency between continuous features and hash codes. This method can also avoid the vanishing gradient problem during training. Finally, extensive experiments for image hashing retrieval and cross-modal hashing retrieval on public datasets demonstrate that the proposed method can yield compact hash codes and outperforms the state-of-the-art baselines.

Deep collaborative graph hashing for discriminative image retrieval

The most striking success of deep hashing for large-scale image retrieval benefits from its powerful discriminative representation of deep learning and the attractive computational efficiency of compact hash code learning. Most existing deep semantic-preserving hashing regard the available semantic labels as the ground truth for classification or transform them into prevalent pairwise similarities. However, such strategies fail to capture the interactive correlations between the visual semantics embedded in images and the given category-level labels. Moreover, they utilize the fixed piecewise or pairwise semantics as the optimization objectives, which suffers from the limited flexibility on semantic representation and adaptive knowledge communication in hash code learning. In this paper, we propose a novel Deep Collaborative Graph Hashing (DCGH), which collectively considers multi-level semantic embeddings, latent common space construction, and intrinsic structure mining in discriminative hash codes learning, for large-scale image retrieval. To the best of our knowledge, this is the first collaborative graph hashing for image retrieval. Specifically, instead of using the conventional single-flow visual network architecture, we design a dual-stream feature encoding network to jointly explore the multi-level semantic information across visual and semantic features. Moreover, a well-established shared latent space is constructed based on space reconstruction to explore the concurrent information and bridge the semantic gap between visual and semantic space. Furthermore, a graph convolutional network is introduced to preserve the latent structural relations in the optimal pairwise similarity-preserving hash codes. The whole learning framework is optimized in an end-to-end fashion. Extensive experiments on different datasets demonstrate that our DCGH can achieve superb image retrieval performance against state-of-the-art supervised hashing methods. The source codes of the proposed DCGH are available at https://github.com/JalinWang/DCGH .

An Efficient and Robust Semantic Hashing Framework for Similar Text Search

Similar text search aims to find texts relevant to a given query from a database, which is fundamental in many information retrieval applications, such as question search and exercise search. Since millions of texts always exist behind practical search engine systems, a well-developed text search system usually consists of recall and ranking stages. Specifically, the recall stage serves as the basis in the system, where the main purpose is to find a small set of relevant candidates accurately and efficiently. Towards this goal, deep semantic hashing, which projects original texts into compact hash codes, can support good search performance. However, learning desired textual hash codes is extremely difficult due to the following problems. First, compact hash codes (with short length) can improve retrieval efficiency, but the demand for learning compact hash codes cannot guarantee accuracy due to severe information loss. Second, existing methods always learn the unevenly distributed codes in the space from a local perspective, leading to unsatisfactory code-balance results. Third, a large fraction of textual data contains various types of noise in real-world applications, which causes the deviation of semantics in hash codes. To this end, in this paper, we first propose a general unsupervised encoder-decoder semantic hashing framework, namely MASH (short for Memory-bAsed Semantic Hashing), to learn the balanced and compact hash codes for similar text search. Specifically, with a target of retaining semantic information as much as possible, the encoder introduces a novel relevance constraint among informative high-dimensional representations to guide the compact hash code learning. Then, we design an external memory where the hashing learning can be optimized in the global space to ensure the code balance of the learning results, which can promote search efficiency. Besides, to alleviate the performance degradation problem of the model caused by text noise, we propose an improved SMASH (short for denoiSing Memory-bAsed Semantic Hashing) model by incorporating a noise-aware encoder-decoder framework. This framework considers the noise degree for each text from the semantic deviation aspect, ensuring the robustness of hash codes. Finally, we conduct extensive experiments in three real-world datasets. The experimental results clearly demonstrate the effectiveness and efficiency of MASH and SMASH in generating balanced and compact hash codes, as well as the superior denoising ability of SMASH.

OMGH: Online Manifold-Guided Hashing for Flexible Cross-Modal Retrieval

Cross-modal hashing has recently gained an increasing attention for its efficiency and fast retrieval speed in indexing the multimedia data across different modalities. Nevertheless, the multimedia data points often emerge in a streaming manner, which often makes the offline hashing methods loss their efficiency and flexibility. Besides, existing online methods often lack of learning capacity to preserve the semantic correlations between newly coming data and existing data, while suffering from the limited discriminative capability and less scalability for handling both labeled and unlabeled data. To alleviate these concerns, this paper proposes an Online Manifold-Guided Hashing (OMGH) framework, which can incrementally learn the compact hash code of streaming data while adaptively optimizing the hash function in a streaming manner. To be specific, OMGH first exploits a matrix tri-factorization framework to learn the discriminative hash codes for streaming multi-modal data. Then, an online anchor-based manifold structure is designed to sparsely represent the old data and adaptively guide the hash code learning process, which can well reduce the complexity in preserving the semantic correlation between the old data and streaming data. Meanwhile, such anchor-based manifold embedding is adaptive to the unsupervised and supervised learning strategies in a flexible way. Besides, an online discrete optimization method is efficiently addressed to incrementally update the hash functions and optimize the hash codes on streaming data points. As a result, the derived hash codes are more semantically meaningful for various online cross-modal retrieval tasks. Extensive experiments verify the advantages of the proposed OMGH model, by achieving and improving the state-of-the-art cross-modal retrieval performances on three benchmark datasets.

Unsupervised Multiview Distributed Hashing for Large-Scale Retrieval

Multi-view hashing (MvH) learns compact hash code by efficiently integrating multi-view data, and has achieved promising performance in large-scale retrieval task. In real-world applications, multi-view data is often stored or collected in different locations, and learning hash code in such case is more challenging yet less studied. In addition, unsupervised MvHs hardly achieve impressive retrieval performance due to absence of supervision. To fulfill this gap, this paper introduces a novel unsupervised multi-view distributed hashing (UMvDisH) to learn hash code from multi-view data, which is distributed in different nodes of a network. UMvDisH jointly performs latent factor model and spectral clustering to generate latent hash code and pseudo label respectively in each node. The consistency between hash code and pseudo label improves discrimination of hash code. The proposed distributed learning problem is divided into a set of decentralized subproblems by imposing local consistency among neighbor nodes. As such, the subproblems can be solved in parallel, and training time can be reduced. The communication cost is low due to no exchange of training data. Experimental results on four benchmark image datasets including a very large-scale image dataset show that UMvDisH achieves comparable retrieval performance and trains faster than state-of-the-art unsupervised MvHs in the distributed setting.

Ensemble learning framework for image retrieval via deep hash ranking

Deep hashing combines feature extraction or representation with hash coding jointly, which can significantly improve the speed of large-scale image retrieval. However, we notice that compared with traditional retrieval methods, due to the reduction of dimension and information loss, the retrieval performance of binaryhash coding has declined to a certain extent. Most hash retrieval algorithms focus on the semantic similarity between image pairs, and ignore the ranking information between the returned samples. The returned samples should not only match the retrieved samples, but also rank the correct samples in front of the returned list. In addition, the performance difference of the deep model used in deep hash retrieval will also limit the efficiency of retrieval. To address such problem, we proposed an ensemble deep neural model robust framework for image retrieval, which can learn compact hash codes containing rich semantic information through hash constraints. The ensemble strategy is introduced, and the weighted voting is applied to integrate the ranking list. Comprehensive experiments on three benchmark datasets show that the proposed method achieves very competitive results. Codes are available at https://github.com/lidonggen-123/Ensemble_Deephash_Image_Retrieval.

Discrete Joint Semantic Alignment Hashing for Cross-Modal Image-Text Search

Supervised cross-modal image-text hashing has aroused extensive concentrations in comprehending the correspondence between vision and language for data search tasks. Existing methods learn the compact hash codes by leveraging a given image-text data pairs or supervised information to explore such correspondence. However, they still confront obvious drawbacks. First, there is no engagement between multiple semantic information that yields the suboptimal search performance. Second, most of them adopt continuous relaxation strategy by discarding the discrete constraints, which results in large binary quantization errors. To deal with these problems, we propose a novel supervised hashing method, termed Discrete Joint Semantic Alignment Hashing (DJSAH). Specifically, it builds a connection between semantics (a.k.a. class labels and pairwise similarities) by the joint semantic alignment learning. And thus the high-level discriminative semantics can be preserved into the hash codes. Besides, a well-designed discrete optimization algorithm with linear computation and memory cost is developed to reduce the information loss of the hash codes with no need for relaxation. Extensive experiments and analyses on three benchmark datasets validate the superiority of the proposed DJSAH against several state-of-the-art hashing methods.

Graph-based discriminative features learning for fine-grained image retrieval

Fine-grained image retrieval has gradually become a hot topic in computer vision , which aims to retrieve images with the same subcategories from general visual categories. Though fine-grained image retrieval has made a breakthrough with the development of convolutional neural networks, its performance is still limited by the low discriminative feature embedding. To solve this problem, most prior works focus on mining more discriminative features with various strategies. In this paper, we propose a novel graph-based discriminative features learning network for fine-grained image retrieval (GDF-Net). We first design a global fine-grained feature aggregation module, which reconstructs the discriminative features through capturing context correlation based on a K-Nearest Neighbor graph. To reduce storage overhead and speed up retrieval, we further design a semantic hash encoding module, which generates a semantically compact hash code under the guidance of Cauchy quantization loss and bit balance loss. Validated by extensive experiments and ablation studies, our method consistently outperforms state-of-the-art generic retrieval methods as well as fine-grained retrieval methods on three datasets, e.g., CUB Birds, Stanford Dogs and Stanford Cars.

Supervised Adaptive Similarity Matrix Hashing.

Compact hash codes can facilitate large-scale multimedia retrieval, significantly reducing storage and computation. Most hashing methods learn hash functions based on the data similarity matrix, which is predefined by supervised labels or a distance metric type. However, this predefined similarity matrix cannot accurately reflect the real similarity relationship among images, which results in poor retrieval performance of hashing methods, especially in multi-label datasets and zero-shot datasets that are highly dependent on similarity relationships. Toward this end, this study proposes a new supervised hashing method called supervised adaptive similarity matrix hashing (SASH) via feature-label space consistency. SASH not only learns the similarity matrix adaptively, but also extracts the label correlations by maintaining consistency between the feature and the label space. This correlation information is then used to optimize the similarity matrix. The experiments on three large normal benchmark datasets (including two multi-label datasets) and three large zero-shot benchmark datasets show that SASH has an excellent performance compared with several state-of-the-art techniques.

Enhanced Deep Discrete Hashing with semantic-visual similarity for image retrieval

Hashing has been shown to be successful in a number of Approximate Nearest Neighbor (ANN) domains, ranging from medicine, computer vision to information retrieval. However, current deep hashing methods either ignore both rich information of labels and visual linkages of image pairs, or leverage relaxation-based algorithms to address discrete problems, resulting in a large information loss. To address the aforementioned problems, in this paper, we propose an Enhanced Deep Discrete Hashing (EDDH) method to leverage both label embedding and semantic-visual similarity to learn the compact hash codes. In EDDH, the discriminative capability of hash codes is enhanced by a distribution-based continuous semantic-visual similarity matrix, where not only the margin between the positive pairs and negative pairs is expanded, but also the visual linkages between image pairs is considered. Specifically, the semantic-visual continuous similarity matrix is constructed by analyzing the asymmetric generalized Gaussian distribution of the visual linkages between pairs with label consideration. Besides, in order to achieve an efficient hash learning framework, EDDH employs an asymmetric real-valued learning structure to learn the compact hash codes. In addition, we develop a fast discrete optimization algorithm, which can directly generate discrete binary codes in single step, and introduce an intermediate term before iterations to avoid the problems caused by directly the use of large semantic-visual similarity matrix, which results in a significant reduction in the computational overhead. Finally, we conducted extensive experiments on three datasets to show that EDDH has a significantly enhanced performance compared to the compared state-of-the-art baselines.

Unsupervised Multi-modal Hashing for Cross-Modal Retrieval

The explosive growth of multimedia data on the Internet has magnified the challenge of information retrieval. Multimedia data usually emerges in different modalities, such as image, text, video, and audio. Unsupervised cross-modal hashing techniques that support searching among multi-modal data have gained importance in large-scale retrieval tasks because of the advantage of low storage cost and high efficiency. Current methods learn the hash functions by transforming high-dimensional data into discrete hash codes. However, the original manifold structure and semantic correlation are not preserved well in compact hash codes. We propose a novel unsupervised cross-modal hashing method to cope with this problem from two perspectives. On the one hand, the semantic correlation in textual space and the locally geometric structure in the visual space are reconstructed by unified hashing features seamlessly and simultaneously. On the other hand, the \(\ell _{2,1}\)-norm penalties are imposed on the projection matrices separately to learn the relevant and discriminative hash codes. The experimental results indicate that our proposed method achieves an improvement of 1%, 6%, 9%, and 2% over the best comparison method on the four publicly available datasets (WiKi, PASCAL-VOC, UCI Handwritten Digit, and NUS-WIDE), respectively. In conclusion, the proposed framework which combines hash functions learning and multimodal graph embedding is effective in learning hash codes and achieves superior retrieval performance compared to state-of-the-art methods.

NSDH: A Nonlinear Supervised Discrete Hashing framework for large-scale cross-modal retrieval

Hashing technology has been widely used in approximate nearest neighbor search algorithms for large-scale cross-modal retrieval due to its significantly reduced storage and high-speed search capabilities. However, most existing supervised cross-modal hashing methods either mainly rely on binary pairwise similarity and fail to exploit the rich semantic information contained in the label matrix, or the use of single linear projections which suffer from limited information completeness. In this paper, we propose a novel method, named Nonlinear Supervised Discrete Hashing (NSDH). Specifically, NSDH consists of two components, (1) a semantic enhancement descriptor consisting of multiple linear projections that is used to extract comprehensive latent representations of heterogeneous multimedia data, which aligns the original heterogeneous features and integrates the rich semantic label matrix; (2) a fast discrete optimization module used to learn discriminative compact hash codes, which preserves the similarity information using an inner product between the real-valued embeddings of the output of the semantic enhancement descriptors. Therefore, NSDH leverages both the label matrix and similarity information in order to enhance the semantic information of the learned hash codes. In this way, the representation learning capability of the output layer of semantic enhancement descriptors can be greatly enhanced and as a result the learned hash codes are more discriminative. In addition, we present a fast discrete optimization algorithm to efficiently learn the binary hash codes. Results from our experiments on two benchmark datasets highlight the superiority of NSDH in comparison to many state-of-the-art cross-modal hashing methods.

Video Moment Localization via Deep Cross-Modal Hashing.

Due to the continuous booming of surveillance and Web videos, video moment localization, as an important branch of video content analysis, has attracted wide attention from both industry and academia in recent years. It is, however, a non-trivial task due to the following challenges: temporal context modeling, intelligent moment candidate generation, as well as the necessary efficiency and scalability in practice. To address these impediments, we present a deep end-to-end cross-modal hashing network. To be specific, we first design a video encoder relying on a bidirectional temporal convolutional network to simultaneously generate moment candidates and learn their representations. Considering that the video encoder characterizes temporal contextual structures at multiple scales of time windows, we can thus obtain enhanced moment representations. As a counterpart, we design an independent query encoder towards user intention understanding. Thereafter, a cross-model hashing module is developed to project these two heterogeneous representations into a shared isomorphic Hamming space for compact hash code learning. After that, we can effectively estimate the relevance score of each "moment-query" pair via the Hamming distance. Besides effectiveness, our model is far more efficient and scalable since the hash codes of videos can be learned offline. Experimental results on real-world datasets have justified the superiority of our model over several state-of-the-art competitors.

Learning Efficient Hash Codes for Fast Graph-Based Data Similarity Retrieval.

Traditional operations, e.g. graph edit distance (GED), are no longer suitable for processing the massive quantities of graph-structured data now available, due to their irregular structures and high computational complexities. With the advent of graph neural networks (GNNs), the problems of graph representation and graph similarity search have drawn particular attention in the field of computer vision. However, GNNs have been less studied for efficient and fast retrieval after graph representation. To represent graph-based data, and maintain fast retrieval while doing so, we introduce an efficient hash model with graph neural networks (HGNN) for a newly designed task (i.e. fast graph-based data retrieval). Due to its flexibility, HGNN can be implemented in both an unsupervised and supervised manner. Specifically, by adopting a graph neural network and hash learning algorithms, HGNN can effectively learn a similarity-preserving graph representation and compute pair-wise similarity or provide classification via low-dimensional compact hash codes. To the best of our knowledge, our model is the first to address graph hashing representation in the Hamming space. Our experimental results reach comparable prediction accuracy to full-precision methods and can even outperform traditional models in some cases. In real-world applications, using hash codes can greatly benefit systems with smaller memory capacities and accelerate the retrieval speed of graph-structured data. Hence, we believe the proposed HGNN has great potential in further research.

Deep Hashing Learning for Visual and Semantic Retrieval of Remote Sensing Images

Driven by the urgent demand for managing remote sensing big data, large-scale remote sensing image retrieval (RSIR) attracts increasing attention in the remote sensing field. In general, existing retrieval methods can be regarded as visual-based retrieval approaches that search and return a set of similar images to a given query image from a database. Although these retrieval methods have delivered good results, there is still a question that needs to be addressed: can we obtain the accurate semantic labels of the returned similar images to further help analyzing and processing imagery? To this end, in this article, we redefine the image retrieval problem as visual and semantic retrieval of images. Especially, we propose a novel deep hashing convolutional neural network (DHCNN) to retrieve similar images and classify their semantic labels simultaneously in a unified framework. In more detail, a convolutional neural network (CNN) is used to extract high-dimensional deep features. Then, a hash layer is perfectly inserted into the network to transfer the deep features into compact hash codes. In addition, a fully connected layer with a softmax function is performed on the hash layer to generate the probability distribution of each class. Finally, a loss function is elaborately designed to consider the label loss of each image and similarity loss of pairs of images simultaneously. Experimental results on three remote sensing data sets demonstrate that the proposed method can achieve state-of-art retrieval and classification performance.

Cluster-wise unsupervised hashing for cross-modal similarity search

Cross-modal hashing similarity retrieval plays dual roles across various applications including search engines and autopilot systems. More generally, these methods also known to reduce the computation and memory storage in a training scheme. The key limitation of current methods are that: (i) they relax the discrete constrains to solve the optimization problem which may defeat the model purpose, (ii) projecting heterogenous data into a latent space may encourage to loss the diverse representations in such data, (iii) transforming real-valued data point to the binary codes always resulting in a loss of information and producing the suboptimal continuous latent space. In this paper, we propose a novel framework to project the original data points from different modalities into its own low-dimensional latent space and finds the cluster centroid points in its a low-dimensional space, using Cluster-wise Unsupervised Hashing (CUH). In particular, the proposed clustering scheme aims to jointly learns the compact hash codes and the corresponding linear hash functions. A discrete optimization framework is developed to learn the unified binary codes across modalities under of the guidance cluster-wise code-prototypes. Extensive experiments over multiple datasets demonstrate the effectiveness of our proposed model in comparison with the state-of-the-art in unsupervised cross-modal hashing tasks.

Relaxed Locality Preserving Supervised Discrete Hashing

Supervised discrete hashing (SDH) has widely attracted attention because it can directly obtain hash codes rather than relaxing the discrete constraint, which reduces information loss during the quantization process. However, SDH and its extensions tend to ignore the underlying geometry structure of data, while in many cases, data is lying on a low-dimensional manifold. Therefore, based on SDH, we propose a novel manifold learning-based supervised discrete hashing method in this paper to further improve the performance. Our goal is to learn compact hash codes by discovering latent information and using the local structure of data. To this end, we utilize the anchor graph to capture local neighborhood relationship embedding in data for manifold learning. Besides, since the previous methods use a strict binary label matrix for classification, we jointly optimize a relaxed label matrix to increase the discriminative ability of the proposed model. As such, the learned hash codes are more suitable for classification. Compared with SDH and other popular hashing methods, the experimental results on four large-scale datasets show that the proposed hashing method has better performance.

Learning discriminative hashing codes for cross-modal retrieval based on multi-view features

Hashing techniques have been applied broadly in retrieval tasks due to their low storage requirements and high speed of processing. Many hashing methods based on a single view have been extensively studied for information retrieval. However, the representation capacity of a single view is insufficient and some discriminative information is not captured, which results in limited improvement. In this paper, we employ multiple views to represent images and texts for enriching the feature information. Our framework exploits the complementary information among multiple views to better learn the discriminative compact hash codes. A discrete hashing learning framework that jointly performs classifier learning and subspace learning is proposed to complete multiple search tasks simultaneously. Our framework includes two stages, namely a kernelization process and a quantization process. Kernelization aims to find a common subspace where multi-view features can be fused. The quantization stage is designed to learn discriminative unified hashing codes. Extensive experiments are performed on single-label datasets (WiKi and MMED) and multi-label datasets (MIRFlickr and NUS-WIDE), and the experimental results indicate the superiority of our method compared with the state-of-the-art methods.