Hashing Learning Method Research Articles

Cross-modal hash retrieval based on semantic multiple similarity learning and interactive projection matrix learning

Cross-modal hash has become a key technology for large datasets retrieval. However, some challenges still need to be tackled: 1) How to effectively embed semantic information into hash learning; 2) How to strengthen cross-modal mutual communication and common learning; 3) How to effectively solve data extension with semantics. In this paper, we propose a novel Extensible Multi-similarity Cross-modal Hash Learning (EMCHL) method. To reduce the differentiation of inter-modal and strengthen cross-modal mutual communication, EMCHL mainly considers three similarity measures: 1) Using Hamming distance to measure the semantic structure self-similarity of label and sample hash codes in unimodal, it can reduce the sample quantization error; 2) the log-likelihood function is adopted to enhance information exchange in cross-modal hash codes. Correspondence cross-modal reciprocal-similarity measurement can reduce the inter-modal quantization code errors; 3) The robust low-rank constraint of the dot product is introduced to perform the cross-modal sparse latent-similarity calculation. It can establish the connection between the common latent semantic space and the hash codes. Besides, we also propose an Interactive Projection Matrix Learning (IPML) extensible mechanism that makes it easier to be applied to large-scale retrieval datasets. Extensive experiment results show the superior performance of EMCHL compared to state-of-the-art cross-modal hash methods. Our method code is released at: https://github.com/Tjeep-Tan/EMCHL.

Cross‐modal person re‐identification based on deep attention hash learning

SummaryPerson re‐identification based on text description is a critical task in modern security systems. However, existing methods primarily focus on performance and overlook the crucial aspect of retrieval efficiency. In this article, we propose a novel two‐stage multimodal re‐discovery algorithm called DCH‐ReID, which leverages attention hashing. First, we introduce a chunked mapping hash learning method that effectively mitigates confusion between hash codes. Second, we propose a hash learning approach based on the channel attention mechanism, assigning higher binary bit weights to important body parts. Finally, to balance retrieval performance and time efficiency, we present a two‐stage retrieval scheme. Through extensive experiments on the CUHK‐PEDES benchmark dataset, we validate that our proposed DCH‐ReID algorithm exhibits superior efficiency and higher accuracy compared to current mainstream text‐based pedestrian re‐identification algorithms.

Attention-Oriented Deep Multi-Task Hash Learning

Hashing has wide applications in image retrieval at large scales due to being an efficient approach to approximate nearest neighbor calculation. It can squeeze complex high-dimensional arrays via binarization while maintaining the semantic properties of the original samples. Currently, most existing hashing methods always predetermine the stable length of hash code before training the model. It is inevitable for these methods to increase the computing time, as the code length converts, caused by the task requirements changing. A single hash code fails to reflect the semantic relevance. Toward solving these issues, we put forward an attention-oriented deep multi-task hash learning (ADMTH) method, in which multiple hash codes of varying length can be simultaneously learned. Compared with the existing methods, ADMTH is one of the first attempts to apply multi-task learning theory to the deep hashing framework to generate and explore multi-length hash codes. Meanwhile, it embeds the attention mechanism in the backbone network to further extract discriminative information. We utilize two common available large-scale datasets, proving its effectiveness. The proposed method substantially improves retrieval efficiency and assures the image characterizing quality.

Feature adaptive multi-view hash for image search

<abstract><p>With the rapid development of network technology and small handheld devices, the amount of data has significantly increased and various kinds of data can be supplied to us at the same time. Recently, hashing technology has become popular in executing large-scale similarity search and image matching tasks. However, most of the prior hashing methods are mainly focused on the choice of the high-dimensional feature descriptor for learning effective hashing functions. In practice, real world image data collected from multiple scenes cannot be descriptive enough by using a single type of feature. Recently, several unsupervised multi-view hashing learning methods have been proposed based on matrix factorization, anchor graph and metric learning. However, large quantization error will be introduced via a sign function and the robustness of multi-view hashing is ignored. In this paper we present a novel feature adaptive multi-view hashing (FAMVH) method based on a robust multi-view quantization framework. The proposed method is evaluated on three large-scale benchmarks CIFAR-10, CIFAR-20 and Caltech-256 for approximate nearest neighbor search task. The experimental results show that our approach can achieve the best accuracy and efficiency in the three large-scale datasets.</p></abstract>

Hybrid-attention based Feature-reconstructive Adversarial Hashing Networks for Cross-modal Retrieval

With the massive growth of data of various modal types, people no longer use a single modal retrieval method, but a cross-modal retrieval method when performing retrieval tasks. Such methods often need to store data efficiently while maintaining the characteristics of fast query. Because the hashing learning method can represent the original high-dimensional data through a simple and compact binary hash code, which can greatly compress the data size and facilitate data storage and mutual retrieval, the cross-modal hashing retrieval has gradually become a hot topic in recent years. However, how to bridge the gap between modalities to improve the retrieval performance further is still a challenging problem. In order to solve this problem, we propose a Hybrid-attention based Feature-reconstructive Adversarial Hashing (HFAH) networks for cross-modal retrieval. First, a label semantic guidance module is introduced to guide the extraction process of text features and image features through the learning of labels, so as to fully maintain the semantic similarity between different modal data. Then, the hybrid-attention module is introduced to make the extracted data contain richer semantic information. Finally, the feature reconstruction network is used to make the relevant degree between similar cross-modal data pairs higher than that between dissimilar data pairs. Related experiments on two benchmark datasets confirm to us that HFAH performs better than several existing cross- modal retrieval methods.

Contrastive hashing with vision transformer for image retrieval

Hashing techniques have attracted considerable attention owing to their advantages of efficient computation and economical storage. However, it is still a challenging problem to generate more compact binary codes for promising performance. In this paper, we propose a novel contrastive vision transformer hashing method, which seamlessly integrates contrastive learning and vision transformers (ViTs) with hash technology into a well-designed model to learn informative features and compact binary codes simultaneously. First, we modify the basic contrastive learning framework by designing several hash layers to meet the specific requirement of hash learning. In our hash network, ViTs are applied as backbones for feature learning, which is rarely performed in existing hash learning methods. Then, we design a multiobjective loss function, in which contrastive loss explores discriminative features by maximizing agreement between different augmented views from the same image, similarity preservation loss performs pairwise semantic preservation to enhance the representative capabilities of hash codes, and quantization loss controls the quantitative error. Hence, we can facilitate end-to-end joint training to improve the retrieval performance. The encouraging experimental results on three widely used benchmark databases demonstrate the superiority of our algorithm compared with several state-of-the-art hashing algorithms.

Deep objectness hashing using large weakly tagged photos

CNN-based hashing methods have greatly boosted the performance of image retrieval, under the strong supervision of large amounts of manually annotated labels. In recent years, a large number of social media images with user tags have been generated on the Internet. These images can be regarded as weakly labeled training data, which can provide rich samples for training hash network, and greatly reduce the cost of obtaining training data. However, there are noise and visual irrelevant tags in user tags, and different tags may describe different objects in the image. In the previous CNN-based hashing method, a training image usually corresponds to a manual label and generates a hash code. These methods are difficult to use the image described by user tags with noise. For solving the above problem, we propose a CNN-based objectness hash learning method using user tags as a guide for training. First of all, the user tags are roughly filtered to remove noise tags that are not related to the visual content of images. Secondly, we quantify user tags into a unified semantic space and extract the highest-frequency words of the semantic space from similar objectness areas as their labels. Then, these objectness areas with their labels are grouped into a series of triple units as training data. So that the generated hash code can inherit the semantic similarity of the objectness areas well, that is, the Hamming distance between hash codes generated by similar objectness areas is closer, the reverse is the farther. Experimental results on NUS-WIDE and Flickr datasets show that our method can effectively extract object-level semantic information from weak user tags, and improve the accuracy of image retrieval.

A feature consistency driven attention erasing network for fine-grained image retrieval

Large-scale fine-grained image retrieval based hashing learning method has two main problems. First, low dimension feature embedding can fasten the retrieval process but bring accuracy decrease due to much information loss. Second, fine-grained images lead to the same category query hash codes mapping into the different cluster in database hash latent space. To handle these issues, we propose a feature consistency driven attention erasing network (FCAENet) for fine-grained image retrieval. For the first issue, we propose an adaptive augmentation module in FCAENet, which is the selective region erasing module (SREM). SREM makes the network more robust on subtle differences of fine-grained task by adaptively covering some regions of raw images. The feature extractor and hash layer can learn more representative hash codes for fine-grained images by SREM. With regard to the second issue, we fully exploit the pair-wise similarity information and add the enhancing space relation loss (ESRL) in FCAENet to make the vulnerable relation stabler between the query hash code and database hash code. We conduct extensive experiments on five fine-grained benchmark datasets (CUB2011, Aircraft, NABirds, VegFru, Food101) for 12bits, 24bits, 32bits, 48bits hash codes. The results show that FCAENet achieves the state-of-the-art (SOTA) fine-grained image retrieval performance based on the hashing learning method.

Cross‐modal retrieval based on deep regularized hashing constraints

Cross-modal retrieval has attracted great attention due to the increasing demand for tremendous amounts of multimodal data in recent years. These retrievals could either be text-to-image or image-to-text. To address the problem of inappropriate information included between images and texts, we propose two cross-modal recovery techniques established on a dual-branch neural network defined on a common subspace and the hashing learning method. First, a cross-modal recovery technique established on a multilabel information deep ranking model (MIDRM) is provided. In this method, we introduce a triplet-loss function into the dual-branch neural network model. This function takes advantage of the semantic information of the bimodal components, focusing on not only the similarities between similar images and text features but also the distances between dissimilar images and texts. Second, we establish a new cross-modal hashing technique said to be the deep regularized hashing constraint (DRHC). In this method, the regularized function is used to replace the binary constraint, and the discrete value is constrained to a certain numerical range so that the network can achieve end-to-end training. Overall, the time complexity is greatly improved, and the occupied storage space is also greatly reduced. Different experiments on our proposed MIDRM and DRHC models demonstrate their superior performance to those of the state-of-the-art methods on two widely used data sets. The experimental results show that our approach also increases the mean average precision of cross-modal recovery.

Joint Specifics and Consistency Hash Learning for Large-Scale Cross-Modal Retrieval.

With the dramatic increase in the amount of multimedia data, cross-modal similarity retrieval has become one of the most popular yet challenging problems. Hashing offers a promising solution for large-scale cross-modal data searching by embedding the high-dimensional data into the low-dimensional similarity preserving Hamming space. However, most existing cross-modal hashing usually seeks a semantic representation shared by multiple modalities, which cannot fully preserve and fuse the discriminative modal-specific features and heterogeneous similarity for cross-modal similarity searching. In this paper, we propose a joint specifics and consistency hash learning method for cross-modal retrieval. Specifically, we introduce an asymmetric learning framework to fully exploit the label information for discriminative hash code learning, where 1) each individual modality can be better converted into a meaningful subspace with specific information, 2) multiple subspaces are semantically connected to capture consistent information, and 3) the integration complexity of different subspaces is overcome so that the learned collaborative binary codes can merge the specifics with consistency. Then, we introduce an alternatively iterative optimization to tackle the specifics and consistency hashing learning problem, making it scalable for large-scale cross-modal retrieval. Extensive experiments on five widely used benchmark databases clearly demonstrate the effectiveness and efficiency of our proposed method on both one-cross-one and one-cross-two retrieval tasks.

Meta-Hashing for Remote Sensing Image Retrieval

With the explosive growth of the volume and resolution of high-resolution remote-sensing (HRRS) images, the management of them becomes a challenging task. The traditional content-based remote-sensing image retrieval (CBRSIR) technologies cannot meet what we expect due to the large volume of image archives and complex contents within HRRS images. As a successful approximate nearest neighborhood (ANN) search technique, Hash learning has received wide attention, especially when deep convolutional neural networks (DCNNs) appear. Due to DCNNs&#x2019; strong capacity of feature learning, many DCNN-based hashing methods have been proposed and achieved good performance for large-scale CBRSIR tasks. Nevertheless, their limitation is that a large of labeled training samples should be collected for training the deep models. To overcome this limitation, this article, therefore, develops a new supervised hash learning method for the large-scale HRRS CBRSIR task based on meta-learning, which could achieve well-retrieval performance with a few labeled training samples. First, taking the characteristics of HRRS into account, we develop a self-adaptive convolution (SAP-Conv) block and design a hashing net based on the block. SAP-Conv can learn robust features from HRRS images by exploring their multiscale information. Second, to enhance the generalization of the hashing net under a few labeled training samples, the hash learning is formulated in a meta-way, and we name it meta-hashing. Meta-hashing can effectively preserve the similarities between support and query set, and the similarities between samples within support set by the developed loss function. To further improve the performance of meta-hashing, we expand it to a dynamic version named dynamic-meta-hashing, in which the numbers of support and query are changeable in the training phase. Experimental results counted on the three widely used HRRS datasets demonstrate our dynamic-meta-hashing and meta-hashing can achieve promising performance in large-scale HRRS CBRSIR tasks based on a few training samples. Our source codes are available at <uri>https://github.com/TangXu-Group/Meta-hashing</uri>.

Discrete hashing with triple supervision learning

In recent years, discrete supervised hashing methods have attracted increasing attention because of their high retrieval efficiency and precision. However, in these methods, some effective semantic information is typically neglected, which means that all the information is not sufficiently utilized. Moreover, these methods often only decompose the first-order features of the original data, ignoring the more fine-grained higher-order features. To address these problems, we propose a supervised hashing learning method called discrete hashing with triple supervision learning (DHTSL). Specifically, we integrate three aspects of semantic information into this method: (1) the bidirectional mapping of semantic labels; (2) pairwise similarity relations; (3) second-order features from the original data. We also design a discrete optimization method to solve the proposed objective function. Moreover, an out-of-sample extension strategy that can better maintain the independence and balance of hash codes is employed to improve retrieval performance. Extensive experiments on three widely used datasets demonstrate its superior performance.

Deep Multi-View Enhancement Hashing for Image Retrieval.

Hashing is an efficient method for nearest neighbor search in large-scale data space by embedding high-dimensional feature descriptors into a similarity preserving Hamming space with a low dimension. However, large-scale high-speed retrieval through binary code has a certain degree of reduction in retrieval accuracy compared to traditional retrieval methods. We have noticed that multi-view methods can well preserve the diverse characteristics of data. Therefore, we try to introduce the multi-view deep neural network into the hash learning field, and design an efficient and innovative retrieval model, which has achieved a significant improvement in retrieval performance. In this paper, we propose a supervised multi-view hash model which can enhance the multi-view information through neural networks. This is a completely new hash learning method that combines multi-view and deep learning methods. The proposed method utilizes an effective view stability evaluation method to actively explore the relationship among views, which will affect the optimization direction of the entire network. We have also designed a variety of multi-data fusion methods in the Hamming space to preserve the advantages of both convolution and multi-view. In order to avoid excessive computing resources on the enhancement procedure during retrieval, we set up a separate structure called memory network which participates in training together. The proposed method is systematically evaluated on the CIFAR-10, NUS-WIDE and MS-COCO datasets, and the results show that our method significantly outperforms the state-of-the-art single-view and multi-view hashing methods.

Robust semi-supervised non-negative matrix factorization for binary subspace learning

Non-negative matrix factorization and its extensions were applied to various areas (i.e., dimensionality reduction, clustering, etc.). When the original data are corrupted by outliers and noise, most of non-negative matrix factorization methods cannot achieve robust factorization and learn a subspace with binary codes. This paper puts forward a robust semi-supervised non-negative matrix factorization method for binary subspace learning, called RSNMF, for image clustering. For better clustering performance on the dataset contaminated by outliers and noise, we propose a weighted constraint on the noise matrix and impose manifold learning into non-negative matrix factorization. Moreover, we utilize the discrete hashing learning method to constrain the learned subspace, which can achieve a binary subspace from the original data. Experimental results validate the robustness and effectiveness of RSNMF in binary subspace learning and image clustering on the face dataset corrupted by Salt and Pepper noise and Contiguous Occlusion.

Discrete Two-Step Cross-Modal Hashing through the Exploitation of Pairwise Relations.

The cross-modal hashing method can map heterogeneous multimodal data into a compact binary code that preserves semantic similarity, which can significantly enhance the convenience of cross-modal retrieval. However, the currently available supervised cross-modal hashing methods generally only factorize the label matrix and do not fully exploit the supervised information. Furthermore, these methods often only use one-directional mapping, which results in an unstable hash learning process. To address these problems, we propose a new supervised cross-modal hash learning method called Discrete Two-step Cross-modal Hashing (DTCH) through the exploitation of pairwise relations. Specifically, this method fully exploits the pairwise similarity relations contained in the supervision information: for the label matrix, the hash learning process is stabilized by combining matrix factorization and label regression; for the pairwise similarity matrix, a semirelaxed and semidiscrete strategy is adopted to potentially reduce the cumulative quantization errors while improving the retrieval efficiency and accuracy. The approach further combines an exploration of fine-grained features in the objective function with a novel out-of-sample extension strategy to enable the implicit preservation of consistency between the different modal distributions of samples and the pairwise similarity relations. The superiority of our method was verified through extensive experiments using two widely used datasets.

Variable length deep cross-modal hashing based on Cauchy probability function

With the rapid development of multimedia technology, considerable achievement has been achieved in image retrieval technology. On the one hand, users’ demand for cross-modal retrieval is increasing rapidly. On the other hand, deep hashing algorithms as one of the most prominent high-dimensional reduction methods have received extensive attention. Under such background, the cross-modal retrieval method based on deep hashing came into being. Although the cross-modal hashing learning method has moved a long way in recent years, it still has space of promotion. First, existing cross-modal based hashing algorithms can only map the feature vectors into binary codes with a fixed length. The length of hash codes cannot be modified according to the practical situation. Second, the saturation level of the probability function used in existing methods is too high to concentrate relevant samples very well, which results in low retrieval efficiency. To solve the above problem, a novel variable-length hash code based on adaptive weight was proposed in this paper. The length of hash codes could be adjusted according to the importance of each bit. And a novel probability function based on Cauchy distribution was proposed to generate compact binary codes and make hashing retrieval more efficient. The experiment shows that the proposed cross-modal image retrieval algorithm based on deep hashing outperforms existing related algorithms on the accuracy of retrieval.

Deep Hash Learning for Remote Sensing Image Retrieval

The content-based remote sensing image retrieval (CBRSIR) has attracted increasing attention with the number of remote sensing (RS) images growing explosively. Benefiting from the strong capacity of the deep convolutional neural network (DCNN), the performance of CBRSIR has been improved in recent years. Although great successes have been obtained, learning the RS images' representative features and enhancing the retrieval efficiency for the large-scale CBRSIR tasks are still two challenging problems. In this article, we propose a new CBRSIR method named feature and hash (FAH) learning, which consists of a deep feature learning model (DFLM) and an adversarial hash learning model (AHLM). The DFLM aims at learning the RS images' dense features to guarantee the retrieval precision. In the DFLM, the DCNN and the proposed feature aggregation are integrated to capture the multiscale features. Then, the discrimination of the obtained features can be highlighted by the attention map in the developed attention branch. The AHLM maps the dense features onto the compact hash codes so that the retrieval efficiency can be improved. The AHLM contains a hash learning submodel and an adversarial regularization submodel. In particular, the hash learning submodel learns the real-valued hash codes that are similarity preserved by semantic supervisions. The adversarial regularization submodel regularizes the real-valued hash codes to learn the discrete uniform distribution with possible values 0 and 1. In this way, the hash codes are coding-balanced and the quantization errors are reduced. Encouraging experimental results counted on three public benchmark data sets demonstrate that our FAH can achieve competitive performance in the CBRSIR task compared with many existing hash learning methods.

Low-Rank Hypergraph Hashing for Large-Scale Remote Sensing Image Retrieval

As remote sensing (RS) images increase dramatically, the demand for remote sensing image retrieval (RSIR) is growing, and has received more and more attention. The characteristics of RS images, e.g., large volume, diversity and high complexity, make RSIR more challenging in terms of speed and accuracy. To reduce the retrieval complexity of RSIR, a hashing technique has been widely used for RSIR, mapping high-dimensional data into a low-dimensional Hamming space while preserving the similarity structure of data. In order to improve hashing performance, we propose a new hash learning method, named low-rank hypergraph hashing (LHH), to accomplish for the large-scale RSIR task. First, LHH employs a l2-1 norm to constrain the projection matrix to reduce the noise and redundancy among features. In addition, low-rankness is also imposed on the projection matrix to exploit its global structure. Second, LHH uses hypergraphs to capture the high-order relationship among data, and is very suitable to explore the complex structure of RS images. Finally, an iterative algorithm is developed to generate high-quality hash codes and efficiently solve the proposed optimization problem with a theoretical convergence guarantee. Extensive experiments are conducted on three RS image datasets and one natural image dataset that are publicly available. The experimental results demonstrate that the proposed LHH outperforms the existing hashing learning in RSIR tasks.

Large-Scale Remote Sensing Image Retrieval Based on Semi-Supervised Adversarial Hashing

Remote sensing image retrieval (RSIR), a superior content organization technique, plays an important role in the remote sensing (RS) community. With the number of RS images increases explosively, not only the retrieval precision but also the retrieval efficiency is emphasized in the large-scale RSIR scenario. Therefore, the approximate nearest neighborhood (ANN) search attracts the researchers’ attention increasingly. In this paper, we propose a new hash learning method, named semi-supervised deep adversarial hashing (SDAH), to accomplish the ANN for the large-scale RSIR task. The assumption of our model is that the RS images have been represented by the proper visual features. First, a residual auto-encoder (RAE) is developed to generate the class variable and hash code. Second, two multi-layer networks are constructed to regularize the obtained latent vectors using the prior distribution. These two modules mentioned are integrated under the generator adversarial framework. Through the minimax learning, the class variable would be a one-hot-like vector while the hash code would be the binary-like vector. Finally, a specific hashing function is formulated to enhance the quality of the generated hash code. The effectiveness of the hash codes learned by our SDAH model was proved by the positive experimental results counted on three public RS image archives. Compared with the existing hash learning methods, the proposed method reaches improved performance.

Neighborhood kinship preserving hashing for supervised learning

Most existing hashing methods rarely utilize the label information to learn the hashing function. However the label information of the training data is very important for classification. In this paper, we develop a new neighbor kinship preserving hashing based on a learned robust distance metric, which can pull the intra-class neighborhood samples as close as possible and push the inter-class neighborhood samples as far as possible, such that the discriminant information of the training data is incorporated into the learning framework. Furthermore, the discriminant information is inherited into the hamming space by the proposed neighbor kinship preserving hashing which can obtain highly similar binary representation for kinship neighbor pairs and highly different binary representation for non-kinship neighbor pairs. Moreover, the proposed priori intervention iterative optimization algorithm can better apply the learned discriminant information for classification and matching. Experimental results clearly demonstrate that our method achieves leading performance compared with the state-of-the-art supervised hashing learning methods.