Compact Binary Codes Research Articles

SemanticHash: Hash Coding Via Semantics-Guided Label Prototype Learning

In this article, we propose SemanticHash, a simple and effective deep neural network model, to leverage semantic word embeddings (e.g., BERT) in hash codes learning. Both images and class labels are compressed into <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -bit binary vectors by using the visual (or the semantic) hash functions, which are jointly learned and aligned to optimize the semantic consistency. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -dimensional class label prototypes—projected from semantic word embeddings—guide the hash mapping on the image side and vice versa, creating the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -bit image hash codes being aligned with their semantic prototypes and therefore more discriminative. Extensive experimental results on four benchmarks, CIFAR10, NUS-WIDE, ImageNet, and MS-COCO datasets, demonstrate the effectiveness of our approach. We also perform studies to analyze the effects of quantization and word semantic spaces and to explain the relations among the learned class prototypes. Finally, the generalization capability of the proposed approach is further demonstrated. It achieves competitive performance in comparison with state-of-the-art unsupervised and zero-shot hashing methods. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement</i>—Hash code learning is an important technology that enables efficient image retrieval on large-scale data. While existing hashing algorithms can effectively generate compact binary codes in a supervised learning setting trained with a moderate-size dataset, they are demanding to be scalable to large datasets and do not generalize to unseen datasets. The proposed approach overcomes these limitations. Compared with state-of-the-art ones, our solution achieves 2.1% of average performance improvement on four moderate-size benchmarks and 4.7% of improvement on ImageNet, a large-scale dataset with over 1.2 M training images. With superior performance on popular benchmarks for binary hash code learning, the technology introduced performs well on cross-dataset and zero-shot (i.e., the testing concepts are unseen during training) scenarios too. Our approach attains over 17.7% of zero-shot retrieval performance improvement when compared to the state-of-the-art in the area. This article thus provides a powerful solution to utilize massive data for fast and accurate image retrieval in the big data era.

Similarity-preserving hash for content-based audio retrieval using unsupervised deep neural networks

Due to its efficiency in storage and search speed, binary hashing has become an attractive approach for a large audio database search. However, most existing hashing-based methods focus on data-independent scheme where random linear projections or some arithmetic expression are used to construct hash functions. Hence, the binary codes do not preserve the similarity and may degrade the search performance. In this paper, an unsupervised similarity-preserving hashing method for content-based audio retrieval is proposed. Different from data-independent hashing methods, we develop a deep network to learn compact binary codes from multiple hierarchical layers of nonlinear and linear transformations such that the similarity between samples is preserved. The independence and balance properties are included and optimized in the objective function to improve the codes. Experimental results on the Extended Ballroom dataset with 8 genres of 3,000 musical excerpts show that our proposed method significantly outperforms state-of-the-art data-independent method in both effectiveness and efficiency.

Compact and Cancelable Fingerprint Binary Codes Generation via One Permutation Hashing

Representing fingerprint templates in binary form can provide outstanding merits compared to the conventional minutiae-based fingerprint recognition system. The existing fixed-length fingerprint feature extraction methods either suffer from redundant feature magnitude or lack of template security. In this letter, we present a compact (128 bytes) and cancelable fingerprint binary codes generation scheme which enables accurate and efficient comparison as well as high security. This binary representation is also available for advanced encryption schemes (e.g., fuzzy commitment). Specifically, a kernel learning-based real-valued fingerprint feature is converted into compact and cancelable binary code via one permutation hashing. A partial Haar transform is deployed to further strengthen the irreversibility of the whole system. Experimental results on six benchmark datasets FVC2002 and FVC2004 coupled with security analysis demonstrate the superiority of the proposed method compared with several state-of-the-arts.

Deep Semantic Hashing Using Pairwise Labels

Data hashing has been widely used to approximate large-scale similarity searches. Original text data can be represented using compact binary codes through hashing. Recent advances in neural network architecture have demonstrated the effectiveness of this method and its ability to learn hash functions more accurately. Most previous studies have been focused on encoding explicit supervised features, such as pointwise labels. Owing to the special nature of textual data, previous semantic text hashing approaches have only utilized pointwise label information. The purpose of the learning hash code developed in the present study is to make similar or related text have similar hash codes. Separate label learning for each datum is the easiest means of achieving this objective, but some inconsistencies remain. However, pairwise label information reflects the similarity more intuitively than pointwise label data. This paper proposes a supervised semantic text hashing method that utilizes pairwise label information. Several different methods based on the variational auto-encoder model are employed to calculate the pairwise similarity of text pairs. Because the similarity calculation process does not require additional parameters, the entire learning process is faster and more efficient than those in the existing methods. The experimental results obtained using public datasets show that the proposed method can exploit pairwise label information sufficiently well to outperform previous state-of-the-art hashing approaches. This report also describes variants involving different technique combinations, presents analyses of the efficiencies of these approaches, and discusses methods of improving their efficiencies.

Discrete Two-Step Cross-Modal Hashing through the Exploitation of Pairwise Relations.

The cross-modal hashing method can map heterogeneous multimodal data into a compact binary code that preserves semantic similarity, which can significantly enhance the convenience of cross-modal retrieval. However, the currently available supervised cross-modal hashing methods generally only factorize the label matrix and do not fully exploit the supervised information. Furthermore, these methods often only use one-directional mapping, which results in an unstable hash learning process. To address these problems, we propose a new supervised cross-modal hash learning method called Discrete Two-step Cross-modal Hashing (DTCH) through the exploitation of pairwise relations. Specifically, this method fully exploits the pairwise similarity relations contained in the supervision information: for the label matrix, the hash learning process is stabilized by combining matrix factorization and label regression; for the pairwise similarity matrix, a semirelaxed and semidiscrete strategy is adopted to potentially reduce the cumulative quantization errors while improving the retrieval efficiency and accuracy. The approach further combines an exploration of fine-grained features in the objective function with a novel out-of-sample extension strategy to enable the implicit preservation of consistency between the different modal distributions of samples and the pairwise similarity relations. The superiority of our method was verified through extensive experiments using two widely used datasets.

Deep Unsupervised Hashing for Large-Scale Cross-Modal Retrieval Using Knowledge Distillation Model.

Cross-modal hashing encodes heterogeneous multimedia data into compact binary code to achieve fast and flexible retrieval across different modalities. Due to its low storage cost and high retrieval efficiency, it has received widespread attention. Supervised deep hashing significantly improves search performance and usually yields more accurate results, but requires a lot of manual annotation of the data. In contrast, unsupervised deep hashing is difficult to achieve satisfactory performance due to the lack of reliable supervisory information. To solve this problem, inspired by knowledge distillation, we propose a novel unsupervised knowledge distillation cross-modal hashing method based on semantic alignment (SAKDH), which can reconstruct the similarity matrix using the hidden correlation information of the pretrained unsupervised teacher model, and the reconstructed similarity matrix can be used to guide the supervised student model. Specifically, firstly, the teacher model adopted an unsupervised semantic alignment hashing method, which can construct a modal fusion similarity matrix. Secondly, under the supervision of teacher model distillation information, the student model can generate more discriminative hash codes. Experimental results on two extensive benchmark datasets (MIRFLICKR-25K and NUS-WIDE) show that compared to several representative unsupervised cross-modal hashing methods, the mean average precision (MAP) of our proposed method has achieved a significant improvement. It fully reflects its effectiveness in large-scale cross-modal data retrieval.

Fast Unmediated Hashing for Cross-Modal Retrieval

Cross-modal hashing is for the purpose of compressing heterogeneous multi-modal data into compact binary codes for the cross-modal retrieval, where accuracy and efficiency are two primary issues. To achieve high accuracy and efficiency, we put forward a novel method named Fast Unmediated Hashing (FUH) for cross-modal retrieval. For this method, motivated by the fact that label vector is a natural binary representation of samples for retrieval, we directly learn the cross-modal hash codes from semantic labels without any intermediate representation. This will capture more relations among different modalities, and reduce the number of variables. However, directly learning hash codes from labels would weaken the discrimination of hash codes. To address this issue, double supervision involving label information and pairwise similarity is proposed to enhance the discrimination. In addition, to decrease the training time, we present a strategy to bypass the similarity matrix-related operation in each iteration of optimization, thus some other related terms can also be computed offline to lower training complexity. Compared to several state-of-the-art techniques on three public datasets, the experimental results have manifested the superiority of FUH concerning efficiency and accuracy.

Adversarial Multi-Label Variational Hashing.

In this paper, we propose an adversarial multi-label variational hashing (AMVH) method to learn compact binary codes for efficient image retrieval. Unlike most existing deep hashing methods which only learn binary codes from specific real samples, our AMVH learns hash functions from both synthetic and real data which make our model effective for unseen data. Specifically, we design an end-to-end deep hashing framework which consists of a generator network and a discriminator-hashing network by enforcing simultaneous adversarial learning and discriminative binary codes learning to learn compact binary codes. The discriminator-hashing network learns binary codes by optimizing a multi-label discriminative criterion and minimizing the quantization loss between binary codes and real-value codes. The generator network is learned so that latent representations can be sampled in a probabilistic manner and used to generate new synthetic training sample for the discriminator-hashing network. Experimental results on several benchmark datasets show the efficacy of the proposed approach.

Variable length deep cross-modal hashing based on Cauchy probability function

With the rapid development of multimedia technology, considerable achievement has been achieved in image retrieval technology. On the one hand, users’ demand for cross-modal retrieval is increasing rapidly. On the other hand, deep hashing algorithms as one of the most prominent high-dimensional reduction methods have received extensive attention. Under such background, the cross-modal retrieval method based on deep hashing came into being. Although the cross-modal hashing learning method has moved a long way in recent years, it still has space of promotion. First, existing cross-modal based hashing algorithms can only map the feature vectors into binary codes with a fixed length. The length of hash codes cannot be modified according to the practical situation. Second, the saturation level of the probability function used in existing methods is too high to concentrate relevant samples very well, which results in low retrieval efficiency. To solve the above problem, a novel variable-length hash code based on adaptive weight was proposed in this paper. The length of hash codes could be adjusted according to the importance of each bit. And a novel probability function based on Cauchy distribution was proposed to generate compact binary codes and make hashing retrieval more efficient. The experiment shows that the proposed cross-modal image retrieval algorithm based on deep hashing outperforms existing related algorithms on the accuracy of retrieval.

Deep video code for efficient face video retrieval

In this paper, we address one specific video retrieval problem in terms of human face. Given one query in forms of either a frame or a sequence from a person, we search the database and return the most relevant face videos, i.e., ones have the same class label with the query. Such problem is very challenging due to the large intra-class variations and the high request on the efficiency of video representations in terms of both time and space. To handle such challenges, this paper proposes a novel Deep Video Code (DVC) method which encodes video faces into compact binary codes. Specifically, we devise an end-to-end convolutional neural network (CNN) framework that takes face videos as training inputs, models each of them as a unified representation by temporal feature pooling operation, and finally projects the high-dimensional representations of both frames and videos into Hamming space to generate binary codes. In such Hamming space, distance of dissimilar pairs is larger than that of similar pairs by a margin. To this end, a novel bounded triplet hashing loss is elaborately designed, which takes all dissimilar pairs into consideration for each anchor point in a mini-batch, and the optimization of the loss function is smoother and more stable. Extensive experiments on challenging video face databases and general image/video datasets with comparison to the state-of-the-arts verify the effectiveness of our method in different kinds of retrieval scenarios.

Unsupervised Deep Multi-Similarity Hashing With Semantic Structure for Image Retrieval

With the advance of Convolutional Neural Network, deep hashing methods have shown the great promising performance in large-scale image retrieval. Without depending on extensive human-annotated data, unsupervised hashing is more applicable to image retrieval tasks compared to supervised methods. However, due to the lack of fine-grained supervised signals and multi-similarity constraints, most state-of-the-art unsupervised deep hashing algorithms cannot ensure the correct fine-grained similarity ranking for image pairs. In this paper, we propose a novel unsupervised deep multi-similarity hashing framework to learn compact binary codes by jointly exploiting global-aware and spatial-aware representations, called Unsupervised Deep Multi-Similarity Hashing with Semantic Structure (UDMSH). Specifically, to obtain distinguishing characteristics, we develop a sub-network by jointly learning global semantic structures from Convolutional Neural Network (CNN) and inherent spatial structures from Fully Convolutional Network (FCN). By computing the cosine distance for deep features from image pairs, we construct a similarity matrix with semantic structure, then utilize this matrix to guide hash code learning process. Based on it, we carefully design a multi-level pairwise loss to preserve the correct fine-grained similarity ranking. Furthermore, we introduce Hamming-isometric mapping into unsupervised hashing framework to decrease the quantization errors. Extensive experiments on three widely used benchmarks prove that our proposed UDMSH outperforms several state-of-the-art unsupervised hashing with respect to different evaluation metrics.

Collaborative Generative Hashing for Marketing and Fast Cold-Start Recommendation

Cold-start has being a critical issue in recommender systems with the explosion of data in e-commerce. Most existing studies proposed to alleviate the cold-start problem are also known as hybrid recommender systems that learn representations of users and items by combining user-item interactive and user/item content information. However, previous hybrid methods regularly suffered poor efficiency bottlenecking in online recommendations with large-scale items, because they were designed to project users and items into continuous latent space where the online recommendation is expensive. To this end, we propose a collaborative generated hashing (CGH) framework to improve the efficiency by denoting users and items as binary codes, then fast hashing search techniques can be used to speed up the online recommendation. In addition, the proposed CGH can generate potential users or items for marketing application where the generative network is designed with the principle of minimum description length, which is used to learn compact and informative binary codes. Extensive experiments on two public datasets show the advantages for recommendations in various settings over competing baselines and analyze its feasibility in marketing application.

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Cross-modal hashing facilitates mapping of heterogeneous multimedia data into a common Hamming space, which can be utilized for fast and flexible retrieval across different modalities. In this paper, we propose a novel cross-modal hashing architecture-deep neural decoder cross-modal hashing (DNDCMH), which uses a binary vector specifying the presence of certain facial attributes as an input query to retrieve relevant face images from a database. The DNDCMH network consists of two separate components: an attribute-based deep cross-modal hashing (ADCMH) module, which uses a margin (m)-based loss function to efficiently learn compact binary codes to preserve similarity between modalities in the Hamming space, and a neural error correcting decoder (NECD), which is an error correcting decoder implemented with a neural network. The goal of NECD network in DNDCMH is to error correct the hash codes generated by ADCMH to improve the retrieval efficiency. The NECD network is trained such that it has an error correcting capability greater than or equal to the margin (m) of the margin-based loss function. This results in NECD can correct the corrupted hash codes generated by ADCMH up to the Hamming distance of m. We have evaluated and compared DNDCMH with state-of-the-art cross-modal hashing methods on standard datasets to demonstrate the superiority of our method.

Learning to Hash for Personalized Image Authentication

This paper takes a fresh look at the image authentication problem and proposes an alternative framework for personalized authentication based on the hash learning technology. Conventional image authentication methods tend to provide a general authentication framework for all images with the fixed quantization strategy and fixed control parameters determined based on given limited images and attacks. However, they may suffer from more or less misjudgments in practice, not to mention performance degradation when encountering out-of-sample images. Instead of proposing a new feature extraction algorithm, a novel personalized authentication framework which incorporates the distance metric learning technology and supervised quantization strategy to the process of image authentication is proposed in this paper. The tamper detection task is reformulated as a new supervised manipulation classification problem. For each input image, various content-preserving and content-changing samples are generated automatically firstly. Then, feature representations of all samples can be obtained by existing feature extraction methods. After that, a weighted large margin for manipulation classification (WLMMC) scheme is proposed to learn an effective feature mapping space to improve the classification performance between content-changing samples and content-preserving samples. During the quantization stage, a novel supervised personalized quantization strategy (SPQ), which is motivated by the observation that different attacks have different degrees of influence on feature components, is proposed to learn more compact yet discriminative binary codes for each input image. Effectiveness of the proposed framework is qualitatively and quantitatively demonstrated on a variety of images. Extensive experiments show that the proposed framework can significantly improve the authentication performance over the state-of-the-art techniques while achieve more compact hash codes flexibly as required.

Scalable deep asymmetric hashing via unequal-dimensional embeddings for image similarity search

In recent years, Hashing has become a popular technique used to support large-scale image retrieval, due to its significantly reduced storage, high search speed and capability of mapping high dimensional original features into compact similarity-preserving binary codes. Although effectiveness achieved, most existing hashing methods are still some limitations, including: (1) Many supervised hashing methods only transform the label information into pairwise similarities to guide the hash code learning process, which will lead to the loss of rich semantic information between image pairs. (2) Some pioneer hashing methods use a relaxation-based strategy to solve discrete problems, resulting in a large quantization error. (3) Some supervised hashing methods handle the hashing learning procedure based on an asymmetric learning manner, and although this partially solves the problems of low efficiency and accuracy of symmetric learning strategy, they are all based on the embeddings of equal dimension, which leads to the reduction in the models representation ability and an increase in potential noise. To overcome the above limitations, in this paper, we propose a novel yet simple but effective hashing method, named Scalable Deep Asymmetric Hashing (SDAH). Specifically, SDAH is an end-to-end deep hashing method based on a fast iterative optimization strategy, which utilizes two real-valued embeddings of unequal dimensions, i.e., real-valued embeddings of images and labels, to flexibly perform asymmetric similarity computation. It can circumvent the use of the large pairwise similarity matrix by introducing an intermediate label matrix term which results in a remarkable reduction in the memory space cost. By doing this, the learned hash codes are more semantically informative for image retrieval tasks. Experimental results on several benchmark datasets highlight the superiority of SDAH in comparison with many state-of-the-art hashing methods.

3D palmprint recognition using complete block wise descriptor

Recent years have witnessed a growing interesting in developing automatic 3D palmprint recognition systems. The key problem of 3D palmprint recognition is to contrive an effective representation. In this paper, we propose a 3D palmprint recognition method by completely fusing of multiple block wise features. First, we propose a shape index multi-direction binary code strategy to extract 2D texture-level features of 3D palmprint. Then, we use the compact surface type binary code scheme to characterize 3D structure-level features of 3D palmprint. Finally, we integrate them together to construct the complete block wise descriptor for 3D palmprint recognition. Experiments conducted on the public available 3D palmprint database validate the effectiveness of our proposed method.

Proximity Preserving Binary Code Using Signed Graph-Cut

We introduce a binary embedding framework, called Proximity Preserving Code (PPC), which learns similarity and dissimilarity between data points to create a compact and affinity-preserving binary code. This code can be used to apply fast and memory-efficient approximation to nearest-neighbor searches. Our framework is flexible, enabling different proximity definitions between data points. In contrast to previous methods that extract binary codes based on unsigned graph partitioning, our system models the attractive and repulsive forces in the data by incorporating positive and negative graph weights. The proposed framework is shown to boil down to finding the minimal cut of a signed graph, a problem known to be NP-hard. We offer an efficient approximation and achieve superior results by constructing the code bit after bit. We show that the proposed approximation is superior to the commonly used spectral methods with respect to both accuracy and complexity. Thus, it is useful for many other problems that can be translated into signed graph cut.

A supervised deep convolutional based bidirectional long short term memory video hashing for large scale video retrieval applications

Recently, large scale video content retrieval has gained more attention due to a large amount of user-generated images and video content available over the internet. Hashing is one of the effective techniques that encode the high dimensional features vectors into compact binary codes. The aim of hashing is to generate the short binary codes and map the similar hash code values to retrieve similar video from the database with minimum distance measure. Deep learning-based hashing networks are employed to learn the representative video feature to estimate the hash functions. However, the existing hashing approaches fail due to the frame-level feature representation and does not well exploits the effective temporal features in visual search. Furthermore, significant loss of features during the dimensionality reduction step causes low accuracy. Hence, it is essential to develop a deep learning-based hashing framework that should exploits both the strong spatial and temporal features for the scalable video search. The main objective is to learn the high dimensional features from the entire video and derive compact binary codes to retrieve the similar videos for the input query sequence. In this paper, we propose a joint network model of supervised stacked heterogeneous convolutional multi-kernel (Stacked HetConv-MK)-bidirectional Long Short Term Memory (BiDLSTM) network model that effectively encodes the rich structural as well as the discriminative features from the video sequence to estimate the compact binary codes. Initially, the video frames are passed to the stacked convolution networks with heterogeneous convolutional kernel size and residual learning to extract the spatial features at different views from the video sequences and to improve the learning efficiency. Then, the bidirectional network computes the sequence in both forward and backward directions and obtains the series of hidden state output. Finally, the fully connected structure with an activation unit performs hashing to learn the multiple codes for each video. Experimental analysis is performed on three datasets and the result shows a better accuracy measure than the other state-of-art approaches.

Binary multi-view perceptual hashing for image authentication

This paper presents a novel Binary Multi-View Perceptual Hashing (BMVPH) scheme for image authentication, which provides compact and efficient representations and can easily scale to large data. We apply virtual prior attacks (e.g. additive noise, blurring, compression, logo-insert etc.) on original images to generate simulated distorted copies. The original images and the corresponding distorted copies provide the so-called training set. For perceptual hashing learning, we formulate BMVPH by two key components: collaborative binary representation learning (CBRL) and perpetual content authentication learning (PCAL), into a unified learning framework. Our BMVPH scheme collaboratively encodes the multi-view features into a compact common binary code space while considering the perceptual content similarity at the same time. The experimental results show that when compared with the state-of-the-art methods, the proposed algorithm can achieve higher discrimination and better perceptual robustness. In particular, the Area Under ROC Curve (AUC) increases on average of 3.8% as compared with other state-of-the-art methods.

Deep Semantic Reconstruction Hashing for Similarity Retrieval

Hashing has shown enormous potentials in preserving semantic similarity for large-scale data retrieval. Existing methods widely retain the similarity within two binary codes towards their discrete semantic affinity, i.e., 1 or -1. However, such a discrete reconstruction approach has obvious drawbacks. First, two unrelated dissimilar samples would have similar binary codes when both of them are the most dissimilar with an anchor sample. Second, the fine-grained semantic similarity cannot be shown in the generated binary codes among data with multiple semantic concepts. Furthermore, existing approaches generally adopt a point-wise error-minimizing strategy to enforce the real-valued codes close to its associated discrete codes, resulting in the well-learned paired semantic similarity being unintentionally damaged when performing quantization. To address these issues, we propose a novel deep hashing method with pairwise similarity-preserving quantization constraint, termed Deep Semantic Reconstruction Hashing (DSRH), which defines a high-level semantic affinity within each data pair to learn compact binary codes. Specifically, DSRH is expected to learn the specific binary codes whose similarity can reconstruct their high-level semantic similarity. Besides, we adopt a pairwise similarity-preserving quantization constraint instead of the traditional point-wise quantization technique, which is conducive to maintain the well-learned paired semantic similarity when performing quantization. Extensive experiments are conducted on four representative image retrieval benchmarks, and the proposed DSRH outperforms the state-of-the-art deep-learning methods with respect to different evaluation metrics.