Binary Code Learning Research Articles

A Scalable Optimization Mechanism for Pairwise Based Discrete Hashing.

Maintaining the pairwise relationship among originally high-dimensional data into a low-dimensional binary space is a popular strategy to learn binary codes. One simple and intuitive method is to utilize two identical code matrices produced by hash functions to approximate a pairwise real label matrix. However, the resulting quartic problem in term of hash functions is difficult to directly solve due to the non-convex and non-smooth nature of the objective. In this paper, unlike previous optimization methods using various relaxation strategies, we aim to directly solve the original quartic problem using a novel alternative optimization mechanism to linearize the quartic problem by introducing a linear regression model. Additionally, we find that gradually learning each batch of binary codes in a sequential mode, i.e. batch by batch, is greatly beneficial to the convergence of binary code learning. Based on this significant discovery and the proposed strategy, we introduce a scalable symmetric discrete hashing algorithm that gradually and smoothly updates each batch of binary codes. To further improve the smoothness, we also propose a greedy symmetric discrete hashing algorithm to update each bit of batch binary codes. Moreover, we extend the proposed optimization mechanism to solve the non-convex optimization problems for binary code learning in many other pairwise based hashing algorithms. Extensive experiments on benchmark single-label and multi-label databases demonstrate the superior performance of the proposed mechanism over recent state-of-the-art methods on two kinds of retrieval tasks: similarity and ranking order. The source codes are available on https://github.com/xsshi2015/Scalable-Pairwise-based-Discrete-Hashing.

Graph-based Multi-view Binary Learning for image clustering

Hashing techniques, also known as binary code learning, have recently attracted increasing attention in large-scale data analysis and storage. Generally, most existing hash clustering methods are single-view ones, which lack complete structure or complementary information from multiple views. For clustering tasks, most hashing research mainly mapped the original data into the Hamming space while heavily ignoring the original feature space structure. To solve these problems, we propose a novel binary code algorithm for clustering, which adopts graph embedding to preserve the original data structure, called Graph-based Multi-view Binary Learning (GMBL). The binary code learning combines the graph structure of original data and the complementary information of different views for clustering. Specifically, GMBL preserves the local linear relationship utilizing the Laplacian matrix aim to maintain the graph-based structure of the original data in Hamming space. Moreover, by automatically assigning weights for each view to improve clustering performance, which takes distinctive contributions of multiple views into considerations. Besides, an alternating iterative optimization method is designed to solve the resulting optimization problems. Extensive experimental results on five public datasets are provided to reveal the effectiveness of the algorithm and its superior performance over other state-of-the-art alternatives.

Inductive Structure Consistent Hashing via Flexible Semantic Calibration.

Semantic-preserving hashing establishes efficient multimedia retrieval by transferring knowledge from original data to hash codes so that the latter can preserve the underlying visual and semantic similarities. However, it becomes a crucial bottleneck: how to effectively bridge the trilateral domain gaps (i.e., the visual, semantic, and hashing spaces) to further improve the retrieval accuracy. In this article, we propose an inductive structure consistent hashing (ISCH) method, which can interactively coordinate the semantic correlations between the visual feature space, the binary class space, and the discrete hashing space. Specifically, an inductive semantic space is formulated by a simple multilayer stacking class-encoder, which transforms the naive class information into flexible semantic embeddings. Meanwhile, we design a semantic dictionary learning model to facilitate the bilateral visual-semantic bridging and guide the class-encoder toward reliable semantics, which could well alleviate the visual-semantic bias problem. In particular, the visual descriptors and respective semantic class representations are regularized with a coinciding alignment module. In order to generate privileged hash codes, we further explore semantic and prototype binary code learning to jointly quantify the semantic and latent visual representations into unified discrete hash codes. Moreover, an efficient optimization algorithm is developed to address the resulting discrete programming problem. Comprehensive experiments conducted on four large-scale data sets, i.e., CIFAR-10, NUSWIDE, ImageNet, and MSCOCO, demonstrate the superiority of our method over the state-of-the-art alternatives against different evaluation protocols.

Discriminative dual-stream deep hashing for large-scale image retrieval

Deep hashing has been an important research topic for using deep learning to boost performance of hash learning. Most existing deep supervised hashing methods mainly focus on how to effectively preserve the similarity in hash coding solely depending on pairwise supervision. However, such pairwise similarity-preserving strategy cannot fully explore the semantic information in most cases, which results in information loss. To address this problem, this paper proposes a discriminative dual-stream deep hashing (DDDH) method, which integrates the pairwise similarity loss and the classification loss into a unified framework to take full advantage of label information. Specifically, the pairwise similarity loss aims to preserve the similarity and structural information of high-dimensional original data. Meanwhile, the designed classification loss can enlarge the margin between different classes which improves the discrimination of learned binary codes. Moreover, an effective optimization algorithm is employed to train the hash code learning framework in an end-to-end manner. The results of extensive experiments on three image datasets demonstrate that our method is superior to several state-of-the-art deep and non-deep hashing methods. Ablation studies and analysis further show the effectiveness of introducing the classification loss in the overall hash learning framework.

Deep balanced discrete hashing for image retrieval

Hashing has been widely used for large-scale multimedia retrieval because of its advantages in storage and retrieval efficiency. Traditional supervised hash methods represent an image as a feature vector and then perform a separate quantization step to generate a binary code. Due to the difficulty of discrete optimization of hash codes, continuous relaxation is generally used to replace discrete optimization. However, the process of continuous relaxation leads to inevitable quantization error. To avoid this drawback, a deep balanced discrete hashing method is proposed, which uses discrete gradient propagation with the straight-through estimator. The proposed method does not use the traditional continuous relaxation strategy, thereby reducing the quantization error caused by continuous relaxation. And the proposed method uses supervised information to directly guide the discrete coding and deep feature learning process. In the proposed method, the last layer of the Convolutional Neural Network (CNN) outputs the binary code directly. In the loss function, discrete values are calculated by combining the pairwise loss and a balance controlling term. The learned binary hash code maintains the similar relationship and label consistency at the same time. While maintaining the pairwise similarity, the proposed method keeps the balance of hash codes to improve retrieval performance. Extensive experiments show that the proposed method outperforms the state-of-the-art hashing methods on four image retrieval benchmark datasets.

A General Framework for Deep Supervised Discrete Hashing

With the rapid growth of image and video data on the web, hashing has been extensively studied for image or video search in recent years. Benefiting from recent advances in deep learning, deep hashing methods have shown superior performance over the traditional hashing methods. However, there are some limitations of previous deep hashing methods (e.g., the semantic information is not fully exploited). In this paper, we develop a general deep supervised discrete hashing framework based on the assumption that the learned binary codes should be ideal for classification. Both the similarity information and the classification information are used to learn the hash codes within one stream framework. We constrain the outputs of the last layer to be binary codes directly, which is rarely investigated in deep hashing algorithms. Besides, both the pairwise similarity information and the triplet ranking information are exploited in this paper. In addition, two different loss functions are presented: $${l_2}$$ loss and hinge loss, which are carefully designed for the classification term under the one stream framework. Because of the discrete nature of hash codes, an alternating minimization method is used to optimize the objective function. Experimental results have shown that our approach outperforms current state-of-the-art methods on benchmark datasets.

Supervised deep semantics-preserving hashing for real-time pulmonary nodule image retrieval

Hashing-based medical image retrieval has drawn extensive attention recently, which aims at providing effective aided diagnosis for medical personnel. In the paper, a novel deep hashing framework is proposed in the medical image retrieval, where the processes of deep feature extraction, binary code learning, and deep hash function learning are jointly carried out in supervised fashion. Particularly, the discrete constrained objective function in the hash code learning is optimized iteratively, where the binary code can be directly solved with no need for relaxation. In the meantime, the semantic similarity is maintained by fully exploring supervision information during the discrete optimization, where the neighborhood structure of training data is preserved by applying a graph regularization term. Additionally, to gain the fine-grained ranking of the returned medical images sharing the same Hamming distance, a novel image re-ranking scheme is proposed to refine the similarity measurement by jointly considering Euclidean distance between the real-valued feature descriptors and their category information between those images. Extensive experiments on the pulmonary nodule image dataset demonstrate that the proposed method can achieve better retrieval performance over the state of the arts.

Unsupervised deep hashing by joint optimization for pulmonary nodule image retrieval

Abstract In recent years, hash-based image retrieval has attracted great attention due to the rapid growth of medical images. In the paper, an end-to-end unsupervised deep hashing is proposed, where feature extraction and binary optimization are carried out by joint optimization. Our method consists of five components: a shared deep convolution neural network for learning image representations, a deconvolution module for reconstructing the original images, a classification module for leveraging semantic supervision by pseudo labels, a binary code learning module for encoding images features into binary codes, and a joint loss function for deep hash function learning. In addition, the real-valued features balanced in different dimensions by a rotation matrix are quantized directly into discrete binary codes in an alternating optimization approach to minimize the quantization loss. Experiments have been performed on the pulmonary nodule images dataset and the results demonstrate the proposed method can yield better retrieval performance by comparing with the state-of-the-art unsupervised hashing methods.

Unified Binary Generative Adversarial Network for Image Retrieval and Compression

Binary codes have often been deployed to facilitate large-scale retrieval tasks, but not that often for image compression. In this paper, we propose a unified framework, BGAN+, that restricts the input noise variable of generative adversarial networks to be binary and conditioned on the features of each input image, and simultaneously learns two binary representations per image: one for image retrieval and the other serving as image compression. Compared to related methods that attempt to learn a single binary code serving both purposes, we demonstrate that choosing for two codes leads to more effective representations due to less concessions needed when balancing the requirements. The added value of using a unified framework compared to two separate frameworks lies in the synergy in data representation that is beneficial for both learning processes. When devising this framework, we also address another challenge in learning binary codes, namely that of learning supervision. While the most striking successes in image retrieval using binary codes have mostly involved discriminative models requiring labels, the proposed BGAN+ framework learns the binary codes in an unsupervised fashion, yet more effectively than the state-of-the-art supervised approaches. The proposed BGAN+ framework is evaluated on three benchmark datasets for image retrieval and two datasets on image compression. The experimental results show that BGAN+ outperforms the existing retrieval methods with significant margins and achieves promising performance for image compression, especially for low bit rates.

Deep learned compact binary descriptor with a lightweight network-in-network architecture for visual description

Binary descriptors have been widely used for real-time image retrieval and correspondence matching. However, most of the learned descriptors are obtained using a large deep neural network (DNN) with several million parameters, and the learned binary codes are generally not invariant to many geometrical variances which is crucial for accurate correspondence matching. To address this problem, we proposed a new learning approach using a lightweight DNN architecture via a stack of multiple multilayer perceptrons based on the network in network (NIN) architecture, and a restricted Boltzmann machine (RBM). The latter is used for mapping the features to binary codes, and carry out the geometrically invariant correspondence matching task. Our experimental results on several benchmark datasets (e.g., Brown, Oxford, Paris, INRIA Holidays, RomePatches, HPatches, and CIFAR-10) show that the proposed approach produces the learned binary descriptor that outperforms other baseline self-supervised binary descriptors in terms of correspondence matching despite the smaller size of its DNN. Most importantly, the proposed approach does not freeze the features that are obtained while pre-training the NIN model. Instead, it fine-tunes the features while learning the features needed for binary mapping through the RBM. Additionally, its lightweight architecture makes it suitable for resource-constrained devices.

On Aggregation of Unsupervised Deep Binary Descriptor with Weak Bits.

Despite the thrilling success achieved by existing binary descriptors, most of them are still in the mire of three limitations: 1) vulnerable to the geometric transformations; 2) incapable of preserving the manifold structure when learning binary codes; 3) NO guarantee to find the true match if multiple candidates happen to have the same Hamming distance to a given query. All these together make the binary descriptor less effective, given large-scale visual recognition tasks. In this paper, we propose a novel learning-based feature descriptor, namely Unsupervised Deep Binary Descriptor (UDBD), which learns transformation invariant binary descriptors via projecting the original data and their transformed sets into a joint binary space. Moreover, we involve a ℓ2,1-norm loss term in the binary embedding process to gain simultaneously the robustness against data noises and less probability of mistakenly flipping bits of the binary descriptor, on top of it, a graph constraint is used to preserve the original manifold structure in the binary space. Furthermore, a weak bit mechanism is adopted to find the real match from candidates sharing the same minimum Hamming distance, thus enhancing matching performance. Extensive experimental results on public datasets show the superiority of UDBD in terms of matching and retrieval accuracy over state-of-the-arts.

Online Adaptive Supervised Hashing for Large-Scale Cross-Modal Retrieval

In recent years, with the continuous growth of multimedia data on the Internet, multimodal hashing has attracted increasing attention for its efficiency in large-scale cross-modal retrieval. Typically, most existing multimodal hashing methods are batch-based methods that cannot deal with the growing streaming data. Online multimodal hashing adopts online learning strategy to learn hash models incrementally, which can process large-scale streaming data. However, existing supervised online multimodal hashing methods still suffer from several limitations: (1) most methods cannot update hash codes of old data when hash model changes, which will hinder the retrieval accuracy. (2) the discrete optimizations of most methods for learning binary hash codes are less efficient or less effective. To address the above limitations, an efficient supervised online multimodal hashing method termed Online Adaptive Supervised Hashing (OASH) is proposed in this paper. OASH regresses class labels and training data to binary hash codes to learn discrete hash codes and hash functions with an efficient online optimization scheme. It learns hash functions incrementally with newly arriving data and updates hash codes with the latest hash model. By adaptively updating hash functions and hash codes with new data rather than accessing to old data, it gains much performance enhancement and computation savings. Extensive experiments on three benchmark data sets validate the superiority of OASH over state-of-the-art methods in both accuracy and efficiency.

Binary Codes Based on Non-Negative Matrix Factorization for Clustering and Retrieval

Traditional non-negative matrix factorization methods cannot learn the subspace from the high-dimensional data space composed of binary codes. One hopes to discover a compact parts-based representation composed of binary codes, which can uncover the intrinsic information and simultaneously respect the geometric structure of the original data. For this purpose, we introduce discrete hashing methods and propose a novel non-negative matrix factorization to generate binary codes from the original data. In this paper, we construct an affinity graph to encode the geometrical structure of the original data, and the learned binary code subspace achieved by matrix factorization respects the structure. The proposed problem can be formulated as a mixed integer optimization problem. Therefore, we transform it into several sub-problems including an integer optimization problem, two convex problems with the non-negative constraint and a quadratic programming problem. Optimizing each sub-problem alternately until we achieve a local optimal solution. Image clustering and retrieval on image datasets show the excellent performance of our method in comparison to other dimensional reduction methods.

Similarity-Preserving Linkage Hashing for Online Image Retrieval.

Online image hashing aims to update hash functions on-the-fly along with newly arriving data streams, which has found broad applications in computer vision and beyond. To this end, most existing methods update hash functions simply using discrete labels or pairwise similarity to explore intra-class relationships, which, however, often deteriorates search performance when facing a domain gap or semantic shift. One reason is that they ignore the particular semantic relationships among different classes, which should be taken into account in updating hash functions. Besides, the common characteristics between the label vectors (can be regarded as a sort of binary codes) and to-be-learned binary hash codes have left unexploited. In this paper, we present a novel online hashing method, termed Similarity Preserving Linkage Hashing (SPLH), which not only utilizes pairwise similarity to learn the intra-class relationships, but also fully exploits a latent linkage space to capture the inter-class relationships and the common characteristics between label vectors and to-be-learned hash codes. Specifically, SPLH first maps the independent discrete label vectors and binary hash codes into a linkage space, through which the relative semantic distance between data points can be assessed precisely. As a result, the pairwise similarities within the newly arriving data stream are exploited to learn the latent semantic space to benefit binary code learning. To learn the model parameters effectively, we further propose an alternating optimization algorithm. Extensive experiments conducted on three widely-used datasets demonstrate the superior performance of SPLH over several state-of-the-art online hashing methods.

Fast Class-Wise Updating for Online Hashing.

Online image hashing has received increasing research attention recently, which processes large-scale data in a streaming fashion to update the hash functions on-the-fly. To this end, most existing works exploit this problem under a supervised setting, i.e., using class labels to boost the hashing performance, which suffers from the defects in both adaptivity and efficiency: First, large amounts of training batches are required to learn up-to-date hash functions, which leads to poor online adaptivity. Second, the training is time-consuming, which contradicts with the core need of online learning. In this paper, a novel supervised online hashing scheme, termed Fast Class-wise Updating for Online Hashing (FCOH), is proposed to address the above two challenges by introducing a novel and efficient inner product operation. To achieve fast online adaptivity, a class-wise updating method is developed to decompose the binary code learning and alternatively renew the hash functions in a class-wise fashion, which well addresses the burden on large amounts of training batches. Quantitatively, such a decomposition further leads to at least 75 percent storage saving. To further achieve online efficiency, we propose a semi-relaxation optimization, which accelerates the online training by treating different binary constraints independently. Without additional constraints and variables, the time complexity is significantly reduced. Such a scheme is also quantitatively shown to well preserve past information during updating hashing functions. We have quantitatively demonstrated that the collective effort of class-wise updating and semi-relaxation optimization provides a superior performance comparing to various state-of-the-art methods, which is verified through extensive experiments on three widely-used datasets.

Sequential binary code selection for robust object tracking

The appearance model, which is required to be adaptive to the non-stationary environment, is the essential step in object tracking but normally suffers from imbalance between effectiveness and efficiency. In this paper, a novel method named as sequential binary code selection (SBC) is proposed to learn a set of compact binary codes for image patch representation. Using the sparse projections, the high dimensional feature can be speedily embedded into the compact binary codes with preserving both the label information and geometrical distance. By the sequential learning, the latter learned binary code which corrects the errors made by the previous codes is more discriminative to the present environment. Furthermore, though binary code selection, the most compact and least redundant hash codes from the candidate pool will be selected and kept. Experimental results illustrate the effectiveness of the SBC, as well as the state-of-the-art tracking performance of the proposed approach.

Concatenation hashing: A relative position preserving method for learning binary codes

Hashing methods perform the efficient nearest neighbor search by mapping high-dimensional data to binary codes. Compared to projection-based hashing methods, hashing methods that adopt the clustering technique can encode the complex relationship of the data into binary codes. However, their search performance is affected by the boundary of the cluster. Two similar data points may be assigned to two different clusters and then encoded into two much different binary codes. In this paper, we propose a new hashing method based on the clustering technique and it can alleviate the effect from the cluster boundary. It is from an observation that the relative positions of any two close data points to each cluster center are close. An alternating optimization is developed to simultaneously discover the cluster structures of the data and learn the hash functions to preserve the relative positions of the data to each cluster center. To integrate the information in each cluster, the corresponding binary code of each data point is obtained by concatenating the substrings learnt by the hash functions in each cluster. The experiments show that our method is competitive to or better than the state-of-the-art hashing methods.

Bidirectional Discrete Matrix Factorization Hashing for Image Search.

Unsupervised image hashing has recently gained significant momentum due to the scarcity of reliable supervision knowledge, such as class labels and pairwise relationship. Previous unsupervised methods heavily rely on constructing sufficiently large affinity matrix for exploring the geometric structure of data. Nevertheless, due to lack of adequately preserving the intrinsic information of original visual data, satisfactory performance can hardly be achieved. In this article, we propose a novel approach, called bidirectional discrete matrix factorization hashing (BDMFH), which alternates two mutually promoted processes of 1) learning binary codes from data and 2) recovering data from the binary codes. In particular, we design the inverse factorization model, which enforces the learned binary codes inheriting intrinsic structure from the original visual data. Moreover, we develop an efficient discrete optimization algorithm for the proposed BDMFH. Comprehensive experimental results on three large-scale benchmark datasets show that the proposed BDMFH not only significantly outperforms the state-of-the-arts but also provides the satisfactory computational efficiency.

Supervised discrete cross-modal hashing based on kernel discriminant analysis

Cross-modal hashing methods have drawn considerable attention due to the rapid growth of multi-modal data. To obtain efficient binary codes in a low-dimensional Hamming space, most existing approaches relaxed the discrete constraint, which could cause quantization loss and even result in performance degradation. In order to avoid this bottleneck, some scholars employed iterative discrete cyclic coordinate descent (DCC) to learn hash codes bit by bit, but this was very time-consuming. To counter this problem, a simple yet novel supervised discrete cross-modal hashing framework is represented to directly learn the unified discrete binary codes with a close-form, rather than bit by bit. Furthermore, to preserve label separability, the kernel discriminant analysis is fused into the proposed framework to enrich the discrimination ability of the learned binary codes. The goal of the proposed method is to obtain the common discrete binary codes of different modalities in a shared latent Hamming space so that the different modalities of a sample can be effectively connected. Experimental study shows the encouraging results of the proposed algorithm in comparisons to the state-of-the-art baseline approaches on four real-world datasets. Especially on the LabelMe dataset, the superiority of the proposed method is obvious, with an average improvement of 9% over the best available results.

Learning binary code for fast nearest subspace search

Subspace is widely used to represent objects under different viewpoints, illuminations, identities, and more. Due to the growing amount and dimensionality of visual contents, fast search in a large-scale database with high-dimensional subspaces is an important task in many applications, such as image retrieval, clustering, video retrieval, and visual recognition. This can be facilitated by approximate nearest subspace (ANS) search which requires effective subspace representation. All existing methods for this problem represent a subspace by a point in the Euclidean or the Grassmannian space before applying the approximate nearest neighbor (ANN) search. However, the efficiency of these methods is not guaranteed because the subspace representation step can be very time consuming when coping with high-dimensional data. Moreover, the subspace to point transforming process may cause subspace structural information loss which influences the search accuracy. In this paper, we present a new approach for hashing-based ANS search which can directly binarize a subspace without transforming it into a vector. The proposed method learns the binary codes for subspaces following a similarity preserving criterion, and simultaneously leverages the learned binary codes to train matrix classifiers as hash functions. Experiments on face and action recognition and video retrieval applications show that our method outperforms several state-of-the-art methods in both efficiency and accuracy. Moreover, we also compare our method with vector-based hashing methods. Results also show the superiority of our subspace matrix based search scheme.