Compact Binary Codes Research Articles

Deep Supervised Hashing for Fast Image Retrieval

In this paper, we present a new hashing method to learn compact binary codes for highly efficient image retrieval on large-scale datasets. While the complex image appearance variations still pose a...

Unsupervised Binary Representation Learning with Deep Variational Networks

Learning to hash is regarded as an efficient approach for image retrieval and many other big-data applications. Recently, deep learning frameworks are adopted for image hashing, suggesting an alternative way to formulate the encoding function other than the conventional projections. Although deep learning has been proved to be successful in supervised hashing, existing unsupervised deep hashing techniques still cannot produce leading performance compared with the non-deep methods, as it is hard to unveil the intrinsic structure of the whole sample space by simply regularizing the output codes within each single training batch. To tackle this problem, in this paper, we propose a novel unsupervised deep hashing model, named deep variational binaries (DVB). The conditional auto-encoding variational Bayesian networks are introduced in this work to exploit the feature space structure of the training data using the latent variables. Integrating the probabilistic inference process with hashing objectives, the proposed DVB model estimates the statistics of data representations, and thus produces compact binary codes. Experimental results on three benchmark datasets, i.e., CIFAR-10, SUN-397 and NUS-WIDE, demonstrate that DVB outperforms state-of-the-art unsupervised hashing methods with significant margins.

Codebook-Free Compact Descriptor for Scalable Visual Search

The MPEG compact descriptors for visual search (CDVS) is a standard toward image matching and retrieval. To achieve high retrieval accuracy over a large scale image/video dataset, recent research efforts have demonstrated that employing extremely high-dimensional descriptors such as the Fisher vector (FV) and the vector of locally aggregated descriptors (VLAD) can yield good performance. Since the FV (or VLAD) possesses high discriminability but small visual vocabulary, it has been adopted by CDVS to construct a global compact descriptor. In this paper, we study the development of global compact descriptors in the completed CDVS standard and the emerging compact descriptors for video analysis (CDVA) standard, in which we formulate the FV (or VLAD) compression as a resource-constrained optimization problem. Accordingly, we propose a codebook-free aggregation method via dual selection to generate a global compact visual descriptor, which supports fast and accurate feature matching free of large visual codebooks, fulfilling the low memory requirement of mobile visual search at significantly reduced latency. Specifically, we investigate both sample-specific Gaussian component redundancy and bit dependency within a binary aggregated descriptor to produce compact binary codes. Our technique contributes to the scalable compressed Fisher vector (SCFV) adopted by the CDVS standard. Moreover, the SCFV descriptor is currently serving as the frame-level hand-crafted video feature, which inspires the inheritance of CDVS descriptors for the emerging CDVA standard. Furthermore, we investigate the positive complementary effect of our standard compliant compact descriptor and deep learning based features extracted from convolutional neural networks with significant mean average precision gains. Extensive evaluation over benchmark databases shows the significant merits of the codebook-free binary codes for scalable visual search.

An End-to-End-Based Low Dimensional Binary Embedding for Chrysanthemum Phenotypic Petal Similarity Evaluation

Plant phenotyping is widely used to identify the genotype differences and the environmental conditions of plants. These pieces of information can be gathered by various sensors conveniently and easily. The similarity evaluation of phenotypic data is essential for category classifications and stress sensitivity analysis of plants. Chrysanthemum is a flower with great varieties and petal types. The similarity evaluation of chrysanthemum flowers plays a key role in the phenotypic research. For improving the performance and effectiveness of phenotypic similarity evaluation of high-throughput chrysanthemum flower images, this paper proposes an end-to-end based low dimensional binary embedding framework by using deep learning and hash encoding approaches. Within this framework, compact binary codes are learned by deep convolutional neural networks. Firstly, features of chrysanthemum images are extracted by deep neural networks, then a binary layer is embedded for converting the features into lower dimensional bit codes. Finally, the phenotypic similarity of the chrysanthemum flower is evaluated in Hamming space with the binary codes. Extensive experiments are conducted and the results show that the performance and effectiveness of similarity evaluations are improved greatly. In summary, our research provides an end-to-end pipeline for the similarity evaluation of chrysanthemum petal phenotypes, and the proposed method is practically applicable to the high-throughput similarity evaluation. The proposed method can be used as a foundation for the in-depth research on chrysanthemum flower petals as well.

Angular Deep Supervised Hashing for Image Retrieval

Deep learning based image hashing methods learn hash codes by using powerful feature extractors and nonlinear transformations to achieve highly efficient image retrieval. For most end-to-end deep hashing methods, the supervised learning process relies on pair-wise or triplet-wise information to provide an internal relationship of similarity data. However, the use of pair-wise and triplet loss function is limited not only by expensive training costs but also by quantization errors. In this paper, we propose a novel semantic learning based hashing method for image retrieval to optimize the deep features structure in the hash space from a perspective of angular view. Specifically, we proposed an angular hashing loss function that explicitly improve intra-class compactness and inter-class separability between features in hash space. Geometrically, angular hashing loss can be regarded as imposing hash constraints on hypersphere manifold. In order to solve the training problem on the multi-label case, we further designed a dynamic Softmax training strategy that can directly train the network using gradient descent method. Extensive experiments on two well-known datasets of CIFAR-10 and NUS-WIDE demonstrate that the proposed Angular Deep Supervised Hashing (ADSH) method can generate high-quality and compact binary codes, which can achieve state-of-the-art performance as compared with conventional image hashing and deep learning-based hashing methods.

Balanced Deep Supervised Hashing

Recently, Convolutional Neural Network (CNN) based hashing method has achieved its promising performance for image retrieval task. However, tackling the discrepancy between quantization error minimization and discriminability maximization of network outputs simultaneously still remains unsolved. Motivated by the concern, we propose a novel Balanced Deep Supervised Hashing (BDSH) based on variant posterior probability to learn compact discriminability-preserving binary code for large scale image data. Distinguished from the previous works, BDSH can search an equilibrium point within the discrepancy. Towards the goal, a delicate objective function is utilized to maximize the discriminability of the output space with the variant posterior probability of the pair-wise label. A quantization regularizer is utilized as a relaxation from real-value outputs to the desired discrete values (e.g., -1/+1). Extensive experiments on the benchmark datasets show that our method can yield state-of-the-art image retrieval performance from various perspectives.

Medical Image Retrieval Based on Convolutional Neural Network and Supervised Hashing

In recent years, with extensive application in image retrieval and other tasks, a convolutional neural network (CNN) has achieved outstanding performance. In this paper, a new content-based medical image retrieval (CBMIR) framework using CNN and hash coding is proposed. The new framework adopts a Siamese network in which pairs of images are used as inputs, and a model is learned to make images belonging to the same class have similar features by using weight sharing and a contrastive loss function. In each branch of the network, CNN is adapted to extract features, followed by hash mapping, which is used to reduce the dimensionality of feature vectors. In the training process, a new loss function is designed to make the feature vectors more distinguishable, and a regularization term is added to encourage the real value outputs to approximate the desired binary values. In the retrieval phase, the compact binary hash code of the query image is achieved from the trained network and is subsequently compared with the hash codes of the database images. We experimented on two medical image datasets: the cancer imaging archive-computed tomography (TCIA-CT) and the vision and image analysis group/international early lung cancer action program (VIA/I-ELCAP). The results indicate that our method is superior to existing hash methods and CNN methods. Compared with the traditional hashing method, feature extraction based on CNN has advantages. The proposed algorithm combining a Siamese network with the hash method is superior to the classical CNN-based methods. The application of a new loss function can effectively improve retrieval accuracy.

Learning binary hash codes for finger vein image retrieval

High dimensional real-valued features have been shown to be effective for finger vein identification, but result in high computational cost especially in a large-scale finger vein database. Therefore compact binary codes are generally used to obtain fast query speed as well as reduce storage requirements. However, only a few strategies are available for finger vein retrieval in Hamming space, and existing binary representation of finger veins can become unstable and undiscriminating due to finger vein variations induced by translation, rotation and illumination. In this paper, we propose a binary hash codes learning algorithm to map finger vein images in the original feature space to Hamming space. First, to obtain the discriminative finger vein image features, a novel finger vein image representation method called Nonlinearly Subspace Coding (NSC) is proposed. The codebook is a union of low-dimensional linear subspaces instead of visual words. For a given local finger vein texton, its top-k nearest subspaces are found and the texton is nonlinearly mapped into these subspaces. Then, a finger vein Binary Hash Codes (BHC) learning method is proposed by jointly considering the discriminability and the stability of the binary code. Experimental results on a two-session public finger vein database and a large fused finger vein database demonstrate the effectiveness and efficiency of our binary hash coding learning algorithm for large-scale finger vein retrieval.

Deep Variational and Structural Hashing.

In this paper, we propose a deep variational and structural hashing (DVStH) method to learn compact binary codes for multimedia retrieval. Unlike most existing deep hashing methods which use a series of convolution and fully-connected layers to learn binary features, we develop a probabilistic framework to infer latent feature representation inside the network. Then, we design a struct layer rather than a bottleneck hash layer, to obtain binary codes through a simple encoding procedure. By doing these, we are able to obtain binary codes discriminatively and generatively. To make it applicable to cross-modal scalable multimedia retrieval, we extend our method to a cross-modal deep variational and structural hashing (CM-DVStH). We design a deep fusion network with a struct layer to maximize the correlation between image-text input pairs during the training stage so that a unified binary vector can be obtained. We then design modality-specific hashing networks to handle the out-of-sample extension scenario. Specifically, we train a network for each modality which outputs a latent representation that is as close as possible to the binary codes which are inferred from the fusion network. Experimental results on five benchmark datasets are presented to show the efficacy of the proposed approach.

Fried Binary Embedding: From High-Dimensional Visual Features to High-Dimensional Binary Codes.

Most existing binary embedding methods prefer compact binary codes ( -dimensional) to avoid high computational and memory cost of projecting high-dimensional visual features ( -dimensional, ). We argue that long binary codes ( ) are critical to fully utilize the discriminative power of high-dimensional visual features, and can achieve better results in various tasks such as approximate nearest neighbor search. Generating long binary codes involves large projection matrix and high-dimensional matrix-vector multiplication, thus is memory and compute intensive. We propose Fried binary embedding (FBE) and Supervised Fried Binary Embedding (SuFBE), to tackle these problems. FBE is suitable for most of the practical applications in which the labels of training data are not given, while SuFBE can significantly boost the accuracy in the cases that the training labels are available. The core idea is to decompose the projection matrix using adaptive Fastfood transform, which is the multiplication of several structured matrices. As a result, FBE and SuFBE can reduce the computational complexity from to , and memory cost from to , respectively. More importantly, by using the structured matrices, FBE and SuFBE can well regulate projection matrix by reducing its tunable parameters and lead to even better accuracy than using either unconstrained projection matrix (like ITQ) or sparse matrix such as SP and SSP with the same long code length. Experimental comparisons with state-of-the-art methods over various visual applications demonstrate both the efficiency and performance advantages of FBE and SuFBE.

Deep Learning Hash for Wireless Multimedia Image Content Security

With the explosive growth of the wireless multimedia data on the wireless Internet, a large number of illegal images have been widely disseminated in wireless networks, which seriously endangers the content security of wireless networks. However, how to identify and classify illegal images quickly, accurately, and in real time is a key challenge for wireless multimedia networks. To avoid illegal images circulating on the Internet, each image needs to be detected, extracted features, and compared with the image in the feature library to verify the legitimacy of the image. An improved image deep learning hash (IDLH) method to learn compact binary codes for image search is proposed in this paper. Specifically, there are three major processes of IDLH: the feature extraction, deep secondary search, and image classification. IDLH performs image retrieval by the deep neural networks (DNN) as well as image classification with the binary hash codes. Different from other deep learning-hash methods that often entail heavy computations by using a conventional classifier, exemplified by K nearest neighbor (K-NN) and support vector machines (SVM), our method learns classifiers using binary hash codes, which can be learned synchronously in training. Finally, comprehensive experiments are conducted to evaluate IDLH method by using CIFAR-10 and Caltech 256 image library datasets, and the results show that the retrieval performance of IDLH method can effectively identify illegal images.

Deep learning compact binary codes for fingerprint indexing

With the rapid growth in fingerprint databases, it has become necessary to develop excellent fingerprint indexing to achieve efficiency and accuracy. Fingerprint indexing has been widely studied with real-valued features, but few studies focus on binary feature representation, which is more suitable to identify fingerprints efficiently in large-scale fingerprint databases. In this study, we propose a deep compact binary minutia cylinder code (DCBMCC) as an effective and discriminative feature representation for fingerprint indexing. Specifically, the minutia cylinder code (MCC), as the state-of-the-art fingerprint representation, is analyzed and its shortcomings are revealed. Accordingly, we propose a novel fingerprint indexing method based on deep neural networks to learn DCBMCC. Our novel network restricts the penultimate layer to directly output binary codes. Moreover, we incorporate independence, balance, quantization-loss-minimum, and similarity-preservation properties in this learning process. Eventually, a multi-index hashing (MIH) based fingerprint indexing scheme further speeds up the exact search in the Hamming space by building multiple hash tables on binary code substrings. Furthermore, numerous experiments on public databases show that the proposed approach is an outstanding fingerprint indexing method since it has an extremely small error rate with a very low penetration rate.

Hadamard Coding for Supervised Discrete Hashing.

In this paper, we propose a learning-based supervised discrete hashing method. Binary hashing is widely used for large-scale image retrieval as well as video and document searches because the compact binary code representation is essential for data storage and reasonable for query searches using bit-operations. The recently proposed supervised discrete hashing (SDH) method efficiently solves mixed-integer programming problems by alternating optimization and the discrete cyclic coordinate descent (DCC) method. Based on some preliminary experiments, we show that the SDH method can be simplified without performance degradation. We analyze the simplified model and provide a mathematically exact solution thereof; we reveal that the exact binary code is provided by a "Hadamard matrix." Therefore, we named our method Hadamard codedsupervised discrete hashing (HC-SDH). In contrast to SDH, our model does not require an alternating optimization algorithm and does not depend on initial values. HC-SDH is also easier to implement than iterative quantization (ITQ). Experimental results involving a large-scale database show that Hadamard coding outperforms conventional SDH in terms of precision, recall, and computational time. On the large datasets SUN-397 and ImageNet, HC-SDH provides a superior mean average of precision (mAP) and top-accuracy compared to the conventional SDH methods with the same code length and FastHash. The training time of HC-SDH is 170 times faster than conventional SDH and the testing time including the encoding time is seven times faster than FastHash which encodes using a binary-tree.

Deep hashing network for material defect image classification

Common non‐destructive material testing technology has some well‐known problems such as slow detection, low detection accuracy, and low level of information obtained. To solve these problems, this study applied recent advances in convolution neural networks to propose an effective deep learning network using casting datasets. The approach achieves non‐destructive material testing with automatic, intelligent detection technology. For most existing deep learning networks, an image is eventually transformed into a multidimensional visual feature vector for comparison and classification. However, such vectors may not optimally improve detection precision and speed, and can lead to significant storage problems. A deep hashing network is proposed in which images are mapped into compact binary codes. There are three key components: (i) a sub‐network with multiple convolution‐pooling layers to capture image representations; (ii) a hashing layer to generate compact binary hash codes; (iii) an encoder module to divide the image feature vector from the output of the sub‐network above into multiple branches, each encoded into one hash bit. Extensive experiments using a casting dataset show promising performance compared with the state‐of‐the‐art approach.

Efficient Conversion of Deep Features to Compact Binary Codes Using Fourier Decomposition for Multimedia Big Data

Exponential growth of multimedia data has been witnessed in recent years from various industries, such as e-commerce, health, transportation, and social networks, etc. Access to desired data in such gigantic datasets require sophisticated and efficient retrieval methods. In the last few years, neuronal activations generated by a pretrained convolutional neural network (CNN) have served as generic descriptors for various tasks including image classification, object detection and segmentation, and image retrieval. They perform incredibly well compared to hand-crafted features. However, these features are usually high dimensional, requiring a lot of memory and computations for indexing and retrieval. For very large datasets, utilization of these high dimensional features in raw form becomes infeasible. In this paper, a highly efficient method is proposed to transform high dimensional deep features into compact binary codes using bidirectional Fourier decomposition. This compact bit code saves memory and eases computations during retrieval. Further, these codes can also serve as hash codes, allowing very efficient access to images in large datasets using approximate nearest neighbor (ANN) search techniques. Our method does not require any training and achieves considerable retrieval accuracy with short length codes. It has been tested on features extracted from fully connected layers of a pretrained CNN. Experiments conducted with several large datasets reveal the effectiveness of our approach for a wide variety of datasets.

Binary Multi-View Clustering.

Clustering is a long-standing important research problem, however, remains challenging when handling large-scale image data from diverse sources. In this paper, we present a novel Binary Multi-View Clustering (BMVC) framework, which can dexterously manipulate multi-view image data and easily scale to large data. To achieve this goal, we formulate BMVC by two key components: compact collaborative discrete representation learning and binary clustering structure learning, in a joint learning framework. Specifically, BMVC collaboratively encodes the multi-view image descriptors into a compact common binary code space by considering their complementary information; the collaborative binary representations are meanwhile clustered by a binary matrix factorization model, such that the cluster structures are optimized in the Hamming space by pure, extremely fast bit-operations. For efficiency, the code balance constraints are imposed on both binary data representations and cluster centroids. Finally, the resulting optimization problem is solved by an alternating optimization scheme with guaranteed fast convergence. Extensive experiments on four large-scale multi-view image datasets demonstrate that the proposed method enjoys the significant reduction in both computation and memory footprint, while observing superior (in most cases) or very competitive performance, in comparison with state-of-the-art clustering methods.

Privacy-preserving image retrieval for mobile devices with deep features on the cloud

With the prevalent use of mobile cameras to capture images, the demands for efficient and effective methods for indexing and retrieval of personal image collections on mobile devices have also risen. In this paper, we propose to represent images with hash codes, which is a compressed representation of deep convolutional features using deep auto-encoder on the cloud. To ensure user's privacy, the image is first encrypted using a light-weight encryption algorithm on mobile device prior to offloading it to the cloud for features extraction. This approach eliminates the computationally expensive process of features extraction on resource constrained devices. A pre-trained convolutional neural network (CNN) is used to extract features which are then transformed to compact binary codes using a deep auto-encoder. The hash codes are then sent back to the mobile device where they are stored in a hash table along with image location. Approximate nearest neighbor (ANN) search approach is utilized to efficiently retrieve the desired images without exhaustive searching of the entire image collection. The proposed method is evaluated against three different publicly available image datasets namely Corel-10K, GHIM-10K, and Product image dataset. Experimental results demonstrate that features representation using CNN and auto-encoder shows much better results than several state-of-the-art hashing schemes for image retrieval on mobile devices.

Nonlinear Structural Hashing for Scalable Video Search

In this paper, we propose a nonlinear structural hashing approach to learn compact binary codes for scalable video search. Unlike most existing video hashing methods which consider image frames within a video separately for binary code learning, we develop a multi-layer neural network to learn compact and discriminative binary codes by exploiting both the structural information between different frames within a video and the nonlinear relationship between video samples. To be specific, we learn these binary codes under two different constraints at the output of our network: 1) the distance between the learned binary codes for frames within the same scene is minimized and 2) the distance between the learned binary matrices for a video pair with the same label is less than a threshold and that for a video pair with different labels is larger than a threshold. To better measure the structural information of the scenes from videos, we employ a subspace clustering method to cluster frames into different scenes. Moreover, we design multiple hierarchical nonlinear transformations to preserve the nonlinear relationship between videos. Experimental results on three video data sets show that our method outperforms state-of-the-art hashing approaches on the scalable video search task.

Sub-Selective Quantization for Learning Binary Codes in Large-Scale Image Search.

Recently with the explosive growth of visual content on the Internet, large-scale image search has attracted intensive attention. It has been shown that mapping high-dimensional image descriptors to compact binary codes can lead to considerable efficiency gains in both storage and performing similarity computation of images. However, most existing methods still suffer from expensive training devoted to large-scale binary code learning. To address this issue, we propose a sub-selection based matrix manipulation algorithm, which can significantly reduce the computational cost of code learning. As case studies, we apply the sub-selection algorithm to several popular quantization techniques including cases using linear and nonlinear mappings. Crucially, we can justify the resulting sub-selective quantization by proving its theoretic properties. Extensive experiments are carried out on three image benchmarks with up to one million samples, corroborating the efficacy of the sub-selective quantization method in terms of image retrieval.

Discriminative Deep Quantization Hashing for Face Image Retrieval.

This paper proposes a new discriminative deep quantization hashing (DDQH) approach for large-scale face image retrieval by learning discriminative and compact binary codes. It jointly explores the discrete code learning, batch normalization quantization (BNQ) module, and end-to-end learning in one unified framework, which can guarantee the optimal compatibility of hash coding and feature learning. To learn multiscale and robust facial features, a deep network properly stacking several convolution-pooling layers and pooling layers is designed, and the facial features are obtained by fusing the outputs of the last convolutional layer and the last pooling layer. Besides, the prediction errors of the learned binary codes are minimized to learn discriminative binary codes of images. To obtain higher retrieval accuracies, a BNQ module is utilized to control quantization at a moderate level. Experiments are conducted on two widely used data sets, and the proposed DDQH method achieves encouraging improvements over some state-of-the-art hashing approaches.