Discrete Binary Codes Research Articles

Graph Convolutional Network Discrete Hashing for Cross-Modal Retrieval.

With the rapid development of deep neural networks, cross-modal hashing has made great progress. However, the information of different types of data is asymmetrical, that is to say, if the resolution of an image is high enough, it can reproduce almost 100% of the real-world scenes. However, text usually carries personal emotion and it is not objective enough, so we generally think that the information of image will be much richer than text. Although most of the existing methods unify the semantic feature extraction and hash function learning modules for end-to-end learning, they ignore this issue and do not use information-rich modalities to support information-poor modalities, leading to suboptimal results, although they unify the semantic feature extraction and hash function learning modules for end-to-end learning. Furthermore, previous methods learn hash functions in a relaxed way that causes nontrivial quantization losses. To address these issues, we propose a new method called graph convolutional network (GCN) discrete hashing. This method uses a GCN to bridge the information gap between different types of data. The GCN can represent each label as word embedding, with the embedding regarded as a set of interdependent object classifiers. From these classifiers, we can obtain predicted labels to enhance feature representations across modalities. In addition, we use an efficient discrete optimization strategy to learn the discrete binary codes without relaxation. Extensive experiments conducted on three commonly used datasets demonstrate that our proposed method graph convolutional network-based discrete hashing (GCDH) outperforms the current state-of-the-art cross-modal hashing methods.

Efficient Supervised Graph Embedding Hashing for large-scale cross-media retrieval

Recently, graph based hashing has gained much attention due to its effectiveness in multi-media retrieval. Although several graph embedding based works have been designed and achieved promising performance, there are still some issues that need to be further studied, including, (1) one significant drawback of graph embedding is its expensive memory and computation cost caused by the graph Laplacian matrix; (2) most pioneer works fail to fully explore the available class labels in training procedure, which generally makes them suffer from unsatisfactory retrieval performance. To overcome these drawbacks, we propose a simple yet effective supervised cross-media hashing scheme, termed Efficient Supervised Graph Embedding Hashing (ESGEH), which can simultaneously learn hash functions and discrete binary codes efficiently. Specifically, ESGEH leverages both class label based semantic embedding and graph embedding to generate a sharing semantic subspace, and class labels are also incorporated to minimize the quantization error for better approximating the generated binary codes. In order to reduce the computational sources, a well-designed intermediate terms decomposition is proposed to avoid explicitly computing the graph Laplacian matrix. Finally, an iterative discrete optimal algorithm is derived to solve above problem, and each subproblem can yield a closed-form solution. Extensive experimental results on four public datasets demonstrate the superiority of the proposed approach over several existing cross-media hashing methods in terms of both accuracy and efficiency.

Enhanced Deep Discrete Hashing with semantic-visual similarity for image retrieval

Hashing has been shown to be successful in a number of Approximate Nearest Neighbor (ANN) domains, ranging from medicine, computer vision to information retrieval. However, current deep hashing methods either ignore both rich information of labels and visual linkages of image pairs, or leverage relaxation-based algorithms to address discrete problems, resulting in a large information loss. To address the aforementioned problems, in this paper, we propose an Enhanced Deep Discrete Hashing (EDDH) method to leverage both label embedding and semantic-visual similarity to learn the compact hash codes. In EDDH, the discriminative capability of hash codes is enhanced by a distribution-based continuous semantic-visual similarity matrix, where not only the margin between the positive pairs and negative pairs is expanded, but also the visual linkages between image pairs is considered. Specifically, the semantic-visual continuous similarity matrix is constructed by analyzing the asymmetric generalized Gaussian distribution of the visual linkages between pairs with label consideration. Besides, in order to achieve an efficient hash learning framework, EDDH employs an asymmetric real-valued learning structure to learn the compact hash codes. In addition, we develop a fast discrete optimization algorithm, which can directly generate discrete binary codes in single step, and introduce an intermediate term before iterations to avoid the problems caused by directly the use of large semantic-visual similarity matrix, which results in a significant reduction in the computational overhead. Finally, we conducted extensive experiments on three datasets to show that EDDH has a significantly enhanced performance compared to the compared state-of-the-art baselines.

Object classification of remote sensing images based on optimized projection supervised discrete hashing

Recently, with the number of large-scale remote sensing (RS) images increasing, the demand for large-scale RS image object classification is growing, and many researchers are interested. Hashing, as a result of its low memory requirements and high time efficiency, has widely solved the problem of large-scale RS images. Supervised hashing methods mainly leverage RS image label information to learn hashing function; however, the similarity of the original feature space cannot be well preserved, which cannot meet the accurate requirements of RS images object classification. To address the aforementioned analysis, we propose a method named optimized projection supervised discrete hashing (OPSDH), which jointly learns optimized projection constraint and discrete binary codes generation model. It uses an effective optimized projection method to further constraint on the supervised hashing learn, and generated hash codes preserve the similarity based on the data label while retaining the original feature space’s similarity. The experimental results show that OPSDH reaches improved performance compared with existing hashing methods and demonstrates that the proposed OPSDH is more efficient for operational applications.

Unsupervised deep hashing by joint optimization for pulmonary nodule image retrieval

Abstract In recent years, hash-based image retrieval has attracted great attention due to the rapid growth of medical images. In the paper, an end-to-end unsupervised deep hashing is proposed, where feature extraction and binary optimization are carried out by joint optimization. Our method consists of five components: a shared deep convolution neural network for learning image representations, a deconvolution module for reconstructing the original images, a classification module for leveraging semantic supervision by pseudo labels, a binary code learning module for encoding images features into binary codes, and a joint loss function for deep hash function learning. In addition, the real-valued features balanced in different dimensions by a rotation matrix are quantized directly into discrete binary codes in an alternating optimization approach to minimize the quantization loss. Experiments have been performed on the pulmonary nodule images dataset and the results demonstrate the proposed method can yield better retrieval performance by comparing with the state-of-the-art unsupervised hashing methods.

Correlation Filtering-Based Hashing for Fine-Grained Image Retrieval

The low storage and strong representation capabilities of hash codes for image retrievalhas made hashing technologies very popular. Several existing deep hashing methods focuson the task of general image retrieval, while neglecting the task of fine-grained image retrieval. Recently, some fine-grained hashing methods have been proposed to capture the subtle differences, which mainly utilize the single-modality visual features to solve the discriminative region localization while ignoring the semantic information. In this letter, we propose a correlation filtering hashing (CFH) method to learn discrete binary codes, which can adequately take advantage of the cross-modal correlation between the semantic information and the visual features for discriminative region localization. Specifically, we utilize a feature pyramid network to learn multi-level visual features. Subsequently, the label vector is embedded into the visual space, which can be used as a correlation filter on the feature maps to capture the latent location of objects. Finally, weperform global average pooling over the output maps and concatenate the features of different levels to produce the hash codes of query images. Extensive experiments on two fine-grained datasets show that the proposed CFH outperforms the state-of-the-art hashing methods.

Unsupervised Deep Cross-modal Hashing with Virtual Label Regression

Unsupervised cross-modal hashing learns hash codes without dependence on the semantic labels. It has the desirable advantage of well scalability and thus can effectively support large-scale cross-media retrieval. However, how to directly learn discriminative discrete hash codes under the unsupervised learning paradigm is still an open challenging problem. In this paper, we aim to attack this problem by proposing an Unsupervised Deep Cross-modal Hashing with Virtual Label Regression (UDCH-VLR) method. We propose a novel unified learning framework to jointly perform deep hash function training, virtual label learning and regression. Specifically, we learn unified hash codes via collaborative matrix factorization on the multi-modal deep representations to preserve the multi-modal shared semantics. Further, we incorporate the virtual label learning into the objective functions and simultaneously regress the learned virtual labels to the hash codes. Finally, instead of simply exploiting the existing shallow features and relaxing the binary constraints, we devise an alternative optimization strategy to directly update the deep hash functions and discrete binary codes. Under such circumstance, the discriminative capability of hash codes can be progressively enhanced with iterative learning. Extensive experiments on three publicly available cross-media retrieval datasets demonstrate that our approach outperforms state-of-the-art methods.

Supervised discrete cross-modal hashing based on kernel discriminant analysis

Cross-modal hashing methods have drawn considerable attention due to the rapid growth of multi-modal data. To obtain efficient binary codes in a low-dimensional Hamming space, most existing approaches relaxed the discrete constraint, which could cause quantization loss and even result in performance degradation. In order to avoid this bottleneck, some scholars employed iterative discrete cyclic coordinate descent (DCC) to learn hash codes bit by bit, but this was very time-consuming. To counter this problem, a simple yet novel supervised discrete cross-modal hashing framework is represented to directly learn the unified discrete binary codes with a close-form, rather than bit by bit. Furthermore, to preserve label separability, the kernel discriminant analysis is fused into the proposed framework to enrich the discrimination ability of the learned binary codes. The goal of the proposed method is to obtain the common discrete binary codes of different modalities in a shared latent Hamming space so that the different modalities of a sample can be effectively connected. Experimental study shows the encouraging results of the proposed algorithm in comparisons to the state-of-the-art baseline approaches on four real-world datasets. Especially on the LabelMe dataset, the superiority of the proposed method is obvious, with an average improvement of 9% over the best available results.

Asymmetric Deep Semantic Quantization for Image Retrieval

Due to its fast retrieval and storage efficiency capabilities, hashing has been widely used in nearest neighbor retrieval tasks. By using deep learning based techniques, hashing can outperform non-learning based hashing technique in many applications. However, we argue that the current deep learning based hashing methods ignore some critical problems (e.g., the learned hash codes are not discriminative due to the hashing methods being unable to discover rich semantic information and the training strategy having difficulty optimizing the discrete binary codes). In this paper, we propose a novel image hashing method, termed as \textbf{\underline{A}}symmetric \textbf{\underline{D}}eep \textbf{\underline{S}}emantic \textbf{\underline{Q}}uantization (\textbf{ADSQ}). \textbf{ADSQ} is implemented using three stream frameworks, which consist of one \emph{LabelNet} and two \emph{ImgNets}. The \emph{LabelNet} leverages the power of three fully-connected layers, which are used to capture rich semantic information between image pairs. For the two \emph{ImgNets}, they each adopt the same convolutional neural network structure, but with different weights (i.e., asymmetric convolutional neural networks). The two \emph{ImgNets} are used to generate discriminative compact hash codes. Specifically, the function of the \emph{LabelNet} is to capture rich semantic information that is used to guide the two \emph{ImgNets} in minimizing the gap between the real-continuous features and the discrete binary codes. Furthermore, \textbf{ADSQ} can utilize the most critical semantic information to guide the feature learning process and consider the consistency of the common semantic space and Hamming space. Experimental results on three benchmarks (i.e., CIFAR-10, NUS-WIDE, and ImageNet) demonstrate that the proposed \textbf{ADSQ} can outperforms current state-of-the-art methods.

Robust and Flexible Discrete Hashing for Cross-Modal Similarity Search

Multimodal hashing approaches have gained great success on large-scale cross-modal similarity search applications, due to their appealing computation and storage efficiency. However, it is still a challenge work to design binary codes to represent the original features with good performance in an unsupervised manner. We argue that there are some limitations that need to be further considered for unsupervised multimodal hashing: 1) most existing methods drop the discrete constraints to simplify the optimization, which will cause large quantization error; 2) many methods are sensitive to outliers and noises since they use $\ell _{2}$ -norm in their objective functions which can amplify the errors; and 3) the weight of each modality, which greatly influences the retrieval performance, is manually or empirically determined and may not fully fit the specific training set. The above limitations may significantly degrade the retrieval accuracy of unsupervised multimodal hashing methods. To address these problems, in this paper, a novel hashing model is proposed to efficiently learn robust discrete binary codes, which is referred as Robust and Flexible Discrete Hashing (RFDH). In the proposed RFDH model, binary codes are directly learned based on discrete matrix decomposition, so that the large quantization error caused by relaxation is avoided. Moreover, the $\ell _{2,1}$ -norm is used in the objective function to improve the robustness, such that the learned model is not sensitive to data outliers and noises. In addition, the weight of each modality is adaptively adjusted according to training data. Hence the important modality will get large weights during the hash learning procedure. Owing to above merits of RFDH, it can generate more effective hash codes. Besides, we introduce two kinds of hash function learning methods to project unseen instances into hash codes. Extensive experiments on several well-known large databases demonstrate superior performance of the proposed hash model over most state-of-the-art unsupervised multimodal hashing methods.

Partial randomness hashing applied to remote sensing object classification

Recently, object classification of remote sensing images has attracted more and more research interests due to the development of satellite and aerial vehicle technologies. Hashing learning is an efficient method to handle the huge amount of the remote sensing data. We proposed a hashing learning method named partial randomness supervised discrete hashing (PRSDH), which combines data-dependent methods and data-independent methods. It jointly learns a discrete binary codes generation and partial random constraint optimization model. By random projection, the computation complexity is reduced effectively. With the weight matrix derived from the training data, the semantic similarity between the data can be well preserved while generating the hashing codes. For the discrete constraint problem, this paper adopts the discrete cyclic coordinate descent algorithm to optimize the codes bit by bit. The experimental results show that PRSDH outperforms other comparative methods and demonstrate that PRSDH has good adaptability to the characteristic of remote sensing object.

Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features.

Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep learning architectures can learn more effective image representation features. However, these methods only use semantic features to generate hash codes by shallow projection but ignore texture details. In this paper, we proposed a novel hashing method, namely hierarchical recurrent neural hashing (HRNH), to exploit hierarchical recurrent neural network to generate effective hash codes. There are three contributions of this paper. First, a deep hashing method is proposed to extensively exploit both spatial details and semantic information, in which, we leverage hierarchical convolutional features to construct image pyramid representation. Second, our proposed deep network can exploit directly convolutional feature maps as input to preserve the spatial structure of convolutional feature maps. Finally, we propose a new loss function that considers the quantization error of binarizing the continuous embeddings into the discrete binary codes, and simultaneously maintains the semantic similarity and balanceable property of hash codes. Experimental results on four widely used data sets demonstrate that the proposed HRNH can achieve superior performance over other state-of-the-art hashing methods.Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep learning architectures can learn more effective image representation features. However, these methods only use semantic features to generate hash codes by shallow projection but ignore texture details. In this paper, we proposed a novel hashing method, namely hierarchical recurrent neural hashing (HRNH), to exploit hierarchical recurrent neural network to generate effective hash codes. There are three contributions of this paper. First, a deep hashing method is proposed to extensively exploit both spatial details and semantic information, in which, we leverage hierarchical convolutional features to construct image pyramid representation. Second, our proposed deep network can exploit directly convolutional feature maps as input to preserve the spatial structure of convolutional feature maps. Finally, we propose a new loss function that considers the quantization error of binarizing the continuous embeddings into the discrete binary codes, and simultaneously maintains the semantic similarity and balanceable property of hash codes. Experimental results on four widely used data sets demonstrate that the proposed HRNH can achieve superior performance over other state-of-the-art hashing methods.

Discrete Multimodal Hashing With Canonical Views for Robust Mobile Landmark Search

Mobile landmark search (MLS) recently receives increasing attention for its great practical values. However, it still remains unsolved due to two important challenges. One is high bandwidth consumption of query transmission, and the other is the huge visual variations of query images sent from mobile devices. In this paper, we propose a novel hashing scheme, named as canonical view based discrete multi-modal hashing (CV-DMH), to handle these problems via a novel three-stage learning procedure. First, a submodular function is designed to measure visual representativeness and redundancy of a view set. With it, canonical views, which capture key visual appearances of landmark with limited redundancy, are efficiently discovered with an iterative mining strategy. Second, multi-modal sparse coding is applied to transform visual features from multiple modalities into an intermediate representation. It can robustly and adaptively characterize visual contents of varied landmark images with certain canonical views. Finally, compact binary codes are learned on intermediate representation within a tailored discrete binary embedding model which preserves visual relations of images measured with canonical views and removes the involved noises. In this part, we develop a new augmented Lagrangian multiplier (ALM) based optimization method to directly solve the discrete binary codes. We can not only explicitly deal with the discrete constraint, but also consider the bit-uncorrelated constraint and balance constraint together. Experiments on real world landmark datasets demonstrate the superior performance of CV-DMH over several state-of-the-art methods.

Robust discrete code modeling for supervised hashing

Recent years have witnessed the promising efficacy and efficiency of hashing (also known as binary code learning) for retrieving nearest neighbor in large-scale data collections. Particularly, with supervision knowledge (e.g., semantic labels), we may further gain considerable performance boost. Nevertheless, most existing supervised hashing schemes suffer from the following limitations: (1) severe quantization error caused by continuous relaxation of binary codes; (2) disturbance of unreliable codes in subsequent hash function learning; and (3) erroneous guidance derived from imprecise and incomplete semantic labels. In this work, we propose a novel supervised hashing approach, termed as Robust Discrete Code Modeling (RDCM), which directly learns high-quality discrete binary codes and hash functions by effectively suppressing the influence of unreliable binary codes and potentially noisily-labeled samples. RDCM employs ℓ2, p norm, which is capable of inducing sample-wise sparsity, to jointly perform code selection and noisy sample identification. Moreover, we preserve the discrete constraint in RDCM to eliminate the quantization error. An efficient algorithm is developed to solve the discrete optimization problem. Extensive experiments conducted on various real-life datasets show the superiority of the proposed RDCM approach as compared to several state-of-the-art hashing methods.

Asymmetric Discrete Graph Hashing

Recently, many graph based hashing methods have been emerged to tackle large-scale problems. However, there exists two major bottlenecks: (1) directly learning discrete hashing codes is an NP-hardoptimization problem; (2) the complexity of both storage and computational time to build a graph with n data points is O(n2). To address these two problems, in this paper, we propose a novel yetsimple supervised graph based hashing method, asymmetric discrete graph hashing, by preserving the asymmetric discrete constraint and building an asymmetric affinity matrix to learn compact binary codes.Specifically, we utilize two different instead of identical discrete matrices to better preserve the similarity of the graph with short binary codes. We generate the asymmetric affinity matrix using m (m << n) selected anchors to approximate the similarity among all training data so that computational time and storage requirement can be significantly improved. In addition, the proposed method jointly learns discrete binary codes and a low-dimensional projection matrix to further improve the retrieval accuracy. Extensive experiments on three benchmark large-scale databases demonstrate its superior performance over the recent state of the arts with lower training time costs.

Manifold-ranking embedded order preserving hashing for image semantic retrieval

Due to the storage and computational efficiency of hashing technology, it has proven a valuable tool for large scale similarity search. In many cases, the large scale data in real-world lie near some (unknown) low-dimensional and non-linear manifold. Moreover, Manifold Ranking approach can preserve the global topological structure of the data set more effectively than Euclidean Distance-based Ranking approach, which fails to preserve the semantic relevance degree. However, most existing hashing methods ignore the global topological structure of the data set. The key issue is how to incorporate the global topological structure of data set into learning effective hashing function. In this paper, we propose a novel unsupervised hashing approach, namely Manifold-Ranking Embedded Order Preserving Hashing (MREOPH). A manifold ranking loss is introduced to solve the issue of global topological structure preserving. An order preserving loss is introduced to ensure the consistency between manifold ranking and hamming ranking. A hypercubic quantization loss is introduced to learn discrete binary codes. The information theoretic regularization term is taken into consideration for preserving desirable properties of hash codes. Finally, we integrate them in a joint optimization framework for minimizing the information loss in each processing. Experimental results on three datasets for semantic search clearly demonstrate the effectiveness of the proposed method.

Robust Discrete Spectral Hashing for Large-Scale Image Semantic Indexing

In big data era, the ever-increasing image data has posed significant challenge on modern image retrieval. It is of great importance to index images with semantic keywords efficiently and effectively, especially confronted with fast-evolving property of the web. Learning-based hashing has shown its power in handling large-scale high-dimensional applications, such as image retrieval. Existing solutions normally separate the process of learning binary codes and hash functions into two independent stages to bypass challenge of the discrete constraints on binary codes. In this work, we propose a novel unsupervised hashing approach, namely robust discrete hashing (RDSH), to facilitate large-scale semantic indexing of image data. Specifically, RDSH simultaneously learns discrete binary codes as well as robust hash functions within a unified model. In order to suppress the influence of unreliable binary codes and learn robust hash functions, we also integrate a flexible $\ell _{2,p}$ loss with nonlinear kernel embedding to adapt to different noise levels. Finally, we devise an alternating algorithm to efficiently optimize RDSH model. Given a test image, we first conduct $r$ -nearest-neighbor search based on Hamming distance of binary codes, and then propagate semantic keywords of neighbors to the test image. Extensive experiments have been conducted on various real-world image datasets to show its superiority to the state-of-the-arts in large-scale semantic indexing.

Code Consistent Hashing Based on Information-Theoretic Criterion

Learning based hashing techniques have attracted broad research interests in the Big Media research area. They aim to learn compact binary codes which can preserve semantic similarity in the Hamming embedding. However, the discrete constraints imposed on binary codes typically make hashing optimizations very challenging. In this paper, we present a code consistent hashing ( CCH ) algorithm to learn discrete binary hash codes. To form a simple yet efficient hashing objective function, we introduce a new code consistency constraint to leverage discriminative information and propose to utilize the Hadamard code which favors an information-theoretic criterion as the class prototype. By keeping the discrete constraint and introducing an orthogonal constraint, our objective function can be minimized efficiently. Experimental results on three benchmark datasets demonstrate that the proposed CCH outperforms state-of-the-art hashing methods in both image retrieval and classification tasks, especially with short binary codes.

Optical signal processors based on a time-spectrum convolution

We show that a fiber-optics architecture conventionally used for microwave photonic filtering can implement a time-spectrum convolution (TSC) process of general interest for a wide range of fundamental optical signal processing and analysis operations. This process is practically implemented by temporally modulating a specially filtered broadband incoherent light source followed by propagation through a suitable linear dispersive medium. The TSC concept allows the time-domain realization of fundamental analog processing operations over both temporal and spectral intensity waveforms. Three particular, relevant signal-processing operations are proposed and experimentally demonstrated here to illustrate the broad potential of application of the TSC concept, namely time-integration, spectrum-integration, and time-frequency correlation of discrete binary codes.

Energy-efficient interspike interval codes

We investigate the energy efficiency of interspike interval (ISI) neural codes. Using the hypothesis that nature maximizes the energy efficiency of information processing, it is possible to derive neuronal firing frequencies which maximize the information/energy ratio. With simple assumptions about the encoded ISI and noise distributions, we show that ISI codes can be at least as efficient as discrete binary and frequency codes and that their predicted optimal frequencies are in the same range.