Large-scale Remote Sensing Image Retrieval Research Articles

Deep Contrastive Self-Supervised Hashing for Remote Sensing Image Retrieval

Hashing has been widely used for large-scale remote sensing image retrieval due to its outstanding advantages in storage and search speed. Recently, deep hashing methods, which produce discriminative hash codes by building end-to-end deep convolutional networks, have shown promising results. However, training these networks requires numerous labeled images, which are scarce and expensive in remote sensing datasets. In order to solve this problem, we propose a deep unsupervised hashing method, namely deep contrastive self-supervised hashing (DCSH), which uses only unlabeled images to learn accurate hash codes. It eliminates the need for label annotation by maximizing the consistency of different views generated from the same image. More specifically, we assume that the hash codes generated from different views of the same image are similar, and those generated from different images are dissimilar. On the basis of the hypothesis, we can develop a novel loss function containing the temperature-scaled cross-entropy loss and the quantization loss to train the developed deep network end-to-end, resulting in hash codes with semantic similarity preserved. Our proposed network contains four parts. First, each image is transformed into two different views using data augmentation. After that, they are fed into an encoder with the same shared parameters to obtain deep discriminate features. Following this, a hash layer converts the high-dimensional image representations into compact binary codes. Lastly, a novel hash function is introduced to train the proposed network end-to-end and thus guide generated hash codes with semantic similarity. Extensive experiments on two popular benchmark datasets of the UC Merced Land Use Database and the Aerial Image Dataset have demonstrated that our DCSH has significant superiority in remote sensing image retrieval compared with state-of-the-art unsupervised hashing methods.

Multiview Graph Convolutional Hashing for Multisource Remote Sensing Image Retrieval

Recently, hashing has been successfully applied for large-scale remote sensing image retrieval (LSRSIR) due to its advantage in terms of computation and storage. In LSRSIR, existing hashing methods mainly focus on single-source remotely sensed data. They cannot effectively fuse multisource remotely sensed data, which has a large potential for LSRSIR. To fulfill this gap, this letter proposes a novel deep hashing method, dubbed Multiview Graph Convolutional Hashing (MGCH) that can successfully fuse multisource remote sensing image. Since graph convolutional network (GCN) has been applied as an effective means that expresses and integrates relationships into features, MGCH applies a GCN to explore inherent structural similarity among multiview data, which will help to generate discriminative hash codes. An asymmetric scheme is developed that optimizes the proposed deep model in an end-to-end manner to improve training efficiency. We evaluate the proposed method by fusing two different kinds of RS images, i.e., multispectral (MUL) image and panchromatic (PAN) image. The experimental results on the dual-source RS image data set (DSRSID) show that the proposed MGCH outperforms state-of-the-art multiview hashing methods.

Deep Hashing Learning for Visual and Semantic Retrieval of Remote Sensing Images

Driven by the urgent demand for managing remote sensing big data, large-scale remote sensing image retrieval (RSIR) attracts increasing attention in the remote sensing field. In general, existing retrieval methods can be regarded as visual-based retrieval approaches that search and return a set of similar images to a given query image from a database. Although these retrieval methods have delivered good results, there is still a question that needs to be addressed: can we obtain the accurate semantic labels of the returned similar images to further help analyzing and processing imagery? To this end, in this article, we redefine the image retrieval problem as visual and semantic retrieval of images. Especially, we propose a novel deep hashing convolutional neural network (DHCNN) to retrieve similar images and classify their semantic labels simultaneously in a unified framework. In more detail, a convolutional neural network (CNN) is used to extract high-dimensional deep features. Then, a hash layer is perfectly inserted into the network to transfer the deep features into compact hash codes. In addition, a fully connected layer with a softmax function is performed on the hash layer to generate the probability distribution of each class. Finally, a loss function is elaborately designed to consider the label loss of each image and similarity loss of pairs of images simultaneously. Experimental results on three remote sensing data sets demonstrate that the proposed method can achieve state-of-art retrieval and classification performance.

Global Optimization: Combining Local Loss With Result Ranking Loss in Remote Sensing Image Retrieval

With the explosive growth of remote sensing big data, large-scale remote sensing image retrieval (RSIR) has become one of the most challenging tasks in data mining, attracting more and more attention from researchers. In recent years, deep learning has achieved fruitful results in RSIR. However, there are three major problems. The first problem is due to the imbalanced categories in RSIR data sets. Accordingly, a novel sample mining method is proposed. This method is based on top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> “misplaced” samples (Top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> MS) along the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> nearest neighbor decision boundary, rather than setting boundary threshold of sample mining and reducing the impact of artificial and objective factors on network training as usual. The second problem comes from the inconsistency between the loss reduction direction and the optimization direction. In response, we additionally propose a novel result ranking loss (RRL) according to the retrieval results in a batch. RRL is used as a plug-in and combined with other local losses. As a result, the retrieval accuracy improved significantly. For the third problem, processing a large number of high-resolution images will increase training time and cause network freeze. To solve the problem, this article proposes a global optimization model based on feature space and retrieval results (FSRR). More specifically, FSRR consists of local loss based on feature space and RRL based on retrieval results, and can be optimized in an end-to-end manner. Experimental results show that our proposed algorithm achieves the best performance when compared with other state-of-the-art deep RSIR algorithms on two benchmark data sets.

Hashing Nets for Hashing: A Quantized Deep Learning to Hash Framework for Remote Sensing Image Retrieval

Fast and accurate remote sensing image retrieval from large data archives has been an important research topic in the remote sensing research literature. Recently, hashing-based remote sensing image retrieval has attracted extreme attention because of its efficient search capabilities. Especially, deep remote sensing image hashing algorithms have been developed based on convolutional neural networks (CNNs) and have shown effective retrieval performance. However, implementing a deep hashing network tends to be highly expensive in terms of storage space and computing resources to be suitable for on-orbit remote sensing image retrieval, which usually operates on resource-limited devices such as satellites and unmanned aerial vehicles (UAVs). To address this limitation, we propose to hash a deep network that in turn hashes remote sensing images. Specifically, we develop a quantized deep learning to hash (QDLH) framework for large-scale remote sensing image retrieval. The weights and activation functions in the QDLH framework are binarized to low-bit representations, which require comparatively much less storage space and computing resources. The QDLH results in a lightweight deep neural network for effective remote sensing image hashing. We conduct extensive experiments on two public remote sensing image data sets by incorporating several state-of-the-art network architectures into our QDLH methodology for remote sensing image hashing. The experimental results demonstrate that the proposed QDLH is effective in saving hardware resources in terms of both storage and computation. Moreover, superior remote sensing image retrieval performance is also achieved by our QDLH, compared with state-of-the-art deep remote sensing image hashing methods.

Distribution Consistency Loss for Large-Scale Remote Sensing Image Retrieval

Remote sensing images are featured by massiveness, diversity and complexity. These features put forward higher requirements for the speed and accuracy of remote sensing image retrieval. The extraction method plays a key role in retrieving remote sensing images. Deep metric learning (DML) captures the semantic similarity information between data points by learning embedding in vector space. However, due to the uneven distribution of sample data in remote sensing image datasets, the pair-based loss currently used in DML is not suitable. To improve this, we propose a novel distribution consistency loss to solve this problem. First, we define a new way to mine samples by selecting five in-class hard samples and five inter-class hard samples to form an informative set. This method can make the network extract more useful information in a short time. Secondly, in order to avoid inaccurate feature extraction due to sample imbalance, we assign dynamic weight to the positive samples according to the ratio of the number of hard samples and easy samples in the class, and name the loss caused by the positive sample as the sample balance loss. We combine the sample balance of the positive samples with the ranking consistency of the negative samples to form our distribution consistency loss. Finally, we built an end-to-end fine-tuning network suitable for remote sensing image retrieval. We display comprehensive experimental results drawing on three remote sensing image datasets that are publicly available and show that our method achieves the state-of-the-art performance.

Large-Scale Remote Sensing Image Retrieval Based on Semi-Supervised Adversarial Hashing

Remote sensing image retrieval (RSIR), a superior content organization technique, plays an important role in the remote sensing (RS) community. With the number of RS images increases explosively, not only the retrieval precision but also the retrieval efficiency is emphasized in the large-scale RSIR scenario. Therefore, the approximate nearest neighborhood (ANN) search attracts the researchers’ attention increasingly. In this paper, we propose a new hash learning method, named semi-supervised deep adversarial hashing (SDAH), to accomplish the ANN for the large-scale RSIR task. The assumption of our model is that the RS images have been represented by the proper visual features. First, a residual auto-encoder (RAE) is developed to generate the class variable and hash code. Second, two multi-layer networks are constructed to regularize the obtained latent vectors using the prior distribution. These two modules mentioned are integrated under the generator adversarial framework. Through the minimax learning, the class variable would be a one-hot-like vector while the hash code would be the binary-like vector. Finally, a specific hashing function is formulated to enhance the quality of the generated hash code. The effectiveness of the hash codes learned by our SDAH model was proved by the positive experimental results counted on three public RS image archives. Compared with the existing hash learning methods, the proposed method reaches improved performance.

Learning Source-Invariant Deep Hashing Convolutional Neural Networks for Cross-Source Remote Sensing Image Retrieval

Due to the urgent demand for remote sensing big data analysis, large-scale remote sensing image retrieval (LSRSIR) attracts increasing attention from researchers. Generally, LSRSIR can be divided into two categories as follows: uni-source LSRSIR (US-LSRSIR) and cross-source LSRSIR (CS-LSRSIR). More specifically, US-LSRSIR means the inquiry remote sensing image and images in the searching data set come from the same remote sensing data source, whereas CS-LSRSIR is designed to retrieve remote sensing images with a similar content to the inquiry remote sensing image that are from a different remote sensing data source. In the literature, US-LSRSIR has been widely exploited, but CS-LSRSIR is rarely discussed. In practical situations, remote sensing images from different kinds of remote sensing data sources are continually increasing, so there is a great motivation to exploit CS-LSRSIR. Therefore, this paper focuses on CS-LSRSIR. To cope with CS-LSRSIR, this paper proposes source-invariant deep hashing convolutional neural networks (SIDHCNNs), which can be optimized in an end-to-end manner using a series of well-designed optimization constraints. To quantitatively evaluate the proposed SIDHCNNs, we construct a dual-source remote sensing image data set that contains eight typical land-cover categories and 10 000 dual samples in each category. Extensive experiments show that the proposed SIDHCNNs can yield substantial improvements over several baselines involving the most recent techniques.

Large-Scale Remote Sensing Image Retrieval by Deep Hashing Neural Networks

As one of the most challenging tasks of remote sensing big data mining, large-scale remote sensing image retrieval has attracted increasing attention from researchers. Existing large-scale remote sensing image retrieval approaches are generally implemented by using hashing learning methods, which take handcrafted features as inputs and map the high-dimensional feature vector to the low-dimensional binary feature vector to reduce feature-searching complexity levels. As a means of applying the merits of deep learning, this paper proposes a novel large-scale remote sensing image retrieval approach based on deep hashing neural networks (DHNNs). More specifically, DHNNs are composed of deep feature learning neural networks and hashing learning neural networks and can be optimized in an end-to-end manner. Rather than requiring to dedicate expertise and effort to the design of feature descriptors, we can automatically learn good feature extraction operations and feature hashing mapping under the supervision of labeled samples. To broaden the application field, DHNNs are evaluated under two representative remote sensing cases: scarce and sufficient labeled samples. To make up for a lack of labeled samples, DHNNs can be trained via transfer learning for the former case. For the latter case, DHNNs can be trained via supervised learning from scratch with the aid of a vast number of labeled samples. Extensive experiments on one public remote sensing image data set with a limited number of labeled samples and on another public data set with plenty of labeled samples show that the proposed remote sensing image retrieval approach based on DHNNs can remarkably outperform state-of-the-art methods under both of the examined conditions.

Multiple Feature Hashing Learning for Large-Scale Remote Sensing Image Retrieval

Driven by the urgent demand of remote sensing big data management and knowledge discovery, large-scale remote sensing image retrieval (LSRSIR) has attracted more and more attention. As is well known, hashing learning has played an important role in coping with big data mining problems. In the literature, several hashing learning methods have been proposed to address LSRSIR. Until now, existing LSRSIR methods take only one type of feature descriptor as the input of hashing learning methods and ignore the complementary effects of multiple features, which may represent remote sensing images from different aspects. Different from the existing LSRSIR methods, this paper proposes a flexible multiple-feature hashing learning framework for LSRSIR, which takes multiple complementary features as the input and learns the hybrid feature mapping function, which projects multiple features of the remote sensing image to the low-dimensional binary (i.e., compact) feature representation. Furthermore, the compact feature representations can be directly utilized in LSRSIR with the aid of the hamming distance metric. In order to show the superiority of the proposed multiple feature hashing learning method, we compare the proposed approach with the existing methods on two publicly available large-scale remote sensing image datasets. Extensive experiments demonstrate that the proposed approach can significantly outperform the state-of-the-art approaches.

Partial Randomness Hashing for Large-Scale Remote Sensing Image Retrieval

With the rapid progress of satellite and aerial vehicle technologies, large-scale remote sensing (RS) image retrieval has recently become an important research issue in geosciences. Hashing-based searching approaches have been widely employed in content-based image retrieval tasks. However, most hash schemes compromise between learning efficiency and retrieval accuracy, and can thus barely satisfy the precise requirements in RS data analysis. To address these shortcomings, we introduce a partial randomness scheme for learning hash functions, which is referred to as partial randomness hashing (PRH). Specifically, for constructing hash functions, a part of model parameter values are randomly generated and the remaining ones are trained based on RS images. The randomness enables an efficient hash function construction and the trained model parameters encode characteristics from RS images. The coplay between random and trained model parameters results in both efficient and effective learning scheme for constructing hash functions. Experiments on two large public RS image data sets have shown that our PRH method outperforms state of the arts in terms of both learning efficiency and retrieval accuracy.