Remote Sensing Image Classification Research Articles

Integrating IoT and honey badger algorithm based ensemble learning for accurate vehicle detection and classification

The fusion of Internet of Things (IoT) and deep learning (DL) methods has proven valuable in automating vehicle detection and classification tasks on remote sensing images (RSI). This technology has broad applications, including traffic monitoring, urban planning, and transportation management. Recent advancements have demonstrated the efficacy of DL models like convolutional neural networks (CNN) in RSI classification tasks. In this aspect, this study proposes a novel honey badger optimization algorithm with an ensemble learning-based vehicle detection and classification (HBOAEL-VDC) technique. The purpose of the study is to design ensemble DL models for accurate vehicle identification and classification processes. To accomplish this, the HBOAEL-VDC technique makes use of an improved RetinaNet model for the detection of objects, i.e., vehicles on the RSI. Moreover, the classification of detected vehicles takes place using the ensemble learning process, comprising three DL models, namely gated recurrent unit (GRU), long short-term memory (LSTM), and bidirectional long short-term memory (BiLSTM). Furthermore, the HBOA-based parameter tuning process gets carried out to adjust the hyperparameter values of the DL models and thereby improve the classification results. The simulation outcome of the HBOAEL-VDC approach is tested on benchmark RSI databases. The experimentation outcomes reported the enhanced vehicle classification performance of the HBOAEL-VDC approach over other recent DL models.

Enhancing hyperspectral remote sensing image classification using robust learning technique

Advanced sensor tech integrates into diverse applications, including remote sensing, robotics, and IoT. Combining artificial intelligence (AI) with sensors enhances their capabilities, creating smart sensors, revolutionizing remote sensing and Internet of Things (IoT). This synergy forms a potent technology in the field. This study carries out a comprehensive analysis of the progress made in Hyperspectral sensors and AI-based classification techniques that are employed in remote sensing fields that utilize hyperspectral images. The classification of images obtained from Hyperspectral Sensors (HSS) has emerged as a prominent research subject within the domain of remote sensing. HSS offer a wealth of information across numerous spectral bands, supporting diverse applications such as land cover classification, environmental monitoring, agricultural assessment, change detection, and more. However, the abundance of data present in HSS also poses the challenge called the curse of dimensionality. The reduction of data dimensionality is crucial before applying any machine learning model to achieve optimal results. The present study introduces a new hybrid strategy combining the Back-Propagation algorithm with a variable adaptive momentum (BPVAM) and principal component analysis (PCA) for the purpose of classifying hyperspectral images. PCA is first applied to obtain an optimal set of discriminative features by eliminating highly correlated and redundant features. These features are then fed into the BPVAM model for classification. The addition of the momentum term in the weight update equation of the backpropagation algorithm helped achieve faster convergence with high accuracy. The proposed model was subjected to evaluation through experiments conducted on two benchmark datasets. These results indicated that the hybrid model based on BPVAM with PCA is an efficient technique for HSS classification.

CamoNet: A Target Camouflage Network for Remote Sensing Images Based on Adversarial Attack

Object detection algorithms based on convolutional neural networks (CNNs) have achieved remarkable success in remote sensing images (RSIs), such as aircraft and ship detection, which play a vital role in military and civilian fields. However, CNNs are fragile and can be easily fooled. There have been a series of studies on adversarial attacks for image classification in RSIs. However, the existing gradient attack algorithms designed for classification cannot achieve excellent performance when directly applied to object detection, which is an essential task in RSI understanding. Although we can find some works on adversarial attacks for object detection, they are weak in concealment and easily detected by the naked eye. To handle these problems, we propose a target camouflage network for object detection in RSIs, called CamoNet, to deceive CNN-based detectors by adding imperceptible perturbation to the image. In addition, we propose a detection space initialization strategy to maximize the diversity in the detector’s outputs among the generated samples. It can enhance the performance of the gradient attack algorithms in the object detection task. Moreover, a key pixel distillation module is employed, which can further reduce the modified pixels without weakening the concealment effect. Compared with several of the most advanced adversarial attacks, the proposed attack has advantages in terms of both peak signal-to-noise ratio (PSNR) and attack success rate. The transferability of the proposed target camouflage network is evaluated on three dominant detection algorithms (RetinaNet, Faster R-CNN, and RTMDet) with two commonly used remote sensing datasets (i.e., DOTA and DIOR).

Local Differential Privacy Based Membership-Privacy-Preserving Federated Learning for Deep-Learning-Driven Remote Sensing

With the development of deep learning, image recognition based on deep learning is now widely used in remote sensing. As we know, the effectiveness of deep learning models significantly benefits from the size and quality of the dataset. However, remote sensing data are often distributed in different parts. They cannot be shared directly for privacy and security reasons, and this has motivated some scholars to apply federated learning (FL) to remote sensing. However, research has found that federated learning is usually vulnerable to white-box membership inference attacks (MIAs), which aim to infer whether a piece of data was participating in model training. In remote sensing, the MIA can lead to the disclosure of sensitive information about the model trainers, such as their location and type, as well as time information about the remote sensing equipment. To solve this issue, we consider embedding local differential privacy (LDP) into FL and propose LDP-Fed. LDP-Fed performs local differential privacy perturbation after properly pruning the uploaded parameters, preventing the central server from obtaining the original local models from the participants. To achieve a trade-off between privacy and model performance, LDP-Fed adds different noise levels to the parameters for various layers of the local models. This paper conducted comprehensive experiments to evaluate the framework’s effectiveness on two remote sensing image datasets and two machine learning benchmark datasets. The results demonstrate that remote sensing image classification models are susceptible to MIAs, and our framework can successfully defend against white-box MIA while achieving an excellent global model.

Comparing Three Methods of Selecting Training Samples in Supervised Classification of Multispectral Remote Sensing Images.

Selecting training samples is crucial in remote sensing image classification. In this paper, we selected three images-Sentinel-2, GF-1, and Landsat 8-and employed three methods for selecting training samples: grouping selection, entropy-based selection, and direct selection. We then used the selected training samples to train three supervised classification models-random forest (RF), support-vector machine (SVM), and k-nearest neighbor (KNN)-and evaluated the classification results of the three images. According to the experimental results, the three classification models performed similarly. Compared with the entropy-based method, the grouping selection method achieved higher classification accuracy using fewer samples. In addition, the grouping selection method outperformed the direct selection method with the same number of samples. Therefore, the grouping selection method performed the best. When using the grouping selection method, the image classification accuracy increased with the increase in the number of samples within a certain sample size range.

Multi-scale fusion for few-shot remote sensing image classification

ABSTRACT Few-shot image classification is currently valuable research in the field of remote sensing image applications. The difficulty in obtaining training data for a large number of labelled remote sensing images leads to difficulties in applying traditional deep learning-based remote sensing image classification methods. Existing few-shot remote sensing image classification methods usually use a large number of base datasets to train network models in the pre-training phase and perform few-shot classification tasks after model fine-tuning on new datasets in the meta-testing phase. However, an insufficient number of remote sensing images lead to inaccurate features being extracted by the model, and the large amount of new class data for fine-tuning is also not easily accessible. In addition, using a pre-trained model to extract features from the new dataset will create a”negative transfer” problem. In this paper, we aim to address the above challenges in two ways. First, we use a subset of ImageNet, a readily available few-shot natural image dataset, for model pre-training, and eliminate the fine-tuning operation to simulate a real-world application scenario where remote sensing data is extremely scarce. Second, due to the significant differences in scale and style between the ImageNet dataset and the remote sensing dataset, which lead to a serious negative transfer problem in the model, we design a multi-scale feature fusion module to comprehensively decide the labels of query samples considering all scales to compensate for the scale differences and alleviate the”negative transfer” problem. We conducted experiments on four benchmark remote sensing datasets and achieved satisfactory performance.

Research on Land Use Classification Method of High Resolution Remote Sensing Image Based on SVM

Considering the high spatial resolution remote sensing image has huge amounts of data and complex spectral distribution and the characteristics of the space characteristics of the rich, in combination with support vector machine (SVM) in tackling small sample, nonlinear and high dimensional pattern recognition problems show unique advantages, in this chapter will experiment data by using support vector machine (SVM) for high spatial resolution remote sensing image classification.

RIC-CNN: Rotation-Invariant Coordinate Convolutional Neural Network

Due to the lack of rotation invariance in traditional convolution operations, even acting a slight rotation on the input can severely degrade the performance of Convolutional Neural Networks (CNNs). To address this, we propose a Rotation-Invariant Coordinate Convolution (RIC-C), which achieves natural invariance to arbitrary rotations around the input center without additional trainable parameters or data augmentation. We first evaluate the rotational invariance of RIC-C using the MNIST dataset and compare its performance with most previous rotation-invariant CNN models. RIC-C achieves state-of-the-art classification on the MNIST-rot test set without data augmentation and with lower computational costs. Then, the interchangeability of RIC-C with traditional convolution operations is demonstrated by seamlessly integrating it into common CNN models like VGG, ResNet, and DenseNet. We conduct remote sensing image classification on the NWPU VHR-10, MTARSI and AID datasets and patch matching experiments on the UBC benchmark dataset, showing that RIC-C significantly enhances the performance of CNN models across different applications, especially when training data is limited. Our codes can be downloaded from https://github.com/HanlinMo/Rotation-Invariant-Coordinate-Convolutional-Neural-Network.git.

Land Cover Classification of Remote Sensing Images Based on Hierarchical Convolutional Recurrent Neural Network

Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have gained improved results in remote sensing image data classification. Multispectral image classification can benefit from the rich spectral information extracted by these models for land cover classification. This paper proposes a classification model called a hierarchical convolutional recurrent neural network (HCRNN) to combine the CNN and RNN modules for pixel-level classification of multispectral remote sensing images. In the HCRNN model, the original 13-band information from Sentinel-2 is transformed into a 1D multispectral sequence using a fully connected layer. It is then reshaped into a 3D multispectral feature matrix. The 2D-CNN features are extracted and used as inputs to the corresponding hierarchical RNN. The feature information at each level is adapted to the same convolution size. This network structure fully leverages the advantages of CNNs and RNNs to extract temporal and spatial features from the spectral data, leading to high-precision pixel-level multispectral remote sensing image classification. The experimental results demonstrate that the overall accuracy of the HCRNN model on the Sentinel-2 dataset reaches 97.62%, which improves the performance by 1.78% compared to the RNN model. Furthermore, this study focused on the changes in forest cover in the study area of Laibin City, Guangxi Zhuang Autonomous Region, which was 7997.1016 km2, 8990.4149 km2, and 8103.0020 km2 in 2017, 2019, and 2021, respectively, with an overall trend of a small increase in the area covered.

Object-oriented polarimetric SAR image classification via the combination of a pixel-based classifier and a region growing technique

ABSTRACT Land-cover type interpretation by the use of remote sensing image classification techniques is always a hot topic. In this paper, an object-oriented method is presented for fully polarimetric synthetic aperture radar (SAR) image classification. Differing from most of the traditional object-oriented classification algorithms, the proposed method employs an innovative classification strategy that combines a pixel-based classifier and a region growing technique. Firstly, taking each individual pixel as a seed pixel, the homogeneous areas are extracted by a region growing technique. Then, using the information of the pixel-based classification result, the pixels located in each homogeneous area are all assigned to a certain class. Finally, the majority voting strategy is deployed to determine the final class label of each pixel. The experiments conducted on two fully polarimetric SAR images reveal that the proposed classification scheme can obtain pleasing classification accuracy and can provide the classification maps with more homogeneous regions than pixel-based classification.

A Hybrid Binary Dragonfly Algorithm with an Adaptive Directed Differential Operator for Feature Selection

Feature selection is a typical multiobjective problem including two conflicting objectives. In classification, feature selection aims to improve or maintain classification accuracy while reducing the number of selected features. In practical applications, feature selection is one of the most important tasks in remote sensing image classification. In recent years, many metaheuristic algorithms have attempted to explore feature selection, such as the dragonfly algorithm (DA). Dragonfly algorithms have a powerful search capability that achieves good results, but there are still some shortcomings, specifically that the algorithm’s ability to explore will be weakened in the late phase, the diversity of the populations is not sufficient, and the convergence speed is slow. To overcome these shortcomings, we propose an improved dragonfly algorithm combined with a directed differential operator, called BDA-DDO. First, to enhance the exploration capability of DA in the later stages, we present an adaptive step-updating mechanism where the dragonfly step size decreases with iteration. Second, to speed up the convergence of the DA algorithm, we designed a new differential operator. We constructed a directed differential operator that can provide a promising direction for the search, then sped up the convergence. Third, we also designed an adaptive paradigm to update the directed differential operator to improve the diversity of the populations. The proposed method was tested on 14 mainstream public UCI datasets. The experimental results were compared with seven representative feature selection methods, including the DA variant algorithms, and the results show that the proposed algorithm outperformed the other representative and state-of-the-art DA variant algorithms in terms of both convergence speed and solution quality.

Active learning based on similarity level histogram and adaptive-scale sampling for very high resolution image classification

In remote sensing image classification, active learning aims to obtain an excellent classification model by selecting informative or representative training samples. However, due to the complexity of remote sensing images, the same class of ground objects usually have different spectral representations. The existing active learning methods may not take into account diverse representations of the same targets, which leads to a possible lack of intra-class diversity in the collected samples. To alleviate this problem, we propose an active learning method based on similarity level histogram (SLH) and adaptive-scale sampling to improve very high resolution remote sensing image classification. Specifically, we construct a SLH for each class of ground objects to effectively consider the intra-class diversity of the same target. To avoid the problem of sample imbalance caused by over-sampling or under-sampling, we design an adaptive-scale sampling strategy. Then, we utilize active learning to mine representative samples from each SLH warehouse according to adaptive-scale sampling strategies until the iteration condition is satisfied. Experiments show that the proposed algorithm can achieve better classification performance with limited training samples and is competitive with other methods based on four sets of publicly available data.

Classification of multi-modal remote sensing images based on knowledge graph

ABSTRACT With the development of remote sensing (RS) technology, single-modal data alone has gradually become difficult to meet the requirement for high accuracy of RS image classification. As a result, multi-modal RS image classification has become a hot research topic. However, multi-modal RS image classification faces challenges in effectively utilizing advanced semantic information regarding the relationships between land cover classes and extracting discriminative features from the data. Based on this, a multi-modal RS image classification method based on knowledge graph (KG) is proposed. Firstly, graph topology is used to align the hyperspectral image (HSI) and multispectral image (MSI) features extracted by graph convolution networks. Constraint alignment is performed on different feature graphs to reduce the difficulty of fusion and the false recognition rate. Then, we use self-attention and cross-attention to purposefully fuse HSI and MSI to obtain discriminative features rich in two modal information and achieve feature weighted fusion. Finally, a KG based on object spatial relationships is constructed to obtain spatial relationships between different classes to assist in multi-modal RS image classification. The experimental results on the Houston and Ausburg datasets demonstrate that the proposed method achieves overall accuracy of 90.40% and 90.85%, respectively, both of which are more than 3% higher than existing classification methods. The results indicate that our method has better classification performance and can provide a useful reference for RS image classification research.

Application of Deep Learning in Multitemporal Remote Sensing Image Classification

The rapid advancement of remote sensing technology has significantly enhanced the temporal resolution of remote sensing data. Multitemporal remote sensing image classification can extract richer spatiotemporal features. However, this also presents the challenge of mining massive data features. In response to this challenge, deep learning methods have become prevalent in machine learning and have been widely applied in remote sensing due to their ability to handle large datasets. The combination of remote sensing classification and deep learning has become a trend and has developed rapidly in recent years. However, there is a lack of summary and discussion on the research status and trends in multitemporal images. This review retrieved and screened 170 papers and proposed a research framework for this field. It includes retrieval statistics from existing research, preparation of multitemporal datasets, sample acquisition, an overview of typical models, and a discussion of application status. Finally, this paper discusses current problems and puts forward prospects for the future from three directions: adaptability between deep learning models and multitemporal classification, prospects for high-resolution image applications, and large-scale monitoring and model generalization. The aim is to help readers quickly understand the research process and application status of this field.

Deep Learning in Image Recognition

With the rapid development of high-tech in China, the field of artificial intelligence has also been promoted. Image recognition is an important topic in the field of artificial intelligence, which mainly includes two modules: classification recognition and feature extraction; At the same time, as an important research direction of AI, deep learning has made rapid progress in recent years. It has been widely used in image recognition, speech recognition and other fields and has achieved great success. This paper analyzes the application of deep learning in image recognition, mainly from the aspects of face recognition, remote sensing image classification, etc. Its purpose is to provide help for relevant practitioners, so as to promote the development of image recognition in the tide of AI development.

Information Leakage in Deep Learning-Based Hyperspectral Image Classification: A Survey

In deep learning-based hyperspectral remote sensing image classification tasks, random sampling strategies are typically used to train model parameters for testing and evaluation. However, this approach leads to strong spatial autocorrelation between the training set samples and the surrounding test set samples, and some unlabeled test set data directly participate in the training of the network. This leaked information makes the model overly optimistic. Models trained under these conditions tend to overfit to a single dataset, which limits the range of practical applications. This paper analyzes the causes and effects of information leakage and summarizes the methods from existing models to mitigate the effects of information leakage. Specifically, this paper states the main issues in this area, where the issue of information leakage is addressed in detail. Second, some algorithms and related models used to mitigate information leakage are categorized, including reducing the number of training samples, using spatially disjoint sampling strategies, few-shot learning, and unsupervised learning. These models and methods are classified according to the sample-related phase and the feature extraction phase. Finally, several representative hyperspectral image classification models experiments are conducted on the common datasets and their effectiveness in mitigating information leakage is analyzed.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

Classification of hyperspectral images is an essential application of deep learning techniques. However, standard deep learning approaches require a large number of labelled samples for model training, and classification performance can be enhanced. In this study, we propose the use of residual generative adversarial networks for the classification of hyperspectral images based on deep learning Convolutional Neural Networks, a large number of labelled samples, and the high classification accuracy required for the classification task. The method is based on generative adversarial networks, with a 6-layer residual network containing up-sampling and convolutional layers replacing the inverse convolutional layer network structure of the generator to enhance the data generation capability, and an 18-layer residual convolutional network replacing the convolutional layer network structure of the discriminator to enhance the feature extraction capability. The hyperspectral image classification method with residual generation adversarial network enhances the information exchange between the shallow network and the deep network by adding a residual structure to the network, extracts the deep features of hyperspectral images and improves the accuracy of hyperspectral image classification. Extensive experiments on several benchmark hyperspectral datasets have shown that the method outperforms comparative methods by 0.7% to 22.3% on OA.

Improved CNN-based CatBoost model for license plate remote sensing image classification

The rapid development of artificial intelligence technology has brought profound technical changes to many traditional industries, among which intelligent transportation has become a hot spot for development in the traditional transportation field. Remote sensing license plate image recognition technology is extensively emphasized in such domains as intelligent transportation and intelligent vehicle management. However, in the practical traffic environment, low visibility scenes caused by complex environmental factors such as rain, snow, haze and cloudy days influence the recognition and classification of license plates, while the distortion of license plate images that may be caused by irregular movements of vehicles bring challenges to license plate recognition classification. The CNN-CatBoost model proposed in this paper divides the license plate recognition classification into two stages. The first stage uses the excellent performance of convolutional neural network in processing image data to extract various license plate image features; the second stage uses the CatBoost module to further process the image feature data and finally obtain the remote sensing license plate information. The model achieves outstanding results in the experiments and has practical application value. Through comparison with other network models, the CNN-CatBoost model proposed in this paper has superior performance.

Study on High-resolution Remote Sensing Image Scene Classification Using Transfer Learning

Remote sensing image classification (RSIC) has been increasingly concerned and becoming a challenging task. Recently, deep convolutional neural networks (DCNN) offer the effective classification method include the capacity to handle high-dimensional data and to distinguish classes with very complex characteristics on the remote sensing community. However, the focus of these methods is on publicly available data sets in the field of remote sensing, there are few studies on RSIC composed of different benchmark datasets, which the complexity, diversity, and similarity of data greatly increase the difficulty of classification. In this paper, we reconstructed and selected one new dataset from two standard benchmark remote sensing datasets: UC Merged Land-Use and NWPU-RESISC45. We utilize three transfer learning frameworks to extract the high-level feature map and feed feature information into the proposed model for partial and full fine-tuning. Data augmentation technology is used to increase the number of training samples and dropout strategies to prevent overfitting. The experimental results demonstrate that the proposed methodology achieved remarkable performance in scene classification of overall accuracy: 90.1%,91.0%,93.3 with VggNet, DesNet, InceptionNet, respectively.

Sphere2Vec: A general-purpose location representation learning over a spherical surface for large-scale geospatial predictions

Generating learning-friendly representations for points in space is a fundamental and long-standing problem in machine learning. Recently, multi-scale encoding schemes (such as Space2Vec and NeRF) were proposed to directly encode any point in 2D or 3D Euclidean space as a high-dimensional vector, and has been successfully applied to various (geo)spatial prediction and generative tasks. However, all current 2D and 3D location encoders are designed to model point distances in Euclidean space. So when applied to large-scale real-world GPS coordinate datasets (e.g., species or satellite images taken all over the world), which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D). To solve these problems, we propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface. We developed a unified view of distance-reserving encoding on spheres based on the Double Fourier Sphere (DFS). We also provide theoretical proof that the Sphere2Vec encoding preserves the spherical surface distance between any two points, while existing encoding schemes such as Space2Vec and NeRF do not. Experiments on 20 synthetic datasets show that Sphere2Vec can outperform all baseline models including the state-of-the-art (SOTA) 2D location encoder (i.e., Space2Vec) and 3D encoder NeRF on all these datasets with up to 30.8% error rate reduction. We then apply Sphere2Vec to three geo-aware image classification tasks - fine-grained species recognition, Flickr image recognition, and remote sensing image classification. Results on 7 real-world datasets show the superiority of Sphere2Vec over multiple 2D and 3D location encoders on all three tasks. Further analysis shows that Sphere2Vec outperforms other location encoder models, especially in the polar regions and data-sparse areas because of its nature for spherical surface distance preservation. Code and data of this work are available at https://gengchenmai.github.io/sphere2vec-website/.