Remote Sensing Image Classification Research Articles

An Efficient Multispectral Image Classification and Optimization Using Remote Sensing Data

A significant amount of effort and cost is required to collect training samples for remote sensing image classifications. The study of remote sensing and how to read multispectral images is becoming more important. High-dimensional multispectral images are created by the various bands that show how materials behave. The need for more information about things and the improvement of sensor resolutions have led to the creation of multispectral data with a higher size. In recent years, it has been shown that the high dimensionality of these data makes it hard to preprocess them in multiple ways. Recent research has demonstrated that one of the most crucial methods to address this issue is by adopting a variety of learning strategies. But as the data gets more complicated, these methodologies are not adequate to support. The proposed methodology shows that the classification experiment using remote sensing images indicates the maximum likelihood classifier with different deep learning models; weight vector (WV) AdaBoost and ADAM can greatly limit overfitting, and it obtains high classification accuracy. Proposed VGG16 and Inception v3 increase classification accuracy along with optimization process produce 96.08%.

Cost-effective land cover classification for remote sensing images

Land cover maps are of vital importance to various fields such as land use policy development, ecosystem services, urban planning and agriculture monitoring, which are mainly generated from remote sensing image classification techniques. Traditional land cover classification usually needs tremendous computational resources, which often becomes a huge burden to the remote sensing community. Undoubtedly cloud computing is one of the best choices for land cover classification, however, if not managed properly, the computation cost on the cloud could be surprisingly high. Recently, cutting the unnecessary computation long tail has become a promising solution for saving cost in the cloud. For land cover classification, it is generally not necessary to achieve the best accuracy and 85% can be regarded as a reliable land cover classification. Therefore, in this paper, we propose a framework for cost-effective remote sensing classification. Given the desired accuracy, the clustering algorithm can stop early for cost-saving whilst achieving sufficient accuracy for land cover image classification. Experimental results show that achieving 85%-99.9% accuracy needs only 27.34%-60.83% of the total cloud computation cost for achieving a 100% accuracy. To put it into perspective, for the US land cover classification example, the proposed approach can save over $1,593,490.18 for the government in each single-use when the desired accuracy is 90%.

Geometric-Spectral Reconstruction Learning for Multi-Source Open-Set Classification With Hyperspectral and LiDAR Data

Dear Editor, This letter presents an open-set classification method of remote sensing images (RSIs) based on geometric-spectral reconstruction learning. More specifically, in order to improve the ability of RSI classification model to adapt to the open-set environment, an open-set classification method based on geometric and spectral feature fusion is proposed. This method proposes to realize RSI open-set classification based on geometric and spectral features with hyper-spectral and light detection and ranging (LiDAR) data for the first time. In a variety of data sources of remote sensing, hyperspectral images (HSIs) and LiDAR data can provide rich spectral and geometric information for target objects. This letter combines both HSIs and LiDAR data to realize the recognition of unknown classes and the classification of known classes. Experiments show that the proposed method is better than previous state-of-the-art methods.

Climate Change Analysis Based on Satellite Multispectral Image Processing in Feature Selection Using Reinforcement Learning

Currently private and government agencies use remote sensing images (RSI) for various applications from military applications to agriculture growth. The images can be multispectral, panchromatic, ultra-spectral, or hyperspectral of terra bytes. RSI classification is considered one important application for remote sensing. Climate change detection especially affects numerous aspects of day-to-day lives, for instance, forestry management, weather forecasting, transportation, agriculture, road condition monitoring, and the detection of the natural atmosphere. Conversely, certain research works had a focus on classification of actual weather phenomenon images, generally depending on visual observations from humans. The conventional artificial visual difference between weather phenomena will take more time and error-prone. This paper develops a new reinforcement learning based climate change analysis on satellite multispectral image processing (RLCCA-SMSIP) technique. In order to properly determine climate change, the RLCCA-SMSIP technique employs residual network (ResNet-101) model for feature extraction. Next, deep reinforcement learning (DRL) approach is utilized for climate classification. Finally, parameter selection of the RLCCA-SMSIP technique involves sine cosine algorithm (SCA) for DRL model. For assuring the enhanced outcomes of the presented RLCCA-SMSIP model, comprehensive comparison results are assessed. The obtained values denote the supremacy of the RLCCA-SMSIP model on climate classification.

A General Self-Supervised Framework for Remote Sensing Image Classification

This paper provides insights into the interpretation beyond simply combining self-supervised learning (SSL) with remote sensing (RS). Inspired by the improved representation ability brought by SSL in natural image understanding, we aim to explore and analyze the compatibility of SSL with remote sensing. In particular, we propose a self-supervised pre-training framework for the first time by applying the masked image modeling (MIM) method to RS image research in order to enhance its efficacy. The completion proxy task used by MIM encourages the model to reconstruct the masked patches, and thus correlate the unseen parts with the seen parts in semantics. Second, in order to figure out how pretext tasks affect downstream performance, we find the attribution consensus of the pre-trained model and downstream tasks toward the proxy and classification targets, which is quite different from that in natural image understanding. Moreover, this transferable consensus is persistent in cross-dataset full or partial fine-tuning, which means that SSL could boost general model-free representation beyond domain bias and task bias (e.g., classification, segmentation, and detection). Finally, on three publicly accessible RS scene classification datasets, our method outperforms the majority of fully supervised state-of-the-art (SOTA) methods with higher accuracy scores on unlabeled datasets.

Towards explainable AI for hyperspectral image classification in Edge Computing environments

Several Remote Sensing applications in Hyperspectral Imaging rely on Artificial Intelligence techniques, particularly Deep Neural Networks, which often outperform other algorithms. However, despite their effectiveness, these techniques are considered opaque for the non-linearity of the underlying model, making a conscious results interpretation difficult even for experts in the application field. The present work aims to describe the results of our experiments toward Explainable Artificial Intelligence techniques for hyperspectral remote sensing image classification in Edge Computing environments. The proposed technique extends the traditional 2D Gradient-weighted Class Activation Mapping (Grad-CAM) to 3D Convolutional Neural Networks. Moreover, we use spectral-cumulative Grad-CAM and class probability as complementary methods. The experimental results confirm that the observer can interpret the choices made within neural network layers by visualizing the activation volumes provided by the proposed method.

A meta-methodology for preserving narrow objects when using spatial contextual classifiers for remote sensing data

ABSTRACT In the field of land-cover classification with remote sensing images, methods that analyse purely the spectral information of individual pixels generally produce noisy results, due to salt-and-pepper effects. The use of methods that also incorporate spatial contextual information into classification, often defined as contextual or spectral-spatial approaches, is an effective strategy for reducing the occurrence of punctual noises and, consequently, improving accuracy. However, contextual methods still present a critical limitation: an over smoothing performance on certain classes can cause the loss of details on important spatial structures. They may overlook salient punctual and linear objects that can be efficiently classified using purely spectral information, particularly over areas of rapid class transition. This issue is commonly observed with the classification of medium spatial-resolution images that include narrow class structures, such as rivers and roads. To solve this problem, we present a strategy for contextual classification that allows adjusting a trade-off between noise smoothing and the preservation of small spatial details. The proposed strategy comprises a meta-methodology, in the sense that it does not depend on specific pixel-based and contextual classifiers. The meta-methodology for improving contextual classification methods (Meta-CTX) consists in performing a separability analysis, at the pixel level, based on the class membership estimates provided by a pixel-based classifier. The Meta-CTX analyses the distance between class membership estimates in order to identify pixels that are expected to be accurately classified using purely spectral information. The Meta-CTX preserves the per-pixel classification of these pixels. It uses spatial contextual information only to classify pixels that are more susceptible to classification errors. The experimental results indicate that the Meta-CTX can efficiently combine noise smoothing with the preservation of small spatial details in remote sensing image classification.

Remote-Sensing Cross-Domain Scene Classification: A Dataset and Benchmark

Domain adaptation for classification has achieved significant progress in natural images but not in remote-sensing images due to huge differences in data-imaging mechanisms between different modalities and inconsistencies in class labels among existing datasets. More importantly, the lack of cross-domain benchmark datasets has become a major obstacle to the development of scene classification in multimodal remote-sensing images. In this paper, we present a cross-domain dataset of multimodal remote-sensing scene classification (MRSSC). The proposed MRSSC dataset contains 26,710 images of 7 typical scene categories with 4 distinct domains that are collected from Tiangong-2, a Chinese manned spacecraft. Based on this dataset, we evaluate several representative domain adaptation algorithms on three cross-domain tasks to build baselines for future research. The results demonstrate that the domain adaptation algorithm can reduce the differences in data distribution between different domains and improve the accuracy of the three tasks to varying degrees. Furthermore, MRSSC also achieved fairly results in three applications: cross-domain data annotation, weakly supervised object detection and data retrieval. This dataset is believed to stimulate innovative research ideas and methods in remote-sensing cross-domain scene classification and remote-sensing intelligent interpretation.

Classification of Land Cover Remote-Sensing Images Based on Pattern Recognition

With the development of remote sensing technology, remote sensing image data plays an active role in the dynamic monitoring of global resource changes and land cover utilization. Remote sensing image land cover classification is an important application direction of remote sensing data; how to further improve the accuracy of remote sensing image land cover classification is very important for the effective application of remote sensing data. The traditional remote sensing image land cover classification is mainly to classify remote sensing data according to the spectral data of ground objects. However, due to the complex environment of remote sensing images and the dynamic changes of the environment, traditional classification methods based only on pixel spectral data are often unable to achieve. A satisfactory classification result is achieved. In addition, some researchers have also proposed to combine pixel neighborhood texture information to supplement spectral feature data. Although the traditional classification method based on spectral features solves the problem of time-consuming visual interpretation, to a certain extent, due to the limited semantic expression ability and poor generalization ability of the design features, the classification accuracy is still not very satisfactory. This paper mainly studies the classification method of land cover remote sensing image based on pattern recognition. This paper is based on the experimental results of remote sensing data in Nanjing Yuhuatai District in 2018 and 2019. The ground resolution of the data is 2.5 meters. Data is projected, corrected, and equalized. Half of the images covering 43.75 square kilometers are used as training samples, and the remaining 50 square kilometers are used for detection. In the classification results of this IndianPines data, OA only increased by nearly 10% to 86.2%, AA increased by 13%, r was 82.77%, and Kappa coefficient was 0.84. In the classification results of Salinas data, both OA and AA increase by about 5%, and the optimization effect is not obvious.

Implementation of Remote-Sensing Data Processing Platform Based on Computable Storage

Previous researches on accelerating remote sensing data processing are based on traditional von Neumann architecture, which separates storage and computation. Under the architecture, data must be obtained from the storage device first and then transmitted to Field Programmable Gate Array (FPGA) through the system bus. The power consumption caused by the data handling is huge, even exceeding the energy consumption required for data processing. In order to reduce the migration of remote sensing data and alleviate the problems of storage wall and power wall under von Neumann architecture, we design a remote sensing data processing platform based on the system architecture of computable storage, which uses Solid-State Disk (SSD) with computing capability to process the remote sensing data and realize accelerated remote sensing data processing. Based on this platform, applications related to remote sensing data processing such as compression, target detection, and image classification are deployed in SSD to improve the information acquisition rate in remote sensing data. Experimental results show that after compression being offloaded to SSD computing performance is improved by 2.27 times compared with the host CPU. Compared with the host GPU, the target detection speed is improved by 6.25% and the power consumption is reduced by 66.7%. Compared with the host, the detection speed of remote sensing image classification is improved by 78.8%, the power consumption is reduced by 70%, achieving the expected classification effect. The Remote Sensing data Processing Platform based on Computable Storage (CSRSPP) distributes various computing tasks to the SSD for execution, which not only improves the processing speed of computing tasks, but also greatly reduces the power consumption of the platform.

Remote sensing image classification based on object-oriented convolutional neural network

Remote sensing image classification is of great importance for urban development and planning. The need for higher classification accuracy has led to improvements in classification technology. In this research, Landsat 8 images are used as experimental data, and Wuhan, Chengde and Tongchuan are selected as research areas. The best neighborhood window size of the image patch and band combination method are selected based on two sets of comparison experiments. Then, an object-oriented convolutional neural network (OCNN) is used as a classifier. The experimental results show that the classification accuracy of the OCNN classifier is 6% higher than that of an SVM classifier and 5% higher than that of a convolutional neural network classifier. The graph of the classification results of the OCNN is more continuous than the plots obtained with the other two classifiers, and there are few fragmentations observed for most of the category. The OCNN successfully solves the salt and pepper problem and improves the classification accuracy to some extent, which verifies the effectiveness of the proposed object-oriented model.

Multi-Class Pixel Certainty Active Learning Model for Classification of Land Cover Classes Using Hyperspectral Imagery

An accurate identification of objects from the acquisition system depends on the clear segmentation and classification of remote sensing images. With the limited financial resources and the high intra-class variations, the earlier proposed algorithms failed to handle the sub-optimal dataset. The building of an efficient training set iteratively in active learning (AL) approaches improves classification performance. The heuristics-based AL provides better results with the inheritance of contextual information and the robustness to noise variations. The uncertainty exists pixel variations make the heuristics-based AL fail to handle the remote sensing image classification. Previously, we focused on the extraction of clear textural pattern information by using the extended differential pattern-based relevance vector machine (EDP-AL). This paper extends that work into the novel pixel-certainty activity learning (PCAL) based on the information about textural patterns obtained from the extended differential pattern (EDP). Initially, distributed intensity filtering (DIF) is used to eliminate noise from the image, and then histogram equalization (HE) is used to improve the image quality. The EDP is used to merge and classify different labels for each image sample, and this algorithm expresses the textural information. The PCAL technique is used to classify the HSI patterns that are important in remote sensing applications using this pattern collection. Pavia University and Indian Pines (IP) are the datasets used to validate the performance of the proposed PCAL (PU). The ability of PCAL to accurately categorize land cover types is demonstrated by a comparison of the proposed PCAL with existing algorithms in terms of classification accuracy and the Kappa coefficient.

A Classification Feature Optimization Method for Remote Sensing Imagery Based on Fisher Score and mRMR

In object-oriented remote sensing image classification experiments, the dimension of the feature space is often high, leading to the “dimension disaster”. If a reasonable feature selection method is adopted, the classification efficiency and accuracy of the classifier can be improved. In this study, we took GF-2 remote sensing imagery as the research object and proposed a feature dimension reduction algorithm combining the Fisher Score and the minimum redundancy maximum relevance (mRMR) feature selection method. First, the Fisher Score was used to construct a feature index importance ranking, following which the mRMR algorithm was used to select the features with the maximum correlation and minimum redundancy between categories. The feature set was optimized using this method, and remote sensing images were automatically classified based on the optimized feature subset. Experimental analysis demonstrates that, compared with the traditional mRMR, Fisher Score, and ReliefF methods, the proposed Fisher Score–mRMR (Fm) method provides higher accuracy in remote sensing image classification. In terms of classification accuracy, the accuracy of the Fm feature selection method with RT and KNN classifiers is improved compared with that of single feature selection method, reaching 95.18% and 96.14%, respectively, and the kappa coefficient reaches 0.939 and 0.951, respectively.

GPCNet: global-context pyramidal and class-balanced network for high-resolution SAR remote sensing image classification

The description of context information affected by speckle and class imbalance under labeled data makes the pixelwise classification for high-resolution (HR) synthetic aperture radar (SAR) image a challenging task. To address these issues, we propose a global-context pyramidal and class-balanced network (GPCNet) for HR SAR image classification. The proposed structure follows an encoder–decoder architecture. In the encoder module, the multiscale convolutional and global-local cross-channel attention (GCA) blocks are employed to capture the global-context and distinguishable deep feature statistics, while reducing the impacts of the random fluctuation in the homogeneous region. The channel information of different scale convolutional layers is efficiently learned by local cross-channel interaction in the GCA block. Besides, a sampled class-balanced loss, associating with the effective number, is utilized for alleviating the class imbalance of HR SAR images. The experiments carried out on a TerraSAR-X image classification dataset demonstrate GPCNet is able to yield superior performance when compared with other related networks.

Remote Sensing Image Classification: A Comprehensive Review and Applications

Remote sensing is mainly used to investigate sites of dams, bridges, and pipelines to locate construction materials and provide detailed geographic information. In remote sensing image analysis, the images captured through satellite and drones are used to observe surface of the Earth. The main aim of any image classification-based system is to assign semantic labels to captured images, and consequently, using these labels, images can be arranged in a semantic order. The semantic arrangement of images is used in various domains of digital image processing and computer vision such as remote sensing, image retrieval, object recognition, image annotation, scene analysis, content-based image analysis, and video analysis. The earlier approaches for remote sensing image analysis are based on low-level and mid-level feature extraction and representation. These techniques have shown good performance by using different feature combinations and machine learning approaches. These earlier approaches have used small-scale image dataset. The recent trends for remote sensing image analysis are shifted to the use of deep learning model. Various hybrid approaches of deep learning have shown much better results than the use of a single deep learning model. In this review article, a detailed overview of the past trends is presented, based on low-level and mid-level feature representation using traditional machine learning concepts. A summary of publicly available image benchmarks for remote sensing image analysis is also presented. A detailed summary is presented at the end of each section. An overview regarding the current trends of deep learning models is presented along with a detailed comparison of various hybrid approaches based on recent trends. The performance evaluation metrics are also discussed. This review article provides a detailed knowledge related to the existing trends in remote sensing image classification and possible future research directions.

(Retracted) Support vector machine parallelized remote sensing image classification algorithm based on big data

With the development of big data technology, machine learning classification methods have been widely used in the classification and recognition of remote sensing images. For remote sensing big data, how to quickly and efficiently use machine learning classification algorithms to classify remote sensing images is an urgent problem. It is a general term for the theory, method, technology, and activities of obtaining valuable information based on massive remote sensing data sets, synthesizing auxiliary data from other sources, and using big data thinking and means. The purpose of this paper is to study the support vector machine (SVM) parallelized remote sensing image classification algorithm based on big data. We propose a parallel nesting of GPU in MPI multiprocesses based on the big data framework, which can more effectively improve the calculation processing speed and build a high-performance SVM parallel computing framework based on the big data framework. The optimization problem of SVM considers both empirical risk and structural risk minimization and requires maximum edge distance when constructing hyperplane decision boundaries, so there is ample space between the interval boundaries to accommodate the test samples. Based on this framework, we improve the machine learning SVM algorithm and realize the high-performance parallel computing of SVM classification algorithm on this platform. It is an efficient hybrid parallel mode to nest GPU in MPI multiprocess in parallel. When the number of nodes is 2, 4, and 6, the speedup of the SVM classification algorithm is 1.52, 2.24, and 2.55.

Assessing the effects of convolutional neural network architectural factors on model performance for remote sensing image classification: An in-depth investigation

Although the application of deep learning in remote sensing (RS) has achieved fruitful results, systematic research on exploring the model performance and guiding the design of new convolutional neural network (CNN) architectures is still lacking. This subject is of great concern to researchers or practitioners in this field because existing CNN structures may not be adequate to deal with complex RS scenarios. In this study, an empirical formula of CNN model performance is delivered based on a literature review. Extensive experiments are conducted on six public RS data sets to investigate the influences of three architectural factors, namely, network depth, width, and cardinality. Two types of CNN architectures, i.e., VGG and ResNet, are adopted as baselines. We monitor and visualize the data distributions and gradients of the utilized CNNs to prevent the gradient vanishing or exploding problem. Grad-CAM is adopted to open the black box of CNNs and to illustrate the effects of adjusting architectural factors. Our experiments indicate that (1) increasing the network depth is beneficial to the semantic feature learning capacity of a CNN model, but excessive depth also leads to a decline of overall accuracy; (2) a partly widening strategy is effective because it can improve the model performance while maintaining the network complexity; and (3) network cardinality has huge potential in achieving a balance between model efficiency and accuracy. Suggestions for improving the CNN model performance and developing new structures are summarized in this paper.

Dual-Coupled CNN-GCN-Based Classification for Hyperspectral and LiDAR Data.

Deep learning techniques have brought substantial performance gains to remote sensing image classification. Among them, convolutional neural networks (CNN) can extract rich spatial and spectral features from hyperspectral images in a short-range region, whereas graph convolutional networks (GCN) can model middle- and long-range spatial relations (or structural features) between samples on their graph structure. These different features make it possible to classify remote sensing images finely. In addition, hyperspectral images and light detection and ranging (LiDAR) images can provide spatial-spectral information and elevation information of targets on the Earth’s surface, respectively. These multi-source remote sensing data can further improve classification accuracy in complex scenes. This paper proposes a classification method for HS and LiDAR data based on a dual-coupled CNN-GCN structure. The model can be divided into a coupled CNN and a coupled GCN. The former employs a weight-sharing mechanism to structurally fuse and simplify the dual CNN models and extracting the spatial features from HS and LiDAR data. The latter first concatenates the HS and LiDAR data to construct a uniform graph structure. Then, the dual GCN models perform structural fusion by sharing the graph structures and weight matrices of some layers to extract their structural information, respectively. Finally, the final hybrid features are fed into a standard classifier for the pixel-level classification task under a unified feature fusion module. Extensive experiments on two real-world hyperspectral and LiDAR data demonstrate the effectiveness and superiority of the proposed method compared to other state-of-the-art baseline methods, such as two-branch CNN and context CNN. In particular, the overall accuracy (99.11%) on Trento achieves the best classification performance reported so far.

Enhanced Feature Selection Method Using Wrapper-based Random Search Strategy and Mutual Information for Remote Sensing Image Classification

Remote sensing image classification is one of the useful image processing tasks and finds application in many real-life scenarios. Image feature is important and unavoidable in classification. It is impossible to get a clear classification result without proper features. So after feature extraction, selecting an efficient and relevant feature is inevitable. Thus, our system proposes a new way of selecting features by a wrapper-based method that works using a randomized search strategy. The process is done in an orderly manner. In each step, the features that contribute less to classification are rejected to bring out the most relevant features. Three steps are followed: (1) Randomized selection (2) Warm start and (3) Cool down. The Randomized selection selects relevant features based on the random search method from the full feature set. Among the selected features, the important features are selected, based on the Mutual Information using the Warm start. Some important features, missed out in Randomized selection, are picked up in Cool down. The system finds out 25% of the most relevant features with greater classification accuracy when compared with classification accuracy obtained using 100% features. The proposed system has been checked for its efficiency with the help of remote sensing-based datasets from the UCI repository and it is found to be more efficient than the other existing methods. The results produced with the selected features are of high accuracy and low computational cost.

A Novel Weakly Supervised Remote Sensing Landslide Semantic Segmentation Method: Combining CAM and cycleGAN Algorithms

With the development of deep learning algorithms, more and more deep learning algorithms are being applied to remote sensing image classification, detection, and semantic segmentation. The landslide semantic segmentation of a remote sensing image based on deep learning mainly uses supervised learning, the accuracy of which depends on a large number of training data and high-quality data annotation. At this stage, high-quality data annotation often requires the investment of significant human effort. Therefore, the high cost of remote sensing landslide image data annotation greatly restricts the development of a landslide semantic segmentation algorithm. Aiming to resolve the problem of the high labeling cost of landslide semantic segmentation with a supervised learning method, we proposed a remote sensing landslide semantic segmentation with weakly supervised learning method combing class activation maps (CAMs) and cycle generative adversarial network (cycleGAN). In this method, we used the image level annotation data to replace pixel level annotation data as the training data. Firstly, the CAM method was used to determine the approximate position of the landslide area. Then, the cycleGAN method was used to generate the fake image without a landslide, and to make the difference with the real image to obtain the accurate segmentation of the landslide area. Finally, the pixel-level segmentation of the landslide area on remote sensing image was realized. We used mean intersection-over-union (mIOU) to evaluate the proposed method, and compared it with the method based on CAM, whose mIOU was 0.157, and we obtain better result with mIOU 0.237 on the same test dataset. Furthermore, we made a comparative experiment using the supervised learning method of a u-net network, and the mIOU result was 0.408. The experimental results show that it is feasible to realize landslide semantic segmentation in a remote sensing image by using weakly supervised learning. This method can greatly reduce the workload of data annotation.