High-resolution Image Classification Research Articles

High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

The classification of high-resolution (HR) synthetic aperture radar (SAR) images is of great importance for SAR scene interpretation and application. However, the presence of intricate spatial structural patterns and complex statistical nature makes SAR image classification a challenging task, especially in the case of limited labeled SAR data. This paper proposes a novel HR SAR image classification method, using a multi-scale deep feature fusion network and covariance pooling manifold network (MFFN-CPMN). MFFN-CPMN combines the advantages of local spatial features and global statistical properties and considers the multi-feature information fusion of SAR images in representation learning. First, we propose a Gabor-filtering-based multi-scale feature fusion network (MFFN) to capture the spatial pattern and get the discriminative features of SAR images. The MFFN belongs to a deep convolutional neural network (CNN). To make full use of a large amount of unlabeled data, the weights of each layer of MFFN are optimized by unsupervised denoising dual-sparse encoder. Moreover, the feature fusion strategy in MFFN can effectively exploit the complementary information between different levels and different scales. Second, we utilize a covariance pooling manifold network to extract further the global second-order statistics of SAR images over the fusional feature maps. Finally, the obtained covariance descriptor is more distinct for various land covers. Experimental results on four HR SAR images demonstrate the effectiveness of the proposed method and achieve promising results over other related algorithms.

Scene Classification of Optical High-resolution Remote Sensing Images Using Vision Transformer and Graph Convolutional Network

Scene Classification of Optical High-resolution Remote Sensing Images Using Vision Transformer and Graph Convolutional Network

Comparative Analysis of Feature Extraction and Pixel-based Classification of High-Resolution Satellite Images Using Geospatial Techniques

Two common techniques for classifying satellite imagery are pixel-based and Feature extraction image analysis methods. Typically, for agreements reached imaging, pixel-based analysis is used, whereas high-resolution imagery is suitable for Feature extraction analysis. However, In the classification of moderate images, image segmentation's ability depending on criteria such as shape, color, texture, and spatial features in Feature extraction image analysis implies it can perform better than pixel-based analysis. A comparative study of the two methods was performed using Sentinel-2 imagery from 18 May 2020 to categorize LU/LC in the City of Baghdad, Iraq. After calculating LU/LC for Baghdad images' capital, a supervised classification was performed using the two methods. The images used have been the support vector machines (SVM) and the maximum likelihood classification (MLC) for pixel-based method and Feature extraction method, which is available in ENVI and ArcGIS software packages, respectively. Land cover and land use classes included five Groups (vegetation area, asphalt road, soil area, water body, and built-up) was found that the Feature extraction methodology produced higher overall accuracy and Kappa index in the city of Baghdad image. The highest achieved accuracy for the Feature extraction technique was overall accuracy 95% with Kappa index 0.94 of SVM and overall accuracy of 92% with Kappa index 0.90 of MLC. In comparison, the highest accuracy for the pixel-based was overall accuracy 88% with Kappa index 0.84 of SVM and overall accuracy 86% with Kappa index 0.82 of MLC.

Integrating Backdating and Transfer Learning in an Object-Based Framework for High Resolution Image Classification and Change Analysis

Classification and change analysis based on high spatial resolution imagery are highly desirable for urban landscapes. However, methods with both high accuracy and efficiency are lacking. Here, we present a novel approach that integrates backdating and transfer learning under an object-based framework. Backdating is used to optimize the target area to be classified, and transfer learning is used to select training samples for classification. We further compare the new approach with that of using backdating or transfer learning alone. We found: (1) The integrated new approach had higher overall accuracy for both classifications (85.33%) and change analysis (88.67%), which were 2.0% and 4.0% higher than that of backdating, and 9.3% and 9.0% higher than that of transfer learning, respectively. (2) Compared to approaches using backdating alone, the use of transfer learning in the new approach allows automatic sample selection for supervised classification, and thereby greatly improves the efficiency of classification, and also reduces the subjectiveness of sample selection. (3) Compared to approaches using transfer learning alone, the use of backdating in the new approach allows the classification focusing on the changed areas, only 16.4% of the entire study area, and therefore greatly improves the efficiency and largely avoid the false change. In addition, the use of a reference map for classification can improve accuracy. This new approach would be particularly useful for large area classification and change analysis.

A Multispectral and Multiangle 3-D Convolutional Neural Network for the Classification of ZY-3 Satellite Images Over Urban Areas

The recent availability of high-resolution multiview ZY-3 satellite images, with angular information, can provide an opportunity to capture 3-D structural features for classification. In high-resolution image classification over urban areas, objects with diverse vertical structures make urban landscape more heterogeneous in 3-D space and consequently can make the classification challenging. In this article, a novel multiangle gray-level cooccurrence tensor feature is proposed based on the multiview bands of the ZY-3 imagery, namely, GLCM^MA–T. The GLCM^MA–T feature captures the distributions of the gray-level spatial variation under different viewing angles, which can depict the 3-D textures and structures of urban objects. The spectral and GLCM^MA–T tensor features are interpreted by two 3-D convolutional neural network (CNN) streams and then concatenated as the input to the fully connected layer. This novel multispectral and multiangle 3-D convolutional neural network (M²-3-DCNN) combines the spectral and angular information, and the fused feature has the potential to provide a comprehensive description of urban objects with complex vertical structures. The experimental results on ZY-3 multiview images from four test areas indicate that the proposed method can significantly improve the classification accuracy when compared with several state-of-the-art multiangle features and deep-learning-based image classification methods.

Attention-Based Pyramid Network for Segmentation and Classification of High-Resolution and Hyperspectral Remote Sensing Images

Unlike conventional natural (RGB) images, the inherent large scale and complex structures of remote sensing images pose major challenges such as spatial object distribution diversity and spectral information extraction when existing models are directly applied for image classification. In this study, we develop an attention-based pyramid network for segmentation and classification of remote sensing datasets. Attention mechanisms are used to develop the following modules: (i) a novel and robust attention-based multi-scale fusion method effectively fuses useful spatial or spectral information at different and same scales; (ii) a region pyramid attention mechanism using region-based attention addresses the target geometric size diversity in large-scale remote sensing images; and (iii) cross-scale attention in our adaptive atrous spatial pyramid pooling network adapts to varied contents in a feature-embedded space. Different forms of feature fusion pyramid frameworks are established by combining these attention-based modules. First, a novel segmentation framework, called the heavy-weight spatial feature fusion pyramid network (FFPNet), is proposed to address the spatial problem of high-resolution remote sensing images. Second, an end-to-end spatial-spectral FFPNet is presented for classifying hyperspectral images. Experiments conducted on ISPRS Vaihingen and ISPRS Potsdam high-resolution datasets demonstrate the competitive segmentation accuracy achieved by the proposed heavy-weight spatial FFPNet. Furthermore, experiments on the Indian Pines and the University of Pavia hyperspectral datasets indicate that the proposed spatial-spectral FFPNet outperforms the current state-of-the-art methods in hyperspectral image classification.

A Deep Learning Model With Capsules Embedded for High-Resolution Image Classification

Classification of remote sensing (RS) images is a key technology for extracting information on ground objects using RS methods. Inspired by the success of deep learning (DL) in artificial intelligence, researchers have proposed different algorithms based on DL to improve the performance of classification. At present, a DL model represented by the convolutional neural networks (CNNs) can extract the abstract feature, but it loses the spatial context of the ground objects. To solve the problem of lack of spatial information in CNNs, the Capsule network takes the form of vectors that convey location transformation information. This article proposes using a Capsules-Unet model, which incorporates Capsules within the U-net architecture for classification of RS images. The aim is to train better models by encapsulating the multidimensional features of the objects in the form of Capsules, and to reduce parameter space by improving the dynamic routing algorithm. Experiments are conducted on ISPRS Vaihingen and Potsdam datasets. Capsules-Unet slightly outperforms all other approaches with far fewer parameters, a reduction in parameters of over 81.8% compared with U-net and over 13.8% compared with Capsule network.

Dehesa environment mapping with transference of a Random Forest classifier to neighboring ultra-high spatial resolution imagery at class and macro-class land cover levels

Accurate vegetation cover maps of forested areas are crucial for ecosystems monitoring, as well as for management of water balance, flood and fire risk, and other forest-associated resources. With this regard, remote sensing techniques have been used for land cover mapping worldwide. Here, we propose a vegetation-mapping methodology in a dehesa environment using ultra-high spatial resolution imagery (UHSR) with a spatial resolution of 0.25 m and four bands in the visible and near-infrared spectrum. Land cover categories were defined by their runoff generation capability and considered two levels of disaggregation: among species (macro-class level) and within species (class level). Additionally, we developed a method to reduce field campaigns and manual work by transferring random forest classifiers trained with a group of images (training group) to neighboring images (validation group). The training group was remarkably accurate, achieving an overall accuracy of 91.6% (k = 0.89) at the class level and 95.8% (k = 0.94) at the macro-class level. The results for the validation group were also very high, with an overall accuracy of 78.3% (k = 0.74) at the class level and 86.3% (k = 0.82) at the macro-class level. Moreover, we found that the blue band, soil color index, and texture features have a great influence on species discrimination, especially within shrub species in dehesa environments. Notably, having accurate land cover maps is crucial, given that the use of a global database led to underestimating the potential runoff in the most representative land cover in the dehesa environment. Future research will focus on the automatic generation of new samples extracted from the classified UHSR images, which could be used as training datasets for the supervised classification of other high spatial resolution images (e.g., Sentinel imagery) for regional-scale hydrological models.

Pixel-Wise Classification of High-Resolution Ground-Based Urban Hyperspectral Images with Convolutional Neural Networks

Using ground-based, remote hyperspectral images from 0.4–1.0 micron in ∼850 spectral channels—acquired with the Urban Observatory facility in New York City—we evaluate the use of one-dimensional Convolutional Neural Networks (CNNs) for pixel-level classification and segmentation of built and natural materials in urban environments. We find that a multi-class model trained on hand-labeled pixels containing Sky, Clouds, Vegetation, Water, Building facades, Windows, Roads, Cars, and Metal structures yields an accuracy of 90–97% for three different scenes. We assess the transferability of this model by training on one scene and testing to another with significantly different illumination conditions and/or different content. This results in a significant (∼45%) decrease in the model precision and recall as does training on all scenes at once and testing on the individual scenes. These results suggest that while CNNs are powerful tools for pixel-level classification of very high-resolution spectral data of urban environments, retraining between scenes may be necessary. Furthermore, we test the dependence of the model on several instrument- and data-specific parameters including reduced spectral resolution (down to 15 spectral channels) and number of available training instances. The results are strongly class-dependent; however, we find that the classification of natural materials is particularly robust, especially the Vegetation class with a precision and recall >94% for all scenes and model transfers and >90% with only a single training instance.

Cloud Image Classification Method Based on Deep Convolutional Neural Network

Compared with satellite remote sensing images, ground-based invisible images have limited swath, but featured in higher resolution, more distinct cloud features, and the cost is greatly reduced, conductive to continuous meteorological observation of local areas. For the first time, this paper proposed a high-resolution cloud image classification method based on deep learning and transfer learning technology for ground-based invisible images. Due to the limited amount of samples, traditional classifiers such as support vector machine can't effectively extract the unique features of different types of clouds, and directly training deep convolutional neural networks leads to over-fitting. In order to prevent the network from over-fitting, this paper proposed applying transfer learning method to fine-tune the pre-training model. The proposed network achieved as high as 85.19% test accuracy on 6-type cloud images classification task. The networks proposed in this paper can be applied to classify digital photos captured by cameras directly, which will reduce the cost of system greatly.

Deep Multibranch Fusion Residual Network for Insect Pest Recognition

Earlier insect pest recognition is one of the critical factors for agricultural yield. Thus, an effective method to recognize the category of insect pests has become significant issues in the agricultural field. In this article, we proposed a new residual block to learn multiscale representation. In each block, it contains three branches: one is parameter-free, and the others contain several successive convolution layers. Moreover, we proposed a module and embedded it into the new residual block to recalibrate the channelwise feature response and to model the relationship of the three branches. By stacking this kind of block, we constructed the deep multibranch fusion residual network (DMF-ResNet). For evaluating the model performance, we first test our model on CIFAR-10 and CIFAR-100 benchmark data sets. The experimental results show that DMF-ResNet outperforms the baseline models significantly. Then, we construct DMF-ResNet with different depths for high-resolution image classification tasks and apply it to recognize insect pests. We evaluate the model performance on the IP102 data set, and the experimental results show that DMF-ResNet could achieve the best accuracy performance than the baseline models and other state-of-the-art methods. Based on these empirical experiments, we demonstrate the effectiveness of our approach.

Semantic segmentation of land cover from high resolution multispectral satellite images by spectral-spatial convolutional neural network

Research to improve the accuracy of very high-resolution satellite image classification algorithms is still one of the hot topics in the field of remote sensing. Successful results of deep learning methods in areas such as image classification and object detection have led to the application of these methods to remote sensing problems. Recently, Convolutional Neural Networks (CNNs) are among the most common deep learning methods used in image classification, however, the use of CNN’s in satellite image classification is relatively new. Due to the high computational complexity of 3D CNNs, which aim to extract both spatial and spectral information, 2D CNNs focussing on the extraction of spatial information are often preferred. High-resolution satellite images, however, contain crucial spectral information as well as spatial information. In this study, a 3D-2D CNN model using both spectral and spatial information was applied to extract more accurate land cover information from very high-resolution satellite images. The model was applied on a Worldview-2 satellite image including agricultural product areas such as tea, hazelnut groves and land use classes such as buildings and roads. The results of the CNN based model were also compared against those of the Support Vector Machine (SVM) and Random Forest (RF) algorithms. The post-classification accuracies were obtained using 800 control points generated by a web interface created for crowdsourcing purposes. The classification accuracy was 95.6% for the 3D-2D CNN model, 89.2% for the RF and 86.4% for the SVM.

RETRACTED: Zhu et al. Attention-Based Deep Feature Fusion for the Scene Classification of High-Resolution Remote Sensing Images. Remote Sens. 2019, 11, 1996

We have been made aware that the innovative contributions, research method and the majority of the content of this article [...]

Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network

Urban land cover classification for high-resolution images is a fundamental yet challenging task in remote sensing image analysis. Recently, deep learning techniques have achieved outstanding performance in high-resolution image classification, especially the methods based on deep convolutional neural networks (DCNNs). However, the traditional CNNs using convolution operations with local receptive fields are not sufficient to model global contextual relations between objects. In addition, multiscale objects and the relatively small sample size in remote sensing have also limited classification accuracy. In this paper, a relation-enhanced multiscale convolutional network (REMSNet) method is proposed to overcome these weaknesses. A dense connectivity pattern and parallel multi-kernel convolution are combined to build a lightweight and varied receptive field sizes model. Then, the spatial relation-enhanced block and the channel relation-enhanced block are introduced into the network. They can adaptively learn global contextual relations between any two positions or feature maps to enhance feature representations. Moreover, we design a parallel multi-kernel deconvolution module and spatial path to further aggregate different scales information. The proposed network is used for urban land cover classification against two datasets: the ISPRS 2D semantic labelling contest of Vaihingen and an area of Shanghai of about 143 km2. The results demonstrate that the proposed method can effectively capture long-range dependencies and improve the accuracy of land cover classification. Our model obtains an overall accuracy (OA) of 90.46% and a mean intersection-over-union (mIoU) of 0.8073 for Vaihingen and an OA of 88.55% and a mIoU of 0.7394 for Shanghai.

Classification of Large-Scale High-Resolution SAR Images With Deep Transfer Learning

The classification of large-scale high-resolution SAR land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover dataset collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100,000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover dataset (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes' problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing dataset, a robust performance on highly imbalanced classes, and is alleviating the over-fitting problem caused by label noise. What's more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

Semantic segmentation of very high-resolution remote sensing image based on multiple band combinations and patchwise scene analysis

Large intraclass variance and low interclass variance are among the most challenging problems in very high-resolution (VHR) image classification. Semantic segmentation constructed in a deep convolution neural network is used as a classification algorithm conducted via end-to-end training, which combines spectral–spatial features and context information. However, large-scale remote sensing images cannot be directly processed because they are limited by GPU memory and segmentation algorithm. At the same time, classification using single band combinations is also unsatisfactory due to the extraordinary complex features of VHR images. Therefore, a method is proposed based on multiple band combinations and patchwise scene analysis. A complex remote sensing image can be considered as into a combination of simple scenes from multiple patchwise images. And optimal band combinations of each patchwise image are selected according to their scene. The segmentation results of each patchwise image are merged to get the desired results according to geographical coordinates. Our method is validated on the ISPRS 2-D Semantic Labeling dataset of Potsdam, on which results competitive with the state-of-the-art are obtained. The proposed scheme has strong universality and can be used for large-scale high-resolution remote sensing image classification.

Multilayer Feature Fusion Network for Scene Classification in Remote Sensing

The scene classification of high spatial resolution (HSR) images is a challenging task in the remote sensing community. How to construct a discriminative representation of the HSR scene is a key step to improve classification performance. In this letter, we propose a novel feature extraction method termed multilayer feature fusion network (MF2Net) for scene classification. At first, the transferred VGGNet-16 model is employed as a feature extractor to acquire multilayer convolutional features. Then, several layers including pooling, transformation, and fusion layers are designed to process hierarchical features in four branches, and the prediction probability can be obtained for classification. Finally, the proposed model is optimized by fine-tuning techniques, where a novel data augmentation approach is explored to improve generalization ability. As a result, MF2Net effectively applies useful information from multilayers to improve the accuracy of scene classification. The experimental results on AID and NWPU-RESISC45 data sets exhibit that the MF2Net method obtains quite competitive classification results compared with many state-of-the-art methods.

Bidirectional Grid Fusion Network for Accurate Land Cover Classification of High-Resolution Remote Sensing Images

Land cover classification has achieved significant advances by employing deep convolutional network (ConvNet) based methods. Following the paradigm of learning deep models, land cover classification is modeled as semantic segmentation of very high resolution remote sensing images. In order to obtain accurate segmentation results, high-level categorical semantics and low-level spatial details should be effectively fused. To this end, we propose a novel bidirectional gird fusion network to aggregate the multilevel features across the ConvNet. Specifically, the proposed model is characterized by a bidirectional fusion architecture, which enriches diversity of feature interaction by encouraging bidirectional information flow. In this way, our model gains mutual benefits between top-down and bottom-up information flows. Moreover, a grid fusion architecture is then followed for further feature refinement in a dense and hierarchical fusion manner. Finally, effective feature upsampling is also critical for the multiple fusion operations. Consequently, a content-aware feature upsampling kernel is incorporated for further improvement. Our whole model consistently achieves significant improvement over state-of-the-art methods on two major datasets, ISPRS and GID.

Scene Classification of Remote Sensing Images Based on Saliency Dual Attention Residual Network

Scene classification of high-resolution Remote Sensing Images (RSI) is one of basic challenges in RSI interpretation. Existing scene classification methods based on deep learning have achieved impressive performances. However, since RSI commonly contain various types of ground objects and complex backgrounds, most of methods cannot focus on saliency features of scene, which limits the classification performances. To address this issue, we propose a novel Saliency Dual Attention Residual Network (SDAResNet) to extract both cross-channel and spatial saliency information for scene classification of RSI. More specifically, the proposed SDAResNet consists of spatial attention and channel attention, in which spatial attention is embedded in low-level feature to emphasize saliency location information and suppress background information, and channel attention is integrated to high-level features to extract saliency meaningful information. Additionally, several image classification tricks are used to further improve classification accuracy. Finally, Extensive experiments on two challenging benchmark RSI datasets are presented to demonstrate that our methods outperform most of state-of-the-art approaches significantly.

Classification of High-Resolution Remote Sensing Images in the Feilaixia Reservoir Based on a Fully Convolutional Network

In technologically underdeveloped areas, water pollution threatens the living environment of local residents, so remote sensing monitoring of the features around reservoirs is necessary. Fully convolutional networks (FCNs) offer great potential for extracting high-resolution features due to their unlimited input image size and higher accuracy compared to convolutional neural networks. Therefore, a proposal to classify WorldView-2 images is implemented with a sixty-eight thousand iterations of fine-tuning and fully trained combined training method based on a fully convolutional network (SEFCN). The chosen images depict the urban area of Yingde, which is located to the northeast of the Feilaixia Reservoir, Qingyuan, Guangdong Province, China. The SEFCN combines an FCN-32s and FCN-16s to better integrate the deep features and shallow features and effectively improve the classification accuracy. Additionally, the loss value fully converges with enough iterations, and the overfitting caused by the superposition of two full trainings is avoided. The SEFCN model achieves the highest accuracy among all compared classification models and obtains the best classified results on the WorldView-2 images, as demonstrated by attaining the highest F1 scores in each category. After the classified images are optimized using a conditional random field, the current status of the study area is analyzed, and several suggestions for land use in the urban area of Yingde are made. The experiments still have deficiencies in the application of high-resolution remote sensing image classification with different sensors, and the classification results can be optimized and improved in other aspects.