High Resolution Remote Sensing Research Articles

HRU-Net: High-Resolution Remote Sensing Image Road Extraction Based on Multi-Scale Fusion

Road extraction from high-resolution satellite images has become a significant focus in the field of remote sensing image analysis. However, factors such as shadow occlusion and spectral confusion hinder the accuracy and consistency of road extraction in satellite images. To overcome these challenges, this paper presents a multi-scale fusion-based road extraction framework, HRU-Net, which exploits the various scales and resolutions of image features generated during the encoding and decoding processes. First, during the encoding phase, we develop a multi-scale feature fusion module with upsampling capabilities (UMR module) to capture fine details, enhancing shadowed areas and road boundaries. Next, in the decoding phase, we design a multi-feature fusion module (MPF module) to obtain multi-scale spatial information, enabling better differentiation between roads and objects with similar spectral characteristics. The network simultaneously integrates multi-scale feature information during the downsampling process, producing high-resolution feature maps through progressive cross-layer connections, thereby enabling more effective high-resolution prediction tasks. We conduct comparative experiments and quantitative evaluations of the proposed HRU-Net framework against existing algorithms (U-Net, ResNet, DeepLabV3, ResUnet, HRNet) using the Massachusetts Road Dataset. On this basis, this paper selects three network models (U-Net, HRNet, and HRU-Net) to conduct comparative experiments and quantitative evaluations on the DeepGlobe Road Dataset. The experimental results demonstrate that the HRU-Net framework outperforms its counterparts in terms of accuracy and mean intersection over union. In summary, the HRU-Net model proposed in this paper skillfully exploits information from different resolution feature maps, effectively addressing the challenges of discontinuous road extraction and reduced accuracy caused by shadow occlusion and spectral confusion factors. In complex satellite image scenarios, the model accurately extracts comprehensive road regions.

Multi-Level Difference Network for Change Detection from Very High-Resolution Remote Sensing Images: A Case Study in Open-Pit Mines

Automatic change detection based on remote sensing is playing an increasingly important role in the national economy construction. To address the problem of limited change detection accuracy in existing single-level difference networks, this study proposes the Multi-level Difference Network (MDNet) for automatic change detection of ground targets from very high-resolution (VHR) remote sensing images. An early-difference network and a late-difference network are combined by MDNet to extract multi-level change features. The early-difference network can focus on change information throughout to reduce the spurious changes in the change detection results, and the late-difference network can provide deep features of a single image for reducing rough boundaries and scattered holes in the change detection results, thus improving the accuracy. However, not all high-level features extracted by MDNet contribute to the recognition of image differences, and the multi-level change features suffer from cross-channel heterogeneity. Stacking them directly on channels does not make effective use of change information, thus limiting the performance of MDNet. Therefore, the Multi-level Change Features Fusion Module (MCFFM) is proposed in this study for the effective fusion of multi-level change features. In the experiments, the publicly available open-pit mine change detection (OMCD) dataset was used first to achieve a change detection of open-pit mines over a large area, with an F1-score of 89.2%, increasing by 1.3% to 5.9% compared to the benchmark methods. Then, a self-made OMCD dataset was used to achieve an F1-score of 92.8% for the localized and fine-scale change detection in open-pit mines, which is an improvement of 0.7% to 5.4% compared to the benchmark methods. Finally, the Season-varying Change Detection Dataset is used to verify that the MDNet proposed can detect changes in other scenarios very well. The experimental results show that the proposed MDNet has significantly improved the performance of change detection on the three datasets compared with six advanced deep learning models, which will contribute to the development of change detection with VHR remote sensing images.

Advancing Soil Organic Carbon and Total Nitrogen Modelling in Peatlands: The Impact of Environmental Variable Resolution and vis-NIR Spectroscopy Integration

Visible and near-infrared (vis-NIR) spectroscopy has proven to be a straightforward method for sample preparation and scaling soil testing, while the increasing availability of high-resolution remote sensing (RS) data has further facilitated the understanding of spatial variability in soil organic carbon (SOC) and total nitrogen (TN) across landscapes. However, the impact of combining vis-NIR spectroscopy with high-resolution RS data for SOC and TN prediction remains an open question. This study evaluated the effects of incorporating a high-resolution LiDAR-derived digital elevation model (DEM) and a medium-resolution SRTM-derived DEM with vis-NIR spectroscopy for predicting SOC and TN in peatlands. A total of 57 soil cores, comprising 262 samples from various horizons (<2 m), were collected and analysed for SOC and TN content using traditional methods and ASD Fieldspec® 4. The 262 observations, along with elevation data from LiDAR and SRTM, were divided into 80% training and 20% testing datasets. By employing the Cubist modelling approach, the results demonstrated that incorporating high-resolution LiDAR data with vis-NIR spectra improved predictions of SOC (RMSE: 4.60%, RPIQ: 9.00) and TN (RMSE: 3.06 g kg−1, RPIQ: 7.05). In conclusion, the integration of LiDAR and soil spectroscopy holds significant potential for enhancing soil mapping and promoting sustainable soil management.

Multi-Level Perceptual Network for Urban Building Extraction from High-Resolution Remote Sensing Images

Building extraction from high-resolution remote sensing images benefits various practical applications. However, automation of this process is challenging due to the variety of building surface coverings, complex spatial layouts, different types of structures, and tree occlusion. In this study, we propose a multilayer perception network for building extraction from high-resolution remote sensing images. By constructing parallel networks at different levels, the proposed network retains spatial information of varying feature resolutions and uses the parsing module to perceive the prominent features of buildings, thus enhancing the model's parsing ability to target scale changes and complex urban scenes. Further, a structure-guided loss function is constructed to optimize building extraction edges. Experiments on multi-source remote sensing data sets show that our proposed multi-level perception network presents a superior performance in building extraction tasks.

MS-AGAN: Road Extraction via Multi-Scale Information Fusion and Asymmetric Generative Adversarial Networks from High-Resolution Remote Sensing Images under Complex Backgrounds

Extracting roads from remote sensing images is of significant importance for automatic road network updating, urban planning, and construction. However, various factors in complex scenes (e.g., high vegetation coverage occlusions) may lead to fragmentation in the extracted road networks and also affect the robustness of road extraction methods. This study proposes a multi-scale road extraction method with asymmetric generative adversarial learning (MS-AGAN). First, we design an asymmetric GAN with a multi-scale feature encoder to better utilize the context information in high-resolution remote sensing images (HRSIs). Atrous spatial pyramid pooling (ASPP) and feature fusion are integrated into the asymmetric encoder–decoder structure to avoid feature redundancy caused by multi-level cascading operations and enhance the generator network’s ability to extract fine-grained road information at the pixel level. Second, to maintain road connectivity, topologic features are considered in the pixel segmentation process. A linear structural similarity loss (LSSIM) is introduced into the loss function of MS-AGAN, which guides MS-AGAN to generate more accurate segmentation results. Finally, to fairly evaluate the performance of deep models under complex backgrounds, the Bayesian error rate (BER) is introduced into the field of road extraction for the first time. Experiments are conducted via Gaofen-2 (GF-2) high-resolution remote sensing images with high vegetation coverage in the Daxing District of Beijing, China, and the public DeepGlobe dataset. The performance of MS-AGAN is compared with a list of advanced models, including RCFSNet, CoANet, UNet, DeepLabV3+, and DiResNet. The final results show that (1) with respect to road extraction performance, the Recall, F1, and IoU values of MS-AGAN on the Daxing dataset are 2.17%, 0.04%, and 2.63% higher than the baselines. On DeepGlobe, the Recall, F1, and IoU of MS-AGAN improve by 1.12%, 0.42%, and 0.25%, respectively. (2) On road connectivity, the Conn index of MS-AGAN from the Daxing dataset is 46.39%, with an improvement of 0.62% over the baselines, and the Conn index of MS-AGAN on DeepGlobe is 70.08%, holding an improvement of 1.73% over CoANet. The quantitative and qualitative analyses both demonstrate the superiority of MS-AGAN in preserving road connectivity. (3) In particular, the BER of MS-AGAN is 20.86% over the Daxing dataset with a 0.22% decrease compared to the best baselines and 11.77% on DeepGlobe with a 0.85% decrease compared to the best baselines. The proposed MS-AGAN provides an efficient, cost-effective, and reliable method for the dynamic updating of road networks via HRSIs.

Building Detection in High-Resolution Remote Sensing Images by Enhancing Superpixel Segmentation and Classification Using Deep Learning Approaches

Accurate building detection is a critical task in urban development and digital city mapping. However, current building detection models for high-resolution remote sensing images are still facing challenges due to complex object characteristics and similarities in appearance. To address this issue, this paper proposes a novel algorithm for building detection based on in-depth feature extraction and classification of adaptive superpixel shredding. The proposed approach consists of four main steps: image segmentation into homogeneous superpixels using a modified Simple Linear Iterative Clustering (SLIC), in-depth feature extraction using an variational auto-encoder (VAE) scale on the superpixels for training and testing data collection, identification of four classes (buildings, roads, trees, and shadows) using extracted feature data as input to an Convolutional Neural Network (CNN), and extraction of building shapes through regional growth and morphological operations. The proposed approach offers more stability in identifying buildings with unclear boundaries, eliminating the requirement for extensive prior segmentation. It has been tested on two datasets of high-resolution aerial images from the New Zealand region, demonstrating superior accuracy compared to previous works with an average F1 score of 98.83%. The proposed approach shows potential for fast and accurate urban monitoring and city planning, particularly in urban areas.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

With the explosion in the volume of collected high-resolution aerial image data, the development of effective image retrieval methods for remote sensing (RS) has become a popular area of research. The problem of content-based image retrieval (CBIR) for high-resolution remote sensing (HRRS) requires robust feature extraction and representation to extract high-level semantics followed by a similarity measurement method to compute the similarity between images based on these semantics. In this paper, we propose a novel sparse graph learning-based triplet network for deep metric learning-based HRRS CBIR. First, the graph representations of the aerial images are constructed via local feature descriptors and unsupervised segmentation that provide an efficient graph-structured representation of the images. Then, a novel attention graph convolutional network is developed to extract high-level semantics from the spatial patterns of the images’ segmented regions by aggregating region-node features. The attention mechanism and the pyramid pooling structure of the graph network help the proposed model extract powerful spatial features by preserving the structural information of the images while reducing the data’s dimensionality. Finally, a novel task-driven dictionary learning (TDDL) method based on triplet loss constructs a sparse metric space for the similarity measurement of the images in which the computed sparse codes for the images promote intra-class similarity between same-class images and inter-class dissimilarity between images of different classes. The proposed sparse features reduce the dimensionality of model parameters which mitigates overfitting and improves the generalization of the overall framework for deep metric learning. Our TDDL method adopts a bi-level optimization strategy so that the dictionary can be trained alongside the parameters of the graph representation network in an end-to-end fashion. Finally, we conduct extensive evaluations of the performance of our proposed architecture on real-world datasets compared to state-of-the-art models and obtain superior results.

Emerging Issues in Mapping Urban Impervious Surfaces Using High-Resolution Remote Sensing Images

Urban impervious surface (UIS) is a key parameter in climate change, environmental change, and sustainability. UIS extraction has been evolving rapidly in the past decades. However, high-resolution impervious surface mapping is a long-term need. There is an urgent requirement for impervious surface mapping from high-resolution remote sensing imagery. In this paper, we compare current extraction methods in terms of extraction units and extraction models and summarize their strengths and limitations. We discuss the challenges in impervious surface estimation from high spatial resolution remote sensing imagery in terms of selection of spatial resolution, spectral band, and extraction method. The uncertainties caused by clouds and snow, shadows, and vegetation occlusion are also analyzed. Automated sample labeling and remote sensing domain knowledge are the main directions in impervious surface extraction using deep learning methods. We should also focus on using continuous time series of high-resolution imagery and multi-source satellite imagery for dynamic monitoring of impervious surfaces.

Dynamic High-Resolution Network for Semantic Segmentation in Remote-Sensing Images

Semantic segmentation of remote-sensing (RS) images is one of the most fundamental tasks in the understanding of a remote-sensing scene. However, high-resolution RS images contain plentiful detailed information about ground objects, which scatter everywhere spatially and have variable sizes, styles, and visual appearances. Due to the high similarity between classes and diversity within classes, it is challenging to obtain satisfactory and accurate semantic segmentation results. This paper proposes a Dynamic High-Resolution Network (DyHRNet) to solve this problem. Our proposed network takes HRNet as a super-architecture, aiming to leverage the important connections and channels by further investigating the parallel streams at different resolution representations of the original HRNet. The learning task is conducted under the framework of a neural architecture search (NAS) and channel-wise attention module. Specifically, the Accelerated Proximal Gradient (APG) algorithm is introduced to iteratively solve the sparse regularization subproblem from the perspective of neural architecture search. In this way, valuable connections are selected for cross-resolution feature fusion. In addition, a channel-wise attention module is designed to weight the channel contributions for feature aggregation. Finally, DyHRNet fully realizes the dynamic advantages of data adaptability by combining the APG algorithm and channel-wise attention module simultaneously. Compared with nine classical or state-of-the-art models (FCN, UNet, PSPNet, DeepLabV3+, OCRNet, SETR, SegFormer, HRNet+FCN, and HRNet+OCR), DyHRNet has shown high performance on three public challenging RS image datasets (Vaihingen, Potsdam, and LoveDA). Furthermore, the visual segmentation results, the learned structures, the iteration process analysis, and the ablation study all demonstrate the effectiveness of our proposed model.

Delineation of agricultural fields in arid regions from Worldview-2 datasets based on image textural properties

Barren lands are being transformed into agricultural fields with the growing demand for agriculture-based products. Hence, monitoring these regions for better planning and management is crucial. Surveying with high-resolution RS (remote sensing) satellites like Worldview-2 provides a faster and cheaper solution than conventional surveys. In the study, the arid region comprising cropland and barrenlands are efficiently and autonomously delineated using its spectral and textural properties using state-of-the-art random forest (RF) ensemble classifiers. The textural information window size is optimized and at a GLCM (gray-level co-occurrence matrix) window size of 13, a stable trend in classification accuracy was observed. A further rise in window sizes did not improve the classification accuracy; beyond GLCM 19, a decline in accuracy was observed. Comparing GLCM-13 RF with the no-GLCM RF classifier, the GLCM-based classifiers performed better; thus, the textural information assisted in removing isolated crop-classified outputs that are falsely predicted pixel groups. Still, it also obscured information about barren lands present within croplands. Delineation accuracy was 93.8% for the no-GLCM RF classifier, whereas, for the GLCM-13 RF classifier, an accuracy of 97.3% was observed. Thus, overall, a 3.5% improvement in accuracy was observed while using the GLCM RF classifier with window size 13. The textural information with proper calibration over high-spatial resolution datasets improves crop delineation in the present study. Henceforth, a more accurate cropland identification will provide a better estimate of the actual cropland area in such an arid region, which will assist in formulating a better resource management policy.

High-resolution remote sensing images semantic segmentation using improved UNet and SegNet

Semantic segmentation for high-resolution (HR) remote sensing (RS) images under the condition of a small number of training samples with imprecise labels is a challenging task. In this paper, we propose a novel method based on improved UNet and SegNet. First, a batch normalization layer is introduced into the original UNet to accelerate the convergence speed and an ELU activation function is selected instead of ReLU to avoid the neuron death. Second, to enhance the conventional SegNet, the encoder is reconstructed and a skip connection is designed to reuse the deep features, which is also beneficial for the network convergence. Third, a joint model is constructed by combining the improved UNet and SegNet and meanwhile a voting strategy is proposed to compute the final results. Experiments on the real-world HR RS images verify the effectiveness and superiority of the proposed method.

Study on Historical Cities Conservation Monitoring Supported by High-Resolution Remote Sensing

Historical cities are important types of cultural heritage sites, and their preservation conditions and conservation status require efficient dynamic monitoring. As the national strategy of China has been vigorously developed in recent years, high-resolution remote sensing technology provides new opportunities for dynamic monitoring of historical and cultural heritage. Based on demand analysis of the historical city conservation and support capacities of high-resolution remote sensing technology, this paper developed a general framework of historical cities conservation monitoring supported by high-resolution remote sensing technology, combined with specific case studies. The framework includes three processes: remote sensing image interpolation for historical cities, temporal and spatial change analysis of conservation elements, and quantitative evaluation of the conservation status. The aims of this study are to explore high-resolution remote sensing technology for conservation monitoring of historical cities and promote the conservation and legacy of historical cities.

RSCDNet: A Robust Deep Learning Architecture for Change Detection From Bi-Temporal High Resolution Remote Sensing Images

Accurate change detection from high-resolution satellite and aerial images is of great significance in remote sensing for precise comprehension of Land cover (LC) variations. The current methods compromise with the spatial context; hence, they fail to detect and delineate small change areas and are unable to capture the difference between features of the bi-temporal images. This paper proposes Remote Sensing Change Detection Network (RSCDNet) - a robust end-to-end deep learning architecture for pixel-wise change detection from bi-temporal high-resolution remote-sensing (HRRS) images. The proposed RSCDNet model is based on an encoder-decoder framework integrated with the Modified Self-Attention (MSA) andthe Gated Linear Atrous Spatial Pyramid Pooling (GL-ASPP) blocks; both efficient mechanisms to regulate the field-of-view while finding the most suitable trade-off between accurate localization and context assimilation. The paper documents the design and development of the proposed RSCDNet model and compares its qualitative and quantitative results with state-of-the-art HRRS change detection architectures. The above mentioned novelties in the proposed architecture resulted in an F1-score of 98%, 98%, 88%, and 75% on the four publicly available HRRS datasets namely, Staza-Tisadob, Onera, CD-LEVIR, and WHU. In addition to the improvement in the performance metrics, the strategic connections in the proposed GL-ASPP and MSA units significantly reduce the prediction time per image (PTPI) and provide robustness against perturbations. Experimental results yield that the proposed RSCDNet model outperforms the most recent change detection benchmark models on all four HRRS datasets.

The Vanishing and Renewal Landscape of Urban Villages Using High-Resolution Remote Sensing: The Case of Haidian District in Beijing

How to recognize the land use change in urban villages during dynamic transformation in Haidian District, Beijing, has become a hot topic with the promotion of urban renewal. The GF-1 high-resolution remote sensing images of 2013, 2015, and 2020 were used in this study to reflect the land use change in urban villages before and after urban renewal by using a hierarchical machine learning recognition method based on scene-based and random forest classification. The overall scale of urban village blocks in Haidian was 10.46 km2, showing the distribution pattern along the traffic arteries in 2013. In 2015, it dropped to 10.11 km2. The scale of urban village blocks in 2020 decreased to 1.02 km2, 9.75% of that in 2013. Three kinds of urban village renewal logic are revealed by further taking Chuanying Village as an example: “urban village–blue–green space”, “urban village–real estate”, and “urban village–municipal facilities”.

An Improved YOLOv5 Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images

Tailings ponds’ failure and environmental pollution make tailings monitoring very important. Remote sensing technology can quickly and widely obtain ground information and has become one of the important means of tailings monitoring. However, the efficiency and accuracy of traditional remote sensing monitoring technology have difficulty meeting the management needs. At the same time, affected by factors such as the geographical environment and imaging conditions, tailings have various manifestations in remote sensing images, which all bring challenges to the accurate acquisition of tailings information in large areas. By improving You Only Look Once (YOLO) v5s, this study designs a deep learning-based framework for the large-scale extraction of tailings ponds information from the entire high-resolution remote sensing images. For the improved YOLOv5s, the Swin Transformer is integrated to build the Swin-T backbone, the Fusion Block of efficient Reparameterized Generalized Feature Pyramid Network (RepGFPN) in DAMO-YOLO is introduced to form the RepGFPN Neck, and the head is replaced with Decoupled Head. In addition, sample boosting strategy (SBS) and global non-maximum suppression (GNMS) are designed to improve the sample quality and suppress repeated detection frames in the entire image, respectively. The model test results based on entire Gaofen-6 (GF-6) high-resolution remote sensing images show that the F1 score of tailings ponds is significantly improved by 12.22% compared with YOLOv5, reaching 81.90%. On the basis of both employing SBS, the improved YOLOv5s boots the mAP@0.5 of YOLOv5s by 5.95%, reaching 92.15%. This study provides a solution for tailings ponds’ monitoring and ecological environment management.

Estimating lichen α- and β-diversity using satellite data at different spatial resolutions

Understanding biodiversity patterns and its environmental drivers is crucial to meet conservation targets and develop effective monitoring tools. Inconspicuous species such as lichens require special attention since they are ecologically important but sensitive species that are often overlooked in conservation planning. Remote sensing (RS) can be particularly beneficial for these species as in combination with modelling techniques it allows planners to assess and better understand biodiversity patterns. This study aims to model the lichen α-diversity (species richness) and β-diversity (species turnover) biodiversity components using high resolution RS variables across a subarctic region in Northern Quebec (∼190.25 km2). Two sensors, one commercial (WorldView-3, WV3) and another freely accessible (Sentinel-2, S2), at different resolutions (1.2 m and 10 m, respectively) were tested separately to develop our variables and feed the models. Lichens were sampled in 45 plots across different habitat types, ranging from forested habitats (coniferous, deciduous) to wetlands (bogs, fens) and rocky outcrops. Two sets of uncorrelated variables (Red and NIR; EVI2) from each sensor were parallelly used to build the α- and β-diversity models (8 models in total) through Poisson regressions and generalized dissimilarity modelling (GDM), respectively. Red and NIR variables were useful for modeling the two biodiversity components at both resolutions, providing information on stand canopy closure and structure, respectively. EVI2, especially from WV3, was only informative for assessing β-diversity, providing similar information than Red. Poisson models explained up to 32 % of the variation in lichen α-diversity, with Red, NIR and EVI2, either from WV3 or S2, showing negative relationships with lichen richness. GDMs described well the relationship between β-diversity and spectral dissimilarity (R2 from 0.25 to 0.30), except for the S2 EVI2 model (R2 = 0.07), confirming that more spectrally and thus environmentally different areas tend to harbor different lichen communities. While WV3 often outperformed the S2 sensor, the latter still provides a powerful tool for the study of lichens and their conservation. This study contributes to improve our knowledge and to inform on the use of RS to understand biodiversity patterns of inconspicuous species, which we consider to be an essential step to enhance their representation in conservation planning.

Automatic Extraction of Bare Soil Land from High-Resolution Remote Sensing Images Based on Semantic Segmentation with Deep Learning

Accurate monitoring of bare soil land (BSL) is an urgent need for environmental governance and optimal utilization of land resources. High-resolution imagery contains rich semantic information, which is beneficial for the recognition of objects on the ground. Simultaneously, it is susceptible to the impact of its background. We propose a semantic segmentation model, Deeplabv3+-M-CBAM, for extracting BSL. First, we replaced the Xception of Deeplabv3+ with MobileNetV2 as the backbone network to reduce the number of parameters. Second, to distinguish BSL from the background, we employed the convolutional block attention module (CBAM) via a combination of channel attention and spatial attention. For model training, we built a BSL dataset based on BJ-2 satellite images. The test result for the F1 of the model was 88.42%. Compared with Deeplabv3+, the classification accuracy improved by 8.52%, and the segmentation speed was 2.34 times faster. In addition, compared with the visual interpretation, the extraction speed improved by 11.5 times. In order to verify the transferable performance of the model, Jilin-1GXA images were used for the transfer test, and the extraction accuracies for F1, IoU, recall and precision were 86.07%, 87.88%, 87.00% and 95.80%, respectively. All of these experiments show that Deeplabv3+-M-CBAM achieved efficient and accurate extraction results and a well transferable performance for BSL. The methodology proposed in this study exhibits its application value for the refinement of environmental governance and the surveillance of land use.

A Convolutional Neural Network for Coastal Aquaculture Extraction from High-Resolution Remote Sensing Imagery

Aquaculture has important economic and environmental benefits. With the development of remote sensing and deep learning technology, coastline aquaculture extraction has achieved rapid, automated, and high-precision production. However, some problems still exist in extracting large-scale aquaculture based on high-resolution remote sensing images: (1) the generalization of large-scale models caused by the diversity of remote sensing in breeding areas; (2) the confusion of breeding target identification caused by the complex background interference of land and sea; (3) the boundary of the breeding area is difficult to extract accurately. In this paper, we built a comprehensive sample database based on the spatial distribution of aquaculture, and expanded the sample database by using confusing land objects as negative samples. A multi-scale-fusion superpixel segmentation optimization module is designed to solve the problem of inaccurate boundaries, and a coastal aquaculture network is proposed. Based on the coastline aquaculture dataset that we labelled and produced ourselves, we extracted cage culture areas and raft culture areas near the coastline of mainland China based on high-resolution remote sensing images. The overall accuracy reached 94.64% and achieved a state-of-the-art performance.

Semantic Segmentation of High-Resolution Remote Sensing Images Based on Sparse Self-Attention and Feature Alignment

Semantic segmentation of high-resolution remote sensing images (HRSI) is significant, yet challenging. Recently, several research works have utilized the self-attention operation to capture global dependencies. HRSI have complex scenes and rich details, and the implementation of self-attention on a whole image will introduce redundant information and interfere with semantic segmentation. The detail recovery of HRSI is another challenging aspect of semantic segmentation. Several networks use up-sampling, skip-connections, parallel structure, and enhanced edge features to obtain more precise results. However, the above methods ignore the misalignment of features with different resolutions, which affects the accuracy of the segmentation results. To resolve these problems, this paper proposes a semantic segmentation network based on sparse self-attention and feature alignment (SAANet). Specifically, the sparse position self-attention module (SPAM) divides, rearranges, and resorts the feature maps in the position dimension and performs position attention operations (PAM) in rearranged and restored sub-regions, respectively. Meanwhile, the proposed sparse channel self-attention module (SCAM) groups, rearranges, and resorts the feature maps in the channel dimension and performs channel attention operations (CAM) in the rearranged and restored sub-channels, respectively. SPAM and SCAM effectively model long-range context information and interdependencies between channels, while reducing the introduction of redundant information. Finally, the feature alignment module (FAM) utilizes convolutions to obtain a learnable offset map and aligns feature maps with different resolutions, helping to recover details and refine feature representations. Extensive experiments conducted on the ISPRS Vaihingen, Potsdam, and LoveDA datasets demonstrate that the proposed method precedes general semantic segmentation- and self-attention-based networks.

Semantic understanding of high spatial resolution remote sensing images using directional geospatial relationships

ABSTRACT Semantic understanding of high spatial resolution remote sensing (RS) images can be divided into object detection, object labelling, identification of geospatial relationships, and semantic description generation. Geographical relations represent the spatial distribution dependencies between geospatial entities such as points, lines, and polygons, and the topologies among them. Geospatial relations play a very important role in describing the relations between geographic objects. These relations can be broadly classified as topological, directional, and proximity relations. These relations describe the adjacency and association relations between geospatial objects. An approach to identify an appropriate directional geospatial relationship between geo-objects present in high spatial resolution RS images is proposed in this paper. Geospatial objects in the form of the closed boundary are taken as input and relationship triplets are generated. Two approaches have been used in the identification of directional relationships and the results of both approaches are compared. The first approach is based on the centroid of the objects and the second considers whole objects while calculating the direction. These relations are then further represented using a knowledge graph, where nodes represent objects and edges represent their relationship. Knowledge graph plays a very important role in overall scene understanding. It shows the association of all objects with each other. These relationships are then represented in the form of descriptions by using template-based sentence generation. Results show that these directional relationships are accurately identified between each pair of objects using both approaches, but relations generated by considering whole objects are closer to human cognition. Semantic understanding of remote sensing images is of great significance in different applications such as urban surveys, urban planning, and management, military intelligence, etc.