Aerial Image Dataset Research Articles

Building Detection in High-Resolution Remote Sensing Images by Enhancing Superpixel Segmentation and Classification Using Deep Learning Approaches

Accurate building detection is a critical task in urban development and digital city mapping. However, current building detection models for high-resolution remote sensing images are still facing challenges due to complex object characteristics and similarities in appearance. To address this issue, this paper proposes a novel algorithm for building detection based on in-depth feature extraction and classification of adaptive superpixel shredding. The proposed approach consists of four main steps: image segmentation into homogeneous superpixels using a modified Simple Linear Iterative Clustering (SLIC), in-depth feature extraction using an variational auto-encoder (VAE) scale on the superpixels for training and testing data collection, identification of four classes (buildings, roads, trees, and shadows) using extracted feature data as input to an Convolutional Neural Network (CNN), and extraction of building shapes through regional growth and morphological operations. The proposed approach offers more stability in identifying buildings with unclear boundaries, eliminating the requirement for extensive prior segmentation. It has been tested on two datasets of high-resolution aerial images from the New Zealand region, demonstrating superior accuracy compared to previous works with an average F1 score of 98.83%. The proposed approach shows potential for fast and accurate urban monitoring and city planning, particularly in urban areas.

MKIoU loss: toward accurate oriented object detection in aerial images

Oriented bounding box regression is crucial for oriented object detection. However, regression-based methods often suffer from boundary problems and the inconsistency between loss and evaluation metrics. A modulated Kalman intersection over union (IoU) loss of approximate SkewIoU is proposed, named MKIoU. To avoid boundary problems, we convert the oriented bounding box to Gaussian distribution then use the Kalman filter to approximate the intersection area. However, there exists significant difference between the calculated and actual intersection areas. Thus, we propose a modulation factor to adjust the sensitivity of angle deviation and width-height offset to loss variation, making the loss more consistent with the evaluation metric. Furthermore, the Gaussian modeling method avoids the boundary problem but causes the angle confusion of square objects simultaneously. Thus, the Gaussian angle loss (GA loss) is presented to solve this problem by adding a corrected loss for square targets. The proposed GA loss can be easily extended to other Gaussian-based methods. Experiments on three publicly available aerial image datasets, DOTA, UCAS-AOD, and HRSC2016, show the effectiveness of the proposed method.

Defending against Poisoning Attacks in Aerial Image Semantic Segmentation with Robust Invariant Feature Enhancement

The outstanding performance of deep neural networks (DNNs) in multiple computer vision in recent years has promoted its widespread use in aerial image semantic segmentation. Nonetheless, prior research has demonstrated the high susceptibility of DNNs to adversarial attacks. This poses significant security risks when applying DNNs to safety-critical earth observation missions. As an essential means of attacking DNNs, data poisoning attacks destroy model performance by contaminating model training data, allowing attackers to control prediction results by carefully crafting poisoning samples. Toward building a more robust DNNs-based aerial image semantic segmentation model, in this study, we proposed a robust invariant feature enhancement network (RIFENet) that can resist data poisoning attacks and has superior semantic segmentation performance. The constructed RIFENet improves the resistance to poisoning attacks by extracting and enhancing robust invariant features. Specifically, RIFENet uses a texture feature enhancement module (T-FEM), structural feature enhancement module (S-FEM), global feature enhancement module (G-FEM), and multi-resolution feature fusion module (MR-FFM) to enhance the representation of different robust features in the feature extraction process to suppress the interference of poisoning samples. Experiments on several benchmark aerial image datasets demonstrate that the proposed method is more robust and exhibits better generalization than other state-of-the-art methods.

Distorted Aerial Images Semantic Segmentation Method for Software-Based Analog Image Receivers Using Deep Combined Learning

Aerial images are important for monitoring land cover and land resource management. An aerial imaging source which keeps its position at a higher altitude, and which has a considerable duration of airtime, employs wireless communications for sending images to relevant receivers. An aerial image must be transmitted from the image source to a ground station where it can be stored and analyzed. Due to transmission errors, aerial images which are received from an image transmitter contain distortions which can affect the quality of the images, causing noise, color shifts, and other issues that can impact the accuracy of semantic segmentation and the usefulness of the information contained in the images. Current semantic segmentation methods discard distorted images, which makes the available dataset small or treats them as normal images, which causes poor segmentation results. This paper proposes a deep-learning-based semantic segmentation method for distorted aerial images. For different receivers, distortions occur differently, and by considering the receiver specificness of the distortions, the proposed method was able to grasp the acceptability for a distorted image using semantic segmentation models trained with large aerial image datasets to build a combined model that can effectively segment a distorted aerial image which was received by an analog image receiver. Two combined deep learning models, an approximating model, and a segmentation model were trained combinedly to maximize the segmentation score for distorted images. The results showed that the combined learning method achieves higher intersection-over-union (IoU) scores than the results obtained by using only a segmentation model.

Deteksi Atap Bangunan Berbasis Citra Udara Menggunakan Google Colab dan Algoritma Deep Learning YOLOv7

This article presents a deep learning-based approach for roof building detection in aerial imagery using the Yolov7 algorithm and Google Colab. We trained the Yolov7 model on a dataset of annotated aerial images and tested its performance on a testing set of 200 images. Our experimental results show that the Yolov7 model achieved an F1 score of 80% for roof building detection on the testing set, indicating that our approach can accurately detect roof buildings in aerial imagery. This approach can be useful in various applications, such as urban planning, disaster management, and infrastructure development. The availability of aerial imagery and the ability to use deep learning algorithms make it possible to automate the process of roof building detection, which can save time and resources. This research can be extended by investigating the use of other deep learning algorithms or combining multiple algorithms to improve the accuracy of roof building detection in aerial imagery.

Enhancing building extraction from remote sensing images through UNet and transfer learning

Performing accurate extraction of buildings from remote sensing (RS) images is a crucial process with widespread applications in urban planning, disaster management, and urban monitoring. However, this task remains challenging due to the diversity and complexity of building shapes, sizes, and textures, as well as variations in lighting and weather conditions. These difficulties motivate our research to propose an improved approach for building extraction using UNet and transfer learning to address these challenges. In this work, we tested seven different backbone architectures within the UNet encoder and found that combining UNet with ResNet101 or ResNet152 yielded the best results. Based on these findings, we combined the superior performance of these base models to create a novel architecture, which achieved significant improvements over previous methods. Specifically, our method achieved a 1.33% increase in Intersection over Union (IoU) compared to the baseline UNet model. Furthermore, it demonstrated a superior performance with a 1.21% increase in IoU over UNet-ResNet101 and a 1.60% increase in IoU over UNet-ResNet152. We evaluated our proposed approach on the INRIA Aerial Image dataset and demonstrated its superiority. Our research addresses a critical need for accurate building extraction from RS images and overcomes the challenges posed by diverse building characteristics.

LAB-Net: Lidar and aerial image-based building segmentation using U-Nets

We describe our approach in the 2022 NORA MapAI competition. The objective of the competition was to construct methods that were able to detect and segment buildings from aerial imaging and laser data. There were two subtasks: (1) building segmentation from aerial imaging; (2) building segmentation from lidar data, optionally combined with aerial images. We trained multiple dynamic U-Net models with self-attention layers. For Task 1, we used a ResNet34-based encoder pre-trained on the ImageNet challenge dataset and further pre-trained the U-Net on another similar aerial image dataset. For Task 2, we adapted the dynamic U-Net to deal with multispectral data. Our ensembles placed us in second place, with the top score on Task 1. The complete source code for reproducing our results is available at https://github.com/HVL-ML/LAB-Net.

BuildSeg: A General Framework for the Segmentation of Buildings

Building segmentation from aerial images and 3D laser scanning (LiDAR) is a challenging task due to the diversity of backgrounds, building textures, and image quality. While current research using different types of convolutional and transformer networks has considerably improved the performance on this task, more precise and accurate segmentation methods for buildings are desirable for applications such as automatic mapping. In this study, we propose a general framework termed BuildSeg employing a generic approach that can be quickly applied to segment buildings. Different data sources were combined to increase generalization performance.
 The approach yields good results for different data sources as shown by experiments on high-resolution multi-spectral and LiDAR imagery of cities in Norway, Denmark, and France.
 We applied ConvNeXt and SegFormer-based models on the high-resolution aerial image dataset from the MapAI-competition. The methods achieved an IoU of 0.7902 and a boundary IoU of 0.6185 on the test set. We used post-processing to account for the rectangular shape of the objects.This increased the boundary IOU from 0.6185 to 0.6189. The code is publicly available.
 

Development of the morphodynamics on Little Ice Age lateral moraines in 10 glacier forefields of the Eastern Alps since the 1950s

Abstract. Since the end of the Little Ice Age (LIA) in the middle of the 19th century, Alpine glaciers have been subject to severe recession that is enhanced by the recent global warming. The melting glaciers expose large areas with loose sediments in the form of lateral moraines, amongst other forms. Due to their instability and high slope angle, the lateral moraines are reworked by geomorphological processes such as debris flows, slides, or fluvial erosion. In this study, the development of the morphodynamics and changes in geomorphological processes on lateral moraines were observed over decades, based on a selection of 10 glacier forefields in the Eastern Alps. To identify geomorphological changes over time, several datasets of archival aerial images reaching back to the 1950s were utilized in order to generate digital elevation models (DEMs) and DEMs of difference. The aerial images were complemented by recent drone images for selected moraine sections, enabling a high-resolution analysis of the processes currently occurring. The results concerning the development of morphodynamics on lateral moraine sections are diverse: some slopes display a stagnation of the erosion rates, whereas the rates of one section increase significantly; however, the majority of the slopes show a decline in morphodynamics over decades but stay on a high level in many cases. In particular, moraine sections with high morphodynamics at the beginning of the observation period mostly show high erosion rates up until present-day measurements, with values up to 11 cm yr−1. These moraine sections also feature heavy gullying on their upper slopes. A correlation between the development of morphodynamics and the time since deglaciation could scarcely be established. In fact, the results instead indicate that characteristics of the lateral moraines such as the initial slope angle at the time of deglaciation have a significant influence on the later morphodynamics. These observations raise concerns as to whether the until now often conducted analyses based on the comparison of lateral moraine sections with different distances to the glacier terminus, assumed to represent varying time spans since deglaciation, can provide sound evidence concerning the process of stabilization.

MCAFNet: Multi-Channel Attention Fusion Network-Based CNN For Remote Sensing Scene Classification

Remote sensing scene images are characterized by intra-class diversity and inter-class similarity. When recognizing remote sensing images, traditional image classification algorithms based on deep learning only extract the global features of scene images, ignoring the important role of local key features in classification, which limits the ability of feature expression and restricts the improvement of classification accuracy. Therefore, this paper presents a multi-channel attention fusion network (MCAFNet). First, three channels are used to extract the features of the image. The channel "spatial attention module" is added after the maximum pooling layer of two channels to get the global and local key features of the image. The other channel uses the original model to extract the deep features of the image. Second, features extracted from different channels are effectively fused by the fusion module. Finally, an adaptive weight loss function is designed to automatically adjust the losses in different types of loss functions. Three challenging data sets, UC Merced Land-Use Dataset (UCM), Aerial Image Dataset (AID), and Northwestern Polytechnic University Dataset (NWPU), are selected for the experiment. Experimental results show that our algorithm can effectively recognize scenes and obtain competitive classification results.

Global Feature Attention Network: Addressing the Threat of Adversarial Attack for Aerial Image Semantic Segmentation

Aerial Image Semantic segmentation based on convolution neural networks (CNNs) has made significant process in recent years. Nevertheless, their vulnerability to adversarial example attacks could not be neglected. Existing studies typically focus on adversarial attacks for image classification, ignoring the negative effect of adversarial examples on semantic segmentation. In this article, we systematically assess and verify the influence of adversarial attacks on aerial image semantic segmentation. Meanwhile, based on the robust characteristics of global features, we construct a novel global feature attention network (GFANet) for aerial image semantic segmentation to solve the threat of adversarial attacks. GFANet uses the global context encoder (GCE) to obtain the context dependencies of global features, introduces the global coordinate attention mechanism (GCAM) to enhance the global feature representation to suppress adversarial noise, and the feature consistency alignment (FCA) is used for feature calibration. In addition, we construct a universal adversarial training strategy to improve the robustness of the semantic segmentation model against adversarial example attacks. Extensive experiments on three aerial image datasets demonstrate that GFANet is more robust against adversarial attacks than existing state-of-the-art semantic segmentation models.

Aerial image recognition in discriminative bi-transformer

Aerial image recognition (AIR) has received increasing interest in the remote sensing community, and complex backgrounds with various types of objects bring huge challenges to recognition. Vision transformer (ViT) can capture long-range contextual relations, but it tends to fall into over-fitting for training samples. This problem is particularly acute in AIR where trainable data is insufficient. While training ViT with label smoothing can alleviate this problem, it is easy to pose the risk of underfitting and relies heavily on data quality. Furthermore, the application of ViT in AIR has always been limited because aerial images generally have greater intra-class variation and inter-class similarity than natural images. Thus, we propose a ViT-based framework called D-BiT to extend ViT to learn discriminative features in aerial images by integrating two heads on the ViT backbone. One is a classifier head with the Discriminative Label Smoothing Module (DLSM) to fully learn effective features without overfitting the distribution of noisy data, even in the condition of insufficient data volume. Another is a projector head with a newly-designed supervised contrastive loss to effectively combine label information and achieve a more compact and reasonable intra-class structure. Experimental results on three popular AIR benchmarks demonstrate the effectiveness of D-BiT: Aerial Image Dataset (AID), UC Merced and Event Recognition in Aerial videos (ERA), where we achieve state-of-the-art (SOTA) performance. In particular, D-BiT can significantly improve the accuracy of ViT for recognizing aerial images.

PCIer: Pavement Condition Evaluation Using Aerial Imagery and Deep Learning

This paper aims to explore and evaluate aerial imagery and deep learning technology in pavement condition evaluation. A convolutional neural network (CNN) model, named PCIer, was designed to process aerial images and produce pavement condition index (PCI) estimations, which are classified into four scales of Good (PCI ≥ 70), Fair (50 ≤ PCI < 70), Poor (25 ≤ PCI < 50), and Very Poor (PCI < 25). In the experiment, the PCI datasets were retrieved from the published pavement condition report by the City of Sacramento, CA. Following the retrieved datasets, the authors also collected the corresponding aerial image datasets containing 100 images for each PCI grade from Google Earth. An 80% proportion of datasets were used for PCIer model training, and the remaining were used for testing. Comparisons showed using a 128-channel heatmap layer in the proposed PCIer model and saving the PCIer model with the best validation accuracy would yield the best performance, with a testing accuracy of 0.97, and a weighted average precision, recall, and F1-score of 0.98, 0.97, and 0.97, respectively. Moreover, future research recommendations are provided in the discussion for improving the effectiveness of pavement evaluation via aerial imagery and deep learning.

A crowdsourced dataset of aerial images with annotated solar photovoltaic arrays and installation metadata

Photovoltaic (PV) energy generation plays a crucial role in the energy transition. Small-scale, rooftop PV installations are deployed at an unprecedented pace, and their safe integration into the grid requires up-to-date, high-quality information. Overhead imagery is increasingly being used to improve the knowledge of rooftop PV installations with machine learning models capable of automatically mapping these installations. However, these models cannot be reliably transferred from one region or imagery source to another without incurring a decrease in accuracy. To address this issue, known as distribution shift, and foster the development of PV array mapping pipelines, we propose a dataset containing aerial images, segmentation masks, and installation metadata (i.e., technical characteristics). We provide installation metadata for more than 28000 installations. We supply ground truth segmentation masks for 13000 installations, including 7000 with annotations for two different image providers. Finally, we provide installation metadata that matches the annotation for more than 8000 installations. Dataset applications include end-to-end PV registry construction, robust PV installations mapping, and analysis of crowdsourced datasets.

BUILDING DETECTION FROM AERIAL IMAGERY USING INCEPTION RESNET UNET AND UNET ARCHITECTURES

Abstract. Buildings are one of the key components in change detection, urban planning, and monitoring. The automatic extraction of the building from high-resolution aerial imagery is still challenging due to the variations in their shapes, structures, textures, and colours. Recently, the convolutional neural networks (CNN) show a significant improvement in object detection and extraction that surpasses other methods. To extract building, in this paper two segmentation architectures, the UNet and the Inception ResNet UNet are implemented and then tested on the Inria aerial image datasets. The Inception ResNet UNet utilizes the Inception architecture and residual blocks. This makes the model wide and deep, though there are a few differences between numbers of UNet and Inception ResNet UNet parameters. The analyses show that UNet has a high rate of metrics in the training progress. However, on the unseen dataset, Inception ResNet UNet extracts buildings more accurately (97.95% accuracy and 0.96 in the dice metric) in comparison with UNet (94.30% accuracy and 0.55 in the dice metric).

Multigranularity Decoupling Network With Pseudolabel Selection for Remote Sensing Image Scene Classification

The existing deep networks have shown excellent performance in remote sensing scene classification, which generally requires a large amount of class-balanced training samples. However, deep networks will result in underfitting with imbalanced training samples since they can easily bias toward the majority classes. To address these problems, a multi-granularity decoupling network (MGDNet) is proposed for remote sensing image scene classification. To begin with, we design a multi-granularity complementary feature representation (MGCFR) method to extract fine-grained features from remote sensing images, which utilizes region-level supervision to guide the attention of the decoupling network. Second, a class-imbalanced pseudo-label selection (CIPS) approach is proposed to evaluate the credibility of unlabeled samples. Finally, the diversity component feature (DCF) loss function is developed to force the local features to be more discriminative. Our model performs satisfactorily on three public datasets: UC Merced (UCM), NWPU-RESISC45, and Aerial Image Dataset (AID). Experimental results show that the proposed model yields superior performance compared with other state-of-the-art methods.

LRAD-Net: An Improved Lightweight Network for Building Extraction From Remote Sensing Images

The building extraction method of remote sensing images that uses deep learning algorithms can solve the problems of low efficiency and poor effect of traditional methods during feature extraction. Although some semantic segmentation networks proposed recently can achieve good segmentation performance in extracting buildings, their huge parameters and large amount of calculation lead to great obstacles in practical application. Therefore, we propose a lightweight network (named LRAD-Net) for building extraction from remote sensing images. LRAD-Net can be divided into two stages: encoding and decoding. In the encoding stage, the lightweight RegNet network with 600 million flop (600 MF) is finally selected as our feature extraction backbone net though lots of experimental comparisons. Then, a multiscale depthwise separable atrous spatial pyramid pooling structure is proposed to extract more comprehensive and important details of buildings. In the decoding stage, the squeeze-and-excitation attention mechanism is applied innovatively to redistribute the channel weights before fusing feature maps with low-level details and high-level semantics, thus can enrich the local and global information of the buildings. What's more, a lightweight residual block with polarized self-attention is proposed, it can incorporate features extracted from the space of maps and different channels with a small number of parameters, and improve the accuracy of recovering building boundary. In order to verify the effectiveness and robustness of proposed LRAD-Net, we conduct experiments on a self-annotated UAV dataset with higher resolution and three public datasets (the WHU aerial image dataset, the WHU satellite image dataset and the Inria aerial image dataset). Compared with several representative networks, LRAD-Net can extract more details of building, and has smaller number of parameters, faster computing speed, stronger generalization ability, which can improve the training speed of the network without affecting the building extraction effect and accuracy.

Superpixel-Based Multiscale CNN Approach Toward Multiclass Object Segmentation From UAV-Captured Aerial Images

Unmanned aerial vehicles (UAVs) are promising remote sensors capable of reforming remote sensing applications. However, for artificial intelligence (AI)-guided tasks, like land cover mapping and ground-object mapping, most deep learning-based architectures fail to extract scale-invariant features, resulting in poor performance accuracy. In this context, the article proposes a superpixel-aided multiscale convolutional neural network (CNN) architecture to avoid misclassification in complex urban aerial images.The proposed framework is a two-tier deep learning-based segmentation architecture. In the first stage, a superpixel-based simple linear iterative cluster (SLIC) algorithm produces superpixel images with crucial contextual information. The second stage comprises a multiscale CNN architecture that uses these information-rich superpixel images to extract scale-invariant features for predicting the object class of each pixel. Two UAV-image-based aerial image datasets: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NITRDrone</i> dataset and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">urban drone dataset</i> (UDD) are considered to perform the experimentation. The proposed model outperforms the considered state-of-the-art methods with an intersection of union (IoU) of 76.39% and 86.85% on UDD and NITRDrone datasets, respectively. Experimentally obtained results prove that the proposed architecture performs superior by achieving better performance accuracy in complex and challenging scenarios.

RT-YOLO: A Residual Feature Fusion Triple Attention Network for Aerial Image Target Detection

In recent years, target detection of aerial images of unmanned aerial vehicle (UAV) has become one of the hottest topics. However, target detection of UAV aerial images often presents false detection and missed detection. We proposed a modified you only look once (YOLO) model to improve the problems arising in object detection in UAV aerial images: (1) A new residual structure is designed to improve the ability to extract features by enhancing the fusion of the inner features of the single layer. At the same time, triplet attention module is added to strengthen the connection between space and channel and better retain important feature information. (2) The feature information is enriched by improving the multi-scale feature pyramid structure and strengthening the feature fusion at different scales. (3) A new loss function is created and the diagonal penalty term of the anchor frame is introduced to improve the speed of training and the accuracy of reasoning. The proposed model is called residual feature fusion triple attention YOLO (RT-YOLO). Experiments showed that the mean average precision (mAP) of RT-YOLO is increased from 57.2% to 60.8% on the vehicle detection in aerial image (VEDAI) dataset, and the mAP is also increased by 1.7% on the remote sensing object detection (RSOD) dataset. The results show that the RT-YOLO outperforms other mainstream models in UAV aerial image object detection.

Weakly Supervised Local–Global Anchor Guidance Network for Landslide Extraction With Image-Level Annotations

Weakly supervised learning using image-level annotations has become a popular choice for reducing labeling efforts of remote sensing object extraction. Existing methods exploit inter-pixel relations within an individual image patch for object localizations. When facing large-scale remote sensing images, it is still challenging to obtain global semantic contexts across image patches for feature representation, resulting in inaccurate object localizations. To remedy these issues, we propose a local-global anchor guidance network (LGAGNet) for weakly supervised landslide extraction. Specifically, a structure-aware object locating (SOL) module is developed to capture the spatial structure of landslide objects and extract local category anchors containing informative feature embeddings. Furthermore, we leverage a global anchor aggregation (GAA) module to excavate semantic patterns across image patches based on a memory bank, which is then used as additional context cues to enhance the feature presentation through a cross-attention mechanism. Finally, a hybrid loss function is designed to guide the network training, considering category-aware semantic contrasts and local activation consistency. Experimental results on high-resolution aerial and satellite image datasets verify the effectiveness of the proposed approach on landslide extraction.