Automatic Building Extraction Research Articles

MAFF-HRNet: Multi-Attention Feature Fusion HRNet for Building Segmentation in Remote Sensing Images

Built-up areas and buildings are two main targets in remote sensing research; consequently, automatic extraction of built-up areas and buildings has attracted extensive attention. This task is usually difficult because of boundary blur, object occlusion, and intra-class inconsistency. In this paper, we propose the multi-attention feature fusion HRNet, MAFF-HRNet, which can retain more detailed features to achieve accurate semantic segmentation. The design of a pyramidal feature attention (PFA) hierarchy enhances the multilevel semantic representation of the model. In addition, we develop a mixed convolutional attention (MCA) block, which increases the capture range of receptive fields and overcomes the problem of intra-class inconsistency. To alleviate interference due to occlusion, a multiscale attention feature aggregation (MAFA) block is also proposed to enhance the restoration of the final prediction map. Our approach was systematically tested on the WHU (Wuhan University) Building Dataset and the Massachusetts Buildings Dataset. Compared with other advanced semantic segmentation models, our model achieved the best IoU results of 91.69% and 68.32%, respectively. To further evaluate the application significance of the proposed model, we migrated a pretrained model based on the World-Cover Dataset training to the Gaofen 16 m dataset for testing. Quantitative and qualitative experiments show that our model can accurately segment buildings and built-up areas from remote sensing images.

BuildMapper: A fully learnable framework for vectorized building contour extraction

Deep learning based methods have significantly boosted the study of automatic building extraction from remote sensing images. However, delineating vectorized and regular building contours like a human does remains very challenging, due to the difficulty of the methodology, the diversity of building structures, and the imperfect imaging conditions. In this paper, we propose the first end-to-end learnable building contour extraction framework, named BuildMapper, which can directly and efficiently delineate building polygons just as a human does. BuildMapper consists of two main components: 1) a contour initialization module that generates initial building contours; and 2) a contour evolution module that performs both contour vertex deformation and reduction, which removes the need for complex empirical post-processing used in existing methods. In both components, we provide new ideas, including a learnable contour initialization method to replace the empirical methods, dynamic predicted and ground truth vertex pairing for the static vertex correspondence problem, and a lightweight encoder for vertex information extraction and aggregation, which benefit a general contour-based method; and a well-designed vertex classification head for building corner vertices detection, which casts light on direct structured building contour extraction. We also built a suitable large-scale building dataset, the WHU-Mix (vector) building dataset, to benefit the study of contour-based building extraction methods. The extensive experiments conducted on the WHU-Mix (vector) dataset, the WHU dataset, and the CrowdAI dataset verified that BuildMapper can achieve a state-of-the-art performance, with a higher mask average precision (AP) and boundary AP than both segmentation-based and contour-based methods. We also confirmed that more than 60.0/50.8% of the building polygons predicted by BuildMapper in the WHU-Mix (vector) test sets I/II, 84.2% in the WHU building test set, and 68.3% in the CrowdAI test set are on par with the manual delineation level.

SEMANTIC SEGMENTATION OF UAV IMAGES BASED ON U-NET IN URBAN AREA

Abstract. Semantic segmentation of aerial data has been one of the leading researches in the field of photogrammetry, remote sensing, and computer vision in recent years. Many applications, including airborne mapping of urban scenes, object positioning in aerial images, automatic extraction of buildings from remote sensing or high-resolution aerial images, etc., require accurate and efficient segmentation algorithms. According to the high potential of deep learning algorithms in the classification of complex scenes, this paper aims to train a deep learning model to evaluate the semantic segmentation accuracy of UAV-based images in urban areas. The proposed method implements a deep learning framework based on the U-Net encoder-decoder architecture, which extracts and classifies features through layers of convolution, max pooling, activation, and concatenation in an end-to-end process. The obtained results compare with two traditional machine learning models, Random Forest (RF) and Multi-Layer Perceptron (MLP). They rely on two steps that involve extracting features and classifying images. In this study, the experiments are performed on the UAVid2020 semantic segmentation dataset from the ISPRS database. Results show the effectiveness of the proposed deep learning framework, so that the U-Net architecture achieved the best results with 75.15% overall accuracy, compared to RF and MLP algorithms with 52.51% and 54.65% overall accuracy, respectively.

Multiconstraint Transformer-Based Automatic Building Extraction From High-Resolution Remote Sensing Images

Multiconstraint Transformer-Based Automatic Building Extraction From High-Resolution Remote Sensing Images

Combining Deep Fully Convolutional Network and Graph Convolutional Neural Network for the Extraction of Buildings from Aerial Images

Deep learning technology, such as fully convolutional networks (FCNs), have shown competitive performance in the automatic extraction of buildings from high-resolution aerial images (HRAIs). However, there are problems of over-segmentation and internal cavity in traditional FCNs used for building extraction. To address these issues, this paper proposes a new building graph convolutional network (BGC-Net), which optimizes the segmentation results by introducing the graph convolutional network (GCN). The core of BGC-Net includes two major modules. One is an atrous attention pyramid (AAP) module, obtained by fusing the attention mechanism and atrous convolution, which improves the performance of the model in extracting multi-scale buildings through multi-scale feature fusion; the other is a dual graph convolutional (DGN) module, the build of which is based on GCN, which improves the segmentation accuracy of object edges by adding long-range contextual information. The performance of BGC-Net is tested on two high spatial resolution datasets (Wuhan University building dataset and a Chinese typical city building dataset) and compared with several state-of-the-art networks. Experimental results demonstrate that the proposed method outperforms several state-of-the-art approaches (FCN8s, DANet, SegNet, U-Net, ARC-Net, BAR-Net) in both visual interpretation and quantitative evaluations. The BGC-Net proposed in this paper has better results when extracting the completeness of buildings, including boundary segmentation accuracy, and shows great potential in high-precision remote sensing mapping applications.

MSFTrans: a multi-task frequency-spatial learning transformer for building extraction from high spatial resolution remote sensing images

ABSTRACT Building extraction is significant in urban planning, economic evaluation, and driverless technology development. However, automatic building extraction from high spatial resolution remote sensing images has been a challenging task due to the various building shapes and colors, imaging conditions, and complex background objects. Current methods in building extraction are generally based on deep convolution networks, and they mostly use an encoder-decoder architecture, wherein detailed building features and small buildings are easily omitted in continuous convolution operations. Moreover, buildings with blurred boundaries are only completely extracted with difficulty. To meet these challenges, we propose a multi-task architecture of frequency-spatial learning Transformer to extract buildings from high spatial resolution remote sensing images. Different from current architecture, we designed a frequency-spatial learning module in the framework of multi-task to synthesize the multi-scale spatial features and frequency decomposition features of high-resolution image. Spiking convolution is proposed in this study to enhance the frequency features of buildings by mimicking the neural transmission in human brains. In this way, multi-scale building features can be better preserved and distinguished from background objects. Moreover, a masked-attention Transformer is adopted to improve multi-scale building mask prediction accuracy by synthesizing successive pixel-wise up-sampled feature maps. We also propose a strategy to evaluate the practical transferability of the proposed method by mimicking practical application cases through training and evaluating images with different spatial resolutions from different study areas and datasets. Experiments using five public building datasets (WHU-Building Satellite Dataset I, WHU-Building Satellite Dataset II, Massachusetts Buildings Dataset, Inria Aerial Image Dataset, xBD Building Dataset) demonstrate the strong potential applicability of our proposed method for practical application cases. Our method outperforms five recently proposed state-of-the-art semantic segmentation methods with 36.60% accuracy improvement on extracted buildings and approximately 53.55% recall progress in extracting small building instances. The implementation code will be released after the paper is published.

Multi-Modal Feature Fusion Network with Adaptive Center Point Detector for Building Instance Extraction

Building information extraction utilizing remote sensing technology has vital applications in many domains, such as urban planning, cadastral mapping, geographic information censuses, and land-cover change analysis. In recent years, deep learning algorithms with strong feature construction ability have been widely used in automatic building extraction. However, most methods using semantic segmentation networks cannot obtain object-level building information. Some instance segmentation networks rely on predefined detectors and have weak detection ability for buildings with complex shapes and multiple scales. In addition, the advantages of multi-modal remote sensing data have not been effectively exploited to improve model performance with limited training samples. To address the above problems, we proposed a CNN framework with an adaptive center point detector for the object-level extraction of buildings. The proposed framework combines object detection and semantic segmentation with multi-modal data, including high-resolution aerial images and LiDAR data, as inputs. Meanwhile, we developed novel modules to optimize and fuse multi-modal features. Specifically, the local spatial–spectral perceptron can mutually compensate for semantic information and spatial features. The cross-level global context module can enhance long-range feature dependence. The adaptive center point detector explicitly models deformable convolution to improve detection accuracy, especially for buildings with complex shapes. Furthermore, we constructed a building instance segmentation dataset using multi-modal data for model training and evaluation. Quantitative analysis and visualized results verified that the proposed network can improve the accuracy and efficiency of building instance segmentation.

Efficient Building Inventory Extraction from Satellite Imagery for Megacities

Accurate building inventories are essential for city planning and disaster risk management. Traditionally generated via census or selected small surveys, these suffer from data quality and/or resolution. High-resolution satellite imagery with object segmentation provides an effective alternative, readily capturing large extents. This study develops a highly automated building extraction methodology for location-based building exposure data from high (0.5 m) resolution satellite stereo imagery. The development relied on Taipei test areas covering 13.5 km2 before application to the megacity of Jakarta. Of the captured Taipei buildings, 48.8% are at one-to-one extraction, improving to 71.9% for larger buildings with total floor area >8000 m2, and to 99% when tightly-spaced building clusters are further included. Mean absolute error in extracted footprint area is 16% for these larger buildings. The extraction parameters are tuned for Jakarta buildings using small test areas before covering Jakarta's 643 km2 with over 1.247 million buildings extracted.

A Multiscale and Multitask Deep Learning Framework for Automatic Building Extraction

Detecting buildings, segmenting building footprints, and extracting building edges from high-resolution remote sensing images are vital in applications such as urban planning, change detection, smart cities, and map-making and updating. The tasks of building detection, footprint segmentation, and edge extraction affect each other to a certain extent. However, most previous works have focused on one of these three tasks and have lacked a multitask learning framework that can simultaneously solve the tasks of building detection, footprint segmentation and edge extraction, making it difficult to obtain smooth and complete buildings. This study proposes a novel multiscale and multitask deep learning framework to consider the dependencies among building detection, footprint segmentation, and edge extraction while completing all three tasks. In addition, a multitask feature fusion module is introduced into the deep learning framework to increase the robustness of feature extraction. A multitask loss function is also introduced to balance the training losses among the various tasks to obtain the best training results. Finally, the proposed method is applied to open-source building datasets and large-scale high-resolution remote sensing images and compared with other advanced building extraction methods. To verify the effectiveness of multitask learning, the performance of multitask learning and single-task training is compared in ablation experiments. The experimental results show that the proposed method has certain advantages over other methods and that multitask learning can effectively improve single-task performance.

Cropland encroachment detection via dual attention and multi-loss based building extraction in remote sensing images.

The United Nations predicts that by 2050, the world’s total population will increase to 9.15 billion, but the per capita cropland will drop to 0.151°hm2. The acceleration of urbanization often comes at the expense of the encroachment of cropland, the unplanned expansion of urban area has adversely affected cultivation. Therefore, the automatic extraction of buildings, which are the main carriers of urban population activities, in remote sensing images has become a more meaningful cropland observation task. To solve the shortcomings of traditional building extraction methods such as insufficient utilization of image information, relying on manual characterization, etc. A U-Net based deep learning building extraction model is proposed and named AttsegGAN. This study proposes an adversarial loss based on the Generative Adversarial Network in terms of training strategy, and the additionally trained learnable discriminator is used as a distance measurer for the two probability distributions of ground truth Pdata and prediction Pg. In addition, for the sharpness of the building edge, the Sobel edge loss based on the Sobel operator is weighted and jointly participated in the training. In WHU building dataset, this study applies the components and strategies step by step, and verifies their effectiveness. Furthermore, the addition of the attention module is also subjected to ablation experiments and the final framework is determined. Compared with the original, AttsegGAN improved by 0.0062, 0.0027, and 0.0055 on Acc, F1, and IoU respectively after adopting all improvements. In the comparative experiment. AttsegGAN is compared with state-of-the-arts including U-Net, DeeplabV3+, PSPNet, and DANet on both WHU and Massachusetts building dataset. In WHU dataset, AttsegGAN achieved 0.9875, 0.9435, and 0.8907 on Acc, F1, and IoU, surpassed U-Net by 0.0260, 0.1183, and 0.1883, respectively, demonstrated the effectiveness of the proposed components in a similar hourglass structure. In Massachusetts dataset, AttsegGAN also surpassed state-of-the-arts, achieved 0.9395, 0.8328, and 0.7130 on Acc, F1, and IoU, respectively, it improved IoU by 0.0412 over the second-ranked PSPNet, and it was 0.0025 and 0.0101 higher than the second place in Acc and F1.

Extracting buildings from high-resolution remote sensing images by deep ConvNets equipped with structural-cue-guided feature alignment

In surveying, mapping and geographic information systems, building extraction from remote sensing imagery is a common task. However, there are still some challenges in automatic building extraction. First, using only single-scale depth features cannot take into account the uncertainty of features such as the hue and texture of buildings in images, and the results are prone to missed detection. Moreover, extracted high-level features often lose structural information and have scale differences with low-level features, which results in less accurate extraction of boundaries. To simultaneously address these problems, we propose pyramid feature extraction (PFE) to construct multi-scale representations of buildings, which is inspired by the feature extraction of scale-invariant feature transform. We also apply attention modules in channel dimension and spatial dimension to PFE and low-level feature maps. Furthermore, we use the structural-cue-guided feature alignment module to learn the correlation between feature maps at different levels, obtaining high-resolution features with strong semantic representation and ensuring the integrity of high-level features in both structural and semantic dimensions. An edge loss is applied to get a highly accurate building boundary. For the WHU Building Dataset, our method achieves an F1 score of 95.3% and an Intersection over Union (IoU) score of 90.9%; for the Massachusetts Buildings Dataset, our method achieves an F1 score of 85.0% and an IoU score of 74.1%.

Transformer for the Building Segmentation of Urban Remote Sensing

The automatic extraction of urban buildings based on remote sensing images is important for urban dynamic monitoring, planning, and management. The deep learning has significantly helped improve the accuracy of building extraction. Most remote sensing image segmentation methods are based on convolution neural networks, which comprise encoding and decoding structures. However, the convolution operation cannot learn the remote spatial correlation. Herein we propose the Shift Window Attention of building SWAB-net based on the transformer model to solve the semantic segmentation of building objects. Moreover, the shift window strategy was adopted to determine buildings using urban satellite images with 4 m resolution to extract the features of sequence images efficiently and accurately. We evaluated the proposed network on SpaceNet 7, and the results of comprehensive analysis showed that the network is conducive for efficient remote sensing image research.

A Method for Fully Automatic Building Footprint Extraction From Remote Sensing Images

Automatically mapping building footprints has a wide range of applications in many fields. In recent years, the automatic building extraction methods based on deep learning show an absolute advantage over the traditional image segmentation methods due to its high speed and high precision. However, the building footprint extracted by deep learning is just an irregular building mask. There is still much work to be done to transform the building mask into a vector building footprint in the usual sense. One of the most important tasks is to determine the orientation of each side of the building. Most of the current methods are based on the building mask to determine the orientation of each side of the building. The biggest disadvantage of this method is that it completely relies on the building mask which is often unsatisfactory. In this case, the article proposes a method to determine the orientation of each side of the building based on the building mask and line segments, thereby effectively avoiding the danger of relying on the building mask. Experiments show that the proposed method can achieve high-speed and high-precision automatic extraction of building footprints from remote sensing images, saving costs.

An End-to-End Atrous Spatial Pyramid Pooling and Skip-Connections Generative Adversarial Segmentation Network for Building Extraction from High-Resolution Aerial Images

Automatic building extraction based on high-resolution aerial imagery is an important challenge with a wide range of practical applications. One of the mainstream methods for extracting buildings from high-resolution images is deep learning because of its excellent deep feature extraction capability. However, existing models suffer from the problems of hollow interiors of some buildings and blurred boundaries. Furthermore, the increase in remote sensing image resolution has also led to rough segmentation results. To address these issues, we propose a generative adversarial segmentation network (ASGASN) for pixel-level extraction of buildings. The segmentation network of this framework adopts an asymmetric encoder–decoder structure. It captures and aggregates multiscale contextual information using the ASPP module and improves the classification and localization accuracy of the network using the global convolutional block. The discriminator network is an adversarial network that correctly discriminates the output of the generator and ground truth maps and computes multiscale L1 loss by fusing multiscale feature mappings. The segmentation network and the discriminator network are trained alternately on the WHU building dataset and the China typical cities building dataset. Experimental results show that the proposed ASGASN can accurately identify different types of buildings and achieve pixel-level high accuracy extraction of buildings. Additionally, compared to available deep learning models, ASGASN also achieved the highest accuracy performance (89.4% and 83.6% IoU on these two datasets, respectively).

Comparison of residual and dense neural network approaches for building extraction from high-resolution aerial images

Applications including change detection, disaster management, and urban planning require precise building information, and therefore automatic building extraction has become a significant research topic. With the improvements in sensor and satellite technologies, more data has become available, and with the increased computational power, deep learning methods have emerged as successful tools. In this study, U-Net and FPN architectures using four different backbones (ResNet-50, ResNeXt-50, SE-ResNext-50, and DenseNet-121), and an Attention Residual U-Net approach were used for building extraction from high-resolution aerial images. Two publicly available datasets, Inria Aerial Image Labeling Dataset and Massachusetts Buildings Dataset were used to train and test the models. According to the results, Attention Residual U-Net model has the highest F1 score with 0.8154, IoU score with 0.7102, and test accuracy with 94.51% on the Inria dataset. On the Massachusetts dataset, FPN Dense-Net-121 model has the highest F1 score with 0.7565 and IoU score with 0.6188, and Attention Residual U-Net model has the highest test accuracy with 92.43%. It has been observed that, FPN with DenseNet backbone can be a better choice when working with small size datasets. On the other hand, Attention Residual U-Net approach achieved higher success when a sufficiently large dataset is provided.

Comparative research on different backbone architectures of DeepLabV3+ for building segmentation

Rapid urban growth and globalization affect land use in cities, and the need for automatic interpretation of remote sensing images is constantly increasing. Deep neural networks are becoming widespread in high-resolution aerial and satellite image sources in Earth observation missions. Various convolutional neural network (CNN) architectures have been implemented in building extraction, but it is still challenging to distinguish building class from other man-made classes in public datasets. Here, we present comparative research for automatic building extraction on different data sources using DeepLabV3+ architecture with ResNet-18, ResNet-50, Xception, and MobileNetv2 models. The CNNs are implemented on Inria Aerial Image Labeling, Massachusetts Buildings, and Wuhan University Building Extraction Datasets in terms of evaluation metrics and training and testing time consumption. Our implementation of the DeepLabV3 + ResNet-50 model performed F1-score of 97.44% in Massachusetts Building dataset and intersection over union as 80.75% in Inria dataset, higher than at least 3% than the previous studies.

Fast Implementation of Automatic Building Extraction Using Color Cluster Analysis from Scenery Image

Fast Implementation of Automatic Building Extraction Using Color Cluster Analysis from Scenery Image

Automatic Building Extraction on Satellite Images Using Unet and ResNet50.

Recently, settlement planning and replanning process are becoming the main problem in rapidly growing cities. Unplanned urban settlements are quite common, especially in low-income countries. Building extraction on satellite images poses another problem. The main reason for the problem is that manual building extraction is very difficult and takes a lot of time. Artificial intelligence technology, which has increased significantly today, has the potential to provide building extraction on high-resolution satellite images. This study proposes the differentiation of buildings by image segmentation on high-resolution satellite images with U-net architecture. The open-source Massachusetts building dataset was used as the dataset. The Massachusetts building dataset includes residential buildings of the city of Boston. It was aimed to remove buildings in the high-density city of Boston. In the U-net architecture, image segmentation is performed with different encoders and the results are compared. In line with the work done, 82.2% IoU accuracy was achieved in building segmentation. A high result was obtained with an F1 score of 0.9. A successful image segmentation was achieved with 90% accuracy. This study demonstrated the potential of automatic building extraction with the help of artificial intelligence in high-density residential areas. It has been determined that building mapping can be achieved with high-resolution antenna images with high accuracy achieved.

On the Exploration of Automatic Building Extraction from RGB Satellite Images Using Deep Learning Architectures Based on U-Net

Detecting and localizing buildings is of primary importance in urban planning tasks. Automating the building extraction process, however, has become attractive given the dominance of Convolutional Neural Networks (CNNs) in image classification tasks. In this work, we explore the effectiveness of the CNN-based architecture U-Net and its variations, namely, the Residual U-Net, the Attention U-Net, and the Attention Residual U-Net, in automatic building extraction. We showcase their robustness in feature extraction and information processing using exclusively RGB images, as they are a low-cost alternative to multi-spectral and LiDAR ones, selected from the SpaceNet 1 dataset. The experimental results show that U-Net achieves a 91.9% accuracy, whereas introducing residual blocks, attention gates, or a combination of both improves the accuracy of the vanilla U-Net to 93.6%, 94.0%, and 93.7%, respectively. Finally, the comparison between U-Net architectures and typical deep learning approaches from the literature highlights their increased performance in accurate building localization around corners and edges.

IRU-Net: An Efficient End-to-End Network for Automatic Building Extraction From Remote Sensing Images

Automatic extraction of buildings from High-Resolution Remote Sensing (RS) Imagery is of great practical interest for numerous applications; including urban planning, change detection, disaster management, estimation of human population, and many other geospatial related applications. This paper proposes a novel efficient Improved ResU-Net architecture called IRU-Net, integrating spatial pyramid pooling module with an encoder-decoder structure, in combination with Atrous convolutions, modified residual connections, and a new skip connection between the encoder-decoder features for automatic extraction of buildings from RS images. Moreover, a new dual loss function called binary cross-entropy-dice-loss (BCEDL) is opted that make cross-entropy (CE) and dice loss (DL) and consider both local information and global information to decrease the class imbalance influence and improve the building extraction results. The proposed model is examined to demonstrate its generalization on two publicly available datasets; the Aerial Images for Roof Segmentation (AIRS) Dataset and the Massachusetts buildings dataset. The proposed IRU-Net achieved an average F-1 accuracy of 92.34% for the Massachusetts dataset and 95.65% for the AIRS dataset. When compared to other state-of-the-art deep learning-based models such as SegNet, U-Net, E-Net, ERFNet and SRI-Net, the overall accuracy improvements of our IRU-Net model is 9.0% (0.9725 vs. 0.8842), 5.2% (0.9725 vs. 0.9218), 3.0% (0.9725 vs. 0.9428), 1.4% (0.9725 vs. 0.9588) and 0.93% (0.9725 vs. 0.9635), for AIRS dataset and 11.6%, 5.9%, 3.1%, 2.7% and 1.4%, for Massachusetts building dataset. These results prove the superiority of the proposed model for building extraction from high-resolution RS images.