Unmanned Aerial Vehicle Remote Sensing Research Articles

Extraction of Canal Distribution Information Based on UAV Remote Sensing System and Object-Oriented Method

At present, the extraction of irrigation canal network distribution information is of great significance for developing a digital twin irrigation district. However, due to the low resolution of remote sensing images, it is difficult to effectively identify the canal networks, especially for channels with a width of less than 1 m, where recognition is insufficient. Therefore, the purpose of this study is to extract canal networks of different widths in an irrigation district in Shaanxi Province as the research area. A rule-based object-oriented classification method was employed, utilizing image data collected by the DJI Mavic 3 multispectral UAV (Unmanned Aerial Vehicle) to explore the accuracy of this method in extracting canal distribution information. Based on UAV multispectral remote sensing imagery, the segmentation parameters for the remote sensing imagery were determined using ENVI 5.6 software, with the segmentation threshold set at 60 and the merging threshold set at 80. By combining the spectral and spatial differences between the canals and other ground objects, rules for extracting canal network distribution information were established, and the information on the distribution of channels in this irrigation area was finally obtained. The experimental results showed a maximum recall rate of 91.88% and a maximum precision rate of 57.59%. The overall recall precision rates for the irrigation district were 85.74% and 55.08%, respectively. This method provides a new solution for identifying and extracting canal systems in irrigation districts, offering valuable insights for acquiring canal distribution information and providing a scientific basis for precision irrigation.

A Low-Cost and Lightweight Real-Time Object-Detection Method Based on UAV Remote Sensing in Transportation Systems

Accurate detection of transportation objects is pivotal for enhancing driving safety and operational efficiency. In the rapidly evolving domain of transportation systems, the utilization of unmanned aerial vehicles (UAVs) for low-altitude detection, leveraging remotely-sensed images and videos, has become increasingly vital. Addressing the growing demands for robust, real-time object-detection capabilities, this study introduces a lightweight, memory-efficient model specifically engineered for the constrained computational and power resources of UAV-embedded platforms. Incorporating the FasterNet-16 backbone, the model significantly enhances feature-processing efficiency, which is essential for real-time applications across diverse UAV operations. A novel multi-scale feature-fusion technique is employed to improve feature utilization while maintaining a compact architecture through passive integration methods. Extensive performance evaluations across various embedded platforms have demonstrated the model’s superior capabilities and robustness in real-time operations, thereby markedly advancing UAV deployment in crucial remote-sensing tasks and improving productivity and safety across multiple domains.

LWSDNet: A Lightweight Wheat Scab Detection Network Based on UAV Remote Sensing Images

Wheat scab can reduce wheat yield and quality. Currently, unmanned aerial vehicles (UAVs) are widely used for monitoring field crops. However, UAV is constrained by limited computational resources on-board the platforms. In addition, compared to ground images, UAV images have complex backgrounds and smaller targets. Given the aforementioned challenges, this paper proposes a lightweight wheat scab detection network based on UAV. In addition, overlapping cropping and image contrast enhancement methods are designed to preprocess UAV remote-sensing images. Additionally, this work constructed a lightweight wheat scab detection network called LWSDNet using mixed deep convolution (MixConv) to monitor wheat scab in field environments. MixConv can significantly reduce the parameters of the LWSDNet network through depthwise convolution and pointwise convolution, and different sizes of kernels can extract rich scab features. In order to enable LWSDNet to extract more scab features, a scab feature enhancement module, which utilizes spatial attention and dilated convolution, is designed to improve the ability of the network to extract scab features. The MixConv adaptive feature fusion module is designed to accurately detect lesions of different sizes, fully utilizing the semantic and detailed information in the network to enable more accurate detection by LWSDNet. During the training process, a knowledge distillation strategy that integrates scab features and responses is employed to further improve the average precision of LWSDNet detection. Experimental results demonstrate that the average precision of LWSDNet in detecting wheat scab is 79.8%, which is higher than common object detection models and lightweight object detection models. The parameters of LWSDNet are only 3.2 million (M), generally lower than existing lightweight object detection networks.

Estimation of Coastal Wetland Vegetation Aboveground Biomass by Integrating UAV and Satellite Remote Sensing Data

Aboveground biomass (AGB) serves as a crucial indicator of the carbon sequestration capacity of coastal wetland ecosystems. Conducting extensive field surveys in coastal wetlands is both time-consuming and labor-intensive. Unmanned aerial vehicles (UAVs) and satellite remote sensing have been widely utilized to estimate regional AGB. However, the mixed pixel effects in satellite remote sensing hinder the precise estimation of AGB, while high-spatial resolution UAVs face challenges in estimating large-scale AGB. To fill this gap, this study proposed an integrated approach for estimating AGB using field sampling, a UAV, and Sentinel-2 satellite data. Firstly, based on multispectral data from the UAV, vegetation indices were computed and matched with field sampling data to develop the Field–UAV AGB estimation model, yielding AGB results at the UAV scale (1 m). Subsequently, these results were upscaled to the Sentinel-2 satellite scale (10 m). Vegetation indices from Sentinel-2 data were calculated and matched to establish the UAV–Satellite AGB model, enabling the estimation of AGB over large regional areas. Our findings revealed the AGB estimation model achieved an R2 value of 0.58 at the UAV scale and 0.74 at the satellite scale, significantly outperforming direct modeling from field data to satellite (R2 = −0.04). The AGB densities of the wetlands in Xieqian Bay, Meishan Bay, and Hangzhou Bay, Zhejiang Province, were 1440.27 g/m2, 1508.65 g/m2, and 1545.11 g/m2, respectively. The total AGB quantities were estimated to be 30,526.08 t, 34,219.97 t, and 296,382.91 t, respectively. This study underscores the potential of integrating UAV and satellite remote sensing for accurately assessing AGB in large coastal wetland regions, providing valuable support for the conservation and management of coastal wetland ecosystems.

URS-YOLOv5s: object detection algorithm for UAV remote sensing images

The application of UAV (Unmanned Aerial Vehicle) remote sensing and aerial photography technology is more and more widely. Aiming at the problem of object detection in remote sensing or aerial images, the paper proposes an object detection algorithm for UAV remote sensing images based on YOLOv5s, called URS-YOLOv5s (UAV Remote Sensing - YOLOv5s). Firstly, the paper designs Cross-Connected Dense Network (CCDNet), where each convolution layer is concatenated to the second convolution layer. Secondly, the paper designs Across-Path Fusion Network (APFNet), which the last feature fusion path is directly fused with the backbone network. It increases the location information and semantic information of deep features. Finally, the loss function of the original algorithm is replaced by EIoU (Focal and Efficient IoU), which makes the calculation method of the overall loss more appropriate. On AI-TOD and Visdrone2019 datasets, experiments show that the accuracy of URS-YOLOv5s on mAP@0.5 is 6.3% and 9% higher than YOLOv5s. In addition, compared with YOLOv3 and YOLOv5l which have good detection effect, URS-YOLOv5s has the characteristics of faster detection speed and lower computational cost. Meanwhile, URS-YOLOv5s is more suitable for deployment to mobile devices, such as drones with limited performance.

Research on Assimilation of Unmanned Aerial Vehicle Remote Sensing Data and AquaCrop Model.

Taking the AquaCrop crop model as the research object, considering the complexity and uncertainty of the crop growth process, the crop model can only achieve more accurate simulation on a single point scale. In order to improve the application scale of the crop model, this study inverted the canopy coverage of a tea garden based on UAV multispectral technology, adopted the particle swarm optimization algorithm to assimilate the canopy coverage and crop model, constructed the AquaCrop-PSO assimilation model, and compared the canopy coverage and yield simulation results with the localized model simulation results. It is found that there is a significant regression relationship between all vegetation indices and canopy coverage. Among the single vegetation index regression models, the logarithmic model constructed by OSAVI has the highest inversion accuracy, with an R2 of 0.855 and RMSE of 5.75. The tea yield was simulated by the AquaCrop-PSO model and the measured values of R2 and RMSE were 0.927 and 0.12, respectively. The canopy coverage R2 of each simulated growth period basically exceeded 0.9, and the accuracy of the simulation results was improved by about 19.8% compared with that of the localized model. The results show that the accuracy of crop model simulation can be improved effectively by retrieving crop parameters and assimilating crop models through UAV remote sensing.

Real-time and Intelligent Moving Targets Tracking based on UAV Remote Sensing Video Camera and Brain-like Computing Chips

Abstract. Moving target tracking technology based on Unmanned Aerial Vehicles (UAV) is widely used in many fields such as automatic inspection and emergency response. The existing moving target tracking methods usually have the problems of large computation and low tracking efficiency. Limited by the computing power of the UAV platform, real-time tracking and analysis of multiple targets based on the video data collected by UAV platform is a difficult task. In this paper, we proposed a novel Target Specific Filtering Tracking with Memory (TSFMTrack) method designed for UAV-based real-time tracking tasks, which involves a Tracklet Filtering Module (TFM) for capturing object appearance features and a Tracklet Matching Module (TMM) for bounding box association in each frame. By experimental comparison with other State-Of-The-Art (SOTA) methods on popular MOT and UAV tracking datasets, the TSFMTrack have shown obvious advantages in accuracy, computational efficiency and reliability. Furthermore, we deployed the TSFMTrack on the brain-inspired chip Lynchip KA200, the experimental results have shown that the TSFMTrack is effective on edge computational platform and suitable for UAV real-time tracking tasks.

MAD-UNet: A Multi-Region UAV Remote Sensing Network for Rural Building Extraction.

For the development of an idyllic rural landscape, an accurate survey of rural buildings is essential. The extraction of rural structures from unmanned aerial vehicle (UAV) remote sensing imagery is prone to errors such as misclassifications, omissions, and subpar edge detailing. This study introduces a multi-scale fusion and detail enhancement network for rural building extraction, termed the Multi-Attention-Detail U-shaped Network (MAD-UNet). Initially, an atrous convolutional pyramid pooling module is integrated between the encoder and decoder to enhance the main network's ability to identify buildings of varying sizes, thereby reducing omissions. Additionally, a Multi-scale Feature Fusion Module (MFFM) is constructed within the decoder, utilizing superficial detail features to refine the layered detail information, which improves the extraction of small-sized structures and their edges. A coordination attention mechanism and deep supervision modules are simultaneously incorporated to minimize misclassifications. MAD-UNet has been tested on a private UAV building dataset and the publicly available Wuhan University (WHU) Building Dataset and benchmarked against models such as U-Net, PSPNet, DeepLabV3+, HRNet, ISANet, and AGSCNet, achieving Intersection over Union (IoU) scores of 77.43% and 91.02%, respectively. The results demonstrate its effectiveness in extracting rural buildings from UAV remote sensing images across different regions.

An Improved TransMVSNet Algorithm for Three-Dimensional Reconstruction in the Unmanned Aerial Vehicle Remote Sensing Domain.

It is important to achieve the 3D reconstruction of UAV remote sensing images in deep learning-based multi-view stereo (MVS) vision. The lack of obvious texture features and detailed edges in UAV remote sensing images leads to inaccurate feature point matching or depth estimation. To address this problem, this study improves the TransMVSNet algorithm in the field of 3D reconstruction by optimizing its feature extraction network and costumed body depth prediction network. The improvement is mainly achieved by extracting features with the Asymptotic Pyramidal Network (AFPN) and assigning weights to different levels of features through the ASFF module to increase the importance of key levels and also using the UNet structured network combined with an attention mechanism to predict the depth information, which also extracts the key area information. It aims to improve the performance and accuracy of the TransMVSNet algorithm's 3D reconstruction of UAV remote sensing images. In this work, we have performed comparative experiments and quantitative evaluation with other algorithms on the DTU dataset as well as on a large UAV remote sensing image dataset. After a large number of experimental studies, it is shown that our improved TransMVSNet algorithm has better performance and robustness, providing a valuable reference for research and application in the field of 3D reconstruction of UAV remote sensing images.

Early Detection of Rice Leaf Blast Disease Using Unmanned Aerial Vehicle Remote Sensing: A Novel Approach Integrating a New Spectral Vegetation Index and Machine Learning

Leaf blast is recognized as one of the most devastating diseases affecting rice production in the world, seriously threatening rice yield. Therefore, early detection of leaf blast is extremely important to limit the spread and propagation of the disease. In this study, a leaf blast-specific spectral vegetation index RBVI = 9.78R816−R724 − 2.08(ρ736/R724) was designed to qualitatively detect the level of leaf blast disease in the canopy of a field and to improve the accuracy of early detection of leaf blast by remote sensing by unmanned aerial vehicle. Stacking integrated learning, AdaBoost, and SVM were used to compare and analyze the performance of the RBVI and traditional vegetation index for early detection of leaf blast. The results showed that the stacking model constructed based on the RBVI spectral index had the highest detection accuracy (OA: 95.9%, Kappa: 93.8%). Compared to stacking, the detection accuracy of the SVM and AdaBoost models constructed based on the RBVI is slightly degraded. Compared with conventional SVIs, the RBVI had higher accuracy in its ability to qualitatively detect leaf blast in the field. The leaf blast-specific spectral index RBVI proposed in this study can more effectively improve the accuracy of UAV remote sensing for early detection of rice leaf blast in the field and make up for the shortcomings of UAV hyperspectral detection, which is susceptible to interference by environmental factors. The results of this study can provide a simple and effective method for field management and timely control of the disease.

Using UAVRS and deep learning to conduct resource surveys of threatened Tibetan medicinal plants in the Qinghai-Tibet Plateau

Gentiana szechenyii Kanitz. and Gentiana veitchiorum Hemsl. are two wild medicinal plants widely used in Tibetan medicine. In recent decades, their wild populations have declined rapidly due to persistent over-harvesting of their flowers. Because their flowers are small and dense, it is difficult to rely on manual counting under wild conditions. The combination of deep learning and unmanned aerial vehicle remote sensing (UAVRS) is a new method for plant surveys. We trained 12 of the most advanced or widely used YOLO models on a custom dataset to achieve quantitative detection of both plant flowers in UAVRS images. The accuracy, precision and recall of G. szechenyii Flower (GSF) detection can reach 97.00%, 90.40% and 95.40%, respectively (based on YOLOv7). Similarly, those of G. veitchiorum Flower (GVF) detection can reach 93.40%, 91.30% and 90.60%, respectively (based on YOLOv5n), and the highest mAP can reach 94.90% (based on YOLOv5m). Based on the detection results, it is calculated that the total yield of dried GSF and GVF that can be harvested as medicinal materials in study area ranges from 1.88 to 2.10 g·m−2 and 4.42–4.62 g·m−2, respectively. The results show that deep learning and UAVRS can be used for quantitative detection of GSF and GVF, which is helpful for further research and protection of these two plants.

Urban Vegetation Classification for Unmanned Aerial Vehicle Remote Sensing Combining Feature Engineering and Improved DeepLabV3+

Addressing the problems of misclassification and omissions in urban vegetation fine classification from current remote sensing classification methods, this research proposes an intelligent urban vegetation classification method that combines feature engineering and improved DeepLabV3+ based on unmanned aerial vehicle visible spectrum images. The method constructs feature engineering under the ReliefF algorithm to increase the number of features in the samples, enabling the deep learning model to learn more detailed information about the vegetation. Moreover, the method improves the classical DeepLabV3+ network structure based on (1) replacing the backbone network using MoblieNetV2; (2) adjusting the atrous spatial pyramid pooling null rate; and (3) adding the attention mechanism and the convolutional block attention module. Experiments were conducted with self-constructed sample datasets, where the method was compared and analyzed with a fully convolutional network (FCN) and U-Net and ShuffleNetV2 networks; the migration of the method was tested as well. The results show that the method in this paper is better than FCN, U-Net, and ShuffleNetV2, and reaches 92.27%, 91.48%, and 85.63% on the accuracy evaluation indices of overall accuracy, MarcoF1, and mean intersection over union, respectively. Furthermore, the segmentation results are accurate and complete, which effectively alleviates misclassifications and omissions of urban vegetation; moreover, it has a certain migration ability that can quickly and accurately classify the vegetation.

Application of Unmanned Aerial Vehicle Remote Sensing for Agricultural Monitoring

Precision agriculture and Smart farming have become the essential backbone for sustainable agricultural production by leveraging cutting-edge remote sensing and communication technologies. With the remarkable development in recent decades, Unmanned Aerial Vehicles (UAV) based sensing technologies have gained rapid proliferation and exploitation in precision agriculture. The unprecedented availability of high resolution (spatial, spectral and temporal) satellite images has promoted the use of remote sensing in many PA applications, including crop monitoring, irrigation management, nutrient application, disease and pest management, and yield prediction. Using thermal infrared or hyperspectral cameras, UAV remote sensing takes extensive pictures of farmland. It also collects the reflection bands of various crops, monitors and analyzes crop growth trends, locates disease and insect pests, and determines the moisture content of the soil based on variations in the bands. This paper will discuss the basic principles of UAV remote sensing technology for monitoring crop growth, disease, and insect pests, the application of soil vegetation index monitoring, and the development prospects of UAV remote sensing in agricultural monitoring.

The Classification Characteristics and Dynamic Changes of Desert Vegetation Based on Unmanned Aerial Vehicle Remote Sensing

The common ground sample method of vegetation survey requires a lot of manpower and time. Meanwhile, many vegetation information may not be revealed because the scale of standard sample is usually only 100 m2. For this reason, this study developed a set of desert vegetation sample survey method based on UAV technology, and further developed a set of vegetation spatial distribution pattern analysis method based on it, and carried out an empirical study with typical shrub communities in Inner Mongolia as an example. It is expected to provide scientific basis for wind and sand control, vegetation restoration and ecological construction in the ecologically fragile area at the edge of Inner Mongolia.

Intelligent Segmentation and Change Detection of Dams Based on UAV Remote Sensing Images

Guilin is situated in the southern part of China with abundant rainfall. There are 137 reservoirs, which are widely used for irrigation, flood control, water supply and power generation. However, there has been a lack of systematic and full-coverage remote sensing monitoring of reservoir dams for a long time. According to the latest public literature, high-resolution unmanned aerial vehicle (UAV) remote sensing has not been used to detect changes on the reservoir dams of Guilin. In this paper, an intelligent segmentation change detection method is proposed to complete the detection of dam change based on multitemporal high-resolution UAV remote sensing data. Firstly, an enhanced GrabCut that fuses the linear spectral clustering (LSC) superpixel mapping matrix and the Sobel edge operator is proposed to extract the features of reservoir dams. The edge operator is introduced into GrabCut to redefine the new energy function’s smooth item, which makes the segmentation results of enhanced GrabCut more robust and accurate. Then, through image registration, the multitemporal dam extraction results are unified to the same coordinate system to complete the difference operation, and finally the dam change results are obtained. The experimental results of two representative reservoir dams in Guilin show that the proposed method can achieve a very high accuracy of change detection, which is an important reference for related research.

Analyses of the Spatial Morphology of Traditional Yunnan Villages Utilizing Unmanned Aerial Vehicle Remote Sensing

Traditional villages are encountering contemporary problems in terms of preservation and comprehension as a result of their complex spatial morphologies, which are distinguished by their extensive cultural and architectural legacy, including distinctive architectural styles and historical significance. This study provides evidence that reinforces the unique characteristics of these settlements, supported by particular data and findings. The primary objective of this study is to employ a quantitative approach in order to assess and conserve the spatial morphology of the aforementioned villages. This will be achieved through the utilization of a novel methodology. The utilization of oblique photography technology, which involves taking photos from an angled perspective, along with the incorporation of many quantitative indices, enables the classification and evaluation of traditional villages. The indices encompass measurements such as length/width and perimeter/area ratios, shape index measurements for assessing form complexity, and fractal dimensions for quantifying pattern complexity. This methodology is employed to examine five discrete villages located in Yunnan Province, thereby uncovering distinctive village layouts and boundary characteristics and providing valuable insights into spatial cohesion. This approach offers a full analysis of traditional village morphologies, thus providing useful insights for the preservation of history and the protection of villages.

Combining vegetation, color, and texture indices with hyperspectral parameters using machine-learning methods to estimate nitrogen concentration in rice stems and leaves

Context or problemNitrogen is one of the important elements of crops, which plays a decisive role in crop growth and development and the formation of yields. Monitoring of rice organ-scale nitrogen concentration based on the unmanned aerial vehicle (UAV) images is of great significance for rice field management and yield prediction. Objective or research questionPrevious studies have focused on the use of traditional statistical methods and chlorophyll-related vegetation indices to construct plant nitrogen concentration, with models lacking generalizability. MethodsIn this study, rice field trials of two varieties (NJ9108, YD6) and nitrogen fertilizer treatments (N0-N3: 0, 105, 210 and 315 kg/ha) were conducted for 3 years with manual sampling and UAV digital and hyperspectral images during key fertility periods. Based on the data of the whole growth periods and combined with vegetation indices (VIs), color indices (CIs), hyperspectral parameters (HPs), texture indices (TIs) and machine-learning algorithms, monitoring models of nitrogen concentration at the organ scale of rice were constructed and used to estimate the N content of multiple organs (leaf and stem) of rice at different periods. Field experiments were used to collect the multi-organ nitrogen concentration of rice and the remote sensing (RS) data of UAV during the critical growth period of the two years (2021, 2022), and machine-learning algorithms were used to construct the estimation models. ResultsThe results showed that VIs had good correlations with leaf nitrogen concentration (LNC), stem nitrogen concentration (SNC) and plant nitrogen concentration (PNC), with correlation coefficients (r) of 0.86, 0.74 and 0.81, respectively. Machine learning estimation models combining multiple types of RS indices were more accurate than single parameter models constructed by traditional statistical methods, with the LNC optimal model (R2 = 0.8, RMSE = 3.83 mg/g), the SNC optimal model (R2 = 0.7, RMSE = 2.43 mg/g) and the PNC optimal model (R2 = 0.7, RMSE = 3.19 mg/g). Validated using data from 2020, the machine-learning models were far more accurate than traditional methods. ConclusionsThese results show that the use of multi-source remote sensing data based on machine-learning algorithms can effectively estimate the nitrogen concentration of organs in rice. ImplicationsThis study provides an accurate, stable and universal method for estimating rice nitrogen concentration in rice organs, which can be used as a reference for estimating rice nitrogen concentration in large fields using UAV RS technology.

Attention-Enhanced One-Shot Attack against Single Object Tracking for Unmanned Aerial Vehicle Remote Sensing Images

Recent studies have shown that deep-learning-based models for processing Unmanned Aerial Vehicle (UAV) remote sensing images are vulnerable to artificially designed adversarial examples, which can lead to incorrect predictions of deep models when facing adversarial examples. Previous adversarial attack methods have mainly focused on the classification and detection of UAV remote sensing images, and there is still a lack of research on adversarial attacks for object tracking in UAV video. To address this challenge, we propose an attention-enhanced one-shot adversarial attack method for UAV remote sensing object tracking, which perturbs only the template frame and generates adversarial samples offline. First, we employ an attention feature loss to make the original frame’s features dissimilar to those of the adversarial frame, and an attention confidence loss to either suppress or enhance different confidence scores. Additionally, by forcing the tracker to concentrate on the background information near the target, a background distraction loss is used to mismatch templates with subsequent frames. Finally, we add total variation loss to generate adversarial examples that appear natural to humans. We validate the effectiveness of our method against popular trackers such as SiamRPN, DaSiamRPN, and SiamRPN++ on the UAV123 remote sensing dataset. Experimental results verify the superior attack performance of our proposed method.

Estimating and Mapping Soil Salinity in Multiple Vegetation Cover Periods by Using Unmanned Aerial Vehicle Remote Sensing

Soil salinization is a severe soil degradation issue in arid and semiarid regions. The distribution of soil salinization can prove useful in mitigating soil degradation. Remote sensing monitoring technology is available for obtaining the distribution of soil salinization rapidly and nondestructively. In this study, experimental data were collected from seven study areas of the Hetao Irrigation District from July to August in 2021 and 2022. The soil salt content (SSC) was considered at various soil depths, and the crop type and time series were considered as environmental factors. We analyzed the effects of various environmental factors on the sensitivity response of unmanned aerial vehicle (UAV)-derived spectral index variables to the SSC and assessed the accuracy of SSC estimations. The five indices with the highest correlation with the SSC under various environmental factors were the input parameters used in modeling based on three machine learning algorithms. The best model was subsequently used to derive prediction distribution maps of the SSC. The results revealed that the crop type and time series did not affect the relationship strength between the SSC and spectral indices, and that the classification of the crop type and time series can considerably enhance the accuracy of SSC estimation. The mask treatment of the soil pixels can improve the correlation between some spectral indices and the SSC. The accuracies of the ANN and RFR models were higher than SVR accuracy (optimal R2 = 0.52–0.79), and the generalization ability of ANN was superior to that of RFR. In this study, considering environmental factors, a UAV remote sensing estimation and mapping method was proposed. The results of this study provide a reference for the high-precision prediction of soil salinization during the vegetation cover period.

A Generative Adversarial Network with Spatial Attention Mechanism for Building Structure Inference Based on Unmanned Aerial Vehicle Remote Sensing Images

The acquisition of building structures has broad applications across various fields. However, existing methods for inferring building structures predominantly depend on manual expertise, lacking sufficient automation. To tackle this challenge, we propose a building structure inference network that utilizes UAV remote sensing images, with the PIX2PIX network serving as the foundational framework. We enhance the generator by incorporating an additive attention module that performs multi-scale feature fusion, enabling the combination of features from diverse spatial resolutions of the feature map. This modification enhances the model’s capability to emphasize global relationships during the mapping process. To ensure the completeness of line elements in the generator’s output, we design a novel loss function based on the Hough transform. A line penalty term is introduced that transforms the output of the generator and ground truth to the Hough domain due to the original loss function’s inability to effectively constrain the completeness of straight-line elements in the generated results in the spatial domain. A dataset of the appearance features obtained from UAV remote sensing images and the internal floor plan structure is made. Using UAV remote sensing images of multi-story residential buildings, high-rise residential buildings, and office buildings as test collections, the experimental results show that our method has better performance in inferring a room’s layout and the locations of load-bearing columns, achieving an average improvement of 11.2% and 21.1% over PIX2PIX in terms of the IoU and RMSE, respectively.