High-resolution Aerial Images Research Articles

Paper] Deep Learning based Hierarchical Object Detection System Adopting a Depth Correction Scheme for High-Resolution Aerial Images Towards Realization of Autonomous UAV Flight

Paper] Deep Learning based Hierarchical Object Detection System Adopting a Depth Correction Scheme for High-Resolution Aerial Images Towards Realization of Autonomous UAV Flight

Accurate Detection and Tracking of Small-Scale Vehicles in High-Altitude Unmanned Aerial Vehicle Bird-View Imagery

Vehicle detection and tracking from unmanned aerial vehicles (UAVs) aerial images are among the main tasks of intelligent traffic systems. Especially in tasks with long distances, extensive backgrounds, and small objects, it increases the difficulty of localization and regression, which can easily lead to missed detections and false positives. This paper proposes a detection-based small-scale vehicle tracking framework that integrates an improved YOLOX network and the DeepSORT algorithm to address these issues. Based on the original YOLOX network, a shallow feature extraction network, 160 × 160 pixels, is added to enhance the ability to extract small-scale object features. A convolutional block attention module (CBAM) is inserted in front of the neck network to select crucial information for vehicle detection tasks while suppressing noncritical ones. EIoU_Loss is introduced as the bounding box regression loss function in training to speed up their convergence and improve the localization accuracy of the small objects. Furthermore, an image segmentation method is proposed to effectively reduce missed and false detection events. It divides the original high-definition image into multiple subimages, first detected and then reassembled. Finally, the improved YOLOX network is used as the detector of the DeepSORT to perform small-scale vehicle detection and tracking tasks in various traffic scenarios. Experiments show that the proposed method can significantly improve the detection accuracy of the network and effectively solve the problems of missed detection and false positives in small-scale vehicle tracking tasks in high-resolution aerial images captured by high-altitude UAVs. Significantly, the algorithm proposed in this paper has sufficient robustness for small-scale tracking tasks of aerial videos captured at different altitudes.

Self-Adaptive-Filling Deep Convolutional Neural Network Classification Method for Mountain Vegetation Type Based on High Spatial Resolution Aerial Images

The composition and structure of mountain vegetation are complex and changeable, and thus urgently require the integration of Object-Based Image Analysis (OBIA) and Deep Convolutional Neural Networks (DCNNs). However, while integration technology studies are continuing to increase, there have been few studies that have carried out the classification of mountain vegetation by combining OBIA and DCNNs, for it is difficult to obtain enough samples to trigger the potential of DCNNs for mountain vegetation type classification, especially using high-spatial-resolution remote sensing images. To address this issue, we propose a self-adaptive-filling method (SAF) to incorporate the OBIA method to improve the performance of DCNNs in mountain vegetation type classification using high-spatial-resolution aerial images. Using this method, SAF technology was employed to produce enough regular sample data for DCNNs by filling the irregular objects created by image segmenting using interior adaptive pixel blocks. Meanwhile, non-sample segmented image objects were shaped into different regular rectangular blocks via SAF. Then, the classification result was defined by voting combining the DCNN performance. Compared to traditional OBIA methods, SAF generates more samples for the DCNN and fully utilizes every single pixel of the DCNN input. We design experiments to compare them with traditional OBIA and semantic segmentation methods, such as U-net, MACU-net, and SegNeXt. The results show that our SAF-DCNN outperforms traditional OBIA in terms of accuracy and it is similar to the accuracy of the best performing method in semantic segmentation. However, it reduces the common pretzel phenomenon of semantic segmentation (black and white noise generated in classification). Overall, the SAF-based OBIA using DCNNs, which is proposed in this paper, is superior to other commonly used methods for vegetation classification in mountainous areas.

High-Resolution Aerial Image by Placing a Ball Lens at a Virtually Conjugate Position in AIRR Optical System

High-Resolution Aerial Image by Placing a Ball Lens at a Virtually Conjugate Position in AIRR Optical System

Evaluating the effects of texture features on Pinus sylvestris classification using high-resolution aerial imagery

Pinus sylvestris, also known as Scots pine, is the most widespread type of pine tree found around the world. Texture features can recognize patterns in an image's texture and so distinguish different species with similar spectral properties. They also record information about the spatial layout of pixels inside an image. Nonetheless, previous research has not addressed the effects of textural variables on Scots pine classification using high-resolution aerial images. In the context of this investigation, we used TreeSatAI, a benchmark dataset with categorized high-resolution aerial imagery, to conduct a comparative analysis, and we examined a comprehensive set of 93 texture features (radiomics) by generating several parameters such as First Order, Gray Level Co-occurrence Matrix (GLCM), Gray Level Dependence Matrix (GLDM), Gray Level Size Zone Matrix (GLSZM), and Neighboring Gray Tone Difference Matrix (NGTDM). We proceeded to train machine learning models utilizing the state-of-the-art algorithms such as XGBoost, LightGBM, and Gradient Boosting Machines (GBM). Upon conducting a more comprehensive analysis, it has come to light that the GLDM and GLRLM features have exhibited greater significance in the realm of satellite image processing research, specifically pertaining to the precise classification of Scots pine. This finding challenges the conventional reliance on GLCM features, which have traditionally been favored in similar studies. This study is remarkable due to its emphasis on the efficacy of employing high-resolution aerial imagery and texture features within the realm of satellite image processing applications.

Improving the performance of learned descriptors in the matching of high spatial resolution aerial images by proposing a large-scale dataset of vertical images

Improving the performance of learned descriptors in the matching of high spatial resolution aerial images by proposing a large-scale dataset of vertical images

Feasibility of Detecting Sweet Potato (Ipomoea batatas) Virus Disease from High-Resolution Imagery in the Field Using a Deep Learning Framework

The sweet potato is an essential food and economic crop that is often threatened by the devastating sweet potato virus disease (SPVD), especially in developing countries. Traditional laboratory-based direct detection methods and field scouting are commonly used to rapidly detect SPVD. However, these molecular-based methods are costly and disruptive, while field scouting is subjective, labor-intensive, and time-consuming. In this study, we propose a deep learning-based object detection framework to assess the feasibility of detecting SPVD from ground and aerial high-resolution images. We proposed a novel object detector called SPVDet, as well as a lightweight version called SPVDet-Nano, using a single-level feature. These detectors were prototyped based on a small-scale publicly available benchmark dataset (PASCAL VOC 2012) and compared to mainstream feature pyramid object detectors using a leading large-scale publicly available benchmark dataset (MS COCO 2017). The learned model weights from this dataset were then transferred to fine-tune the detectors and directly analyze our self-made SPVD dataset encompassing one category and 1074 objects, incorporating the slicing aided hyper inference (SAHI) technology. The results showed that SPVDet outperformed both its single-level counterparts and several mainstream feature pyramid detectors. Furthermore, the introduction of SAHI techniques significantly improved the detection accuracy of SPVDet by 14% in terms of mean average precision (mAP) in both ground and aerial images, and yielded the best detection accuracy of 78.1% from close-up perspectives. These findings demonstrate the feasibility of detecting SPVD from ground and unmanned aerial vehicle (UAV) high-resolution images using the deep learning-based SPVDet object detector proposed here. They also have great implications for broader applications in high-throughput phenotyping of sweet potatoes under biotic stresses, which could accelerate the screening process for genetic resistance against SPVD in plant breeding and provide timely decision support for production management.

Deep Learning Approach for Multi-class Semantic Segmentation of UAV Images

Image understanding plays a very crucial role in remote sensing applications. For this, image semantic segmentation is one of the approaches where each pixel of an image is assigned to particular classes based on various features. Aerial semantic segmentation suffers from the class imbalance problem. Proper differentiation of least represented categories is challenging and a goal for the state-of-art approach. In this work, we present a novel deep learning method to perform this task. We proposed a lightweight encoder-decoder network residual depth separable UNet (RDS-UNet) and conditional random field for effective segmentation on very high-resolution aerial images. We proposed patch-with-multi-class sampling to handle the class imbalance problem without increasing the computational overhead during the training process. We created a semi-precise annotated UAV dataset named NESAC UAV Seg for the aerial semantic segmentation task. We demonstrated the efficacy of our model using the publicly available benchmark Drone Deploy dataset and our NESAC UAV Seg dataset. Our model required approximately half the number of trainable parameters and floating point operations compared to other methods. A detailed ablation study is presented to showcase the effectiveness of various modules utilized in our network.

Forest canopy mortality during the 2018-2020 summer drought years in Central Europe: The application of a deep learning approach on aerial images across Luxembourg

Abstract Efficient monitoring of tree canopy mortality requires data that cover large areas and capture changes over time while being precise enough to detect changes at the canopy level. In the development of automated approaches, aerial images represent an under-exploited scale between high-resolution drone images and satellite data. Our aim herein was to use a deep learning model to automatically detect canopy mortality from high-resolution aerial images after severe drought events in the summers 2018–2020 in Luxembourg. We analysed canopy mortality for the years 2017–2020 using the EfficientUNet++, a state-of-the-art convolutional neural network. Training data were acquired for the years 2017 and 2019 only, in order to test the robustness of the model for years with no reference data. We found a severe increase in canopy mortality from 0.64 km2 in 2017 to 7.49 km2 in 2020, with conifers being affected at a much higher rate than broadleaf trees. The model was able to classify canopy mortality with an F1-score of 66%–71% and we found that for years without training data, we were able to transfer the model trained on other years to predict canopy mortality, if illumination conditions did not deviate severely. We conclude that aerial images hold much potential for automated regular monitoring of canopy mortality over large areas at canopy level when analysed with deep learning approaches. We consider the suggested approach a cost-efficient and -effective alternative to drone and field-based sampling.

Weed mapping in multispectral drone imagery using lightweight vision transformers

In precision agriculture, non-invasive remote sensing can be used to observe crops and weeds in visible and non-visible spectra. This paper proposes a novel approach for weed mapping using lightweight Vision Transformers. The method uses a lightweight Transformer architecture to process high-resolution aerial images obtained from drones and performs semantic segmentation to distinguish between crops and weeds. The method also employs specific architectural designs to enable transfer learning from RGB weights in a multispectral setting. For this purpose, the WeedMap dataset, acquired by drones equipped with multispectral cameras, was used. The experimental results demonstrate the effectiveness of the proposed method, exceeding the state-of-the-art. Our approach also enables more efficient mapping, allowing farmers to quickly and easily identify infested areas and prioritize their control efforts. These results encourage using drones as versatile computer vision flying devices for herbicide management, thereby improving crop yields. The code is available at https://github.com/pasqualedem/LWViTs-for-weedmapping.

Aedes aegypti abundance in urban neighborhoods of Maricopa County, Arizona, is linked to increasing socioeconomic status and tree cover

BackgroundUnderstanding coupled human-environment factors which promote Aedes aegypti abundance is critical to preventing the spread of Zika, chikungunya, yellow fever and dengue viruses. High temperatures and aridity theoretically make arid lands inhospitable for Ae. aegypti mosquitoes, yet their populations are well established in many desert cities.MethodsWe investigated associations between socioeconomic and built environment factors and Ae. aegypti abundance in Maricopa County, Arizona, home to Phoenix metropolitan area. Maricopa County Environmental Services conducts weekly mosquito surveillance with CO2-baited Encephalitis Vector Survey or BG-Sentinel traps at > 850 locations throughout the county. Counts of adult female Ae. aegypti from 2014 to 2017 were joined with US Census data, precipitation and temperature data, and 2015 land cover from high-resolution (1 m) aerial images from the National Agricultural Imagery Program.ResultsFrom 139,729 trap-nights, 107,116 Ae. aegypti females were captured. Counts were significantly positively associated with higher socioeconomic status. This association was partially explained by higher densities of non-native landscaping in wealthier neighborhoods; a 1% increase in the density of tree cover around the trap was associated with a ~ 7% higher count of Ae. aegypti (95% CI: 6–9%).ConclusionsMany models predict that climate change will drive aridification in some heavily populated regions, including those where Ae. aegypti are widespread. City climate change adaptation plans often include green spaces and vegetation cover to increase resilience to extreme heat, but these may unintentionally create hospitable microclimates for Ae. aegypti. This possible outcome should be addressed to reduce the potential for outbreaks of Aedes-borne diseases in desert cities.Graphical

Rethinking urban riparian ecosystems as a frontline strategy to counter climate change: mapping 60 years of carbon sequestration evolution in Padua, Italy

Nature-based solutions (NBSs) are considered essential measures to face climate change in cities. Riparian ecosystems are often undervalued as NBSs even if they represent an “already plugged-in” ecosystem service provider. However, applied research on urban riparian ecosystems is scarce. To fill the gap, this study frames the role of these ecosystems as a frontline mitigation and adaptation strategy by (i) investigating how urban riparian ecosystems experienced a process of urbanization from World War II until today in Padua (Italy), (ii) estimating how effective was carbon sequestration, and (iii) assessing how an afforestation process might contribute to mitigation scenarios. Land use/land cover changes are first mapped with high-resolution aerial images and, by using InVEST models, carbon sequestration is estimated. Results highlight that impervious surfaces notably increased along these ecosystems; hence, carbon sequestration was negative (−928 Mg/C). The future scenario (2022–2050) indicates an increase of carbon stocks (+4,329.95 Mg/C), giving a contribution toward city carbon neutrality. Results suggest that, in the recent past, the urban management and policies did not provide clear and well-defined planning of these areas. On the contrary, at present, urban riparian ecosystems might represent already existing multi-functional NBSs to be restored and enhanced within cities for climate-resilient planning and for increasing the life quality of city dwellers. Finally, our methodology provides a replicable spatial database for supporting urban planning policies.

Spatiotemporal changes of coastal land use land cover and its drivers in Shanghai, China between 1989 and 2015

The coastal zone, an ecotone that provides vital diverse ecosystem services, is one of the most ecologically fragile and sensitive areas on the Earth. It has been strongly influenced by human activities and climate change. Thus, understanding the spatiotemporal changes of land use land cover in the coast is an essential prerequisite for the comprehensive evaluation of coastal ecosystems and the promotion of sustainable development. This study investigated the spatial and temporal changes of land use land cover in the coastal zone of mainland Shanghai by using high spatial resolution aerial images, and the associated driving factors obtained from statistical yearbooks between 1989 and 2015. Our results show that the total land area in the coastal zone exhibited an increasing trend at an annual rate of 7.6% on average. The coastal urban land use also experienced substantial increases, with the degree of urban expansion reaching 42.1% in 2015. The coastal urbanization was at the cost of natural and semi-natural lands. Nearly 64.4% of agricultural land, 24.4% of ocean area within the buffer, 84.9% of fresh water, and 92.8% of the tidal flat had been converted to urban land during 1989–2015. The gradient analysis of land use land cover change along the north-to-south coastal line revealed the spatiotemporal patterns of total land, urbanization degree, and natural and semi-natural lands. The majority of the socioeconomic factors influenced land use land cover change in the coastal zone, positively contributing to the increase of public facilities’ use of land, but negatively affecting the freshwater areas. These findings can provide insights for decision-making in the future for coastal land use land cover planning and management in Shanghai.

Implementing Deep Learning algorithms for urban tree detection and geolocation with high-resolution aerial, satellite, and ground-level images

Urban forests are becoming increasingly important for human well-being as they provide ecosystem services that contribute to improving well-being of city dwellers and to addressing climate change. However, despite their importance, there is an information gap in most of the world's urban forests due to the high cost and complexity of conducting standard forest inventories in urban environments. New technologies based on artificial intelligence can represent a smart and efficient alternative to costly traditional inventories. In this paper, we present an approach based on deep learning algorithms for the detection, counting, and geopositioning of trees using a combination of ground-level and aerial/satellite imagery. We tested several convolutional networks, exploring different combinations of hyperparameters and adjusting the query distance between ground-level images, detection radius, and various resolutions of satellite and aerial images. Our methodology is able to detect and accurately locate 79% of the urban street tree with a positional accuracy of 60 cm to the center of the canopy. Additionally, this approach allows us to determine the availability of photographs of urban trees, indicating from which Google Street View image each tree is visible. Our research provides a scalable and replicable solution to the scarcity of urban tree data and information worldwide, demonstrating the potential of artificial intelligence to revolutionize the way in which we inventory and monitor urban forests.

Detecting School Zones on Florida’s Public Roadways Using Aerial Images and Artificial Intelligence (AI2)

Collecting up-to-date roadway geometry data is essential for transportation agencies so they can undertake planning, maintenance, design, and rehabilitation of roadways. Collection methods can be categorized into two distinct groups: land-based methods (e.g., field inventories, mobile mapping, and image logging); and aerial-based methods (e.g., satellite imagery, drones, and laser scanning). Because using land-based methods for thousands of miles of highways is tedious, costly, and risky for crew members, there is a need to develop better methodologies to collect these data faster, and more safely and cheaply. The increasing availability of high-resolution images and recent advances in computer vision and object detection techniques have enabled the automated extraction of roadway geometry features. This novel study proposes a computer vision-based methodology to detect school zone pavement markings from high-resolution aerial images and determine school zones on Florida’s public roadways. This is critical information for transportation agencies, and they use it for a variety of different purposes: identifying those markings that are old and invisible; comparing the school zone locations with other geometric features such as crosswalks; and analyzing crashes that occur around the zones. Compared with the ground truth data obtained for Leon County, Florida, 94% accuracy was observed at the 90% confidence level. The model was then used to detect school zones in Orange County, Florida, and approximately 500 school zone markings were identified automatically. The road geometry data extracted can be integrated with crash and traffic data to advise policymakers and roadway users.

Traffic perception from aerial images using butterfly fields

Drones are nowadays considered as a valuable solution to monitor urban traffic. Object detectors face numerous challenges when dealing with high-resolution aerial images captured by drones, due to variations in altitude, viewing angle, and weather conditions. To address these challenges, we present an object detector called Butterfly detector that is tailored to detect objects in aerial images. It is an anchor-free method that leverages field-based representations. We introduce Butterfly fields, a type of composite field that describes the spatial information of output features as well as the scale of the detected objects. We employ a voting mechanism between related Butterfly vectors pointing to the object center. We highlight the benefits of our method for urban traffic monitoring by (i) evaluating the recall/precision rate of our detector on two publicly available drone datasets (UAVDT and VisDrone2019), and (ii) measuring the error rate for flow estimations on our newly released EPFL roundabout dataset. We outperform the performance of previous methods while remaining real-time.

Building Detection in High-Resolution Remote Sensing Images by Enhancing Superpixel Segmentation and Classification Using Deep Learning Approaches

Accurate building detection is a critical task in urban development and digital city mapping. However, current building detection models for high-resolution remote sensing images are still facing challenges due to complex object characteristics and similarities in appearance. To address this issue, this paper proposes a novel algorithm for building detection based on in-depth feature extraction and classification of adaptive superpixel shredding. The proposed approach consists of four main steps: image segmentation into homogeneous superpixels using a modified Simple Linear Iterative Clustering (SLIC), in-depth feature extraction using an variational auto-encoder (VAE) scale on the superpixels for training and testing data collection, identification of four classes (buildings, roads, trees, and shadows) using extracted feature data as input to an Convolutional Neural Network (CNN), and extraction of building shapes through regional growth and morphological operations. The proposed approach offers more stability in identifying buildings with unclear boundaries, eliminating the requirement for extensive prior segmentation. It has been tested on two datasets of high-resolution aerial images from the New Zealand region, demonstrating superior accuracy compared to previous works with an average F1 score of 98.83%. The proposed approach shows potential for fast and accurate urban monitoring and city planning, particularly in urban areas.

Disentangling the Benefits of Self-Supervised Learning to Deployment-Driven Downstream Tasks of Satellite Images (Student Abstract)

In this paper, we investigate the benefits of self-supervised learning (SSL) to downstream tasks of satellite images. Unlike common student academic projects, this work focuses on the advantages of the SSL for deployment-driven tasks which have specific scenarios with low or high-spatial resolution images. Our preliminary experiments demonstrate the robust benefits of the SSL trained by medium-resolution (10m) images to both low-resolution (100m) scene classification case (4.25%↑) and very high-resolution (5cm) aerial image segmentation case (1.96%↑), respectively.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

With the explosion in the volume of collected high-resolution aerial image data, the development of effective image retrieval methods for remote sensing (RS) has become a popular area of research. The problem of content-based image retrieval (CBIR) for high-resolution remote sensing (HRRS) requires robust feature extraction and representation to extract high-level semantics followed by a similarity measurement method to compute the similarity between images based on these semantics. In this paper, we propose a novel sparse graph learning-based triplet network for deep metric learning-based HRRS CBIR. First, the graph representations of the aerial images are constructed via local feature descriptors and unsupervised segmentation that provide an efficient graph-structured representation of the images. Then, a novel attention graph convolutional network is developed to extract high-level semantics from the spatial patterns of the images’ segmented regions by aggregating region-node features. The attention mechanism and the pyramid pooling structure of the graph network help the proposed model extract powerful spatial features by preserving the structural information of the images while reducing the data’s dimensionality. Finally, a novel task-driven dictionary learning (TDDL) method based on triplet loss constructs a sparse metric space for the similarity measurement of the images in which the computed sparse codes for the images promote intra-class similarity between same-class images and inter-class dissimilarity between images of different classes. The proposed sparse features reduce the dimensionality of model parameters which mitigates overfitting and improves the generalization of the overall framework for deep metric learning. Our TDDL method adopts a bi-level optimization strategy so that the dictionary can be trained alongside the parameters of the graph representation network in an end-to-end fashion. Finally, we conduct extensive evaluations of the performance of our proposed architecture on real-world datasets compared to state-of-the-art models and obtain superior results.

Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance

Detecting sparse, small, lost persons with only a few pixels in high-resolution aerial images was, is, and remains an important and difficult mission, in which a vital role is played by accurate monitoring and intelligent co-rescuing for the search and rescue (SaR) system. However, many problems have not been effectively solved in existing remote-vision-based SaR systems, such as the shortage of person samples in SaR scenarios and the low tolerance of small objects for bounding boxes. To address these issues, a copy-paste mechanism (ISCP) with semi-supervised object detection (SSOD) via instance segmentation and maximum mean discrepancy distance is proposed (MMD), which can provide highly robust, multi-task, and efficient aerial-based person detection for the prototype SaR system. Specifically, numerous pseudo-labels are obtained by accurately segmenting the instances of synthetic ISCP samples to obtain their boundaries. The SSOD trainer then uses soft weights to balance the prediction entropy of the loss function between the ground truth and unreliable labels. Moreover, a novel evaluation metric MMD for anchor-based detectors is proposed to elegantly compute the IoU of the bounding boxes. Extensive experiments and ablation studies on Heridal and optimized public datasets demonstrate that our approach is effective and achieves state-of-the-art person detection performance in aerial images.