Unmanned Aerial Vehicle Imagery Research Articles

Detection of on‐tree chestnut fruits using deep learning and RGB unmanned aerial vehicle imagery for estimation of yield and fruit load

AbstractThe on‐tree counting of chestnut fruit (bur) is essential for yield estimation and monitoring of fruit load as well as tree health, which is important in determining the management strategy for orchards, markets, and tree health monitoring. However, the practice is still conducted by manual count or farmers’ intuition. Precise and effective counting method is yet to be established. This study attempted to count chestnut fruits with the object detection algorithm, You Only Look Once (YOLO), using the RGB imagery collected with an unmanned aerial vehicle (UAV). The model trained with 500 images (7866 burs labeled with bounding boxes) in YOLO version 4 (v4) could count the number of burs with a R2 value of 0.98 and a root mean square error of 6.3 (8.3% of the mean) relative to the manual count in the data set for each whole tree (n = 53). The R2 of linear regression between the number of burs obtained with the YOLOv4 model and the total yield was 0.76, with a standard error of 1.08 kg tree−1 (26.4% of coefficient of variation), which was equivalent to the in situ burs count by an expert technician. In addition, the number of burs counted with the YOLOv4 model after the adjustment of trunk circumference was negatively correlated with nut weight. Overall, the study revealed that YOLO algorithm coupled with UAV‐based RGB imagery is a precise and efficient method for on‐tree burs detection, which could be used as an indicator of yield and fruit load.

Quantitative Evaluation of Maize Emergence Using UAV Imagery and Deep Learning

Accurate assessment of crop emergence helps breeders select appropriate crop genotypes, and farmers make timely field management decisions to increase maize yields. Crop emergence is conventionally quantified by manual calculations to quantify the number and size of seedlings, which is laborious, inefficient, and unreliable and fails to visualize the spatial distribution and uniformity of seedlings. Phenotyping technology based on remote sensing allows for high-throughput evaluation of crop emergence at the early growth stage. This study developed a system for the rapid estimation of maize seedling emergence based on a deep learning algorithm. The RGB images acquired from an unmanned aerial vehicle (UAV) were used to develop the optimal model for the recognition of seedling location, spacing, and size, and the prediction performance of the system was evaluated in three stations during 2021–2022. A case study was conducted to show the evaluation of the system for maize seedlings and combined with TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) analysis. The results show that the system has good prediction performance for maize seedling count with an average R2 value of 0.96 and an accuracy of 92%; however, shadows and planting density influence its accuracy. The prediction accuracy reduces significantly when the planting density is above 90,000 plants/ha. The distribution characteristics of seedling emergence and growth were also calculated based on the average value and variation coefficient of seedling spacing, seedling area, and seedling length. The estimation accuracies for the average value of seedling spacing, the coefficient of variation of seedling spacing, the average value of the seedling area, the coefficient of variation of the seedling area, and the average value of the seedling length were 87.52, 87.55, 82.69, 84.51, and 90.32%, respectively. In conclusion, the proposed system can quickly analyze the maize seeding growth and uniformity characteristics of experimental plots and locate plots with poor maize emergence.

Estimation of tomato water status with photochemical reflectance index and machine learning: Assessment from proximal sensors and UAV imagery.

Tracking plant water status is a critical step towards the adaptive precision irrigation management of processing tomatoes, one of the most important specialty crops in California. The photochemical reflectance index (PRI) from proximal sensors and the high-resolution unmanned aerial vehicle (UAV) imagery provide an opportunity to monitor the crop water status efficiently. Based on data from an experimental tomato field with intensive aerial and plant-based measurements, we developed random forest machine learning regression models to estimate tomato stem water potential (ψ stem), (using observations from proximal sensors and 12-band UAV imagery, respectively, along with weather data. The proximal sensor-based model estimation agreed well with the plant ψ stem with R 2 of 0.74 and mean absolute error (MAE) of 0.63 bars. The model included PRI, normalized difference vegetation index, vapor pressure deficit, and air temperature and tracked well with the seasonal dynamics of ψ stem across different plots. A separate model, built with multiple vegetation indices (VIs) from UAV imagery and weather variables, had an R 2 of 0.81 and MAE of 0.67 bars. The plant-level ψ stem maps generated from UAV imagery closely represented the water status differences of plots under different irrigation treatments and also tracked well the temporal change among flights. PRI was found to be the most important VI in both the proximal sensor- and the UAV-based models, providing critical information on tomato plant water status. This study demonstrated that machine learning models can accurately estimate the water status by integrating PRI, other VIs, and weather data, and thus facilitate data-driven irrigation management for processing tomatoes.

Combining exposed tree roots and UAV imagery to quantify land denudation in central Mexico

Approximately 42 % of Mexico is affected by soil denudation resulting from moderate to severe sheet erosion and gullying processes. At Huasca de Ocampo (central Mexico), soil degradation has been linked to intense land use dating back to pre-Hispanic times as well as to unfavorable geological, geomorphic, and climatic conditions. Here, we quantify erosion rates with high precision at annual to multi-decadal timescales by combining, for the first time, dendrogeomorphic reconstructions and UAV-based remote sensing. To assess rates of sheet erosion and gullying processes over the longer-term erosion rates (10–60 yrs), we assessed the age and first exposure of 159 roots to determine sheet erosion rates and gullying processes. At shorter timescales (<3 yrs), we employed an Unmanned Aerial Vehicle (UAV) to develop digital surface models (DSMs) for February 2020 and September 2022. Exposed roots provided evidence of sheet erosion ranging between 2.8 and 43.6 mm yr−1 and channel widening ranging between 11 and 270 mm yr−1, with highest erosion rates found along gully slopes. The UAV-based approach pointed to intense gully headcut retreat with rates between 164.8 and 870.4 mm yr−1; within gullies, channel widening rates ranged between 88.7 and 213.6 mm yr−1 and gully incision rates were between 11.8 and 109.8 mm yr−1. The two approaches yielded very comparable results regarding gully erosion and channel widening; this underlines the potential of using exposed roots to quantifying soil degradation processes retrospectively and considerably beyond the period covered by UAV imagery.

Field-scale yield prediction of winter wheat under different irrigation regimes based on dynamic fusion of multimodal UAV imagery

Field-scale crop yield prediction is critical to site-specific field management, which has been facilitated by recent studies fusing unmanned aerial vehicles (UAVs) based multimodal data. However, these studies equivalently stacked multimodal data and underused canopy spatial information. In this study, multimodal imagery fusion (MIF) attention was proposed to dynamically fuse UAV-based RGB, hyperspectral near-infrared (HNIR), and thermal imagery. Based on the MIF attention, a novel model termed MultimodalNet was proposed for field-scale yield prediction of winter wheat. To compare multimodal imagery-based and multimodal features-based methods, a stacking-based ensemble learning model was built using UAV-based canopy spectral, thermal, and texture features. The results showed that the MultimodalNet achieved accurate results at the reproductive stage and performed better than any single modality in the fusion. The MultimodalNet performed best at the flowering stage, with a coefficient of determination of 0.7411 and a mean absolute percentage error of 6.05%. The HNIR and thermal imagery were essential in yield prediction of winter wheat at the reproductive stage. Compared to equivalent stacking fusion, dynamic fusion through adaptively adjusting modality attention improved the model accuracy and adaptability across winter wheat cultivars and water treatments. Equivalently stacking more modalities did not necessarily yield improved performance than dynamically fusing fewer modalities. Methods using multimodal UAV imagery with rich spatial information were more applicable than methods using multimodal features to field-scale yield prediction. This study indicates that the MultimodalNet makes a powerful tool for field-scale yield prediction of winter wheat.

The bigger picture: developing a low-cost graphical user interface to process drone imagery of tidal stream environments

Unmanned Aerial Vehicles (UAVs), or drones, offer the ability to collect cost-effective fine-scale imagery that is suitable for the capture of concurrent hydrodynamic and faunal data within tidal stream environments. This is a necessary stage of information gathering to inform tidal energy device design, advise control and maintenance strategies and better inform environmental consenting processes. For this study a total of sixty-three UAV surveys were undertaken within the Inner Sound of the Pentland Firth, Scotland, UK, over two 4-day periods in 2016 and 2018. The aims of this data collection effort were to characterise bathymetrically driven hydrodynamic features, comprising of kolk-boil distribution, presence, and area, as well as marine life such as seabird distributions, presence, and orientation relative to the flow. To achieve this, a method to extract quantifiable metrics from UAV imagery was required. This paper details the processes and methodology to create a graphical user interface (GUI) to provide these outputs rather than examining specific results. It includes an explanation of the criteria that the GUI needed to meet to be able to process the imagery, a description of the workflow and an explanation of the sub-routines required such as image registration and calibration. The outputs of the GUI, and their relevance to tidal energy developments, are also discussed. Finally, this paper details future work incorporating computer vision techniques to improve the accuracy, reliability, and processing speed of the GUI.&#x0D; &#x0D; &#x0D;

Large wood loads in channels and on floodplains after a 500-year flood using UAV imagery in Mark Twain National Forest, Ozark Highlands, Missouri

Anthropogenic climate change has increased the frequency of large floods in rivers draining the Ozark Highlands. This study assesses the effects of a > 500-yr flood in spring 2017 on riparian forests and large wood loads in the North Fork of the White River watershed, Missouri, for six stream reaches with drainage areas from 5 to 124 km2. Standing trees and large wood (LW) were assessed using unmanned aerial vehicle (UAV) imagery and calibrated by field surveys. Scaled flood magnitude (flood stage/bankfull depth) correlated with percent urban and agricultural land above each reach suggesting that land use may have contributed to forest damage. Canopy loss on the valley floor ranged from 7 to 63 % by reach and correlated with mean and cross-sectional stream power (p < 0.01). Standing tree density after the flood ranged from 50 to 243 trees/ha. The density of LW pieces ranged from 25 to 147 trees/ha. Most LW was aligned with stream flow, not in jams, and located on floodplains below riffles or bar heads, along channel bends, or in chutes. Wood loads on the valley floor increased downstream from 12 to 45 m3/ha. Channel loads were < 30 m3/ha while floodplain, terrace, and chute loads were > 30 m3/ha at drainage areas >50 km2. Channel LW loads increased with flood magnitude and in narrow valleys (p < 0.02), but not drainage area. Increased wood storage occurred on floodplains and terraces, but it is not clear if the stored wood will be available for downstream transport by future floods.

Geomorphological transformations and future deformation estimations of a large potential landslide in the high-order position area of Diexi, China

Landslides in high-order position areas pose a serious threat to residents located below such areas. Therefore, research on the evolution process and underlying dynamic mechanisms is crucial. The majority of relevant studies are based on landslides that have already occurred; however, the investigation of potential landslides is of higher value. In this study, a progressive potential landslide identification method is proposed with the application of SBAS-InSAR (Small Baseline Subset Interferometric Synthetic Aperture Radar) and subsequently combined with Google Earth, GF-1 (Gaofen-1), GF-2 (Gaofen-2), ZY-3 (Ziyuan-3) and UAV (Unmanned Aerial Vehicle) imagery, and DEM (Digital Elevation Model) for further validation. The proposed method is employed to determine the potential landslide in Tuanjiecun by integrating the multi-period data to simultaneously analyze the evolution and mechanism of the potential landslide. The long short-term memory method is then adopted to predict the evolution trend based on accumulative deformation from SBAS-InSAR. The results suggest that the deformation of Tuanjiecun potential landslide will increase to −238.57 mm. This article provides a new approach for disaster prevention and mitigation by determining a potential landslide using composite remote sensing and predicting the development of the potential landslide on indirect contact monitoring.

A novel approach for surveying flowers as a proxy for bee pollinators using drone images

The abundance and diversity of plants and insects are important indicators of biodiversity, overall ecosystem health and agricultural production. Bees in particular are interesting indicators as they provide a key ecosystem service in many agricultural crops. Worldwide, habitat loss and fragmentation, agricultural intensification and climate change are important drivers of plant and bee decline. Monitoring of plants and bees is a crucial first step to safeguard their diversity and the services they provide but traditional in situ methods are time consuming and expensive. Remote sensing and Earth observation have the advantages that they can cover large areas and provides repeated, spatially continuous and standardized information. However, to date it has proven challenging to use these methods to assess small-scaled species-level biodiversity components with this approach. Here we surveyed bees and flowering plants using conventional field methods in 30 grasslands along a land-use intensity gradient in the Southeast of the Netherlands. We took RGB (true colored Red-Green–Blue) images using an Unmanned Aerial Vehicle (UAV) from the same fields and tested whether remote sensing can provide accurate assessments of flower cover and diversity and, by association, bee abundance and diversity. We explored the performance of different machine learning methods: Random Forest (RF), Neural Networks (NNET) and Support-Vector Machine (SVM). To evaluate the effect of the spatial resolution on the accuracy of the estimates, we tested all approaches using images at the original spatial resolution (∼ 0.5 cm) and re-sampled at 1 cm, 2 cm and 5 cm. We generally found significant relationships between UAV RGB derived estimates of flower cover and in situ estimates of flower cover and bee abundance and diversity. The highest resolution images generally resulted in the strongest relationships, with RF and NNET methods producing considerably better results than SVM methods (flower cover RF R2 = 0.8, NNET R2 = 0.79; bee abundance RF R2 = 0.65, NNET R2 = 0.54, bee species richness RF R2 = 0.62, NNET R2 = 0.52; bee species diversity RF R2 = 0.54, NNET R2 = 0.46). Our results suggest that methods based on the coupling of UAV imagery and machine learning methods can be developed into valuable tools for large-scale, standardized and cost-effective monitoring of flower cover and therefore of an important aspect of habitat quality for bees.

Estimation of Off-Target Dicamba Damage on Soybean Using UAV Imagery and Deep Learning.

Weeds can cause significant yield losses and will continue to be a problem for agricultural production due to climate change. Dicamba is widely used to control weeds in monocot crops, especially genetically engineered dicamba-tolerant (DT) dicot crops, such as soybean and cotton, which has resulted in severe off-target dicamba exposure and substantial yield losses to non-tolerant crops. There is a strong demand for non-genetically engineered DT soybeans through conventional breeding selection. Public breeding programs have identified genetic resources that confer greater tolerance to off-target dicamba damage in soybeans. Efficient and high throughput phenotyping tools can facilitate the collection of a large number of accurate crop traits to improve the breeding efficiency. This study aimed to evaluate unmanned aerial vehicle (UAV) imagery and deep-learning-based data analytic methods to quantify off-target dicamba damage in genetically diverse soybean genotypes. In this research, a total of 463 soybean genotypes were planted in five different fields (different soil types) with prolonged exposure to off-target dicamba in 2020 and 2021. Crop damage due to off-target dicamba was assessed by breeders using a 1-5 scale with a 0.5 increment, which was further classified into three classes, i.e., susceptible (≥3.5), moderate (2.0 to 3.0), and tolerant (≤1.5). A UAV platform equipped with a red-green-blue (RGB) camera was used to collect images on the same days. Collected images were stitched to generate orthomosaic images for each field, and soybean plots were manually segmented from the orthomosaic images. Deep learning models, including dense convolutional neural network-121 (DenseNet121), residual neural network-50 (ResNet50), visual geometry group-16 (VGG16), and Depthwise Separable Convolutions (Xception), were developed to quantify crop damage levels. Results show that the DenseNet121 had the best performance in classifying damage with an accuracy of 82%. The 95% binomial proportion confidence interval showed a range of accuracy from 79% to 84% (p-value ≤ 0.01). In addition, no extreme misclassifications (i.e., misclassification between tolerant and susceptible soybeans) were observed. The results are promising since soybean breeding programs typically aim to identify those genotypes with 'extreme' phenotypes (e.g., the top 10% of highly tolerant genotypes). This study demonstrates that UAV imagery and deep learning have great potential to high-throughput quantify soybean damage due to off-target dicamba and improve the efficiency of crop breeding programs in selecting soybean genotypes with desired traits.

Detection of Artificial Seed-like Objects from UAV Imagery

In the last two decades, unmanned aerial vehicle (UAV) technology has been widely utilized as an aerial survey method. Recently, a unique system of self-deployable and biodegradable microrobots akin to winged achene seeds was introduced to monitor environmental parameters in the air above the soil interface, which requires geo-localization. This research focuses on detecting these artificial seed-like objects from UAV RGB images in real-time scenarios, employing the object detection algorithm YOLO (You Only Look Once). Three environmental parameters, namely, daylight condition, background type, and flying altitude, were investigated to encompass varying data acquisition situations and their influence on detection accuracy. Artificial seeds were detected using four variants of the YOLO version 5 (YOLOv5) algorithm, which were compared in terms of accuracy and speed. The most accurate model variant was used in combination with slice-aided hyper inference (SAHI) on full resolution images to evaluate the model’s performance. It was found that the YOLOv5n variant had the highest accuracy and fastest inference speed. After model training, the best conditions for detecting artificial seed-like objects were found at a flight altitude of 4 m, on an overcast day, and against a concrete background, obtaining accuracies of 0.91, 0.90, and 0.99, respectively. YOLOv5n outperformed the other models by achieving a mAP0.5 score of 84.6% on the validation set and 83.2% on the test set. This study can be used as a baseline for detecting seed-like objects under the tested conditions in future studies.

A Lightweight and High-Accuracy Deep Learning Method for Grassland Grazing Livestock Detection Using UAV Imagery

Unregulated livestock breeding and grazing can degrade grasslands and damage the ecological environment. The combination of remote sensing and artificial intelligence techniques is a more convenient and powerful means to acquire livestock information in a large area than traditional manual ground investigation. As a mainstream remote sensing platform, unmanned aerial vehicles (UAVs) can obtain high-resolution optical images to detect grazing livestock in grassland. However, grazing livestock objects in UAV images usually occupy very few pixels and tend to gather together, which makes them difficult to detect and count automatically. This paper proposes the GLDM (grazing livestock detection model), a lightweight and high-accuracy deep-learning model, for detecting grazing livestock in UAV images. The enhanced CSPDarknet (ECSP) and weighted aggregate feature re-extraction pyramid modules (WAFR) are constructed to improve the performance based on the YOLOX-nano network scheme. The dataset of different grazing livestock (12,901 instances) for deep learning was made from UAV images in the Hadatu Pasture of Hulunbuir, Inner Mongolia, China. The results show that the proposed method achieves a higher comprehensive detection precision than mainstream object detection models and has an advantage in model size. The mAP of the proposed method is 86.47%, with the model parameter 5.7 M. The average recall and average precision can be above 85% at the same time. The counting accuracy of grazing livestock in the testing dataset, when converted to a unified sheep unit, reached 99%. The scale applicability of the model is also discussed, and the GLDM could perform well with the image resolution varying from 2.5 to 10 cm. The proposed method, the GLDM, was better for detecting grassland grazing livestock in UAV images, combining remote sensing, AI, and grassland ecological applications with broad application prospects.

Combining unmanned aerial vehicles and satellite imagery to quantify areal extent of intertidal brown canopy‐forming macroalgae

AbstractBrown macroalgae habitats provide a range of ecosystem services, offering coastal protection, supporting and increasing biodiversity, and more recently have been recognized for their potential role as blue carbon habitats. Consequently, accurate areal estimates of these habitats are vitally important. Satellite imagery is often utilized for areal estimates of vegetated habitats due to their ability to capture vast areas but are disadvantaged by their lower resolution. In contrast, imagery collected by unmanned aerial vehicles (UAV) provide high‐resolution datasets but are unable to cover the necessary spatial scale required for calculating areal estimates at regional, national or international scales. This study successfully and accurately corrects the outputs from low‐resolution Sentinel 2 imagery to the standard of high‐resolution UAV imagery by using a novel brown algae index and a simple regression model to provide accurate spatial estimates. This model was applied to rocky shores across Wales, UK to predict a spatial extent of 6.2 km2 for three fucoid macroalgae species; Ascophyllum nodosum, Fucus vesiculosus and F. serratus. The regression model was validated in two ways. First, the data used to create the regression model was split to train and test (50:50) the model, with a root mean square error of ~8%–14%. Secondly, spatial estimates of fucoids in independent aerial imagery were assessed using aerial photography interpretation and compared to that of the regression model (7% difference). The carbon standing stock of fucoids calculated from the spatial estimate (6.2 km2) was found to be significantly lower than that of other marine carbon stores, indicating that fucoids do not significantly contribute as a blue carbon habitat based on biomass alone. This study produces a robust and accurate remote sensing technique to estimate spatial extent of macroalgae at large spatial scales, with possible worldwide applicability.

Leaf area index estimation of pergola-trained vineyards in arid regions using classical and deep learning methods based on UAV-based RGB images

Timely and accurate mapping of leaf area index (LAI) in vineyards plays an important role for management choices in precision agricultural practices. However, only a little work has been done to extract the LAI of pergola-trained vineyards using higher spatial resolution remote sensing data. The main objective of this study was to evaluate the ability of unmanned aerial vehicle (UAV) imageries to estimate the LAI of pergola-trained vineyards using shallow and deep machine learning (ML) methods. Field trials were conducted in different growth seasons in 2021 by collecting 465 LAI samples. Firstly, this study trained five classical shallow ML models and an ensemble learning model by using different spectral and textural indices calculated from UAV imageries, and the most correlated or useful features for LAI estimations in different growth stages were differentiated. Then, due to the classical ML approaches need the arduous computation of multiple indices and feature selection procedures, another ResNet-based convolutional neural network (CNN) model was constructed which can be directly fed by cropped images. Furthermore, this study introduced a new image data augmentation method which is applicable to regression problems. Results indicated that the textural indices performed better than spectral indices, while the combination of them can improve estimation results, and the ensemble learning method showed the best among classical ML models. By choosing the optimal input image size, the CNN model we constructed estimated the LAI most effectively without extracting and selecting the features manually. The proposed image data augmentation method can generate new training images with new labels by mosaicking the original ones, and the CNN model showed improved performance after using this method compared to those using only the original images, or augmented by rotation and flipping methods. This data augmentation method can be applied to other regression models to extract growth parameters of crops using remote sensing data, and we conclude that the UAV imagery and deep learning methods are promising in LAI estimations of pergola-trained vineyards.

LAI estimation through remotely sensed NDVI following hail defoliation in maize (Zea mays L.) using Sentinel-2 and UAV imagery

Extreme events such as hailstorms are a cause for concern in agriculture, leading to both economic and food supply losses. Traditional damage estimation techniques have recently been called into question since damages have rarely been quantified at the large-field scale. Damage-estimation methods used by field inspectors are complex and sometimes subjective and hardly account for damage spatial variability. In this work, a normalized difference vegetation index (NDVI)-based parametric method was applied using both unmanned aerial vehicles (UAV) and Sentinel-2 sensors to estimate the leaf area index (LAI) of maize (Zea mays L.) resulting from simulated hail damage. These methods were then compared to the LAI values generated from the Sentinel-2 Biophysical Processor. A two-year experiment (2020–2021) was conducted during the maize cropping season, with hail events simulated during a range of maize developmental stages (the 8th-leaf, flowering, milky and dough stages) using a 0–40% defoliation gradient of damage intensities performed with the aid of specifically designed prototype machines. The results showed that both sensors were able to accurately estimate LAI in a nonstandard damaged canopy while requiring only the calibration of the extinction coefficient k(vartheta ) in the case of parametric estimations. In this case, the calibration was performed using 2020 data, providing k(vartheta ) values of 0.59 for Sentinel-2 and 0.37 for the UAV sensor. The validation was performed on 2021 data, and showed that the UAV sensor had the best accuracy (R2 of 0.86, root-mean-square error (RMSE) of 0.71). The k(vartheta ) value proved to be sensor-specific, accounting for the NDVI retrieval differences likely caused by the different spatial operational scales of the two sensors. NDVI proved effective in parametrically estimating maize LAI under damaged canopy conditions at different defoliation degrees. The parametric method matched the Sentinel-2 biophysical process-generated LAI well, leading to less underestimations and more accurate LAI retrieval.

Improved Image Synthesis with Attention Mechanism for Virtual Scenes via UAV Imagery

Benefiting from the development of unmanned aerial vehicles (UAVs), the types and number of datasets available for image synthesis have greatly increased. Based on such abundant datasets, many types of virtual scenes can be created and visualized using image synthesis technology before they are implemented in the real world, which can then be used in different applications. To achieve a convenient and fast image synthesis model, there are some common issues such as the blurred semantic information in the normalized layer and the local spatial information of the feature map used only in the generation of images. To solve such problems, an improved image synthesis model, SYGAN, is proposed in this paper, which imports a spatial adaptive normalization module (SPADE) and a sparse attention mechanism YLG on the basis of generative adversarial network (GAN). In the proposed model SYGAN, the utilization of the normalization module SPADE can improve the imaging quality by adjusting the normalization layer with spatially adaptively learned transformations, while the sparsified attention mechanism YLG improves the receptive field of the model and has less computational complexity which saves training time. The experimental results show that the Fréchet Inception Distance (FID) of SYGAN for natural scenes and street scenes are 22.1, 31.2; the Mean Intersection over Union (MIoU) for them are 56.6, 51.4; and the Pixel Accuracy (PA) for them are 86.1, 81.3, respectively. Compared with other models such as CRN, SIMS, pix2pixHD and GauGAN, the proposed image synthesis model SYGAN has better performance and improves computational efficiency.

Multispectral UAV data for detection of weeds in a citrus farm using machine learning and Google Earth Engine: Case study of Morocco

Accurate and timely weed mapping between and within trees is considered one of the major challenges in the site-specific weed management systems. This research presents the first cloud computing approach based on the incorporation of multispectral Unmanned Aerial Vehicle (UAV) imagery in the Google Earth Engine (GEE) programming environment with the aim of improving the mapping of weed patches between and within trees in a citrus farm. For this purpose, the UAV multispectral bands (red, green, blue, near infrared, and red edge), as well as the estimated vegetation height from the UAV Digital Elevation Model (DEM) were analysed in terms of tree and weed discrimination and used as input into Random Forest (RF) and k-nearest neighbors (KNN) machine learning algorithms. From the DEM, the Digital Terrain Model (DTM) was estimated using the Inverse Distance Weighted Interpolation (IDW) of the elevation values of dense points on the soil. The plant height was derived by subtracting the DTM from the DEM, resulting in the Canopy Height Model (CHM). The experimental results show that: (i) the combination of spectral bands and the CHM can classify both trees and weeds with an overall accuracy reached 96.87%; (ii) the RF classifier was more robust compared to KNN in the classification performance; (iii) when compared to the use of UAV spectral bands, the addition of the CHM can improve the accuracy of crop classification by 13.36% (KNN) and 1.79% (RF). Furthermore, the integration of UAV imagery in the GEE was highly efficient in terms of automation of the UAV imagery processing.

Leveraging Saliency in Single-Stage Multi-Label Concrete Defect Detection Using Unmanned Aerial Vehicle Imagery

Visual inspection of concrete structures using Unmanned Areal Vehicle (UAV) imagery is a challenging task due to the variability of defects’ size and appearance. This paper proposes a high-performance model for automatic and fast detection of bridge concrete defects using UAV-acquired images. Our method, coined the Saliency-based Multi-label Defect Detector (SMDD-Net), combines pyramidal feature extraction and attention through a one-stage concrete defect detection model. The attention module extracts local and global saliency features, which are scaled and integrated with the pyramidal feature extraction module of the network using the max-pooling, multiplication, and residual skip connections operations. This has the effect of enhancing the localisation of small and low-contrast defects, as well as the overall accuracy of detection in varying image acquisition ranges. Finally, a multi-label loss function detection is used to identify and localise overlapping defects. The experimental results on a standard dataset and real-world images demonstrated the performance of SMDD-Net with regard to state-of-the-art techniques. The accuracy and computational efficiency of SMDD-Net make it a suitable method for UAV-based bridge structure inspection.

Comparison of high-resolution NAIP and unmanned aerial vehicle (UAV) imagery for natural vegetation communities classification using machine learning approaches

ABSTRACT To map and manage forest vegetation including wetland communities, remote sensing technology has been shown to be a valid and widely employed technology. In this paper, two ecologically different study areas were evaluated using free and widely available high-resolution multispectral National Agriculture Imagery Program (NAIP) and ultra-high-resolution multispectral unmanned aerial vehicle (UAV) imagery located in the Upper Great Lakes Laurentian Mixed Forest. Three different machine learning algorithms, random forest (RF), support vector machine (SVM), and averaged neural network (avNNet), were evaluated to classify complex natural habitat communities as defined by the Michigan Natural Features Inventory. Accurate training sets were developed using both spectral enhancement and transformation techniques, field collected data, soil data, texture, spectral indices, and expert knowledge. The utility of the various ancillary datasets significantly improved classification results. Using the RF classifier, overall accuracies (OA) between 83.8% and 87.7% with kappa (k) values between 0.79 and 0.85 for the NAIP imagery and between 87.3% and 93.7% OA with k values between 0.83 and 0.92 for the UAV dataset were achieved. Based on the results, we concluded RF to be a robust choice for classifying complex forest vegetation including surrounding wetland communities.

CCTseg: A cascade composite transformer semantic segmentation network for UAV visual perception

Semantic segmentation could obtain the pixel level classification of surrounding environments which is an essential task for autonomous vehicles and mobile robots visual perception. Most existing semantic segmentation networks were focused on the visual perception of autonomous vehicles. Little attention is paid to the semantic segmentation for UAV (Unmanned Aerial Vehicle) visual perception, which is crucial to UAV autonomous flight and landing spot searching. Compared with views from autonomous vehicles, the UAV-based views were more challenging for the semantic segmentation task due to images captured by UAV containing large-scale variation of objects size caused by different altitude and angle. The existing semantic segmentation networks for the visual perception of autonomous vehicles are generally inadequate to effectively extract the representative features of UAV images which required contain context information and local information simultaneously. A cascade composite transformer-based semantic segmentation network is proposed in this study for UAV visual perception. A cascade composite encoder is designed which consists of three transformer-based feature extraction backbones and cascade fused low-level features, middle-level features and high-level features to achieve better feature representation capacity. The spatial enhanced transformer block is implemented as the basic feature extraction block of each backbone to make the extracted features contain context information of environments and local information of objects. A symmetric rhombus decoder is proposed to integrate multi-stage features and make fully utilise of middle stage features which contained abundance of useful information, thus accurately pixel level prediction could be obtained in this way. Ablation studies and comparison experiments for the proposed CCTseg have been conducted on two public UAV imagery datasets suitable for UAV autonomous flight and landing spot observing. Experimental results have demonstrated the effectiveness of the proposed network structure and the superiority of proposed network over other state-of-the-art methods for the semantic segmentation of UAV visual perception.