Unmanned Aerial Systems Images Research Articles

Assessing fusarium oxysporum disease severity in cotton using unmanned aerial system images and a hybrid domain adaptation deep learning time series model

Currently, deep learning has achieved remarkable success in estimating plant disease from unmanned aerial system (UAS) images. However, two critical challenges remain unexplored: spatiotemporal variations in disease symptoms and the domain shift between source and target datasets. To overcome these challenges, this paper proposes an approach that incorporates temporal aspects of disease progression using time series analysis. Spatiotemporal information is integrated by combining convolutional neural networks and bidirectional long-short term memory (CNN-BiLSTM) to classify the disease into five severity levels. Various feature extraction methods, including both handcrafted and CNN-based feature extractors, are evaluated. Furthermore, to tackle the problem of domain shift, a feature-level domain adaptation method is proposed. This method aims to learn transferable feature representations that remain consistent despite variations between source and target datasets. This approach enhances the spatiotemporal transferability of the CNN-BiLSTM model, enabling the effective utilisation of historical datasets. The study demonstrates that the CNN-BiLSTM model outperforms traditional time-independent machine-learning methods that rely on handcrafted features. Specifically, the Resnet101-BiLSTM model achieves the highest overall classification accuracy of 89.7% among all tested models evaluated on a one-year dataset. Moreover, it shows superior generalisation with 72.7% accuracy for cross-spatiotemporal disease severity classification using domain adaptation, as demonstrated through two-year experiments. By reducing the domain shift of source and target datasets and harnessing time series high-resolution images obtained throughout the crop growing season, this hybrid approach has substantial potential to advance the assessment of crop disease severity in field conditions.

Data augmentation approach in detecting roof pathologies with UASs images

Machine learning and computer vision techniques contribute to the automation roof pathologies identification from images collected with Unmanned Aerial System (UASs). However, one of the challenges for practical machine learning model tuning is the small-data problem. One strategy is to adopt data augmentation for generating more training data from existing images. This paper evaluates data augmentation in detecting pathologies in roof inspections with UASs images. The study adopted data augmentation for training two models in an image processing system. The training and tests using data augmentation images obtained superior results in accuracy, precision, recall, F-score, negative precision, and specificity metrics compared to the study using only original photos. These results indicate that data augmentation improves the adopted system’s performance in identifying roof pathologies in UAS images. This inspection system proposed with such integrated technologies would make it possible to increase transparency, simplify steps and reduce the time to perform roof inspections, streamlining the preparation of reports and application of corrective actions.

Large-Scale Earthwork Progress Digitalization Practices Using Series of 3D Models Generated from UAS Images

Since the Fourth Industrial Revolution, existing manpower-centric manufacture has been shifting towards technology and data-centric production in all areas of society. The construction sector is also facing a new paradigm called smart construction with a clear purpose of improving productivity and securing safety by applying site management using information and communications technology (ICT). This study aims to develop a framework for earthwork process digitalization based on images acquired by using the unmanned aerial system (UAS). The entire framework includes precise UAS data acquisition, cut-and-fill volume estimation, cross-section drawing, and geo-fencing generation. To this end, homogeneous time-series drone image data were obtained from active road construction sites under earthwork. The developed system was able to generate precise 3D topographical models and estimate cut-and-fill volume changes. In addition, the proposed framework generated cross-sectional views of each area of interest throughout the construction stages and finally created geo-fencing to assist the safe operation of heavy equipment. We expect that the proposed framework can contribute to smart construction areas by automating the process of digitizing earthwork progress.

3D Reconstruction of a Complex Grid Structure Combining UAS Images and Deep Learning

The latest advances in technical characteristics of unmanned aerial systems (UAS) and their onboard sensors opened the way for smart flying vehicles exploiting new application areas and allowing to perform missions seemed to be impossible before. One of these complicated tasks is the 3D reconstruction and monitoring of large-size, complex, grid-like structures as radio or television towers. Although image-based 3D survey contains a lot of visual and geometrical information useful for making preliminary conclusions on construction health, standard photogrammetric processing fails to perform dense and robust 3D reconstruction of complex large-size mesh structures. The main problem of such objects is repeated and self-occlusive similar elements resulting in false feature matching. This paper presents a method developed for an accurate Multi-View Stereo (MVS) dense 3D reconstruction of the Shukhov Radio Tower in Moscow (Russia) based on UAS photogrammetric survey. A key element for the successful image-based 3D reconstruction is the developed WireNetV2 neural network model for robust automatic semantic segmentation of wire structures. The proposed neural network provides high matching quality due to an accurate masking of the tower elements. The main contributions of the paper are: (1) a deep learning WireNetV2 convolutional neural network model that outperforms the state-of-the-art results of semantic segmentation on a dataset containing images of grid structures of complicated topology with repeated elements, holes, self-occlusions, thus providing robust grid structure masking and, as a result, accurate 3D reconstruction, (2) an advanced image-based pipeline aided by a neural network for the accurate 3D reconstruction of the large-size and complex grid structured, evaluated on UAS imagery of Shukhov radio tower in Moscow.

Sorghum Panicle Detection and Counting Using Unmanned Aerial System Images and Deep Learning.

Machine learning and computer vision technologies based on high-resolution imagery acquired using unmanned aerial systems (UAS) provide a potential for accurate and efficient high-throughput plant phenotyping. In this study, we developed a sorghum panicle detection and counting pipeline using UAS images based on an integration of image segmentation and a convolutional neural networks (CNN) model. A UAS with an RGB camera was used to acquire images (2.7 mm resolution) at 10-m height in a research field with 120 small plots. A set of 1,000 images were randomly selected, and a mask was developed for each by manually delineating sorghum panicles. These images and their corresponding masks were randomly divided into 10 training datasets, each with a different number of images and masks, ranging from 100 to 1,000 with an interval of 100. A U-Net CNN model was built using these training datasets. The sorghum panicles were detected and counted by a predicted mask through the algorithm. The algorithm was implemented using Python with the Tensorflow library for the deep learning procedure and the OpenCV library for the process of sorghum panicle counting. Results showed the accuracy had a general increasing trend with the number of training images. The algorithm performed the best with 1,000 training images, with an accuracy of 95.5% and a root mean square error (RMSE) of 2.5. The results indicate that the integration of image segmentation and the U-Net CNN model is an accurate and robust method for sorghum panicle counting and offers an opportunity for enhanced sorghum breeding efficiency and accurate yield estimation.

Determining the Suitable Number of Ground Control Points for UAS Images Georeferencing by Varying Number and Spatial Distribution

Currently, products that are obtained by Unmanned Aerial Systems (UAS) image processing based on structure-from-motion photogrammetry (SfM) are being investigated for use in high precision projects. Independent of the georeferencing process being done directly or indirectly, Ground Control Points (GCPs) are needed to increase the accuracy of the obtained products. A minimum of three GCPs is required to bring the results into a desired coordinate system through the indirect georeferencing process, but it is well known that increasing the number of GCPs will lead to a higher accuracy of the final results. The aim of this study is to find the suitable number of GCPs to derive high precision results and what is the effect of GCPs systematic or stratified random distribution on the accuracy of the georeferencing process and the final products, respectively. The case study involves an urban area of about 1 ha that was photographed with a low-cost UAS, namely, the DJI Phantom 3 Standard, at 28 m above ground. The camera was oriented in a nadiral position and 300 points were measured using a total station in a local coordinate system. The UAS images were processed using the 3DF Zephyr software performing a full BBA with a variable number of GCPs i.e., from four up to 150, while the number and the spatial location of check points (ChPs) was kept constant i.e., 150 for each independent distribution. In addition, the systematic and stratified random distribution of GCPs and ChPs spatial positions was analysed. Furthermore, the point clouds and the mesh surfaces that were automatically derived were compared with a terrestrial laser scanner (TLS) point cloud while also considering three test areas: two inside the area defined by GCPs and one outside the area. The results expressed a clear overview of the number of GCPs needed for the indirect georeferencing process with minimum influence on the final results. The RMSE can be reduced down to 50% when switching from four to 20 GCPs, whereas a higher number of GCPs only slightly improves the results.

Impact of UAS Image Orientation on Accuracy of Forest Inventory Attributes

The quality and accuracy of Unmanned Aerial System (UAS) products greatly depend on the methods used to define image orientations before they are used to create 3D point clouds. While most studies were conducted in non- or partially-forested areas, a limited number of studies have evaluated the spatial accuracy of UAS products derived by using different image block orientation methods in forested areas. In this study, three image orientation methods were used and compared: (a) the Indirect Sensor Orientation (InSO) method with five irregularly distributed Ground Control Points (GCPs); (b) the Global Navigation Satellite System supported Sensor Orientation (GNSS-SO) method using non-Post-Processed Kinematic (PPK) single-frequency carrier-phase GNSS data (GNSS-SO1); and (c) using PPK dual-frequency carrier-phase GNSS data (GNSS-SO2). The effect of the three methods on the accuracy of plot-level estimates of Lorey’s mean height (HL) was tested over the mixed, even-aged pedunculate oak forests of Pokupsko basin located in Central Croatia, and validated using field validation across independent sample plots (HV), and leave-one-out cross-validation (LOOCV). The GNSS-SO2 method produced the HL estimates of the highest accuracy (RMSE%: HV = 5.18%, LOOCV = 4.06%), followed by the GNSS-SO1 method (RMSE%: HV = 5.34%, LOOCV = 4.37%), while the lowest accuracy was achieved by the InSO method (RMSE%: HV = 5.55%, LOOCV = 4.84%). The negligible differences in the performances of the regression models suggested that the selected image orientation methods had no considerable effect on the estimation of HL. The GCPs, as well as the high image overlaps, contributed considerably to the block stability and accuracy of image orientation in the InSO method. Additional slight improvements were achieved by replacing single-frequency GNSS measurements with dual-frequency GNSS measurements and by incorporating PPK into the GNSS-SO2 method.

Mapping marine litter using UAS on a beach-dune system: a multidisciplinary approach

The amount of marine litter, mainly composed by plastic materials, has become a global environmental issue in coastal environments. Traditional monitoring programs are based on in-situ visual census, which require human effort and are time-demanding. Therefore, it is crucial to implement innovative mapping strategies to improve the environmental monitoring of marine litter on the coast.This work presents a procedure for an automated Unmanned Aerial System (UAS)-based marine litter mapping on a beach-dune system. A multidisciplinary framework, which comprises photogrammetry, geomorphology, machine learning and hydrodynamic modelling, was developed to process a block of UAS images. The work shows how each of these scientific methodologies can be complementary to improve and making more efficient the mapping of marine litter items with UAS on coastal environment. The very high-resolution orthophoto produced from UAS images was automatically screened by random forest machine learning method, in order to characterize the marine litter load on beach and dune areas, distinctively. The marine litter objects were identified with a F-test score of 75% when compared to manual procedure. The location of major marine litter loads within the monitored area was found related to beach slope and water level dynamics on the beach profiles, suggesting that UAS flight deployment and post-processing for beach litter mapping can be optimized based on these environmental parameters.The described UAS-based marine litter detection framework is intended to support scientists, engineers and decision makers aiming at monitoring marine and coastal pollution, with the additional aim of optimizing and automating beach clean-up operations.

Vineyard detection from unmanned aerial systems images

In viticulture, the adoption of precision agriculture techniques is nowadays increasingly essential to reach required high product quality standards. New reliable tools for mapping crop variability indexes in a vineyard or a single parcel are necessary to deploy site-specific management practices. In this paper, a new method to automatically detect vine rows in gray-scale aerial images is presented. The developed image processing algorithm is constituted by three main steps based on dynamic segmentation, Hough Space Clustering and Total Least Squares techniques. The procedure’s reliability has also been proven in the presence of disturbance elements, like dense inter-row grassing, bushes and trees shadows, by properly detecting vine rows in the vineyard images. Moreover, its adaptive features allow it to obtain optimal results in the presence of uneven image illumination due, for example, to the presence of clouds or steep terrain slopes. The extracted row and inter-row information, besides being the basis for vineyard characterization maps computation, like vine plants vigor maps, could also be used as a reference for other precision viticulture tasks such as, for example, path planning of unmanned ground vehicles.

Analysis of Point Cloud Generation from UAS Images

Abstract. Unmanned Aerial Systems (UAS) allow for the collection of low altitude aerial images, along with other geospatial information from a variety of companion sensors. The images can then be processed using sophisticated algorithms from the Computer Vision (CV) field, guided by the traditional and established procedures from photogrammetry. Based on highly overlapped images, new software packages which were specifically developed for UAS technology can easily create ground models, such as Point Clouds (PC), Digital Surface Model (DSM), orthoimages, etc. The goal of this study is to compare the performance of three different software packages, focusing on the accuracy of the 3D products they produce. Using a Nikon D800 camera installed on an ocotocopter UAS platform, images were collected during subsequent field tests conducted over the Olentangy River, north from the Ohio State University campus. Two areas around bike bridges on the Olentangy River Trail were selected because of the challenge the packages would have in creating accurate products; matching pixels over the river and dense canopy on the shore presents difficult scenarios to model. Ground Control Points (GCP) were gathered at each site to tie the models to a local coordinate system and help assess the absolute accuracy for each package. In addition, the models were also relatively compared to each other using their PCs.

Potential uses of small unmanned aircraft systems (UAS) in weed research

SummarySmall unmanned aerial systems (UAS) with cameras have not been adopted in weed research, but offer low‐cost sensing with high flexibility in terms of spatial resolution. A small rotary‐wing UAS was tested as part of a search for an inexpensive, user‐friendly and reliable aircraft for practical applications in UAS imagery weed research. In two experiments with post‐emergence weed harrowing in barley, the crop resistance parameter, which reflects the crop response to harrowing, was unaffected by image capture altitude in the range from 1 to 50 m. This corresponded to image spatial resolution in the range from 0.3 to 17.1 mm per pixel. This finding is important because spatial resolution is inversely related to sensing capacity. We captured 20 plots comprising a total of about 0.2 ha in one image at 50 m altitude without losing information about the cultivation impacts on vegetation compared with ground truth data. UAS imagery also gave excellent results in logarithmic sprayer experiments in oilseed rape, where we captured 37 m long plots in each image from an altitude of 35 m. Furthermore, perennial weeds could be mapped from UAS images. These first experiences with a small rotary‐wing UAS show that it is relatively easy to integrate as a tool in weed research and offers great potential for site‐specific weed management.