Camera-equipped Unmanned Aerial Vehicles Research Articles

Simulation-Based Optimization of Path Planning for Camera-Equipped UAVs That Considers the Location and Time of Construction Activities

Automated progress monitoring of construction sites using cameras has been proposed in recent years. Although previous studies have tried to identify the most informative camera views according to 4D BIM to optimize installation plans, video collection using fixed or pan-tilt-zoom cameras is still limited by their inability to adapt to the dynamic construction environment. Therefore, considerable attention has been paid to using camera-equipped unmanned aerial vehicles (CE-UAVs), which provide mobility for the camera, allowing it to fit its field of view automatically to the important parts of the construction site while avoiding occlusions. However, previous studies on optimizing video collection with CE-UAV are limited to the scanning of static objects on construction sites. Given the growing interest in construction activities, the existing methods are inadequate to meet the requirements for the collection of high-quality videos. In this study, the following requirements for and constraints on collecting construction-activity videos have been identified: (1) the FOV should be optimized to cover the areas of interest with the minimum possible occlusion; (2) the path of the UAV should be optimized to allow efficient data collection on multiple construction activities over a large construction site, considering the locations of activities at specific times; and (3) the data collection should consider the requirements for CV processes. Aiming to address these requirements and constraints, a method has been proposed to perform simulation-based optimization of path planning for CE-UAVs to allow automated and effective collection of videos of construction activities based on a detailed 4D simulation that includes a micro-schedule and the corresponding workspaces. This method can identify the most informative views of the workspaces and the optimal path for data capture. A case study was developed to demonstrate the feasibility of the proposed method.

Investigation into Camera Calibration Flight Paths for UAV-Based Corridor Surveys

Abstract. There is increasing adoption of cost-effective nonmetric camera-equipped unmanned aerial vehicles due to the perceived benefits of timesaving, ease of use, and the accuracy of the digital elevation models that can be produced using structure from motion software. The introduction of systematic elevation errors, doming and bowing, has been evidenced by several authors, and various methods have been identified to reduce these errors. This paper aims to analyse the impact of flight plans on these systematic errors using the especially challenging case of a corridor survey. Two sites were flown for the survey using a DJI Zenmuse. The first site, a car park, was utilised for on-the-job pre-calibration of the camera and consisted of several orbit flights and a double grip flight. Subsequently, an adjacent road (a corridor survey overall 428 m long) was also surveyed at 60 m and 80 m heights using varying flight configurations. This study confirms that pre-calibrating the camera's IOPs significantly reduces the root mean squared elevation error (from 0.268 m to 0.034 m) compared to self-calibrated IOPs using the corridor flights. The impact of flight design on elevation errors confirms a single flight path's risk and the benefits of two or more flight paths, including a point-of-interest orbit flight.

Motion Planning of UAV for Port Inspection Based on Extended RRT* Algorithm

A suitable trajectory in a port inspection mission is important for unmanned aerial vehicles (UAVs). Motion planning can help UAVs quickly generate an optimal trajectory that meets the constraints. The motion planning of UAVs is achieved in this paper as follows: firstly, a collision detection (CD) function is applied that evaluates whether the bias_RRT* (rapidly exploring random tree) algorithm needs to be called. Secondly, an isosceles triangle optimization function optimizes the path. Next, a trajectory is generated based on the minimum snap trajectory method. Lastly, the bias_RRT* algorithm and the improved bias_RRT* algorithm are used in the two experimental scenes for path planning comparison, and trajectory planning is carried out. The results show that, in the improved method, the path length and calculation time are shortened, and the trajectory cost and trajectory deviation are also significantly reduced. Overall, it appears that a camera-equipped UAV under the proposed approach can accomplish monitoring tasks more effectively and safety in port environment.

Construction payment automation using blockchain-enabled smart contracts and robotic reality capture technologies

This paper introduces an autonomous payment administration solution, integrating blockchain-enabled smart contracts and robotic reality capture technologies. The construction progress is captured, analyzed, and documented respectively using sensing, machine intelligence, and as-built building information models (BIM). The progress data is stored in a distributed manner using content addressable file sharing; it is then broadcasted to a smart contract which administers payments and transfers lien rights respectively using crypto currencies/tokens and non-fungible tokens (NFT). The method was successfully implemented for payments to 7 subcontractors in two real-world commercial construction projects in Canada and United States where progress was captured using a camera-equipped unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV) equipped with a laser scanner. The method eliminates reliance on today's heavily intermediated payment applications and showed promise for achieving accurate, efficient, and timely payment administration. Future work should further explore the connections between off- and on-chain realities.

Using UAV-Based Photogrammetry to Obtain Correlation between the Vegetation Indices and Chemical Analysis of Agricultural Crops

The optimum corn harvest time differs between individual harvest scenarios, depending on the intended use of the crop and on the technical equipment of the actual farm. It is therefore economically significant to specify the period as precisely as possible. The harvest maturity of silage corn is currently determined from the targeted sampling of plants cultivated over large areas. In this context, the paper presents an alternative, more detail-oriented approach for estimating the correct harvest time; the method focuses on the relationship between the ripeness data obtained via photogrammetry and the parameters produced by the chemical analysis of corn. The relevant imaging methodology utilizing a spectral camera-equipped unmanned aerial vehicle (UAV) allows the user to acquire the spectral reflectance values and to compute the vegetation indices. Furthermore, the authors discuss the statistical data analysis centered on both the nutritional values found in the laboratory corn samples and on the information obtained from the multispectral images. This discussion is associated with a detailed insight into the computation of correlation coefficients. Statistically significant linear relationships between the vegetation indices, the normalized difference red edge index (NDRE) and the normalized difference vegetation index (NDVI) in particular, and nutritional values such as dry matter, starch, and crude protein are evaluated to indicate different aspects of and paths toward predicting the optimum harvest time. The results are discussed in terms of the actual limitations of the method, the benefits for agricultural practice, and planned research.

Automated Geolocation in Urban Environments Using a Simple Camera-Equipped Unmanned Aerial Vehicle: A Rapid Mapping Surveying Alternative?

GNSS positioning accuracy can be degraded in areas where the surrounding object geometry and morphology interacts with the GNSS signals. Specifically, urban environments pose challenges to precise GNSS positioning because of signal interference or interruptions. Also, non-GNSS surveying methods, including total stations and laser scanners, involve time consuming practices in the field and costly equipment. The present study proposes the use of an Unmanned Aerial Vehicle (UAV) for autonomous rapid mapping that resolves the problem of localization for the drone itself by acquiring location information of characteristic points on the ground in a local coordinate system using simultaneous localization and mapping (SLAM) and vision algorithms. A common UAV equipped with a camera and at least a single known point, are enough to produce a local map of the scene and to estimate the relative coordinates of pre-defined ground points along with an additional arbitrary point cloud. The resulting point cloud is readily measurable for extracting and interpreting geometric information from the area of interest. Under two novel optimization procedures performing line and plane alignment of the UAV-camera-measured point geometries, a set of experiments determines that the localization of a visual point in distances reaching 15 m from the origin, delivered a level of accuracy under 50 cm. Thus, a simple UAV with an optical sensor and a visual marker, prove quite promising and cost-effective for rapid mapping and point localization in an unknown environment.

High-Level Multiple-UAV Cinematography Tools for Covering Outdoor Events

Camera-equipped unmanned aerial vehicles (UAVs), or “drones,” are a recent addition to standard audiovisual shooting technologies. As drone cinematography is expected to further revolutionize media production, this paper presents an overview of the state-of-the-art in this area, along with a brief review of current commercial UAV technologies and legal restrictions on their deployment. A novel taxonomy of UAV cinematography visual building blocks, in the context of filming outdoor events where targets (e.g., athletes) must be actively followed, is additionally proposed. Such a taxonomy is necessary for progress in intelligent/autonomous UAV shooting, which has the potential of addressing current technology challenges. Subsequently, the concepts and advantages inherent in multiple-UAV cinematography are introduced. The core of multiple-UAV cinematography consists in identifying different combinations of multiple single-UAV camera motion types, assembled in meaningful sequences. Finally, based on the defined UAV/camera motion types, tools for managing a partially autonomous, multiple-UAV fleet from the director’s point of view are presented. Although the overall focus is on cinematic coverage of sports events, the majority of our contributions also apply in different scenarios, such as movies/TV production, newsgathering, or advertising.

Shot type constraints in UAV cinematography for autonomous target tracking

During the past years, camera-equipped Unmanned Aerial Vehicles (UAVs) have revolutionized aerial cinematography, allowing easy acquisition of impressive footage. In this context, autonomous functionalities based on machine learning and computer vision modules are gaining ground. During live coverage of outdoor events, an autonomous UAV may visually track and follow a specific target of interest, under a specific desired shot type, mainly adjusted by choosing appropriate focal length and UAV/camera trajectory relative to the target. However, the selected UAV/camera trajectory and the object tracker requirements (which impose limits on the maximum allowable focal length) affect the range of feasible shot types, thus constraining cinematography planning. Therefore, this paper explores the interplay between cinematography and computer vision in the area of autonomous UAV filming. UAV target-tracking trajectories are formalized and geometrically modeled, so as to analytically compute maximum allowable focal length per scenario, to avoid 2D visual tracker failure. Based on this constraint, formulas for estimating the appropriate focal length to achieve the desired shot type in each situation are extracted, so as to determine shot feasibility. Such rules can be embedded into practical UAV intelligent shooting systems, in order to enhance their robustness by facilitating on-the-fly adjustment of the cinematography plan.

An evaluation of the effectiveness of low-cost UAVs and structure from motion for geomorphic change detection

The measurement of topography and of topographic change is essential for the study of many geomorphic processes. In recent years, structure from motion (SfM) techniques applied to photographs taken by camera-equipped unmanned aerial vehicles (UAVs) has become a powerful new tool for the generation of high resolution topography. The variety of available UAV systems continues to increase rapidly, but it is not clear whether increased UAV sophistication translates into improved quality of the calculated topography. To evaluate the lower end of the UAV spectrum, a simple low cost UAV was deployed to calculate high resolution topography in the Daan River gorge in western Taiwan, a site with a complicated 3D morphology and a wide range of surface types, making it a challenging site for topographic measurement. Terrestrial lidar surveys were conducted in parallel with UAV surveys in both June and November 2014, enabling an assessment of the reliability of the UAV survey to detect geomorphic changes in the range of 30cm to several meters. A further UAV survey was conducted in June 2015 in order to quantify changes resulting from the 2015 spring monsoon. To evaluate the accuracy of the UAV derived topography, it was compared to terrestrial lidar data collected during the same survey period using the cloud-to-cloud comparison algorithm M3C2. The UAV-generated point clouds match the lidar point clouds well, with RMS errors of 30–40cm; however, the accuracy of the SfM point clouds depends strongly on the characteristics of the surface being considered, with vegetation, water, and small scale texture causing inaccuracies. The lidar and SfM data yield similar maps of change from June to November 2014, with the same areas of geomorphic change detected by both methods. The SfM-generated change map for November 2014 to June 2015 indicates that the 2015 spring monsoon caused erosion throughout the gorge and highlights the importance of event-driven erosion in the Daan River. The results suggest that even very basic UAVs can yield data suitable for measuring geomorphic change on the scale of a channel reach.

Optimal scheduling for replacing perimeter guarding unmanned aerial vehicles

Guarding the perimeter of an area in order to detect potential intruders is an important task in a variety of security-related applications. This task can in many circumstances be performed by a set of camera-equipped unmanned aerial vehicles (UAVs). Such UAVs will occasionally require refueling or recharging, in which case they must temporarily be replaced by other UAVs in order to maintain complete surveillance of the perimeter. In this paper we consider the problem of scheduling such replacements. We present optimal replacement strategies and justify their optimality.

Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): a review of related works

AbstractOver the past few years, the application of camera-equipped Unmanned Aerial Vehicles (UAVs) for visually monitoring construction and operation of buildings, bridges, and other types of civil infrastructure systems has exponentially grown. These platforms can frequently survey construction sites, monitor work-in-progress, create documents for safety, and inspect existing structures, particularly for hard-to-reach areas. The purpose of this paper is to provide a concise review of the most recent methods that streamline collection, analysis, visualization, and communication of the visual data captured from these platforms, with and without using Building Information Models (BIM) as a priori information. Specifically, the most relevant works from Civil Engineering, Computer Vision, and Robotics communities are presented and compared in terms of their potential to lead to automatic construction monitoring and civil infrastructure condition assessment.

Formation Control of Mobile Robots Using Multiple Aerial Cameras

This paper describes a new vision-based control method to drive a set of robots moving on the ground plane to a desired formation. As the main contribution, we propose to use multiple camera-equipped unmanned aerial vehicles (UAVs) as control units. Each camera views, and is used to control, a subset of the ground team. Thus, the method is partially distributed, combining the simplicity of centralized schemes with the scalability and robustness of distributed strategies. Relying on a homography computed for each UAV-mounted camera, our approach is purely image-based and has low computational cost. In the control strategy we propose, if a robot is seen by multiple cameras, it computes its motion by combining the commands it receives. Then, if the intersections between the sets of robots viewed by the different cameras satisfy certain conditions, we formally guarantee the stabilization of the formation, considering unicycle robots. We also propose a distributed algorithm to control the camera motions that preserves these required overlaps, using communications. The effectiveness of the presented control scheme is illustrated via simulations and experiments with real robots.

Tridimensional Reconstruction Applied to Cultural Heritage with the Use of Camera-Equipped UAV and Terrestrial Laser Scanner

No single sensor can acquire complete information by applying one or several multi-surveys to cultural object reconstruction. For instance, a terrestrial laser scanner (TLS) usually obtains information on building facades, whereas aerial photogrammetry is capable of providing the perspective for building roofs. In this study, a camera-equipped unmanned aerial vehicle system (UAV) and a TLS were used in an integrated design to capture 3D point clouds and thus facilitate the acquisition of whole information on an object of interest for cultural heritage. A camera network is proposed to modify the image-based 3D reconstruction or structure from motion (SfM) method by taking full advantage of the flight control data acquired by the UAV platform. The camera network improves SfM performances in terms of image matching efficiency and the reduction of mismatches. Thus, this camera network modified SfM is employed to process the overlapping UAV image sets and to recover the scene geometry. The SfM output covers most information on building roofs, but has sparse resolution. The dense multi-view 3D reconstruction algorithm is then applied to improve in-depth detail. The two groups of point clouds from image reconstruction and TLS scanning are registered from coarse to fine with the use of an iterative method. This methodology has been tested on one historical monument in Fujian Province, China. Results show a final point cloud with complete coverage and in-depth details. Moreover, findings demonstrate that these two platforms, which integrate the scanning principle and image reconstruction methods, can supplement each other in terms of coverage, sensing resolution, and model accuracy to create high-quality 3D recordings and presentations.