Inspection Flights Research Articles

USING UNMANNED AERIAL VEHICLES IN RECOGNIZING TERRAIN ANOMALIES ENCOUNTERED IN THE GAS PIPELINE RIGHT-OF-WAY (ROW)

The objective of the undertaken research was to characterize and evaluate the impact of weather and lighting conditions on recording terrain anomalies in the photographs obtained during a UAV photogrammetric flight. The present work describes the use and capabilities of the UAV in the mapping of photo acquisition conditions similar to those performed during inspection flights with the use of a manned helicopter equipped with a hyperspectral camera, in the target range of visible light. The research was conducted in the southern part of Poland (between Gliwice and Katowice), where 7 routes were selected, differing from one another in terms of terrain anomalies (buildings, types of land areas, vehicles, vegetation). In the studies, which involved photogrammetric flights performed using a UAV, different seasons and times of day as well as changes in light intensity were taken into account. The flight specification was based on the main parameters with the following assumptions: taking only perpendicular (nadir) RGB photographs, flight altitude 120 m AGL, strip width 160 m, GSD ≤0.04 m and overlap ≥83%. The analysis of the photographic material obtained made it possible to correct the catalog of anomalies defined previously, since the recognition of some objects is very difficult, being usually below the orthophotomap resolution. When making and evaluating orthophotomaps, problems with mapping the shape of objects near the edges of the frame were found. When a 12 mm lens is used, these distortions are significant. It was decided that for the purpose of generating training data from orthophotomaps, only the fragments containing objects which shape would be mapped in accordance with the real one would be used. Thus, the effective width of orthophotomaps obtained from simulated flights will be approximately 100 m.

Automated mission planning for aerial large‐scale power plant thermal inspection

AbstractClose‐range aerial inspection of large‐scale industrial facilities such as power plants is complex. It is characterized by a few areas suitable for safe landing, many obstacles, complicated radio communication, and so forth. The paper presented a new offline approach for automated mission planning in such conditions. It is based on novel concepts of ‐shaped flight profile and safe altitude map. These concepts reduce the mission planning task to a multiple traveling salesmen problem on a two‐dimensional map, which is solved using rapidly exploring random tree and genetic algorithms. A computational fluid dynamics‐based local turbulence map concept was proposed to avoid turbulence zones during inspection flights. As an advantage, this map can be generated at least 6.8× faster than the known approaches. Our new multicopter flight time prediction model provides an error of less than 10% in most cases and significantly outperforms those implemented in commercial mission planning software. All the proposed solutions were verified during a 2400‐MW power plant inspection. Our algorithms managed to plan to visit all the inspection points in six routes, approximately 15 min long each. This mission set has a comparable overall duration but better safety and even battery usage compared with the routes planned by a qualified operator. Moreover, the designed aerial inspection system provides higher quality images than manually from the ground using a thermal camera with two times higher sensitivity and equipped with more advanced optics. To our knowledge, this research is the first to report the successful implementation of autonomous unmanned aerial vehicle‐based power equipment diagnostics on a large‐scale thermal power plant.

Genetic Algorithm Based on Opposite Chromosome Multiple Genetic Operations Combined with Task Load Balancing

A collaborative optimization method for task allocation and path planning in multi-UAV execution of multi-objective cooperative inspection tasks is proposed. The method is based on the Opposite Genetic Algorithms (OGA), which combines the actual task completion time with the balance of UAV inspection flight time, fault downtime, and maximum-minimum time load. A Task Balancing Opposite Chromosome Multiple Mutation Operator Genetic Algorithm (TOMGA) is introduced to solve the task load balancing problem by optimizing the task allocation among multiple UAVs with time as the optimization objective. Simulation results demonstrate that, this algorithm can effectively allocate inspection tasks to UAVs and generate initial flight routes. It resolves the issue of task load imbalance, improves the rationality of task allocation, enhances convergence speed, and overcomes the problem of local optima.

Design and Application of a UAV Autonomous Inspection System for High-Voltage Power Transmission Lines

As the scale of the power grid continues to expand, the human-based inspection method struggles to meet the needs of efficient grid operation and maintenance. Currently, the existing UAV inspection system in the market generally has short endurance power time, high flight operation requirements, low degree of autonomous flight, low accuracy of intelligent identification, slow generation of inspection reports, and other problems. In view of these shortcomings, this paper designs an intelligent inspection system based on self-developed UAVs, including autonomous planning of inspection paths, sliding film control algorithms, mobile inspection schemes and intelligent fault diagnosis. In the first stage, basic data such as latitude, longitude, altitude, and the length of the cross-arms are obtained from the cloud database of the power grid, while the lateral displacement and vertical displacement during the inspection drone operation are calculated, and the inspection flight path is generated independently according to the inspection type. In the second stage, in order to make the UAV’s flight more stable, the reference-model-based sliding mode control algorithm is introduced to improve the control performance. Meanwhile, during flight, the intelligent UAV uploads the captured photos to the cloud in real time. In the third stage, a mobile inspection program is designed in order to improve the inspection efficiency. The transfer of equipment is realized in the process of UAV inspection. Finally, to improve the detection accuracy, a high-precision object detector is designed based on the YOLOX network model, and the improved model increased the mAP0.5:0.95 metric by 2.22 percentage points compared to the original YOLOX_m for bird’s nest detection. After a large number of flight verifications, the inspection system designed in this paper greatly improves the efficiency of power inspection, shortens the inspection cycle, reduces the investment cost of inspection manpower and material resources, and successfully fuses the object detection algorithm in the field of high-voltage power transmission lines inspection.

Electromagnetic Radiation from High-Voltage Transmission Lines: Impact on UAV Flight Safety and Performance

Unmanned aerial vehicles (UAVs) have emerged as versatile tools across various sectors, including power distribution infrastructure. While UAVs offer significant advancements in the time and cost-effectiveness of energy distribution infrastructure inspections, such operations present several challenges. The main challenges include the risk of collision with transmission cables themselves and the high potential for electromagnetic interference caused by high-voltage power lines. This research investigates the strength of the electromagnetic field (EMF) emitted by high-voltage power lines while identifying potential concerns for UAVs operating nearby. The research team conducted a series of experiments to quantify the strength of magnetic, electric, and radiofrequency fields induced by power transmission lines. A thorough understanding of the magnitude and potential effects of EMF from power lines is of great importance when conducting UAV flights in their direct vicinity. This understanding is crucial not only for ensuring the safety of infrastructure inspection flights but also for the safety of all UAV flights carried out near high-voltage lines. Our study contributes to the body of knowledge concerning the influence of electromagnetic radiation from high-voltage transmission lines on UAVs, providing valuable insights into mitigating risks associated with UAV flights in critical infrastructure inspections, as well as other operations.

Host Size Preference of the Leafcutter Ant Parasitoid Eibesfeldtphora tonhascai(Diptera: Phoridae).

Eibesfeldtphora tonhascai Brown(Diptera: Phoridae) is a natural enemy of Atta sexdensLinnaeusand Atta laevigata Smithand is considered a promising candidate for the biological control of these ants. The aim of this study was to verify if E. tonhascai has a preference for specific sizes of A. sexdens foragers and whether this parasitoid demonstrated host species preference when comparing parasitism of A. sexdens and A. laevigata. Worker ants were classified into four different size groups and introduced into an attack arena with one E. tonhascai female. To verify host species preference, both A. sexdens and A. laevigata workers were placed in the attack arena together with one E. tonhascai female. A no-choice test was also performed, when each ant species was offered to a phorid female at different times. Eibesfeldtphora tonhascai performed the highest number of inspection flights, attempted attacks, and actual attacks against A. sexdens workers with a cephalic capsule size of 3-5mm. When comparing attacks on A. sexdens and A. laevigata workers, they were equally inspected, attacked, and the total parasitism rates were similar. However, the phorid adult emergence rate was significantly higher in the A. sexdensparasitized workers. We conclude that E. tonhascai females express preference for parasitizing larger hosts (cephalic capsule width of 3-5mm), but do not display a preference for host species when comparing A. sexdens versus A. laevigata, and thus can similarly contribute to both leafcutter ants' natural biological control.

Research on the Application of Computer Vision Technology in Power System UAV Line Inspection

The stability of transmission lines is one of the important factors to ensure stable power supply. In this paper, a set of UAV intelligent power line inspection flight control system is designed according to the special flight requirements of UAV in power line inspection operation based on the principle of photogrammetry. The key technologies of the system include UAV flight attitude control, high-precision time synchronization between multiple sensors, defect detection and intelligent diagnosis, airborne sensor calibration, wireless communication, and ground data processing. In this paper, a camera and an on-board computer are innovatively used for real-time image processing and pattern recognition for wire feature acquisition. At the same time, this paper proposes an improved Canny edge detection algorithm combined with image segmentation technology to achieve wire feature extraction and identification. The field test results show that the system can complete the remote control of the UAV and realize the intelligent navigation and inspection of the transmission line UAV.

Bats (Plecotus auritus) use contact calls for communication among roost mates

Communication between group members is mediated by a diverse range of signals. Contact calls are produced by many species of birds and mammals to maintain group cohesion and associations among individuals. Contact calls in bats are typically relatively low-frequency social calls, produced only for communication. However, echolocation calls (higher in frequency and used primarily for orientation and prey detection) can also facilitate interaction among individuals and location of conspecifics in the roost. We studied calling behaviour of brown long-eared bats (Plecotus auritus) during return to maternity roosts in response to playbacks of social and echolocation calls. We hypothesised that calling by conspecifics would elicit responses in colony members. Bat responses (inspection flights and social calls production) were significantly highest during social call and echolocation call playbacks than during noise (control) playbacks. We suggest that social calling in maternity roosts of brown long-eared bat evolved to maintain associations among roostmates, rather than to find roosts or roostmates, because this species is strongly faithful to roosts and the social groups and roosts are stable over time and space. Living in a stable social group requires recognition of group members and affiliation of social bonds with group members, features that may be mediated by vocal signals.

Flying bats use serial sampling to locate odour sources.

Olfactory tracking generally sacrifices speed for sensitivity, but some fast-moving animals appear surprisingly efficient at foraging by smell. Here, we analysed the olfactory tracking strategies of flying bats foraging for fruit. Fruit- and nectar-feeding bats use odour cues to find food despite the sensory challenges derived from fast flight speeds and echolocation. We trained Jamaican fruit-eating bats (Artibeus jamaicensis) to locate an odour reward and reconstructed their flight paths in three-dimensional space. Results confirmed that bats relied upon olfactory cues to locate a reward. Flight paths revealed a combination of odour- and memory-guided search strategies. During ‘inspection flights’, bats significantly reduced flight speeds and flew within approximately 6 cm of possible targets to evaluate the presence or absence of the odour cue. This behaviour combined with echolocation explains how bats maximize foraging efficiency while compensating for trade-offs associated with olfactory detection and locomotion.

Aerial Robotic Solution for Detailed Inspection of Viaducts

The inspection of public infrastructure, such as viaducts and bridges, is crucial for their proper maintenance given the heavy use of many of them. Current inspection techniques are very costly and manual, requiring highly qualified personnel and involving many risks. This article presents a novel solution for the detailed inspection of viaducts using aerial robotic platforms. The system provides a highly automated visual inspection platform that does not rely on GPS and could even fly underneath the infrastructure. Unlike commercially available solutions, our system automatically references the inspection to a global coordinate system usable throughout the lifespan of the infrastructure. In addition, the system includes another aerial platform with a robotic arm to make contact inspections of detected defects, thus providing information that cannot be obtained only with images. Both aerial robotic platforms feature flexibility in the choice of camera or contact measurement sensors as the situation requires. The system was validated by performing inspection flights on real viaducts.

InDrone: a 2D-based drone flight behavior visualization platform for indoor building inspection

PurposeThe purpose of this research is to explore how to visually represent human decision-making processes during the performance of indoor building inspection flight operations using drones.Design/methodology/approachData from expert pilots were collected using a virtual reality drone flight simulator. The expert pilot data were studied to inform the development of an interactive 2D representation of drone flight spatial and temporal data – InDrone. Within the InDrone platform, expert pilot data were visually encoded to characterize key pilot behaviors in terms of pilots' approaches to view and difficulties encountered while detecting the inspection markers. The InDrone platform was evaluated using a user-center experimental methodology focusing on two metrics: (1) how novice pilots understood the flight approaches and difficulties contained within InDrone and (2) the perceived usability of the InDrone platform.FindingsThe results of the study indicated that novice pilots recognized inspection markers and difficult-to-inspect building areas in 63% (STD = 48%) and 75% (STD = 35%) of the time on average, respectively. Overall, the usability of InDrone presented high scores as demonstrated by the novice pilots during the flight pattern recognition tasks with a mean score of 77% (STD = 15%).Originality/valueThis research contributes to the definition of visual affordances that support the communication of human decision-making during drone indoor building inspection flight operations. The developed InDrone platform highlights the necessity of defining visual affordances to explore drone flight spatial and temporal data for indoor building inspections.

Automated Feature-Driven Flight Planning for Airborne Inspection of Large Linear Infrastructure Assets

Inspecting large-scale physical infrastructure assets such as powerlines from the air involves capturing data of thousands of features of an asset and its nearby surroundings, which makes inspection flight planning challenging. The objective of this article is to address the problems of automatically generating and optimizing flight paths and mission plans for airborne inspection of large linear infrastructure assets. This article proposes an automated feature-driven flight planning method using metaheuristic optimization with consideration of aircraft dynamic, sensor, terrain, and airfield location constraints. Results for flight planning the inspection of one of the world’s largest above-ground powerline networks are shown. Automated planning can reduce the human planning workload and enable the cost-saving benefits of automated aircraft to be realized through efficient deployment. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work extends previous research and development of an automated aircraft guidance system for infrastructure inspection, by addressing the problem of automated flight planning. Often, deploying manned or automated aircraft reliably and efficiently for large-scale inspection is a challenge. Designing efficient flight paths and missions by hand is complex and time-consuming. Traditional “top-down” planning approaches use region or waypoint-based planning, neglecting the spatial layout of the assets to be inspected, possibly leading to more missions than required, inefficient flight paths, or missed data capture. Since inspecting a large asset with few aircraft can take many days, a solution was sought that can plan across multiple time scales (from second to second path planning to daily mission planning). A solution was designed that can build flight plans automatically from the “bottom-up” while optimizing performance objectives and satisfying constraints. Flight plans can either be used to aid pilots conducting an aerial inspection, as input to an automated line-tracking system for aircraft or UAVs or for estimating the cost of large-scale inspection operations. The presented method and results are motivated by the specific problem of large-scale powerline inspection and the approach may be applied to the inspection of other above-ground infrastructures, such as roads and railways.

Incompatimic model of anomaly detection on different panoramas

The development of automatic methods for comparing panoramas obtained at different times during the inspection flight of UAVs of the same area is currently an urgent and popular task. In this connection, a new algorithmic model for detecting anomalies on multi-time panoramas was proposed, based on the comparison of the found singular points and descriptors, establishing their mutual correspondence on panoramas, and highlighting the found differences in non-overlapping areas of anomalies. The strategy aimed at bringing the panoramas to a single view and their subsequent synchronization is proposed. The results of the algorithm are presented, using the example of multi-time panoramas of the selected inspected area. We managed to synchronize the panoramas at different times to minimize differences in the shooting angles and illumination. Perform a search for anomalies on multi-time panoramas, excluding the selection of anomalies of the "noise" type and minor deviations in the color and geometric coordinates of special points. Sort the found anomalies by importance groups.

Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction

AbstractCrack assessment of bridge piers using unmanned aerial vehicles (UAVs) eliminates unsafe factors of manual inspection and provides a potential way for the maintenance of transportation infrastructures. However, the implementation of UAV‐based crack assessment for real bridge piers is hindered by several key issues, including the following: (a) both perspective distortion and the geometry distortion by nonflat structural surfaces usually appear on crack images taken by the UAV system from the pier surface; however, these two kinds of distortions are difficult to correct at the same time; and (b) the crack image taken by a close‐range inspection flight UAV system is partially imaged, containing only a small part of the entire surface of the pier, and thereby hinders crack localization. In this paper, a new image‐based crack assessment methodology for bridge piers using UAV and three‐dimensional (3D) scene reconstruction is proposed. First, the data acquisition of UAV‐based crack assessment is discussed, and the UAV flight path and photography strategy for bridge pier assessment are proposed. Second, image‐based crack detection and 3D reconstruction are conducted to obtain crack width feature pair sequences and 3D surface models, respectively. Third, a new method of projecting cracks onto a meshed 3D surface triangular model is proposed, which can correct both the perspective distortion and geometry distortion by nonflat structural surfaces, and realize the crack localization. Field test investigations of crack assessment of a real bridge pier using a UAV are carried out for illustration, validation, and error analysis of the proposed methodology.

Drone Mission Definition and Implementation for Automated Infrastructure Inspection Using Airborne Sensors.

This paper describes a Mission Definition System and the automated flight process it enables to implement measurement plans for discrete infrastructure inspections using aerial platforms, and specifically multi-rotor drones. The mission definition aims at improving planning efficiency with respect to state-of-the-art waypoint-based techniques, using high-level mission definition primitives and linking them with realistic flight models to simulate the inspection in advance. It also provides flight scripts and measurement plans which can be executed by commercial drones. Its user interfaces facilitate mission definition, pre-flight 3D synthetic mission visualisation and flight evaluation. Results are delivered for a set of representative infrastructure inspection flights, showing the accuracy of the flight prediction tools in actual operations using automated flight control.

The Power Line Inspection Software (PoLIS): A versatile system for automating power line inspection

A large amount of data, provided in the form of video data, is acquired during manned inspections flights of electric power lines. This data is analyzed by expert human inspectors to detect faults in the power lines infrastructure and prepare the inspection reports. This process is extremely time consuming, very expensive and prone to human error. In this paper, we present PoLIS: the Power Line Inspection Software, which has been developed with the objective of assisting the analysis of the data acquired during inspection flights. PoLIS is based on the cooperation between computer vision and machine learning techniques to automatically process video sequences acquired during inspection flights, resulting in a set of representative images per electric tower which we call Key Frames. These representative images can then be used for inspection purposes, leading to a drastic reduction of the human operators’ workload. At the core of the strategy lies an electric tower detector, which is in charge of estimating the location of the towers within the images based on the combination of a sliding window search technique and a supervised classifier. The location of the tower is then tracked using a tracking-by-registration algorithm based on direct methods, estimating the position of the tower in different images. Finally, different criteria are applied for defining whether the image corresponds to a Key Frame image or not. Extensive evaluation of the proposed strategy is conducted using videos acquired during manned helicopter inspections. The videos constituting this database contain several thousand frames representing both medium and high voltage power transmission lines in the infra-red (IR) and visible spectra. The obtained results show that the proposed strategy can reduce the large amount of data present in the inspection videos to a few Key Frames for each tower. It is also demonstrated that the learning-based approach proposed in PoLIS is appropriate for detecting electric towers, a process which is made faster and more robust by coupling it with a tower tracking algorithm. A Graphical User Interface allowing the application of PoLIS to user-provided videos is also presented in this paper, illustrating the whole process and the automated generation of an inspection report.

Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling

This paper presents a novel method for UAV-based 3D modeling of large infrastructure objects, such as pipelines, canals and levees, that combines anomaly detection with automatic on-board 3D view planning. The study begins by assuming that anomaly detections are possible and focuses on quantifying the potential benefits of the combined method and the view planning algorithm. A simulated canal environment is constructed, and several simulated anomalies are created and marked. The algorithm is used to plan inspection flights for the anomaly locations, and simulated images from the flights are rendered and processed to construct 3D models of the locations of interest. The new flights are compared to traditional flights in terms of flight time, data collected and 3D model accuracy. When compared to a low speed, low elevation traditional flight, the proposed method is shown in simulation to decrease total flight time by up to 55%, while reducing the amount of image data to be processed by 89% and maintaining 3D model accuracy at areas of interest.

Distress Calls of a Fast-Flying Bat (Molossus molossus) Provoke Inspection Flights but Not Cooperative Mobbing.

Many birds and mammals produce distress calls when captured. Bats often approach speakers playing conspecific distress calls, which has led to the hypothesis that bat distress calls promote cooperative mobbing. An alternative explanation is that approaching bats are selfishly assessing predation risk. Previous playback studies on bat distress calls involved species with highly maneuverable flight, capable of making close passes and tight circles around speakers, which can look like mobbing. We broadcast distress calls recorded from the velvety free-tailed bat, Molossus molossus, a fast-flying aerial-hawker with relatively poor maneuverability. Based on their flight behavior, we predicted that, in response to distress call playbacks, M. molossus would make individual passing inspection flights but would not approach in groups or approach within a meter of the distress call source. By recording responses via ultrasonic recording and infrared video, we found that M. molossus, and to a lesser extent Saccopteryx bilineata, made more flight passes during distress call playbacks compared to noise. However, only the more maneuverable S. bilineata made close approaches to the speaker, and we found no evidence of mobbing in groups. Instead, our findings are consistent with the hypothesis that single bats approached distress calls simply to investigate the situation. These results suggest that approaches by bats to distress calls should not suffice as clear evidence for mobbing.

Remote Flight Inspection Using Unmanned Aircraft

Aircraft instrument navigation is based on radio signal technology provided by ground navigation aids (navaids). The International Civil Aviation Organization specifies the required navaids availability, quality, accuracy, and integrity for ground installations on service, including the flight inspection operations. Flight inspection is the task of validating the radio signal emitted by navaids. This paper presents the architecture of a novel system for flight inspection based on the use of unmanned aircraft. The proposed architecture is operated remotely and has flight inspection capabilities by using radio signal sensors. It also has a precise positioning system, based on global navigation satellite system, and uses a data link with low bandwidth, long range, and redundancy. Except for congested airports, the flight inspection is done in segregated airspace. These segregated airspace inspection flights could be a catapult for seriously considering unmanned aircraft benefits in civil services. Unmanned aircraft are a ready technology waiting for legal approval to operate cooperatively with manned aircraft in regular airspace. The paper presents a simulation of the flight inspection of a navaid in Huesca, Spain. The simulation is a proof of concept of the proposed architecture.