Inspection Path Research Articles

Path Planning for 5-Axis CMM Inspection Considering Path Reuse

The 5-axis Coordinate Measuring Machine (CMM) is widely used for quality data collection of the machining parts, such as cylinder blocks and heads of the engines. High efficient inspection path planning for multiple feature groups from different stations is one of the key tasks for CMM application. In engineering practice, the inspection planning of diverse feature groups accounts for large labor cost and process development cycle. To improve the efficiency of path generation for the complex machining part, a five-axis CMM inspection path planning method considering path length, probe rotation and path reusability is proposed. Firstly, the measuring points (MPs) are classified based on feasible inspection direction cone and accessibility of the MPs to achieve the minimum times of probe rotation. Then, the rapidly exploring random trees with multi-root node (RRT-MRNC) algorithm is proposed to implement local path planning considering inspection path reuse. Furthermore, intra-group and inter-group path is generated simultaneously based on the proposed enhanced Genetic Algorithm (GA) algorithm. In order to evaluate the effectiveness of the proposed method, the cylinder block path planning case is used. Compared with the benchmark methods, the total planning time based on the proposed planning method for the dynamic tasks was reduced by 55.2% and 54.9% respectively.

Remote Wind Farm Path Planning for Patrol Robot Based on the Hybrid Optimization Algorithm

Globally, wind power plays a leading role in the renewable energy industry. In order to ensure the normal operation of a wind farm, the staff will regularly check the equipment of the wind farm. However, manual inspection has some disadvantages, such as heavy workload, low efficiency and easy misjudgment. In order to realize automation, intelligence and high efficiency of inspection work, inspection robots are introduced into wind farms to replace manual inspections. Path planning is the prerequisite for an intelligent inspection robot to complete inspection tasks. In order to ensure that the robot can take the shortest path in the inspection process and avoid the detected obstacles at the same time, a new path-planning algorithm is proposed. The path-planning algorithm is based on the chaotic neural network and genetic algorithm. First, the chaotic neural network is used for the first step of path planning. The planning results are encoded into chromosomes to replace the individuals with the worst fitness in the genetic algorithm population. Then, according to the principle of survival of the fittest, the population is selected, hybridized, varied and guided to cyclic evolution to obtain the new path. The shortest path obtained by the algorithm can be used for the robot inspection of the wind farms in remote areas. The results show that the proposed new algorithm can generate a shorter inspection path than other algorithms.

Multiple spacecraft coordination and motion planning for full-coverage inspection of large complex space structures

This paper presents a new multiple spacecraft full-coverage inspection path-planning scheme to closely observe any large complex space structure hovering in space. For this purpose, a decentralized multi-spacecraft methodology is proposed, extending our previous work on the inspection of structures using a single inspector spacecraft for a cooperative collection of small spacecraft. Multi-robot systems are more reliable, flexible, and scalable and can accomplish complex missions, which might be impossible or significantly time-consuming for a single robot. To this end, our previously developed asymptotically optimal sampling-based path planner, called the Rapidly Exploring Random Tree, is combined with a multi-spacecraft coordination methodology. The resulting kinodynamic inspection path planner generates 3-dimensional inspection trajectories for each spacecraft to fully observe any points on the boundary of a complex space structure in close proximity while satisfying the kinematic and dynamic constraints. Spacecraft coordination, communication, and task allocations are developed to reduce the total inspection time, crucial during servicing and proximity operations. Simulation results are presented to demonstrate the ability and effectiveness of our planner.

Wall-Climbing Mobile Robot for Inspecting DEMO Vacuum Vessel

The vacuum vessel (VV) inside and outside inspection of the Demonstration Fusion Power Plant (DEMO) is very difficult due to various constraints, such as non-magnet effect material requirements, constrained space, and neutrons on its surfaces. We propose a design method for wall-climbing mobile robots (WMR) based on the vortex principle and investigate key technologies to meet VV inspection requirements. We developed a kinematic model based on the robot’s motion control requirements and a trajectory tracking control algorithm according to the tractrix principle, enabling the robot to follow the path for autonomous inspection. The impeller is designed based on the vortex principle. The aerodynamic characteristics and structural strength of the impeller were also analysed and optimised. A sliding-mode robust pressure control system was designed for the robot’s negative pressure adsorption, and its effectiveness was verified by simulation. Finally, an initial test prototype verified the structural design and vortex adsorption performance. We also addressed the potential applications of the WMR in DEMO and other fusion reactors.

Post-machining allowance optimization of directed energy deposited impeller blades using point cloud registration

Complex free-form parts manufactured by directed energy deposition (DED) have problems of uneven allowance distributions and severe stair-case effects. In order to optimize allowance for post-machining in a hybrid additive and subtractive machine centre, a point cloud registration method is proposed for allowance optimization to match the on-machine measured point cloud of the deposited part with the point cloud of the theoretical model. Firstly, the deposition path is used to construct the inspection path and obtain the maximum and minimum envelops of the deposited part. Secondly, the allowance distribution for the uniform and positive criterion is solved using the pattern search algorithm, after the datum planes are overlapped. Finally, a free-form blade of a centrifugal impeller is selected as a case study for allowance optimization. The proposed method is also compared with an allowance optimization method using genetic algorithm. The results show that the proposed method is more accurate and efficient for rapid allowance optimization for DED complex parts.

A two-module automated scanning inspection planning methodology for complex surfaces on coordinate measuring machine

Coordinate measuring machine (CMM) plays an essential role in the high precision measurement of complex surfaces. With the emergence of the continuous-contact scanning probe, it becomes possible to inspect large-scale scanning lines rapidly without losing accuracy. Focusing on scanning line distribution and inspection path planning, this paper proposes a two-module scanning inspection planning methodology. Firstly, a Cobweb-Like adaptive Sampling strategy (CLS) is designed to sample form errors by distributing scanning lines in a cobweb-shape mode. Secondly, an inspection path planner is utilized to plan local and global inspection paths. An Improved Ant Colony Optimization (IACO) is proposed to generate time-saving paths for discrete scanning lines. The CLS simulations and scanning experiments are conducted. The methodology can capture large form errors with the minimum inspection cost. Unnecessary transition distances are reduced by 31%-75%. This work enhances the inspection capability of the scanning probe on CMM.

Free-form surface inspection path planning using improved ant colony optimisation algorithm

To improve the inspection efficiency of free-form surfaces using coordinate measuring machines (CMMs), this paper proposes a free-form CMM inspection path optimization method based on an improved ant colony optimisation algorithm. The inspection path composition and its mathematical model are first analysed, and then the improved ant colony optimisation algorithm is applied to minimize the objective function. To address the problem that the ant colony optimisation algorithm has difficulty jumping out of local optimal solutions, this paper introduces a negative feedback mechanism in the pheromone update rule to improve the probability of the ant colony optimisation algorithm jumping out of the local optimal solution, so as to obtain the global optimal solution with a higher probability. Experiments showed that the inspection path length optimised by the improved ant colony optimisation algorithm was at least 5.2% and 8.6% shorter than those of the traditional ant colony optimisation and simulated annealing algorithms, respectively.

Enabling human–machine collaboration in infrastructure inspections through mixed reality

In this study, we propose a novel end-to-end system called Human–Machine Collaborative Inspection (HMCI) to enable collaboration between inspectors with Mixed Reality (MR) headsets and a robotic data collection platform (robot) for structural inspections. We utilize the MR headset’s holographic display and precise head tracking to allow inspectors to visualize and localize information (e.g., structural defect) on the real scenes, which are gathered by the robot and processed by an offsite computational server. The primary use case of HMCI is to enable the inspector to visualize, supervise, and improve results produced by automated defect detection algorithms in near real-time. The workflow in HMCI starts with collecting images and depth data to generate 3D maps of the site from the robot. A technique called single-shot localization is developed to create visual anchors for real-time spatial alignment between the robot and the MR headset. The 3D map and images are then sent to the computational server for analysis to detect defects and their locations. Then, the information is received by the MR headset and overlaid on the actual scenes to visualize it with spatial context. An experimental study is conducted in a lab environment to demonstrate HMCI using Microsoft HoloLens 2 (HL2) as the MR headset and Turtlebot2 as the robot. We start with the reconstruction of a 3D environment using a 3D depth sensor (Azure Kinect) on Turtlebot2 and visually detect fiducial markers as regions-of-interest (replicating structural damage) along a predefined inspection path. Then, regions-of-interest are successfully anchored to the real scene and visualized through HL2. To our knowledge, HMCI is one of the first human–machine collaborative systems that can integrate robots and inspectors with the MR headset, which has been developed, tested, and presented for structural inspection.

Extracting Skeletons of Two-Manifold Triangular Mesh Surface for Planning Skeleton-Guided Five-Axis Surface Inspection Path

Abstract Skeleton curve-guided five-axis sweep scanning is an emerging surface inspection technique with marvelous inspection efficiency. The precondition of applying such a technique to a complex surface is to extract the skeleton curves of a two-manifold surface associated with partitioning the surface into compact surfaces. However, the work on extracting skeletons of two-manifold triangle mesh surfaces is scarce, as existing skeletonization methods mostly focus on either 2D planar shapes or 3D solid shapes. In this paper, we present a new approach to extract the skeletons on a two-manifold triangle mesh with boundaries. The skeletons are formed in the most intuitive manner of wave front propagation which is based on computing the initial value problem of geodesics on a triangle mesh. The step-length of wave front propagation is adaptively controlled to guarantee the appropriate density of skeleton points for having good connectivity. Experiments show that, as a direct application of the proposed skeleton generation and its associated surface partitioning result, the five-axis sweep scanning path of complex free-form surface can be generated conveniently. Experimental results also validate that the computation of our proposed approach is simpler and faster than the state-of-the-art geodesic Voronoi diagram (GVD) method when most of the two-manifold triangle mesh is nearly planar. Additionally, the augmented parameter used for tracing the wave front’s geodesic propagation information is helpful for the skeleton-based surface partition, which is necessary for skeleton curve-guided five-axis sweep scanning.

Skeleton Curve-Guided Five-Axis Sweep Scanning for Surface With Multiple Holes

Although surface with multiple holes (SMHs) is widely used in industrial components, the precise inspection of SMH is a knotty task due to the inefficiency of traditional surface inspection technology. Recently, a five-axis surface sweep scanning approach is emerging and shows great potential to boost the surface inspection efficiency by sensing surface in a continuous sweep scanning way. However, an efficient sweep scanning path should cater to the unique kinematic characteristics of the five-axis inspection system. In this article, we present an approach to automatically generate efficient five-axis sweep scanning paths for inspecting SMH. First, the skeleton of SMH is extracted along with the surface partitioned into several patches, based on which a guiding curve-based sweep scanning path can be defined for each surface patch. By modeling the skeleton of SMH as a directed Euler graph and finding a proper sequence to traverse the graph, a continuous five-axis sweep scanning path is then generated to sweep all patches without any transition among them. The nonsweeping time could be eliminated so that the sweep scanning efficiency is drastically improved in this way. Simulation and physical inspection experiments are conducted on two SMHs, showing that our method significantly outperforms existing approaches, such as the popular zigzag method and the method from the leading commercial software of RENISHAW. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article aims to generate an efficient sweep scanning path of a five-axis coordinate measuring machine (CMM) for inspecting the SMHs of a precision component, such as an engine block or a gearbox cover. Although traditional five-axis CMM path generation methods can be applied to SMH inspection, most of them are designed for the point-by-point inspection mode, which intrinsically suffers from extremely low efficiency. Recently, an emerging five-axis surface sweep scanning technology shows great potential for boosting the efficiency of surface inspection by using a continuous surface sensing stylus mounted on a rapid rotary probe head. The realization of high-speed five-axis surface scanning depends on a proper sweep scanning path which takes advantage of the superior kinematic performance of the rotary probe head as much as possible and accordingly avoids assigning heavy workload to the three linear axes with weaker kinematic performance. However, up to date, only a handful of five-axis sweep scanning path planning methods are reported and mainly focus on the simple and compact surface. Therefore, existing methods are incompetent to generate a suitable sweep scanning path for high-speed inspection of SMH. For these reasons, we present a novel and practical approach to automatically generate a continuous five-axis sweep scanning path for complex SMH. The generated path exactly meets the unique kinematic requirements of high-speed five-axis inspection and eliminates all noninspection operations. In this way, the SMH inspection can be accomplished with high efficiency.

Kinodynamic on-orbit inspection path planning for full-coverage inspection in close proximity of space structures

In this paper, a kinodynamic inspection path planner, called Random Kinodynamic Inspection Tree algorithm, is presented to perform in-close proximity full-coverage on-orbit inspection for the space structures, under kinematic/dynamic motion constraints in deep space environments. Inspection, as one of the key elements of proximity operations, would make a significant breakthrough in on-orbit servicing and, therefore, space exploration missions. The presented approach combines a novel coverage planning scheme with a kinodynamic motion planner to quickly and effectively solve the inspection path planning problem while at the same time handling both holonomic and non-holonomic constraints of the environment and the inspector robot. Our planner avoids the previous decoupled two steps method, which usually has been used to solve coverage planning problems, namely the Art Gallery and Traveling salesman problems, which are difficult or infeasible to be applied to a robot with differential constraints working in a high-dimensional environment. By contrast, our planner employs the Rapidly Exploring Random Tree algorithm as an asymptotically-optimal sampling-based technique in cooperation with a Linear Quadratic Minimum Time controller to generate an optimal and smooth inspection trajectory for any space structures, given complete knowledge of the structure and inspector’s dynamics. The algorithm guarantees probabilistic completeness. Simulations are provided as a validation of the achieved inspection performance.

Path planning of patrol robot based on improved discrete electrostatic discharge algorithm

Safety is the premise of the stable and sustainable development of the chemical industry, safety accidents will not only cause casualties and economic losses, but also cause panic among workers and nearby residents. Robot safety inspection based on the fire risk level in a chemical industrial park can effectively reduce process accident losses and can even prevent accidents. The optimal inspection path is an important support for patrol efficiency, therefore, in this study, the fire risk level of each location to be inspected, which is obtained by the electrostatic discharge algorithm (ESDA)–nonparallel support vector machine evaluation model, is combined with the optimisation of the inspection path; that is, the fire risk level is used to guide the inspection path planning. The inspection path planning problem is a typical travelling salesman problem (TSP). The discrete ESDA (DESDA), based on the ESDA, is proposed. In view of the shortcomings of the long convergence time and ease of falling into the local optimum of the DESDA, further improvements are proposed in the form of the IDESDA, in which the greedy algorithm is used for the initial population, the 2-opt algorithm is applied to generate new solutions, and the elite set is joined to provide the best segment for jumping out of the local optimum. In the experiments, 11 public calculation examples were used to verify the algorithm performance. The IDESDA exhibited higher accuracy and better stability when solving the TSP. Its application to chemical industrial parks can effectively solve the path optimisation problem of patrol robots.

Microservices for autonomous UAV inspection with UAV simulation as a service

Autonomous UAV systems are increasingly touted as the dominant future paradigm for inspecting civil infrastructure. Within this study, we have designed and developed a cloud system for high-level path planning, monitoring, and testing of autonomous UAV missions for inspecting infrastructures such as power lines, power towers, bridges, and railways. The software architecture is based on identified system’s functional and non-functional requirements. The cloud system is intended for UAV inspection operators and therefore should support multiple concurrent users. The microservice architecture has assured independence between functionalities, allowing independent scaling resulting in the fast processing time of near-optimal route calculation for UAVs reaching inspection targets. Furthermore, the independence between the services facilitates feature addition and future development. The system robustness is assured by containerizing services and continuously deploying to the Kubernetes cluster distributed across multiple worker nodes. Kubernetes scaling properties have enabled multiple concurrent users regardless of heavy computations for inspection paths. Application load testing has resulted in low processing time when individual services are scaled. Calculated inspection paths are validated for real-world inspection by employing the UAV Gazebo simulation based on a 3D dynamic model with an onboard flight controller, leading the UAV through the waypoints provided by the cloud system. By containerizing the simulation and deploying it within the cluster, we have enabled developers and users to test paths before sending the real-world UAVs to the inspection.

A Point Cloud-Based Feature Recognition and Path Planning Method

Based on the research background of in situ automatic ultrasonic phased array inspection of irregular porous castings, due to the limited in situ inspection stations, complex shape of irregular porous castings, and extreme multireflection structural features, it is necessary to identify the positioning inspection features among multiple features to be inspected and plan the optimal inspection path. This research is interested in the porous location recognition and its detection path planning of irregular porous castings. For this, a point cloud-based multifeature contour recognition and location algorithm were proposed to simultaneously extract and locate the hole feature and cylindrical feature from an irregular porous castings. Furthermore, a detection path planning method was put forward to search the shortest robot’s detection path based on the above acquired features by visual recognition and positioning technology. First, through the calibration of the industrial robot tool coordinate system and the internal and external parameters of the camera, the “EyeinHand” hand-eye conversion relationship was established. Second, the robot vision system collects the point cloud information of the area to be inspected and performs point cloud splicing, the accuracy of the original data, the removal of noise such as invalid points, outliers and internal noise points, on this basis, the boundary curve of the hole to be inspected was extracted, the cylindrical equation was fitted, its geometric center was calculated, and the central coordinates and axis direction of the contour of the hole to be inspected were obtained. Finally, all the detection paths were traversed through the multibranch tree to obtain the optimal detection path of the detection points of multiple targets. The experimental results show that the positioning accuracy of the feature of the hole to be inspected by the vision system is 0.107 mm, the aperture extraction accuracy is 0.002 mm, the cylinder fitting accuracy is 0.04 mm, and the calculation accuracy of the angle between the two axes is within 0.4. When the number of features to be inspected is different, the average moving distance can be saved by 10.7% by using the end effector after path optimization. The feasibility of in situ automatic ultrasonic phased array detection for irregular porous castings using by visual positioning is verified.

Model-Based Part Manufacturing Quality Inspection Path Planning

This article mainly studies the path planning of part manufacturing quality inspection based on models. Therefore, this paper optimizes the inspection path planning by combining the deep learning of the BP neural network in the neural network model, then improves the recognition efficiency of parts with various shapes through the collection of surface point information, and then combines the basic principles of model inspection and quality control principles to improve the accuracy of quality inspection. In order to better design this optimal path, this paper also designs welding basic formation parameter experiments and robustness analysis experiments to verify the influencing factors of the welding process and the specific results of image processing; this paper also designed the part outer diameter quality inspection test analysis to verify the accuracy and coverage of model-based part manufacturing quality inspection. The results obtained through the collection of experiments are finally compared with the traditional part quality inspection path; the experimental results show that compared with the traditional part quality inspection path, the new part quality inspection path can improve the accuracy rate of 5%-17%, the coverage rate of 9%-20%, and efficiency of 3%-17%.

Autonomous aerial flight path inspection using advanced manufacturing techniques

SUMMARYRobotic systems have shown capabilities to perform inspection tasks in dangerous and difficult-to-access environments, such as those found in different components of power plants. However, most of the current robotic inspection technology is designed for specific components. Aerial robots, commonly termed as Drones, have raised an option to inspect a wider range of structural components. Nevertheless, current aerial inspecting technology still relies on a human pilot with limited line of sight, field of view and a reduced perception as the drone flies away, which prevents performing close-quarter inspection in intricate, structurally complex and GPS-denied environments. This work introduces offline inspection path generation methods based on robotics-integrated to manufacturing techniques. One method uses computer-aided manufacturing (CAM) techniques and the other an additive manufacturing (AM) approach to generate the flight path. That is to say, the drone would fly along the path described by a 3D (Three Dimensional) printer’s extruder or a CNC (Computer Numerical Control) machining tool, enabling to fly very close the structure even in physical structures with a complex geometry. Once the trajectories are generated, they are introduced for its validation in the Gazebo robotics simulator. Simulation results demonstrate the proper performance of the method and confirm that this approach can be used for close inspection of structuralcomponents.

View Path Planning via Online Multiview Stereo for 3-D Modeling of Large-Scale Structures

This study addresses a view-path-planning problem during 3-D scanning of a large-scale structure based on multiview stereo (MVS) for unmanned aerial platforms. Recently, most studies have adopted an explore-then-exploit strategy for 3-D scanning. The strategy first generates a coarse model from a simple overhead scanning and then plans an inspection path to cover the entire surface of the coarse model. However, even though the inspection path may be optimal, it is difficult to guarantee a complete and accurate reconstruction result due to defective factors of MVS such as occlusions, textureless surfaces, and insufficient parallaxes. Furthermore, the entire procedure of this strategy is inefficiently slow because of path redundancies and long MVS processing time. Therefore, we propose a novel view-path-planning method for 3-D scanning based on an online MVS reconstruction algorithm. The suggested method incrementally reconstructs the target model online and iteratively plans view paths by analyzing the current partial reconstructions. The method continuously analyzes the quality of the model and detects inaccurately reconstructed surfaces. It then plans an inspection path that provides a complete coverage of the detected surfaces while maximizing the performance of MVS. This method can construct a complete 3-D model in a single scanning trial without the need for rescanning. Extensive experiments show that our method outperforms the other state-of-the-art methods, especially in terms of the model completeness of complex structures.

Inspection Path Planning of Complex Surface Based on One-Step Inverse Approach and Curvature-Oriented Point Distribution

Polyhedral surface has become a popular representation for computer-aided design, manufacturing, and inspection systems due to its simplicity for geometric computation and product shape communication. However, little attempts have been made in relation to inspection planning strategy of freeform surface-based polyhedral models. The intention of the work presented in this article attempts to demonstrate the efficiency and flexibility of a measurement planning method for the inspection of components with polyhedral surface representations. Based on the one-step inverse method, the triangular mesh surface can be flattened onto a plane, which makes it possible to select sampling points and to generate inspection paths in a 2-D parametric domain. Subsequently, a curvature-oriented point distribution strategy is proposed for measurement path generation. Finally, all these 2-D sample points are inverse-mapped onto the original mesh surface. The advantage of this approach is that it reduces complex intersection operations in the 3-D space and strikes a balance between accuracy and efficiency. The effectivity and feasibility of the proposed method are verified by simulation experiments.

Touch Trigger Probe-Based Interference-Free Inspection Path Planning for Free-Form Surfaces by Optimizing the Probe Posture

Free-form surface inspection is a challenging task due to its complex geometry. Coordinate measurement machine (CMM) and on-machine measurement (OMM) equipped with the touch-trigger probe are commonly used for their high accuracy and ease to operate. Path planning plays a pivotal role in measuring free-form surfaces with accuracy and efficiency, which considers accessibility and stability of probe movement, as well as the total inspection time (IT). In this article, an integrated path planning method is proposed for free-form surface inspection by the touch-trigger probe. The accessibility cone is defined to describe the interference-free probe pole directions for gauging an inspection point. A graphic process unit (GPU)-based algorithm that accelerates the computation of accessibility cone is developed. Then, the inspection points are divided into several clusters by a classification algorithm. In each cluster, the inspection points can be detected in the same probe pole direction. Finally, the inspection points in each cluster are sorted according to the spatial location to shorten the total path length, and the global interference-free inspection path is generated. Experimental study is conducted by applying the proposed method and an existing method to an impeller blade, respectively. The variation of probe posture and total IT is estimated and measured. Compared with the existing method, the proposed method prominently reduces motion of rotation axes, resulting in distinct decrease of total measuring time.

Analysis of the Optimization Strategy Based on the Cooperation of Drones and Intelligent Robots for the Inspection of Substations

With the continuous development of the power system, the operation environment and scale of the power grid substation are more complex. It will become an inevitable trend for automated detection methods to replace manual detection methods because of their intelligence and security. However, at present, the use of intelligent robot inspection substation technology is not very mature, there are visual blindness, long inspection cycle, poor inspection effect, and other problems. By the development trend of the substation, this paper proposes a high altitude, low altitude, and ground inspection strategy of a substation based on the cooperation of UAV and intelligent robots. This strategy uses the grid method to construct the two-dimensional planning of the substation and then optimizes the inspection path of UAV and intelligent robots based on the improved ant colony algorithm. The UAV and the intelligent robot cooperate to realize the real-time inspection of the substation, forming a three-dimensional inspection system of "low-mid-high", "dynamic + static". The simulation results show that the inspection strategy using UAVs and intelligent robots is safer and can effectively reduce inspection time.