Feature Point Extraction Method Research Articles

An automatic feature point extraction method based on laser vision for robotic multi-layer multi-pass weld seam tracking

An automatic feature point extraction method based on laser vision for robotic multi-layer multi-pass weld seam tracking

An Efficient and Robust Complex Weld Seam Feature Point Extraction Method for Seam Tracking and Posture Adjustment

To realize high-quality robotic welding, an efficient and robust complex weld seam feature point extraction method based on a deep neural network (Shuffle-YOLO) is proposed for seam tracking and posture adjustment. The Shuffle-YOLO model can accurately extract the feature points of butt joints, lap joints and irregular joints, and the model can also work well despite strong arc radiation and spatters. Based on the nearest neighbor algorithm and cubic B-spline curve fitting algorithm, the position and posture models of the complex spatially curved weld seams are established. The robot welding postures adjustment and high-precision seam tracking of complex spatially curved weld seams are realized. Experiments show that the method proposed in this paper can extract weld seam feature points quickly and robustly, which enables welding robots to accurately track the weld seams and adjust the welding torch postures simultaneously.

A Prediction Framework for State of Health of Lithium-Ion Batteries Based on Improved Support Vector Regression

As one crucial function of battery management system (BMS), the state of health (SOH) prediction of lithium-ion battery is of great significance to system safe operation and battery’s service life. This paper proposes a framework for SOH prediction, which includes the feature points extraction and SOH prediction. Firstly, based on the incremental capacity (IC) curve, the improved incremental capacity (IIC) curve is deduced by taking the derivative of the IC curve, and the grey relational analysis (GRA) is adopted to select the four feature points with the highest grey relational grade (GRG). Then, an improved sparrow search algorithm (ISSA) is proposed to optimize the parameters of the support vector regression (SVR) for more precise SOH prediction. Finally, experiments are carried out and the results show that the proposed feature points extraction method based on the IC/IIC curves and GRA is efficiently to improve the SOH prediction accuracy. Furthermore, compared with three traditional algorithms, the ISSA-SVR can restrict the SOH prediction error within 1.7%, and it also shows the proposed SOH prediction framework has strong robustness and high universality.

A Stable, Efficient, and High-Precision Non-Coplanar Calibration Method: Applied for Multi-Camera-Based Stereo Vision Measurements.

Traditional non-coplanar calibration methods, represented by Tsai's method, are difficult to apply in multi-camera-based stereo vision measurements because of insufficient calibration accuracy, inconvenient operation, etc. Based on projective theory and matrix transformation theory, a novel mathematical model is established to characterize the transformation from targets' 3D affine coordinates to cameras' image coordinates. Then, novel non-coplanar calibration methods for both monocular and binocular camera systems are proposed in this paper. To further improve the stability and accuracy of calibration methods, a novel circular feature points extraction method based on region Otsu algorithm and radial section scanning method is proposed to precisely extract the circular feature points. Experiments verify that our novel calibration methods are easy to operate, and have better accuracy than several classical methods, including Tsai's and Zhang's methods. Intrinsic and extrinsic parameters of multi-camera-systems can be calibrated simultaneously by our methods. Our novel circular feature points extraction algorithm is stable, and with high precision can effectively improve calibration accuracy for coplanar and non-coplanar methods. Real stereo measurement experiments demonstrate that the proposed calibration method and feature extraction method have high accuracy and stability, and can further serve for complicated shape and deformation measurements, for instance, stereo-DIC measurements, etc.

Long-Range Perception System for Road Boundaries and Objects Detection in Trains

This article introduces a long-range sensing system based on millimeter-wave radar, which is used to detect the roadside boundaries and track trains for trains. Due to the high speed and long braking distance of trains, existing commercial vehicle sensing solutions cannot meet their needs for long-range target detection. To address this challenge, this study proposes a long-range perception system for detecting road boundaries and trains based on millimeter-wave radar. The system uses high-resolution, long-range millimeter-wave radar customized for the strong scattering environment of rail transit. First, we established a multipath scattering theory in complex scenes such as track tunnels and fences and used the azimuth scattering characteristics to eliminate false detections. A set of accurate calculation methods of the train’s ego-velocity is proposed, which divides the radar detection point clouds into static target point clouds and dynamic target point clouds based on the ego-velocity of the train. We then used the road boundary curvature, global geometric parallel information, and multi-frame information fusion to extract and fit the boundary in the static target point stably. Finally, we performed clustering and shape estimation on the radar track information to identify the train and judge the collision risk based on the position and speed of the detected train and the extracted boundary information. The paper makes a significant contribution by establishing a multipath scattering theory for complex scenes of rail transit to eliminate radar false detection and proposing a train speed estimation strategy and a road boundary feature point extraction method that adapt to the rail environment. As well as building a perception system and installing it on the train for verification, the main line test results showed that the system can reliably detect the road boundary more than 400 m ahead of the train and can stably detect and track the train.

A Light Field Full-Focus Image Feature Point Matching Method with an Improved ORB Algorithm

Most of the traditional image feature point extraction and matching methods are based on a series of light properties of images. These light properties easily conflict with the distinguishability of the image features. The traditional light imaging methods focus only on a fixed depth of the target scene, and subjects at other depths are often easily blurred. This makes the traditional image feature point extraction and matching methods suffer from a low accuracy and a poor robustness. Therefore, in this paper, a light field camera is used as a sensor to acquire image data and to generate a full-focus image with the help of the rich depth information inherent in the original image of the light field. The traditional ORB feature point extraction and matching algorithm is enhanced with the goal of improving the number and accuracy of the feature point extraction for the light field full-focus images. The results show that the improved ORB algorithm extracts not only most of the features in the target scene but also covers the edge part of the image to a greater extent and produces extracted feature points which are evenly distributed for the light field full-focus image. Moreover, the extracted feature points are not repeated in a large number in a certain part of the image, eliminating the aggregation phenomenon that exists in traditional ORB algorithms.

FPSNET: An Architecture for Neural-Network-Based Feature Point Extraction for SLAM

The hardware architecture of a deep-neural-network-based feature point extraction method is proposed for the simultaneous localization and mapping (SLAM) in robotic applications, which is named the Feature Point based SLAM Network (FPSNET). Some key techniques are deployed to improve the hardware and power efficiency. The data path is devised to reduce overall off-chip memory accesses. The intermediate data and partial sums resulting in the convolution process are stored in available on-chip memories, and optimized hardware is employed to compute the one-point activation function. Meanwhile, address generation units are used to avoid data overlapping in memories. The proposed FPSNET has been designed in 65 nm CMOS technology with a core area of 8.3 mm2. This work reduces the memory overhead by 50% compared to traditional data storage for activation and overall by 35% for on-chip memories. The synthesis and simulation results show that it achieved a 2.0× higher performance compared with the previous design while achieving a power efficiency of 1.0 TOPS/W, which is 2.4× better than previous work. Compared to other ASIC designs with close peak throughput and power efficiency performance, the presented FPSNET has the smallest chip area (at least 42.4% reduction).

A hierarchical visual model for robot automatic arc welding guidance

PurposeThe traditional vision system cannot automatically adjust the feature point extraction method according to the type of welding seam. In addition, the robot cannot self-correct the laying position error or machining error. To solve this problem, this paper aims to propose a hierarchical visual model to achieve automatic arc welding guidance.Design/methodology/approachThe hierarchical visual model proposed in this paper is divided into two layers: welding seam classification layer and feature point extraction layer. In the welding seam classification layer, the SegNet network model is trained to identify the welding seam type, and the prediction mask is obtained to segment the corresponding point clouds. In the feature point extraction layer, the scanning path is determined by the point cloud obtained from the upper layer to correct laying position error. The feature points extraction method is automatically determined to correct machining error based on the type of welding seam. Furthermore, the corresponding specific method to extract the feature points for each type of welding seam is proposed. The proposed visual model is experimentally validated, and the feature points extraction results as well as seam tracking error are finally analyzed.FindingsThe experimental results show that the algorithm can well accomplish welding seam classification, feature points extraction and seam tracking with high precision. The prediction mask accuracy is above 90% for three types of welding seam. The proposed feature points extraction method for each type of welding seam can achieve sub-pixel feature extraction. For the three types of welding seam, the maximum seam tracking error is 0.33–0.41 mm, and the average seam tracking error is 0.11–0.22 mm.Originality/valueThe main innovation of this paper is that a hierarchical visual model for robotic arc welding is proposed, which is suitable for various types of welding seam. The proposed visual model well achieves welding seam classification, feature point extraction and error correction, which improves the automation level of robot welding.

Image copy-move forgery detection based on dynamic threshold with dense points

Copy-move tampering is one of the most popular tampering techniques at present. The tampered region of the image has good fusion with the original image, which increases the difficulty of detection. After years of research, the current detection method based on key points still has the following problems: 1) Failure to achieve forgery detection of small areas/self-similar areas/smooth areas, 2) Lack of reasonable feature point extraction methods, 3) The various stages of copy-move forgery detection (CMFD) work are relatively independent and lack close connections, 4) A fixed threshold is used as the region of interest similarity metric in the matching and localization stages. The failure to consider tampered images and the diversity of tampered regions leads to the limited detection capability of the algorithm. Considering the actual situation of tampering with the picture, to solve the above problems, we propose a copy-move forgery detection method based on the dynamic threshold. First, we determine the point extraction strategy in each super pixel block according to the size of the simple line interface calculation (SLIC) super pixel block and the Weber local descriptor (WLD) descriptor to ensure the reasonable allocation of feature points and reduce unnecessary points. These key points are then characterized by the scaling, flip and rotation invariants of the fractional general Jacobi-Fourier moments (FJFMs). Then, the matching and mismatch filtering thresholds of each feature point are determined through the WLD and SLIC features, and the SLIC feature is used to replace the distance threshold to improve the detection accuracy of small manufacturing areas. Finally, based on the matching results and SLIC features, an effective positioning method is proposed to improve the speed and accuracy of positioning. Experimental results show that the proposed algorithm is superior to the classic methods in recent years in terms of time and accuracy.

A Rapid Temporal Bone Localization Method Based on Machine Visual Detection Markers

The number of hearing-impaired people is increasing year by year; robotic cochlear drilling surgery is one of the safest methods to treat deafness. Looking at the issue of low efficiency of temporal bone posture positioning in cochlear implantation robotic drilling, a novel auxiliary ring marker temporal bone positioning method was proposed to improve temporal bone posture positioning efficiency, optimize the operation time, and reduce auxiliary injuries caused by the surgery. First, the temporal bone visual positioning assistant ring was designed based on the requirements for cochlear robotic drilling surgery. The target detection was conducted on the auxiliary ring and image processing and feature point extraction methods were designed. Then, the three-dimensional coordinates of the measured feature points were obtained by binocular vision, and the auxiliary ring and temporal bone postures were estimated. Finally, the auxiliary ring and temporal bone localization methods were validated. The experiment results indicated that the temporal bone was located quickly and effectively in a total time of about 33 ms, which was faster and more accurate than traditional visual localization methods and could satisfy real-time temporal bone localization during surgery. This study can reduce the time of temporal bone visual positioning in cochlear implant drilling operations, greatly improving the robot’s capabilities to extract visual information during the operation, which has a better auxiliary role for future research and applications of the cochlear implant drilling operation.

Research on a pavement pothole extraction method based on vehicle-borne continuous laser scanning point cloud

Automatic detection and rapid extraction of condition characteristics of pavement potholes provide important support for road health monitoring and maintenance. In this paper, a method of extracting pavement potholes in vehicle-borne continuous laser point cloud combined with two-dimensional and three-dimensional analysis is proposed. This method mainly includes three parts: Firstly, point cloud data preprocessing is carried out to remove non-road point cloud and road noise points; then the transverse and longitudinal sections of the road are extracted according to the scanning line index of the laser point cloud. The Douglas–Pucker operator is used to analyse the profile contour characteristics, and the integral invariant operator is used to identify the initial feature points of the pothole; finally, the local datum surface of the pavement is constructed by fitting, and the continuous depth image of the pavement is generated. The active contour model operator is used to accurately extract the pothole boundary based on the local depth information, and geometric information such as the depth and area of the pothole is obtained. Selecting some road point cloud data for experiments, the results show that the pothole feature point extraction method in this paper is relatively robust, the extraction precision rate reaches 86.1%, and the recall rate reaches 91.2%. By generating a local depth image for precise extraction of the pothole boundary, it can effectively solve the problem of inaccurate extraction of the pothole boundary based on discrete point clouds. Compared with the manual measurement results, the relative error of the area is less than 7%, and the relative error of the depth is less than 6%, which proves the effectiveness of this method.

Improved feature point extraction method of ORB-SLAM2 dense map

PurposeThis paper aims to quickly obtain an accurate and complete dense three-dimensional map of indoor environment with lower cost, which can be directly used in navigation.Design/methodology/approachThis paper proposes an improved ORB-SLAM2 dense map optimization algorithm. This algorithm consists of three parts: ORB feature extraction based on improved FAST-12, feature point extraction with progressive sample consensus (PROSAC) and the dense ORB-SLAM2 algorithm for mapping. Here, the dense ORB-SLAM2 algorithm adds LoopClose optimization thread and dense point cloud map and octree map construction thread. The dense map is computationally expensive and occupies a large amount of memory. Therefore, the proposed method takes higher efficiency, voxel filtering can reduce the memory while ensuring the density of the map and then use the octree format to store the map to further reduce memory.FindingsThe improved ORB-SLAM2 algorithm is compared with the original ORB-SLAM2 algorithm, and the experimental results show that the map through improved ORB-SLAM2 can be directly used in navigation process with higher accuracy, shorter tracking time and smaller memory.Originality/valueThe improved ORB-SLAM2 algorithm can obtain a dense environment map, which ensures the integrity of data. The comparisons of FAST-12 and improved FAST-12, RANSAC and PROSAC prove that the improved FAST-12 and PROSAC both make the feature point extraction process faster and more accurate. Voxel filter helps to take small storage memory and low computation cost, and octree map construction on the dense map can be directly used in navigation.

Tasseled Crop Rows Detection Based on Micro-Region of Interest and Logarithmic Transformation

Machine vision-based navigation in the maize field is significant for intelligent agriculture. Therefore, precision detection of the tasseled crop rows for navigation of agricultural machinery with an accurate and fast method remains an open question. In this article, we propose a new crop rows detection method at the tasseling stage of maize fields for agrarian machinery navigation. The whole work is achieved mainly through image augment and feature point extraction by micro-region of interest (micro-ROI). In the proposed method, we first augment the distinction between the tassels and background by the logarithmic transformation in RGB color space, and then the image is transformed to hue-saturation-value (HSV) space to extract the tassels. Second, the ROI is approximately selected and updated using the bounding box until the multiple-region of interest (multi-ROI) is determined. We further propose a feature points extraction method based on micro-ROI and the feature points are used to calculate the crop rows detection lines. Finally, the bisector of the acute angle formed by the two detection lines is used as the field navigation line. The experimental results show that the algorithm proposed has good robustness and can accurately detect crop rows. Compared with other existing methods, our method's accuracy and real-time performance have improved by about 5 and 62.3%, respectively, which can meet the accuracy and real-time requirements of agricultural vehicles' navigation in maize fields.

A teaching-free welding position guidance method for fillet weld based on laser vision sensing and EGM technology

Position guidance plays an essential role in guaranteeing welding quality. However, the current position guidance methods usually rely on the teaching trajectory or a similarly designed weld path. The innovation of this manuscript is to establish a teaching-free position guidance method combining the position detected by laser vision with the data bidirectionally sent by externally guided motion (EGM) at a high rate. A feature point extraction method based on Steger algorithm and slope determination is proposed. A dense zig-zag path is generated according to the position guidance based on EGM. Meanwhile, the position guidance experiments for fillet weld (horizontal deflection angle α = 9.2°, 13.3° and vertical deflection angle β = 5.5°, 9.2°) are finished. The mean absolute error (MAE) of position guidance in the Y-axis and Z-axis can be controlled in 0.21 and 0.13 mm, respectively. Moreover, the robot welding experiments for fillet weld in three different horizontal and vertical deflection angles (α = 30°, β = 25°, α = 30° and β = 25°) show that this method can obtain good weld surface morphology and metallographic morphology, and have a potential application value in industrial automation.

FEATURE FUSION-BASED FEATURE POINT EXTRACTION AND MATCHING OF THE THREE-DIMENSIONAL HUMAN VERTEBRAE MODEL

In the positioning and registration of a three-dimensional human vertebrae model, the main difficulty encountered is the extraction and matching of the model feature points. To accurately locate the vertebrae, it is necessary to optimize the existing feature point extraction and matching method. In this context, the present study was envisaged to achieve the purpose of accurate extraction and matching by integrating the features of curvature and Euclidean distance. It was found that this method removed the redundant point cloud data in the 3D model, thereby effectively improving the matching rate. Initially, the feature points were manually picked as the circle center, and the mean value ([Formula: see text]) of average curvature ([Formula: see text]) of all points in the specified infinitesimal radius ([Formula: see text]) sphere neighborhood were calculated. When [Formula: see text], it implied the point [Formula: see text] tended to be flat relative to the whole point cloud surface, and was not regarded as a feature point. When [Formula: see text], the point [Formula: see text] exhibited a greater curvature relative to the whole point cloud surface, that is, a more prominent point. In this case, point [Formula: see text] met the basic conditions for becoming a feature point and was included in the candidate feature point set. Subsequently, the Euclidean distance between the points in the candidate feature point set and the manually picked circle center points was calculated individually. The smaller the Euclidean distance between the two points, the higher the degree of feature similarity between the two points, and the stronger the matching of feature points. Therefore, the manually picked points were replaced with the candidate feature point with the closest distance that accurately reflected the change of the local geometric characteristics of the point. It was found that the accuracy of the improved method of the feature point extraction and matching was about 18% higher than that before the improvement, which verifies the effectiveness and robustness of the proposed method.

Feature matching for texture-less endoscopy images via superpixel vector field consistency.

Feature matching is an important technology to obtain the surface morphology of soft tissues in intraoperative endoscopy images. The extraction of features from clinical endoscopy images is a difficult problem, especially for texture-less images. The reduction of surface details makes the problem more challenging. We proposed an adaptive gradient-preserving method to improve the visual feature of texture-less images. For feature matching, we first constructed a spatial motion field by using the superpixel blocks and estimated its information entropy matching with the motion consistency algorithm to obtain the initial outlier feature screening. Second, we extended the superpixel spatial motion field to the vector field and constrained it with the vector feature to optimize the confidence of the initial matching set. Evaluations were implemented on public and undisclosed datasets. Our method increased by an order of magnitude in the three feature point extraction methods than the original image. In the public dataset, the accuracy and F1-score increased to 92.6% and 91.5%. The matching score was improved by 1.92%. In the undisclosed dataset, the reconstructed surface integrity of the proposed method was improved from 30% to 85%. Furthermore, we also presented the surface reconstruction result of differently sized images to validate the robustness of our method, which showed high-quality feature matching results. Overall, the experiment results proved the effectiveness of the proposed matching method. This demonstrates its capability to extract sufficient visual feature points and generate reliable feature matches for 3D reconstruction and meaningful applications in clinical.

Photorealistic simulation platform for autonomous landing of fixed-wing aircraft

To investigate the vision-based autonomous landing of fixed-wing aircraft, we propose a photorealistic simulation platform that leverages ROS, Unreal Engine, and Pixhawk 4. This platform adopts a software-in-the-loop model consisting of rendering, control, communication, and sensor modules. To achieve realistic rendering control, the seawater hydrodynamics and fixed-wing aircraft dynamics are modeled. Based on the platform, we create simulation scenarios under different weather and disturbance conditions for autonomous landing of the aircraft carrier. The platform solves the problem that datasets of related scenes are difficult to collect and experiments are difficult to carry out. To verify the developability of the platform, we design and implement a runway line feature point extraction method (vision-based pose estimation algorithm), and evaluate the performance of the method under various conditions. Experiments show that our software-in-the-loop platform enables vision-based algorithm verification and supports simulation research for the autonomous landing of fixed-wing aircraft.

Point Projection Network: A Multi-View-Based Point Completion Network with Encoder-Decoder Architecture

Recently, unstructured 3D point clouds have been widely used in remote sensing application. However, inevitable is the appearance of an incomplete point cloud, primarily due to the angle of view and blocking limitations. Therefore, point cloud completion is an urgent problem in point cloud data applications. Most existing deep learning methods first generate rough frameworks through the global characteristics of incomplete point clouds, and then generate complete point clouds by refining the framework. However, such point clouds are undesirably biased toward average existing objects, meaning that the completion results lack local details. Thus, we propose a multi-view-based shape-preserving point completion network with an encoder–decoder architecture, termed a point projection network (PP-Net). PP-Net completes and optimizes the defective point cloud in a projection-to-shape manner in two stages. First, a new feature point extraction method is applied to the projection of a point cloud, to extract feature points in multiple directions. Second, more realistic complete point clouds with finer profiles are yielded by encoding and decoding the feature points from the first stage. Meanwhile, the projection loss in multiple directions and adversarial loss are combined to optimize the model parameters. Qualitative and quantitative experiments on the ShapeNet dataset indicate that our method achieves good results in learning-based point cloud shape completion methods in terms of chamfer distance (CD) error. Furthermore, PP-Net is robust to the deletion of multiple parts and different levels of incomplete data.

Research on Vehicle Adaptive Real-Time Positioning Based on Binocular Vision

The positioning of a self-driving car is the basis of vehicle navigation and decision planning. To solve the problem of localization of self-driving cars in scenarios where GPS signals are missing, this article proposes an adaptive real-time positioning method, which is based on the binocular vision simultaneous localization and mapping method. The front-end visual odometry module uses the gray threshold adaptive oriented FAST and rotated BRIEF feature point extraction method for feature matching and pose calculation. The back end performs optimization processing based on the camera pose and the loop closure detection information estimated by the front end. Before optimization, we developed a rough dynamic feature elimination method to increase the robustness and positioning accuracy. Thus, a globally consistent positioning trajectory was obtained. To verify the actual positioning effect of the positioning system, the algorithm was tested using the KITTI data set and an experimental platform was built to test it in outdoor scenarios. The experimental results showed that the loop closure optimization and dynamic point elimination methods improved the positioning accuracy. The loop closure optimization significantly improved the actual positioning accuracy, and the dynamic feature elimination method slightly improved the positioning accuracy in dynamic scenes.

Point cloud segmentation for complex microsurfaces based on feature line fitting

Surfaces based on feature line constraints have higher accuracy than free-form surfaces and can capture other geometric relations of the model. The parts of complex microsurfaces are formed by arrays and crossings of several small surfaces. Many problems can be encountered in identifying feature points and fitting feature lines, which are difficult to solve by reverse engineering. In this study, feature point extraction, feature line fitting, and three-dimensional segmentation were investigated. First, the connection between two surfaces and the corresponding differential geometric quantities were explored. Then, a feature point extraction method for complex models was proposed. Second, the problems of separation, simplification, and combination of feature points for different models were analyzed, and the feature lines used to segment the point cloud were constructed. Finally, a region growth method based on feature line constraints was proposed to segment the point cloud data. Experimental results show that this method can solve the problem of excessive and insufficient segmentation for complex microsurface point cloud data and thus represents a foundation for high-quality model reconstruction.