Feature Point Extraction Research Articles

AA-LMM: Robust Accuracy-Aware Linear Mixture Model for Remote Sensing Image Registration

Remote sensing image registration has been widely applied in military and civilian fields, such as target recognition, visual navigation and change detection. The dynamic changes in the sensing environment and sensors bring differences to feature point detection in amount and quality, which is still a common and intractable challenge for feature-based registration approaches. With such multiple perturbations, the extracted feature points representing the same physical location in space may have different location accuracy. Most existing matching methods focus on recovering the optimal feature correspondences while they ignore the diversities of different points in position, which easily brings the model into a bad local extrema, especially when existing with the outliers and noises. In this paper, we present a novel accuracy-aware registration model for remote sensing. A soft weighting is designed for each sample to preferentially select more reliable sample points. To better estimate the transformation between input images, an optimal sparse approximation is applied to approach the transformation by multiple iterations, which effectively reduces the computation complexity and also improves the accuracy of approximation. Experimental results show that the proposed method outperforms the state-of-the-art approaches in both matching accuracy and correct matches.

A Prediction Framework for State of Health of Lithium-Ion Batteries Based on Improved Support Vector Regression

As one crucial function of battery management system (BMS), the state of health (SOH) prediction of lithium-ion battery is of great significance to system safe operation and battery’s service life. This paper proposes a framework for SOH prediction, which includes the feature points extraction and SOH prediction. Firstly, based on the incremental capacity (IC) curve, the improved incremental capacity (IIC) curve is deduced by taking the derivative of the IC curve, and the grey relational analysis (GRA) is adopted to select the four feature points with the highest grey relational grade (GRG). Then, an improved sparrow search algorithm (ISSA) is proposed to optimize the parameters of the support vector regression (SVR) for more precise SOH prediction. Finally, experiments are carried out and the results show that the proposed feature points extraction method based on the IC/IIC curves and GRA is efficiently to improve the SOH prediction accuracy. Furthermore, compared with three traditional algorithms, the ISSA-SVR can restrict the SOH prediction error within 1.7%, and it also shows the proposed SOH prediction framework has strong robustness and high universality.

An Efficient and Robust Complex Weld Seam Feature Point Extraction Method for Seam Tracking and Posture Adjustment

To realize high-quality robotic welding, an efficient and robust complex weld seam feature point extraction method based on a deep neural network (Shuffle-YOLO) is proposed for seam tracking and posture adjustment. The Shuffle-YOLO model can accurately extract the feature points of butt joints, lap joints and irregular joints, and the model can also work well despite strong arc radiation and spatters. Based on the nearest neighbor algorithm and cubic B-spline curve fitting algorithm, the position and posture models of the complex spatially curved weld seams are established. The robot welding postures adjustment and high-precision seam tracking of complex spatially curved weld seams are realized. Experiments show that the method proposed in this paper can extract weld seam feature points quickly and robustly, which enables welding robots to accurately track the weld seams and adjust the welding torch postures simultaneously.

ConCs-Fusion: A Context Clustering-Based Radar and Camera Fusion for Three-Dimensional Object Detection

Multi-modality three-dimensional (3D) object detection is a crucial technology for the safe and effective operation of environment perception systems in autonomous driving. In this study, we propose a method called context clustering-based radar and camera fusion for 3D object detection (ConCs-Fusion) that combines radar and camera sensors at the intermediate fusion level to achieve 3D object detection. We extract features from heterogeneous sensors and input them as feature point sets into the fusion module. Within the fusion module, we utilize context cluster blocks to learn multi-scale features of radar point clouds and images, followed by upsampling and fusion of the feature maps. Then, we leverage a multi-layer perceptron to nonlinearly represent the fused features, reducing the feature dimensionality to improve model inference speed. Within the context cluster block, we aggregate feature points of the same object from different sensors into one cluster based on their similarity. All feature points within the same cluster are then fused into a radar–camera feature fusion point, which is self-adaptively reassigned to the originally extracted feature points from a simplex sensor. Compared to previous methods that only utilize radar as an auxiliary sensor to camera, or vice versa, the ConCs-Fusion method achieves a bidirectional cross-modal fusion between radar and camera. Finally, our extensive experiments on the nuScenes dataset demonstrate that ConCs-Fusion outperforms other methods in terms of 3D object detection performance.

A Vision-Based Method for Simultaneous Instance Segmentation and Localization of Indoor Objects

Visual-based positioning technology plays a pivotal role in spatial artificial intelligence, facilitating precise perception and comprehension of the physical world for robotic platforms and augmented reality devices. In this study, we propose a binocular camera-based method for spatial localization of targets using CNN for instance segmentation while simultaneously providing target location information. The method encompasses image acquisition and correction, target recognition and segmentation, and stereo matching, among other components. Building upon this foundation, we introduce a pedestrian recognition segmentation network model with an attention mechanism. To accurately locate the target, we employ a multi-feature fusion feature point extraction and matching algorithm that combines edge information with semantic information. Finally, our proposed method is evaluated for dynamic pedestrian targets in indoor environments, achieving a horizontal positioning error of less than 0.25 m.

A comparison of feature extraction methods in image stitching

The usage of feature detectors for image stitching has become a popular research area in computer vision. Various feature extraction algorithms can be used in the process of image stitching process, but they perform varyingly when handling different images and no single algorithm could outperform all others. This paper focuses on the comparison of feature extraction algorithms used for panoramic image stitching. The research utilizes the SIFT, ORB, AKAZE, and BRISK to conduct feature points and match feature points on a group of image sets. The RANSAC algorithm is then used to filter out the outliers and calculate the homograph matrix. Completes the panoramic with image splicing and smoothing through the matrix transformation. Derived from the comparison of the matching and stitching results, the AKAZE detector is found to be the fastest feature point detection and extraction algorithm, while the SIFT detector will provide more feature points to make more accurate matches possible. These findings have implications for the development of efficient and effective computer vision technologies for various applications.

A Stable, Efficient, and High-Precision Non-Coplanar Calibration Method: Applied for Multi-Camera-Based Stereo Vision Measurements.

Traditional non-coplanar calibration methods, represented by Tsai's method, are difficult to apply in multi-camera-based stereo vision measurements because of insufficient calibration accuracy, inconvenient operation, etc. Based on projective theory and matrix transformation theory, a novel mathematical model is established to characterize the transformation from targets' 3D affine coordinates to cameras' image coordinates. Then, novel non-coplanar calibration methods for both monocular and binocular camera systems are proposed in this paper. To further improve the stability and accuracy of calibration methods, a novel circular feature points extraction method based on region Otsu algorithm and radial section scanning method is proposed to precisely extract the circular feature points. Experiments verify that our novel calibration methods are easy to operate, and have better accuracy than several classical methods, including Tsai's and Zhang's methods. Intrinsic and extrinsic parameters of multi-camera-systems can be calibrated simultaneously by our methods. Our novel circular feature points extraction algorithm is stable, and with high precision can effectively improve calibration accuracy for coplanar and non-coplanar methods. Real stereo measurement experiments demonstrate that the proposed calibration method and feature extraction method have high accuracy and stability, and can further serve for complicated shape and deformation measurements, for instance, stereo-DIC measurements, etc.

Image cascade matching method based on an improved random sampling consistency algorithm

In the process of image matching, image quality will have a certain impact on the extraction of image features, directly affecting the matching accuracy of subsequent images. Therefore, an image cascade matching method based on improved random sampling consistency algorithm is proposed. This method first needs to improve the quality of the matched target image through preprocessing results such as image denoising and enhancement, and prepare for feature point extraction during subsequent image cascade matching; Then extract the feature points of the target to be matched in the image, and use the improved random sampling consistency method to quickly estimate and simplify the registration model based on the extraction results. Use similar feature triangles to perform model pre testing, and use the maximum Euclidean distance method to improve the accuracy of image cascade registration, achieving image cascade matching. The experimental results show that the proposed method can achieve image enhancement while removing image noise, improve the quality of the image, comprehensively extract the feature points to be matched in the image, and achieve accurate matching of image feature points.

Machine vision–based automatic planning for microdroplet coating trajectory on irregularly shaped regions of metal surfaces

Metal surface coating technology plays a crucial role in various aspects of metal engineering such as enhancing metal oxidation resistance, corrosion prevention, insulation, decoration, and surface performance. This paper presents a machine vision–based automatic planning algorithm for microdroplet coating trajectory on irregularly shaped regions of metal surfaces, aiming to achieve uniform coating on different surface shapes and sizes. First, the process of microdroplet shape changes on metal surfaces is analyzed, the wetting circle diameter of a fully spread microdroplet is estimated, and the distance between two neighboring microdroplets is confirmed. Then, color pattern image processing (region contour extraction, feature point extraction, deburring, and equidistant offset algorithms) and template matching technology are developed to extract coating trajectory contours and accurately locate the workpieces on the workbench. To ensure a smooth and continuous coating trajectory, rational cubic Bézier curves are employed to interpolate the contour feature points, forming C1 continuous curves. Finally, taking into consideration the distance between two neighboring microdroplets, the continuous curves are equidistant offset and equichord discretized to determine the microdroplet locations. The experimental results demonstrate that the proposed algorithm successfully achieves arbitrary workpiece positioning, automatic planning of coating trajectory and uniform microdroplet coating on different surface shapes and sizes.

Simultaneous localization and mapping of mobile robots with multi-sensor fusion

Abstract Simultaneous localization and map building of mobile robots is an important research direction in robotics, and multi-sensor fusion is one of the key technologies for mobile robots to achieve autonomous navigation. In this paper, firstly, based on the framework of the SLAM system, we analyze the extraction of FAST feature points and BRIEF feature descriptor matching and propose the CSD-ORB algorithm. Secondly, the FKCS-LD algorithm based on the fusion map key frame and center selection strategy is proposed. The suitable multi-sensor fusion SLAM system is selected to build the back-end optimization algorithm based on the bit-pose map. Finally, we complete the comparison experiments of the multi-sensor fusion SLAM system on standard data sets, robot hardware assembly and software environment construction, and actual field testing. The experimental data show that the trajectory calculation accuracy can be further improved by using local map tracking, i.e., the current frame and local keyframe to calculate the bit pose, in which the maximum error is 6 cm, the root mean square error is reduced to 3.6 cm, the maximum value of local optimization is 0.66034, the minimum value is 0.146232, the average is 0.32812, and the median is 0.33023. The experiment proves that the multi-sensor fusion SLAM system can achieve centimeter-level accuracy and has global consistency and real-time performance, and can effectively complete the task of simultaneous robot positioning and map building.

SPSVO: a self-supervised surgical perception stereo visual odometer for endoscopy

AbstractAccurate tracking and reconstruction of surgical scenes is a critical enabling technology toward autonomous robotic surgery. In endoscopic examinations, computer vision has provided assistance in many aspects, such as aiding in diagnosis or scene reconstruction. Estimation of camera motion and scene reconstruction from intra-abdominal images are challenging due to irregular illumination and weak texture of endoscopic images. Current surgical 3D perception algorithms for camera and object pose estimation rely on geometric information (e.g., points, lines, and surfaces) obtained from optical images. Unfortunately, standard hand-crafted local features for pose estimation usually do not perform well in laparoscopic environments. In this paper, a novel self-supervised Surgical Perception Stereo Visual Odometer (SPSVO) framework is proposed to accurately estimate endoscopic pose and better assist surgeons in locating and diagnosing lesions. The proposed SPSVO system combines a self-learning feature extraction method and a self-supervised matching procedure to overcome the adverse effects of irregular illumination in endoscopic images. The framework of the proposed SPSVO includes image pre-processing, feature extraction, stereo matching, feature tracking, keyframe selection, and pose graph optimization. The SPSVO can simultaneously associate the appearance of extracted feature points and textural information for fast and accurate feature tracking. A nonlinear pose graph optimization method is adopted to facilitate the backend process. The effectiveness of the proposed SPSVO framework is demonstrated on a public endoscopic dataset, with the obtained root mean square error of trajectory tracking reaching 0.278 to 0.690 mm. The computation speed of the proposed SPSVO system can reach 71ms per frame.

Target recognition and detection system based on sensor and nonlinear machine vision fusion

Abstract In order to realize the automatic detection system of electric sensor, a method based on sensor and nonlinear machine vision is proposed. Aiming at complex scenes and dynamic changes in target recognition and detection in large-scale industrial field, a target recognition and detection system based on the fusion of vision sensor and nonlinear machine vision is proposed. The system introduces nonlinear features and uses deep neural network to realize multi-scale analysis and recognition of image data on the basis of traditional machine vision. The system uses C++ language development and has a good user interface. The photoelectric sensor weld image is collected by machine vision technology, the target area of the image is detected by Gaussian model, the feature points of the target area are extracted by combining Hessian matrix, the extracted feature points are input into the quantum gate neural network model, and the recognition results are obtained. The simulation results show that the author’s method has the highest value among the three test indicators, with the highest accuracy rate of 97%, the highest recall rate of 98%, and the highest F1 value of 94. The time consumed by the author’s method for automatic identification of photoelectric sensor welding is within 6 s, the time spent on the film wall recognition method for automatic identification of photoelectric sensor welding is within 20 s, and the time spent by the feature extraction and identification method for automatic identification of photoelectric sensor weld is within 22 s. It has been proven that the method based on the fusion of sensors and nonlinear machine vision can achieve an automatic recognition and detection system for electrical sensor welds. The object detection and recognition method proposed in this article can be applied to dynamic changes and complex scenes in various complex backgrounds and has a good application prospect. The system proposed in this article has some limitations, such as the algorithm in the calculation accuracy, real-time, and other aspects that have room for improvement.

TUCNet: A channel and spatial attention-based graph convolutional network for teeth upsampling and completion

With the increasing popularity of the use of 3D scanning equipment in capturing oral cavity in dental health applications, the quality of 3D dental models has become vital in oral prosthodontics and orthodontics. However, the point cloud data obtained can often be sparse and thus missing information. To address this issue, we construct a high-resolution teeth point cloud completion method named TUCNet to fill up the sparse and incomplete oral point cloud collected and output a dense and complete teeth point cloud. First, we propose a Channel and Spatial Attentive EdgeConv (CSAE) module to fuse local and global contexts in the point feature extraction. Second, we propose a CSAE-based point cloud upsample (CPCU) module to gradually increase the number of points in the point clouds. TUCNet employs a tree-based approach to generate complete point clouds, where child points are derived through a splitting process from parent points following each CPCU. The CPCU learns the up-sampling pattern of each parent point by combining the attention mechanism and the point deconvolution operation. Skip connections are introduced between CPCUs to summarize the split mode of the previous layer of CPCUs, which is used to generate the split mode of the current CPCUs. We conduct numerous experiments on the teeth point cloud completion dataset and the PCN dataset. The experimental results show that our TUCNet not only achieves the state-of-the-art performance on the teeth dataset, but also achieves excellent performance on the PCN dataset.

PLI-SLAM: A Tightly-Coupled Stereo Visual-Inertial SLAM System with Point and Line Features

Point feature-based visual simultaneous localization and mapping (SLAM) systems are prone to performance degradation in low-texture environments due to insufficient extraction of point features. In this paper, we propose a tightly-coupled stereo visual-inertial SLAM system with point and line features (PLI-SLAM) to enhance the robustness and reliability of systems in low-texture environments. We improve Edge Drawing lines (EDlines) for line feature detection by introducing curvature detection and a new standard for minimum line segment length to improve the accuracy of the line features, while reducing the line feature detection time. We contribute also with an improved adapting factor based on experiment to adjust the error weight of line features, which further improves the localization accuracy of the system. Our system has been tested on the EuRoC dataset. Tests on public datasets and in real environments have shown that PLI-SLAM achieves high accuracy. Furthermore, PLI-SLAM could still operate robustly even in some challenging environments. The processing time of our method is reduced by 28%, compared to the ORB-LINE-SLAM based on point and line, when using Line Segment Detector (LSD).

RTV-SIFT: Harnessing Structure Information for Robust Optical and SAR Image Registration

Registration of optical and synthetic aperture radar (SAR) images is challenging because extracting located identically and unique features on both images are tricky. This paper proposes a novel optical and SAR image registration method based on relative total variation (RTV) and scale-invariant feature transform (SIFT), named RTV-SIFT, to extract feature points on the edges of structures and construct structural edge descriptors to improve the registration accuracy. First, a novel RTV-Harris feature point detection method by combining the RTV and the multiscale Harris algorithm is proposed to extract feature points on both images’ significant structures. This ensures a high repetition rate of the feature points. Second, the feature point descriptors are constructed on enhanced phase congruency edge (EPCE), which combines the Sobel operator and maximum moment of phase congruency (PC) to extract edges from structured images that enhance robustness to nonlinear intensity differences and speckle noise. Finally, after coarse registration, the position and orientation Euclidean distance (POED) between feature points is utilized to achieve fine feature point matching to improve the registration accuracy. The experimental results demonstrate the superiority of the proposed RTV-SIFT method in different scenes and image capture conditions, indicating its robustness and effectiveness in optical and SAR image registration.

UAV image matching of mountainous terrain using the LoFTR deep learning model

In natural terrain scene UAV image matching, traditional feature point-based methods often have problems such as an unstable number of extracted feature points, difficulty in detecting feature points in weak texture areas, uneven distribution, and low robustness. Deep learning-based image matching methods can produce larger and more reasonably distributed matching pairs, so this research paper tries to perform UAV image matching based on a deep learning LoFTR algorithm for natural terrain scenes. The critical technical process was: first, the LoFTR algorithm was used to generate dense feature matching, and then the epipolar line constraints were used to purify the interior points, specifically, this study used the MAGSAC++ method to estimate the fundamental matrix, eliminate the wrong matching pairs, and finally get reliable matching results. In this research paper, six sets of visible images taken by different UAVs equipped with different sensors in the field were selected as experimental data to test the method and were compared and analyzed with the traditional classical SIFT, ASIFT, and AKAZE algorithms and the KeyNet-AdaLAM deep learning method. The experimental results show that the method in this study obtains a dense number of robust matching pairs with uniform spatial distribution in the UAV image matching of natural scenes mainly in mountainous areas, and the comprehensive performance is higher and more advantageous than the comparison methods.

An Improved Visual SLAM Method with Adaptive Feature Extraction

The feature point method is the mainstream method to accomplish inter-frame estimation in visual Simultaneous Localization and Mapping (SLAM) methods, among which the Oriented FAST and Rotated BRIEF (ORB) feature-based method provides an equilibrium of accuracy as well as efficiency. However, the ORB algorithm is prone to clustering phenomena, and its unequal distribution of extracted feature points is not conducive to the subsequent camera tracking. To solve the above problems, this paper suggests an adaptive feature extraction algorithm that first constructs multiple-scale images using an adaptive Gaussian pyramid algorithm, calculates adaptive thresholds, and uses an adaptive meshing method for regional feature point detection to adapt to different scenes. The method uses Adaptive and Generic Accelerated Segment Test (AGAST) to speed up feature detection and the non-maximum suppression method to filter feature points. The feature points are then divided equally by a quadtree technique, and the orientation of those points is determined by an intensity centroid approach. Experiments were conducted on publicly available datasets, and the outcomes demonstrate the algorithm has good adaptivity and solves the problem of a large number of corner point clusters that may result from using manually set detection thresholds. The RMSE of the absolute trajectory error of SLAM applying this method on four sequences of TUM RGB-D datasets is decreased by 13.88% when compared with ORB-SLAM3. It is demonstrated that the algorithm provides high-quality feature points for subsequent image alignment, and the application to SLAM improves the reliability and accuracy of SLAM.

A Study on the Optimal Grasping Angle Algorithm for Plug Seedlings Based on Machine Vision

During the replanting operation of a seedling tray, the end-effector needs to repeatedly grab the qualified plug seedlings in the supply tray and release them to the target tray for replanting, and in the process of grasping, the end-effector may cause some mechanical damage to the plug seedlings, thus affecting their quality. Therefore, in order to be able to adjust the position of the hand claw grasping point according to the morphological characteristics of the plug seedlings and select the optimal grasping point, this paper proposes research on the optimal grasping angle algorithm for plug seedlings based on machine vision. Firstly, a rotatable three-jaw end-effector is designed, which uses a three-jaw structure for grasping the burrowing seedlings. The three claws are driven with a telescopic cylinder to carry out clamping and relaxing actions. The rotation of the three claws is controlled with the stepper motor to adjust the optimal grasping position. Secondly, based on the pre-processing of an image of the hole tray seedling, the extraction of feature points in the region of interest, and the calculation of localization, the angle between the angular bisector of the cotyledon leaf blade of the hole tray seedling and the horizontal positive direction is solved. In this paper, two methods are designed to calculate the coordinates of feature points: one is the geometric method and the other is the center-of-mass method. Finally, the optimal grasping angle is calculated by analyzing the angle between the angular bisector of the cotyledon leaf blade and the horizontal positive direction of the cavity seedlings. According to the test, the average calculation error of the proposed algorithm is 3.12 degrees, and the average calculation time is 0.512 sec/sheet, which meet the requirements of the replanting operation.

A Novel Visual SLAM Based on Multiple Deep Neural Networks

The current visual simultaneous localization and mapping (SLAM) systems require the use of matched feature point pairs to estimate camera pose and construct environmental maps. Therefore, they suffer from poor performance of the visual feature matchers. To address this problem, a visual SLAM using deep feature matcher is proposed, which is mainly composed of three parallel threads: Visual Odometry, Backend Optimizer and LoopClosing. In the Visual Odometry, the deep feature extractor with convolutional neural networks is utilized for extracting feature points in each image frame. Then, the deep feature matcher is used for obtaining the corresponding feature landmark pairs. Afterwards, a fusion method based on the last and the reference frame is proposed for camera pose estimation. The Backend Optimizer is designed to execute local bundle adjustment for a part of camera poses and landmarks (map points). While LoopClosing, consisting of a lightweight deep loop closure detector and the same matcher as the one used in Visual Odometry is utilized for loop correction based on pose graph. The proposed system has been tested extensively on most of benchmark KITTI odometry dataset. The experimental results show that our system yields better performance than the existing visual SLAM systems. It can not only run in real-time at a speed of 0.08 s per frame, but also reduce estimation error by at least 0.1 m.

Multi-robot raster map fusion without initial relative position

PurposeThe purpose of this study is to solve the problem of an unknown initial position in a multi-robot raster map fusion. The method includes two-dimensional (2D) raster maps and three-dimensional (3D) point cloud maps.Design/methodology/approachA fusion method using multiple algorithms was proposed. For 2D raster maps, this method uses accelerated robust feature detection to extract feature points of multi-raster maps, and then feature points are matched using a two-step algorithm of minimum Euclidean distance and adjacent feature relation. Finally, the random sample consensus algorithm was used for redundant feature fusion. On the basis of 2D raster map fusion, the method of coordinate alignment is used for 3D point cloud map fusion.FindingsTo verify the effectiveness of the algorithm, the segmentation mapping method (2D raster map) and the actual robot mapping method (2D raster map and 3D point cloud map) were used for experimental verification. The experiments demonstrated the stability and reliability of the proposed algorithm.Originality/valueThis algorithm uses a new visual method with coordinate alignment to process the raster map, which can effectively solve the problem of the demand for the initial relative position of robots in traditional methods and be more adaptable to the fusion of 3D maps. In addition, the original data of the map can come from different types of robots, which greatly improves the universality of the algorithm.