Feature Point Extraction Research Articles

DE-RGBD SLAM: Enhancing Static Feature Point Selection in RGB-D Visual SLAM Using Depth Information

Abstract Feature point extraction plays a key role in visual simultaneous localization and mapping (SLAM) systems. And it remains a major challenge to accurately select static feature points in a complex dynamic environment. To address this issue, this paper proposes an RGB-D SLAM method, referred to as DE-RGBD SLAM, which optimizes feature selection by integrating depth information and effectively utilizes depth data and multi-view geometric information to achieve localization and navigation for mobile robots in dynamic environments. Firstly, the method analyzes prominent feature regions in the image based on colour and depth information captured by an RGB-D camera. It sets adaptive FAST corner detection thresholds according to the grayscale information of these regions while masking other areas. Next, the method obtains in-depth information on the detected feature points in the current frame. It combines their pixel coordinates in the image coordinate system to determine the presence of redundant feature points. Notably, the method can detect some dynamic feature points between consecutive frames. Subsequently, in the camera coordinate system, the method compares the depth information of feature points in the depth image with the epipolar depth estimates derived from the essential matrix to determine whether the features are static and eliminate dynamic feature points. This approach significantly enhances the reliability of static feature points. Finally, the accuracy and robustness of the proposed method are validated through experiments conducted on the public TUM dataset and real-world scenarios compared to state-of-the-art visual SLAM systems.

A CAM-UNet for pose determination of supercapacitor cell module assembly based on monocular vision

Purpose This paper aims to solve the problems of low stacking efficiency and long production time in the supercapacitor module assembly process, a stacking system based on monocular vision is proposed, including bracket visual positioning, grasping and stacking, and it is applied in actual production. Design/methodology/approach To enhance the robustness of the workpiece location method and improve the location accuracy, the improved U-Net network and image processing algorithms are used to segment the collected images. In addition, for the extracted feature points, the objective function that can be globally optimized is obtained by parameterizing the rotation matrix to construct a polynomial equation system and, finally, the equation system is solved to obtain the final pose estimation, which could improve the accuracy of workpiece location. Findings The result indicates that the proposed method is successfully performed on the manipulator. Besides, this method can well solve the problem of object reflection on the conveyor belt. The Intersection over Union of the image segmentation of the object is 0.9948, and the Pixel Accuracy is 0.9973, which has a high segmentation accuracy for the image. The error range between the method proposed in this paper and the pose estimation is within 2 mm, and the qualified rate of supercapacitor module stacking products is over 99.8%. Originality/value This paper proposes a method of accurately extracting feature points by integrating an improved U-Net network and image processing and uses the workpiece positioning algorithm of the optimal solution PnP problem algorithm. The calculation results show that the algorithm improves the positioning accuracy of the workpiece, realizes the assembly of stacked supercapacitor modules and is applied in industrial production.

DyPanVO: a robust monocular visual odometry in dynamic environments by utilizing multiple deep neural networks

Abstract Traditional monocular Visual Odometry (VO) is typically based on the assumption of a static environment. However, it performs poorly in dynamic scenes, suffering from error accumulation and scale drift. To address these issues, a novel framework, named DyPanVO, was proposed, which incorporates multiple deep neural networks for feature point extraction, panoptic segmentation, and depth estimation. Firstly, learning-based methods are employed, specifically SuperPoint for robust feature extraction and LightGlue for accurate feature matching. Then, outlier points are eliminated by combining dynamic prior results from panoptic segmentation with epipolar geometry constraints to determine the motion state of objects. Additionally, the introduction of depth information ensures that scale is recovered and fundamentally solves the issue of error accumulation. Two types of experiment (outdoor and indoor scenes) are carried out to show the effectiveness of DyPanVO. Moreover, the comparison results demonstrate that it performs comparably with multi-view geometry-based and outperforms learning-based methods.

Application of Improved Visual SLAM in Intraoperative Navigation of Robotic-Assisted Laparoscopic Surgery

In robot-assisted laparoscopic surgery, precise intraoperative navigation is essential to enhance the accuracy, safety, and efficiency of the procedure. By extracting feature points from the environment, Visual Simultaneous Localization and Mapping (V-SLAM) technology has been widely employed to accomplish self-localization and map generation. In laparoscopic surgery, V-SLAM offers high-precision real-time positional data for surgical robots, thereby improving the accuracy of operative navigation. Nonetheless, the performance of conventional V-SLAM algorithms is significantly compromised by the extremely dynamic characteristics of surgical environments. Widely used visual SLAM systems, such as ORB-SLAM, are susceptible to inaccurate feature point tracking in the presence of dynamic objects and variations in lighting, hence compromising navigation precision and surgical safety. This study assessed the efficacy of visual V-SLAM technology in enhancing intraoperative navigation during robot-assisted laparoscopic surgery, pointing out its contribution to surgical precision, safety, and operational efficiency. The research enhanced the ORB-SLAM2 algorithm to adapt to the dynamically changing conditions encountered during surgical procedures. The experiment validated the efficacy of the enhanced method within a simulated laparoscopic surgery setting and assessed the impact of three variables: light intensity, instrument occlusion, and dynamic tissue deformation, on trajectory error (ATE and RPE) and mapping accuracy. The findings indicated that under situations of low illumination, significant occlusion, and substantial deformation, ATE and RPE mistakes escalated by over 50%, while map accuracy diminished by approximately 40%, leading to heightened potential surgical risks.

A Lightweight Deep-Learning Visual SLAM for Indoor Dynamic Environment Using Yolov10

Vision simultaneous localization and mapping (SLAM) technology has become a key research direction in the field of mobile robotics in recent years. However, the accuracy and stability of traditional vision SLAM technology greatly affected by the dynamic environment, and the mainstream dynamic feature point rejection method combining vision and semantic segmentation techniques is not applicable to edge-end devices with limited resources and high real-time requirements. This research suggests a visual SLAM algorithm based on the YOLOv10n lightweight target detection model and the GCNv2 feature point extraction model to accomplish real-time dynamic feature point rejection to address those problems. To compensate for the detection accuracy and stability issues of the YOLOv10n model while leveraging its real-time advantages, the algorithm also employs multi-target Kalman filtering with data association through the Hungarian algorithm, history window smoothing, and potential dynamic feature point recording methods to enhance robustness. The algorithm is validated on the TUM RGB-D Dataset, and the results demonstrate that the method can effectively reject the dynamic feature points in the dynamic environment, and it has a significant improvement in the accuracy and stability of the visual SLAM system.

Intelligent Perception and Seam Tracking System for Thick Plate Weldments Based on Constant-Focus Optical Path

To achieve efficient and accurate thick plate welding, as well as to precisely extract and plan the paths of complex three-dimensional weld seams in large steel structures, this study introduces a novel vision-guided approach for robotic welding systems utilizing a constant-focus laser sensor. This methodology specifically targets and mitigates several critical shortcomings inherent in conventional vision-guided welding techniques, including limited detection ranges, diminished precision in both detection and tracking, and suboptimal real-time performance. For preprocessed weld images, an improved grayscale extreme centroid method was developed to extract the center of the light stripe. Furthermore, a sophisticated feature point extraction algorithm, which integrates a maximum distance search strategy with a least-squares fitting procedure, was developed to facilitate the precise and timely identification of weld seam characteristic points. To further optimize the outcomes, a cylindrical filtering mechanism was employed to eliminate substantial discrepancies, whereas local Non-Uniform Rational B-Spline (NURBS) curve interpolation was utilized for the generation of smooth and accurate trajectory plans. A spatial vector-based pose adjustment strategy was then implemented to provide robust guidance for the welding robot, ensuring the successful execution of the welding operations. The experimental results indicated that the proposed algorithm achieved a tracking error of 0.3197 mm for welding workpieces with a thickness of 60 mm, demonstrating the method’s substantial potential in the manufacturing sector, especially in the domain of automated welding.

A Similar Feature Point Matching Method for Aerial Electric Power Tower Images Based on a One-Ring Neighborhood of Vertices

Due to the susceptibility of images to various factors such as scale changes, imaging conditions, and image noise, traditional feature point matching methods are difficult to achieve ideal matching accuracy, leading to many challenges in the automatic analysis and processing of power tower images. To improve the matching accuracy of similar feature points in aerial images of electric power tower, this study proposes a method for matching similar feature points in aerial images of electric power tower based on a vertex ring neighborhood, addressing the problem of large matching errors caused by factors such as scale and imaging conditions. This method first adopts the feature point extraction technique of electric power tower image based on decomposition and filtering, and constructs the Laplacian pyramid of the original image. Subsequently, filter banks are used to decompose the pyramid image in different directions, extract local extremum points as candidate feature point sets, and merge them to obtain the final feature point set. On this basis, taking into account the spatial relationships and geometric characteristics around the feature points, a texture mapping algorithm based on vertex ring neighborhood and patch classification is applied to construct a vertex ring neighborhood structure and extract geometrically representative electric power tower features. Finally, by using a texture feature point matching method based on the principle of similarity, the similarity between the real-time image and the reference image features is calculated for matching, and combined with the Babbitt coefficient to remove mismatched feature points, accurate matching of similar feature points in aerial electric power tower images is achieved. The experimental results show that this method has a mean square error of less than 0.1Pixel in matching similar texture feature points of aerial power tower images under various working conditions, significantly improving the matching accuracy and providing an effective tool for automatic analysis and processing of power tower images.

Improved Early-Stage Maize Row Detection Using Unmanned Aerial Vehicle Imagery

Monitoring row centerlines during early growth stages is essential for effective production management. However, detection becomes more challenging due to weed interference and crop row intersection in images. This study proposed an enhanced Region of Interest (ROI)-based approach for detecting early-stage maize rows. It integrated a modified green vegetation index with a dual-threshold algorithm for background segmentation. The median filtering algorithm was also selected to effectively remove most noise points. Next, an improved ROI-based feature point extraction method was used to eliminate residual noises and extract feature points. Finally, the least square method was employed to fit the row centerlines. The detection accuracy of the proposed method was evaluated using the unmanned aerial vehicle (UAV) image data set containing both regular and intersecting crop rows. The average detection accuracy of the proposed approach was between 0.456° and 0.789° (the angle between the fitted centerline and the expert line), depending on whether crop rows were regular/intersecting. Compared to the Hough Transform (HT) algorithm, the results demonstrated that the proposed method achieved higher accuracy and robustness in detecting regular and intersecting crop rows. The proposed method in this study is helpful for refined agricultural management such as fertilization and irrigation. Additionally, it can detect the missing-seedling regions and replenish seedings in time to increase crop yields.

Research on efficient matching method of coal gangue recognition image and sorting image

When the coal gangue sorting robot sorts coal gangue, the position of the target coal gangue will change due to belt slippage, deviation, and speed fluctuations of the belt conveyor. This will cause the robotic to fail in grasping or miss grasping. We have developed a solution to this problem: the IMSSP-Net two-stage network gangue image fast matching method. This method will reacquire the target gangue position information and improve the robot’s grasping precision and efficiency. In the first stage, we use SuperPoint to guarantee the scene adaptability and credibility of feature point extraction. We have enhanced Superpoint’s ability to detect feature points further by using the improved Multi-scale Retinex with Color Restoration enhancement algorithm. In the second stage, we introduce SuperGlue for feature matching to improve the robustness of the matching network. We eliminated erroneous feature matching point pairs and improved the accuracy of image matching by adopting the PROSAC algorithm. We conducted image matching comparison experiments under different object distances, scales, rotation angles, and complex conditions. The experimental platform adopts the double-manipulator truss-type coal gangue sorting robot independently developed by the team. The matching precision, recall, and matching time of the method are 98.2%, 98.3%, and 84.6ms, respectively. The method can meet the requirements of efficient and accurate matching between coal gangue recognition images and sorting images.

A LiDAR SLAM Algorithm Considering Dynamic Extraction of Feature Points in Underground Coal Mine

Abstract. To address the absence of GNSS signals in underground coal mines and the susceptibility of mainstream LiDAR SLAM to degradation due to insufficient feature constraints, this paper proposes a tightly-coupled SLAM algorithm incorporating LiDAR and IMU for use in such environments. The paper initially introduces a dynamic feature point extraction strategy, where the number of corner feature points can be dynamically adjusted based on the occurrence of degradation in the underground coal mine environment. This approach constructs a constraint matrix with rich and robust feature information, enhancing pose estimation accuracy in environments with inadequate feature constraints, mitigating degradation effects, and reducing global cumulative error. Subsequently, the consistent building of the underground coal mine environment is achieved through back-end factor graph optimization. Finally, to validate the effectiveness of the method, experimental analysis is conducted in an underground coal mine. The results demonstrate that LeGO_LOAM exhibits poor robustness in underground coal mines and fails to construct a globally consistent pose estimation and map. Conversely, the pose estimation error of the proposed method in this paper is 50.93% lower in the plane direction and 42.13% lower in the elevation direction compared to LIO_SAM. This underscores the method's significance as crucial technical support for the intelligent perception and positioning of robots in underground coal mines.

Research on a Feature Point Detection Algorithm for Weld Images Based on Deep Learning

Laser vision seam tracking enhances robotic welding by enabling external information acquisition, thus improving the overall intelligence of the welding process. However, camera images captured during welding often suffer from distortion due to strong noises, including arcs, splashes, and smoke, which adversely affect the accuracy and robustness of feature point detection. To mitigate these issues, we propose a feature point extraction algorithm tailored for weld images, utilizing an improved Deeplabv3+ semantic segmentation network combined with EfficientDet. By replacing Deeplabv3+’s backbone with MobileNetV2, we enhance prediction efficiency. The DenseASPP structure and attention mechanism are implemented to focus on laser stripe edge extraction, resulting in cleaner laser stripe images and minimizing noise interference. Subsequently, EfficientDet extracts feature point positions from these cleaned images. Experimental results demonstrate that, across four typical weld types, the average feature point extraction error is maintained below 1 pixel, with over 99% of errors falling below 3 pixels, indicating both high detection accuracy and reliability.

Position Normalization of Propellant Grain Point Clouds

Point cloud data obtained from scanning propellant grains with 3D scanning equipment exhibit positional uncertainty in space, posing significant challenges for calculating the relevant parameters of the propellant grains. Therefore, it is essential to normalize the position of each propellant grain’s point cloud. This paper proposes a normalization algorithm for propellant grain point clouds, consisting of two stages, coarse normalization and fine normalization, to achieve high-precision transformations of the point clouds. In the coarse normalization stage, a layer-by-layer feature points detection scheme based on k-dimensional trees (KD-tree) and k-means clustering (k-means) is designed to extract feature points from the propellant grain point cloud. In the fine normalization stage, a rotation angle compensation scheme is proposed to align the fitted symmetry axis of the propellant grain point cloud with the coordinate axes. Finally, comparative experiments with iterative closest point (ICP) and random sample consensus (RANSAC) validate the efficiency of the proposed normalization algorithm.

Atomic force microscopy wide-field scanning imaging using homography matrix optimization

During the offline combination of multiple atomic force microscopy (AFM) images, changes in probe displacement can be affected by dynamic noise, leading to dislocation and tearing in the stitching. To overcome this, we designed a homography matrix optimization method to describe the relative positional relationship between images by constructing the original image matrix and applying singular value decomposition to denoise the homography matrix. Additionally, we implemented a scale-invariant feature transform (SIFT) to extract feature points. To verify the effectiveness of this method, the SIFT + RANSAC (R), SIFT + affine transformation (AT), and oriented FAST and rotated BRIEF (ORB) algorithms were compared with the proposed algorithm. The experimental results demonstrate that the proposed method precisely computes the transformation matrix, thereby guaranteeing the geometric consistency of mosaic imaging. The proposed method preserves the intricate details of the original image and enhances the stitching quality of wide-field images.

Selective Encryption Algorithm for Vectoral Geographic Data under Feature Point Grouping Strategy

Abstract. Selective encryption technology is a data security protection method that can balance encryption security and efficiency. Currently, there is ongoing research on applying this technology to vector geographic data. However, existing algorithms often use random selection methods for choosing objects to be encrypted, which results in lower security. To address this problem, we proposed a selective encryption algorithm for vector geographic data based on feature point grouping strategy. The study includes four steps: (1) Based on the user-input encryption ratio, calculate the feature point grouping thresholds for each element in the data. (2) Extract feature point sets for each element based on the grouping threshold. (3) Sort and group the feature point sets, reducing the smallest encryption unit to group objects within individual elements. (4) To reduce encryption costs and enhance the algorithm's resistance to attacks, perform frequency domain coefficient encryption and spatial domain coordinate value encryption in a stepwise manner on the group objects, ultimately producing the encryption results. The experiment showed that: (1) The decrypted encrypted data maintains consistency with the original data coordinates, achieving lossless decryption. (2) The correlation between encrypted data and the original data is significantly disrupted, resulting in a high level of randomness and thereby ensuring the algorithm's robust security. (3) The key space is substantial and highly sensitive, capable of withstanding brute force attacks. (4) The algorithm significantly improves encryption efficiency. (5) The algorithm displays resilience against deletion attacks and noise attacks. In conclusions, the proposed method enhances encryption efficiency while upholding a high level of security, thus it is an efficient selective encryption algorithm for vector geographic data.

Combination Pattern Method Using Deep Learning for Pill Classification

The accurate identification of pills is essential for their safe administration in the medical field. Despite technological advancements, pill classification encounters hurdles such as ambiguous images, pattern similarities, mixed pills, and variations in pill shapes. A significant factor is the inability of 2D imaging to capture a pill’s 3D structure efficiently. Additionally, the scarcity of diverse datasets reflecting various pill shapes and colors hampers accurate prediction. Our experimental investigation shows that while color-based classification obtains a 95% accuracy rate, shape-based classification only reaches 66%, underscoring the inherent difficulty distinguishing between pills with similar patterns. In response to these challenges, we propose a novel system integrating Multi Combination Pattern Labeling (MCPL), a new method designed to accurately extract feature points and pill patterns. MCPL extracts feature points invariant to rotation and scale and effectively identifies unique edges, thereby emphasizing pills’ contour and structural features. This innovative approach enables the robust extraction of information regarding various shapes, sizes, and complex pill patterns, considering even the 3D structure of the pills. Experimental results show that the proposed method improves the existing recognition performance by about 1.2 times. By improving the accuracy and reliability of pill classification and recognition, MCPL can significantly enhance patient safety and medical efficiency. By overcoming the limitations inherent in existing classification methods, MCPL provides high-accuracy pill classification, even with constrained datasets. It substantially enhances the reliability of pill classification and recognition, contributing to improved patient safety and medical efficiency.

Thermal Infrared Orthophoto Geometry Correction Using RGB Orthophoto for Unmanned Aerial Vehicle

The geometric correction of thermal infrared (TIR) orthophotos generated by unmanned aerial vehicles (UAVs) presents significant challenges due to low resolution and the difficulty of identifying ground control points (GCPs). This study addresses the limitations of real-time kinematic (RTK) UAV data acquisition, such as network instability and the inability to detect GCPs in TIR images, by proposing a method that utilizes RGB orthophotos as a reference for geometric correction. The accelerated-KAZE (AKAZE) method was applied to extract feature points between RGB and TIR orthophotos, integrating binary descriptors and absolute coordinate-based matching techniques. Geometric correction results demonstrated a significant improvement in regions with stable and changing environmental conditions. Invariant regions exhibited an accuracy of 0.7~2 px (0.01~0.04), while areas with temporal and spatial changes saw corrections within 5~7 px (0.10~0.14 m). This method reduces reliance on GCP measurements and provides an effective supplementary technique for cases where GCP detection is limited or unavailable. Additionally, this approach enhances time and economic efficiency, offering a reliable alternative for precise orthophoto generation across various sensor data.

Parametric indoor 3D reconstruction based on binary image feature point extraction

Abstract In recent years, with the rapid advancement of laser scanning technology, indoor 3D modeling technology has significantly developed, becoming a research hotspot in the field of 3D reconstruction. Addressing the issues of high complexity in decoding point cloud surfaces in traditional indoor reconstruction, this paper proposes an indoor 3D reconstruction method based on binary image feature point extraction. The method utilizes point cloud data for semantic segmentation of the indoor environment to accurately extract the ceiling area, which is then projected onto the XOY plane and converted into a binary image, thereby transforming the 3D problem into a 2D problem. Subsequently, key feature points are identified through feature point detection technology, and systematic model construction is carried out based on the sequence of these feature points and their associated parameters. This method effectively simplifies the complexity of traditional 3D modeling, achieving accurate and parametric indoor 3D reconstruction.

Comparative Analysis of Image Stitching Algorithms for Bridge Inspection Imaging Systems

This paper discusses an innovative vehicle-mounted device designed for efficient inspection of bridge undersides, aiming to address the challenges of manual inspections, such as being labor-intensive and inefficient. The device uses high-precision cameras to capture detailed images of bridge bottoms, facilitating a thorough condition assessment. It evaluates three feature point extraction algorithms for image stitching—Scale-Invariant Feature Transform (SIFT), Harris corner detection, and Speeded Up Robust Features (SURF)—to create complete visual representations of bridge undersides. Field tests on a prefabricated I girder bridge demonstrated the device's effectiveness in gathering and stitching images, with the SIFT algorithm performing best for girder bottoms, Harris corner algorithm for web and flange surfaces, and SURF for rapid image processing. This research provides a foundation for the rapid identification of visible defects on bridge superstructures, significantly benefiting bridge maintenance and safety evaluations.

An adaptive trajectory segmentation and simplification algorithm based on vessel behavioral features

Existing algorithms for trajectory simplification in AIS data processing have issues such as loss of key feature points, high subjectivity in threshold selection, and incomplete consideration of factors. In this paper, an adaptive trajectory segmentation and simplification algorithm based on vessel behavioral features is proposed. The algorithm identifies behavioral feature intervals by extracting vessel behavioral features and constructs an adaptive segmentation threshold function to achieve adaptive segmentation of trajectories and extraction of key feature points. Then, considering the quality, similarity, and spatial variability of trajectory simplification, an adaptive simplification threshold function is constructed to automatically select the best threshold for each sub-trajectory to generate the simplified trajectory. Our experiments demonstrate that the algorithm maintains a high simplification rate while enabling adaptive simplification of the trajectory, and also retains the key feature points and trajectory shapes to the maximum extent. The effectiveness and robustness of the algorithm are verified through comparison with other algorithms, effectively avoiding the distortion of navigational behavior that may be caused by traditional methods. This provides reliable data support for maritime traffic management and decision-making.

Self-supervised fusion network for RGB-D interest point detection and description

Interest point detection and description are highly challenging in indoor environments with repeated and sparse textures and heavy illumination changes (noted as challenging indoor environments, CIE). In such environments, it is a severe problem of mismatched or misaligned feature points, often resulting in unsatisfactory accuracy in indoor applications, such as SLAM. To deal with the issue, we propose a self-supervised RGB-D cross-modal fusion network (RDFNet) for feature extraction. In the RDFNet, a dual-stream structure is introduced to build a pseudo-Siamese network for simultaneously processing color and depth images, while a new two-stage cross-modal reweighted fusion method (TCRF) is developed to fuse RGB and depth features. The TCRF achieves effective fusion in two steps: (1) introducing the reweighting idea and compositely enhancing RGB features by the depth features at both low-level and high-level stages; (2) concatenating the enhanced RGB and depth features together. In addition, we add a uniform distribution loss function to encourage the uniform extraction of feature points. To verify the proposed model performance, a new test dataset of specific indoor scenes is created to evaluate it and compare it to other state-of-the-art methods. Experimental results demonstrate its excellent performance in challenging indoor scenarios.