Feature Point Detection Research Articles

Evaluation and analysis of feature point detection methods based on vSLAM systems

Feature point detection is identified as a significant issue in the field of computer vision. Quantitative conclusions have been established regarding the performance evaluation of feature point detection using manually annotated datasets. However, these datasets, obtained through affine transformations with preset parameters on images, exhibit limitations in terms of variety, quantity, and challenges, differing from real-world application scenarios. In actual scenes, Vision Simultaneous Localization and Mapping (vSLAM) systems are extensively applied, and the precision of their localization and mapping is directly linked to the performance of feature points. To profoundly understand the performance of feature point detection in practical applications, vSLAM systems are chosen for evaluation in this study. More diverse and challenging datasets are utilized, encompassing real datasets covering variations in lighting, rotation, occlusion, and camera angle changes, as well as synthetic datasets reflecting complex conditions such as time, season, motion patterns, and environmental textures. Based on the evaluation results, the applicability of various feature point detection methods in different environments is thoroughly discussed, and the underlying principles are analyzed (Table 13). The conclusions drawn from this research provide references for the development of new feature point detection methods, the selection of such methods in vSLAM systems, and other related studies in the field of computer vision.

Research on Image Stitching Technology for Road Surface Cracks

Image stitching refers to the combination of multiple images with a certain over lappingarea rate acquired by the image acquisition device into one image, which has a full picture of each image. Image stitching technology is an important part in the field of digital image processing. With the increasing requirements for image quality and the limited resolution of ordinary cameras, it is impossible to obtain large angle of view and clear image at the same time. In this paper, the key technologies involved in pavement crack image stitching are intensively studied, including image feature point detection and matching, internal and external camera parameter estimation and optimization, and image fusion. The work done in this paper and the main innovations include: (1)Two main feature detection and description algorithms, SIFT algorithm and ORBalgorithm, are studied. In view of the fact that the SIFT algorithm has redundancy when the Gaussian pyramid is established, this paper appropriately reduces the number of groups and maintains the number and stability of feature point detection. Combined with the advantages and disadvantages of the improved SIFT algorithm and ORB algorithm, and the performance of the two algorithms in the detection of pavement cracks, the improvedSIF'T algorithm is used as the feature point detection algorithm, and rBRlEF algorithm for describing the feature points in the ORB algorithm is used as the feature point description algorithm. The description algorithm is called the BSIF'T algorithm. Experiments show that the robustness of BSIFT algorithm is similar to that of SIFT algorithm, which is better than ORB algorithm and faster than SIF'T algorithm. (2)Image fiusion algorithms are studied, including best seam-line search and image fusion. Aiming at the ghost phenomenon that may occur in overlapping regions, this paper proposes a best seam-line search algorithm based on distance transform.

Combined effect mechanism of linear loading and disturbance with different amplitudes on sandstone

To comprehend the stress state and response characteristics of materials under complex conditions, researchers have decomposed stress states into fundamental paths and investigated diverse path combinations. To ensure comparability, four identical samples were carefully selected from a pool of 100 samples using ultrasonic tests based on the wave speed and waveform characteristics. These samples underwent specially designed stress paths to analyze the combined effects of linear loading and perturbation. Our result analysis centred on the perturbation amplitude and stress levels during composite action, revealing intricate relationships between the stress levels, strain, and nonlinear/linear energy evolution under complex stress paths. Simultaneously, 3D surface fractures were precisely reconstructed using the YOLOv5 and FAST feature point detection algorithms, elucidating the evolving patterns of the fractures. As a result of our study, the rotation trend of the main fracture was validated by integrating mechanics and P-wave reflection rules. Notably, our experimental results closely aligned with the theoretical predictions, showing the reliability of our study. These findings can significantly contribute to guiding safety protocols in the field of underground engineering.

ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples

Current image feature extraction methods fail to adapt to the fine features of apple image texture, resulting in image matching errors and degraded image processing accuracy. A multi-view orthogonal image acquisition system was constructed with apples as the research object. The system consists of four industrial cameras placed around the apple at different angles and one camera placed on top. Following the image acquisition through the system, synthetic image pairs—both before and after transformation—were generated as the input dataset. This generation process involved each image being subjected to random transformations. Through learning to extract more distinctive and descriptive features, the deep learning-based keypoint detection method surpasses traditional techniques by broadening the application range and enhancing detection accuracy. Therefore, a lightweight network called ALIKE-APPLE was proposed for surface feature point detection. The baseline model for ALIKE-APPLE is ALIKE, upon which improvements have been made to the image feature encoder and feature aggregation modules. It comprises an Improved Convolutional Attention Module (ICBAM) and a Boosting Resolution Sampling Module (BRSM). The proposed ICBAM replaced max pooling in the original image feature encoder for downsampling. It enhanced the feature fusion capability of the model by utilizing spatial contextual information and learning region associations in the image. The proposed BRSM replaced the bilinear interpolation in the original feature aggregator for upsampling, overcoming the apple side image’s geometric distortion and effectively preserving the texture details and edge information. The model size was shrunk by optimizing the number of downsampling operations from the image encoder of the original model. The experimental results showed that the average number of observed keypoints and the average matching accuracy were improved by 166.41% and 37.07%, respectively, compared with the baseline model. The feature detection model of ALIKE-APPLE was found to perform better than the optimal SuperPoint. The feature point distribution of ALIKE-APPLE showed an improvement of 10.29% in average standard deviation (Std), 8.62% in average coefficient of variation (CV), and 156.12% in average feature point density (AFPD). Moreover, the mean matching accuracy (MMA) of ALIKE-APPLE improved by 125.97%. Thus, ALIKE-APPLE boasts a more consistent allocation of feature points and greater precision in matching.

Multi-task learning and joint refinement between camera localization and object detection

Visual localization and object detection both play important roles in various tasks. In many indoor application scenarios where some detected objects have fixed positions, the two techniques work closely together. However, few researchers consider these two tasks simultaneously, because of a lack of datasets and the little attention paid to such environments. In this paper, we explore multi-task network design and joint refinement of detection and localization. To address the dataset problem, we construct a medium indoor scene of an aviation exhibition hall through a semi-automatic process. The dataset provides localization and detection information, and is publicly available at https://drive.google.com/drive/folders/1U28zkuN4_I0dbzkqyIAKlAl5k9oUK0jI?usp=sharing for benchmarking localization and object detection tasks. Targeting this dataset, we have designed a multi-task network, JLDNet, based on YOLO v3, that outputs a target point cloud and object bounding boxes. For dynamic environments, the detection branch also promotes the perception of dynamics. JLDNet includes image feature learning, point feature learning, feature fusion, detection construction, and point cloud regression. Moreover, object-level bundle adjustment is used to further improve localization and detection accuracy. To test JLDNet and compare it to other methods, we have conducted experiments on 7 static scenes, our constructed dataset, and the dynamic TUM RGB-D and Bonn datasets. Our results show state-of-the-art accuracy for both tasks, and the benefit of jointly working on both tasks is demonstrated.

The Design and Control of a Biomimetic Binocular Cooperative Perception System Inspired by the Eye Gaze Mechanism.

Research on systems that imitate the gaze function of human eyes is valuable for the development of humanoid eye intelligent perception. However, the existing systems have some limitations, including the redundancy of servo motors, a lack of camera position adjustment components, and the absence of interest-point-driven binocular cooperative motion-control strategies. In response to these challenges, a novel biomimetic binocular cooperative perception system (BBCPS) was designed and its control was realized. Inspired by the gaze mechanism of human eyes, we designed a simple and flexible biomimetic binocular cooperative perception device (BBCPD). Based on a dynamic analysis, the BBCPD was assembled according to the principle of symmetrical distribution around the center. This enhances braking performance and reduces operating energy consumption, as evidenced by the simulation results. Moreover, we crafted an initial position calibration technique that allows for the calibration and adjustment of the camera pose and servo motor zero-position, to ensure that the state of the BBCPD matches the subsequent control method. Following this, a control method for the BBCPS was developed, combining interest point detection with a motion-control strategy. Specifically, we propose a binocular interest-point extraction method based on frequency-tuned and template-matching algorithms for perceiving interest points. To move an interest point to a principal point, we present a binocular cooperative motion-control strategy. The rotation angles of servo motors were calculated based on the pixel difference between the principal point and the interest point, and PID-controlled servo motors were driven in parallel. Finally, real experiments validated the control performance of the BBCPS, demonstrating that the gaze error was less than three pixels.

OrangeStereo: A navel orange stereo matching network for 3D surface reconstruction

Three-dimensional (3D) surface reconstruction is crucial to accurately detecting navel orange external qualities such as size and shape, and binocular stereo vision is promising for practical sorting scenarios due to its high frame rates and ease of deployment. For objects with texture-less surface, such as navel oranges, it is very challenging to match the corresponding points using traditional stereo matching algorithms because the detected feature points have high similarity in color as well as structure. Most of the existing deep learning-based stereo matching algorithms focus on the stereo matching performance of the whole image and lack the attention to the matching performance of the main region. Meanwhile, simple background with less texture will bring more uncertainty to the matching of corresponding points, which affects the robustness of the model. The OrangeStereo was proposed based on the GwcNet structure to achieve a better corresponding points matching effect of navel oranges, which can provide more accurate 3D surface information. The Structural Features Extraction (SFE) module was introduced after the initial feature extraction module to extract stable structural features and avoid the limitations of using single convolutional features. The Attention Weights Generation (AWG) module was utilized at matching cost calculation stage to help suppress redundant information and make the network pay more attention to the matching performance of specific regions. The improved loss function assigned a greater weight to the loss of the navel orange region using semantic information from left image to make the network more focused on the matching effect of the navel orange region. An ablation study was conducted to confirm the efficiency of the SFE module, AWG module, and the improved loss function of the algorithm. The ablation study contained five sub-experiments, including the baseline model, adding the SFE and the AWG modules respectively, and whether to add the improved loss function based on the addition of previous two modules. Additionally, PSMNet, GwcNet, RealtimeStereo, and ACVNet, four typical stereo matching methods, were contrasted with the OrangeStereo. The proposed OrangeStereo achieved better performance with EPE, RMSE, bad-1, bad-3, bad-5, and inference time of 0.60 pixels, 3.53 pixels, 2.37 %, 1.83 %, 1.62 %, and 33 ms in the orange region, respectively. The R2 and RMSE for the depth of orange utilizing our method were 0.982 and 0.81 mm, respectively. The proposed algorithm can be employed to obtain accurate depth information of fruits quickly, and it is expected to be used in the 3D reconstruction of fruits in the actual commercial fruit sorting lines to achieve more accurate external quality assessment.

Blowout Fluid Velocity Measurement Method based on High-speed Sequence Images

Given the high-temperature and high-pressure environment after the blowout, the staff usually cannot calculate and analyze various blowout parameters at close range and thus take effective measures to control accidents quickly. In this paper, a blowout fluid velocity measurement method is proposed based on high-speed sequence images, and simulation experiments are conducted. The scene conditions of blowout accidents are simulated and analyzed. On this basis, cameras are adopted to shoot a large number of simulated blowout fluid videos from different angles under outdoor conditions. Afterward, the video is converted into a multi-frame image, and the blowout fluid image feature point detection and extraction model are established based on the orb (oriented fast and rotated brief) algorithm. On the feature points extracted by the model, the gms algorithm is used to match the features of the blowout fluid images in the front and back frames. Additionally, the wrong matching points are eliminated using the ransac (random sample consensus) algorithm, and the obtained feature point pairs are employed into the flow velocity calculation model for flow velocity calculation. The results demonstrate that the measured fluid velocity is more accurate. The method proposed in this study can provide reliable data support for early blowout emergency rescue.

A Multi-Modal Attention-Based Approach for Points of Interest Detection on 3D Shapes.

Identifying points of interest (POIs) on the surface of 3D shapes is a significant challenge in geometric processing research. The complex connection between POIs and their geometric descriptors, combined with the small percentage of POIs on the shape, makes detecting POIs on any given 3D shape a highly challenging task. Existing methods directly detect POIs from the entire 3D shape, resulting in low efficiency and accuracy. Therefore, we propose a novel multi-modal POI detection method using a coarse-to-fine approach, with the key idea of reducing data complexity and enabling more efficient and accurate subsequent POI detection by first identifying and processing important regions on the 3D shape. It first obtains important areas on the 3D shape through 2D projected images, then processes points within these regions using attention mechanisms. Extensive experiments demonstrate that our method outperforms existing POI detection techniques.

Research on Conservation and Restoration Methods of Museum Artifacts in the Context of Artificial Intelligence

Abstract In the era of artificial intelligence, the conservation and restoration of museum artifacts encounter both new possibilities and hurdles. The limitations of conventional conservation techniques, in terms of efficiency and accuracy, are becoming increasingly apparent. This study explores the potential of 3D reconstruction technologies, SIFT algorithms, and Poisson’s equation to revolutionize the digital restoration of cultural relics. These advanced approaches allow for precise, efficient restorations and the ability to undertake complex projects without risking harm to the original items. Our approach integrates cutting-edge algorithms to tailor 3D reconstruction strategies for museum artifacts, culminating in creating a comprehensive digital artifact database. Through the innovative use of feature point detection and texture synthesis, we have achieved notable successes, including a 12% improvement in 3D model accuracy and a significant enhancement in the automation of the restoration process. With an SSIM value of 0.9964 for restored images, our method demonstrates superiority over traditional restoration techniques, marking a significant stride towards the efficient digital preservation of cultural heritage.

Transformer-based 2D/3D medical image registration for X-ray to CT via anatomical features.

2D/3D medical image registration is one of the key technologies for surgical navigation systems to perform pose estimation and achieve accurate positioning, which still remains challenging. The purpose of this study is to introduce a new method for X-ray to CT 2D/3D registration and conduct a feasibility study. In this study, a 2D/3D affine registration method based on feature point detection is investigated. It combines the morphological and edge features of spinal images to accurately extract feature points from the images, and uses graph neural networks to aggregate anatomical features of different points to increase the local detail information. Meanwhile, global and positional information are extracted by the Swin Transformer. The results indicate that the proposed method has shown improvements in both accuracy and success ratio compared with other methods. The mean target registration error value reached up to 0.31mm; meanwhile, the runtime overhead was much lower, achieving an average runtime of about 0.6s. This ultimately improves the registration accuracy and efficiency, demonstrating the effectiveness of the proposed method. The proposed method can provide more comprehensive image information and shows good prospects for pose estimation and achieving accurate positioning in surgical navigation systems.

SASFF: A Video Synthesis Algorithm for Unstructured Array Cameras Based on Symmetric Auto-Encoding and Scale Feature Fusion.

For the synthesis of ultra-large scene and ultra-high resolution videos, in order to obtain high-quality large-scene videos, high-quality video stitching and fusion are achieved through multi-scale unstructured array cameras. This paper proposes a network model image feature point extraction algorithm based on symmetric auto-encoding and scale feature fusion. By using the principle of symmetric auto-encoding, the hierarchical restoration of image feature location information is incorporated into the corresponding scale feature, along with deep separable convolution image feature extraction, which not only improves the performance of feature point detection but also significantly reduces the computational complexity of the network model. Based on the calculated high-precision feature point pairing information, a new image localization method is proposed based on area ratio and homography matrix scaling, which improves the speed and accuracy of the array camera image scale alignment and positioning, realizes high-definition perception of local details in large scenes, and obtains clearer synthesis effects of large scenes and high-quality stitched images. The experimental results show that the feature point extraction algorithm proposed in this paper has been experimentally compared with four typical algorithms using the HPatches dataset. The performance of feature point detection has been improved by an average of 4.9%, the performance of homography estimation has been improved by an average of 2.5%, the amount of computation has been reduced by 18%, the number of network model parameters has been reduced by 47%, and the synthesis of billion-pixel videos has been achieved, demonstrating practicality and robustness.

Feature detection of mineral zoning in spiral slope flow under complex conditions based on improved YOLOv5 algorithm

In actual processing plants, the quality and efficiency of the traditional spiral slope flow concentrator still rely on workers to observe the changes in the mineral belt. However, in realistic complex working conditions, the formation of mineral separation zones is subject to large uncertainties, and coupled with the limited efforts, experience, and responsibility of workers, it becomes important to free up labour and improve the efficiency and profitability of the beneficiation plant. Therefore, to solve the problem of difficult detection of fuzzy small target mineral separation point features in real scenes, an improved YOLOv5-based algorithm is proposed. Firstly, the dataset quality is well improved by image enhancement and pre-processing techniques, after that an innovative CASM attention mechanism is added to the backbone of the YOLOv5 model, followed by a multi-scale feature output and prediction enhancement in the neck part of the model, and an optimized loss function is designed to optimize the whole feature learning process. The improved effect of the model and the specific detection performance were tested using real mine belt image datasets, the ablation experiment verified the comprehensive effectiveness of the proposed improved method and finally compared it with the existing high-level attention mechanism and target detection algorithms. The experimental results show that the improved YOLOv5 algorithm proposed in this study has the best overall detection performance carrying a MAP of 0.954, which is over 20% better than YOLOv5. It is worth mentioning that the improvement to achieve this performance only increases the parameter values by 0.8M and GFLOPs by 1.8, moreover, in terms of the inference speed, it also achieves a respectable 63 FPS, implying that the proposed improved method achieves a better balance between the performance enhancement and the computational complexity of the model, the overall detection results fully satisfy the industrial requirements.

Body circumference fitting model based on coupled linear regression and body type classification

Body circumference fitting model is a key technology for personalization in smart manufacturing by the apparel industry. First, the paper collected front and side images plus manual measurement data of 122 young men. Second, through contour recognition and feature point detection of the collected images, the width and thickness of each body part plus the height were estimated. Then, a correlation analysis of these known factors was conducted. The paper proposed that weight was included in initial regression model for circumferences as an additional independent variable along with width and thickness. Finally, according body type classification, a multivariate regression model was established. After validation, the values predicted by the multivariate regression model were within ±1.5 cm of the manual values for 90.9% of the individuals on average. The paper provides theoretical support for body circumference fitting and may also be beneficial to the development of the smart manufacturing of garments.

An Improved SIFT Underwater Image Stitching Method

Underwater image stitching is a technique employed to seamlessly merge images with overlapping regions, creating a coherent underwater panorama. In recent years, extensive research efforts have been devoted to advancing image stitching methodologies for both terrestrial and underwater applications. However, existing image stitching methods, which do not utilize detector information, heavily rely on matching feature pairs and tend to underperform in situations where underwater images contain regions with blurred feature textures. To address this challenge, we present an improved scale-invariant feature transform (SIFT) underwater image stitching method. This method enables the stitching of underwater images with arbitrarily acquired images featuring blurred feature contours and that do not require any detector information. Specifically, we perform a coarse feature extraction between the reference and training images, and then we acquire the target image and perform an accurate feature extraction between the reference and target images. In the final stage, we propose an improved fade-in and fade-out fusion method to obtain a panoramic underwater image. The experimental results show that our proposed method demonstrates enhanced robustness, particularly in scenarios where detecting feature points is challenging, when compared to traditional SIFT methods. Additionally, our method achieves higher matching accuracy and produces higher-quality results in the stitching of underwater images.

AA-LMM: Robust Accuracy-Aware Linear Mixture Model for Remote Sensing Image Registration

Remote sensing image registration has been widely applied in military and civilian fields, such as target recognition, visual navigation and change detection. The dynamic changes in the sensing environment and sensors bring differences to feature point detection in amount and quality, which is still a common and intractable challenge for feature-based registration approaches. With such multiple perturbations, the extracted feature points representing the same physical location in space may have different location accuracy. Most existing matching methods focus on recovering the optimal feature correspondences while they ignore the diversities of different points in position, which easily brings the model into a bad local extrema, especially when existing with the outliers and noises. In this paper, we present a novel accuracy-aware registration model for remote sensing. A soft weighting is designed for each sample to preferentially select more reliable sample points. To better estimate the transformation between input images, an optimal sparse approximation is applied to approach the transformation by multiple iterations, which effectively reduces the computation complexity and also improves the accuracy of approximation. Experimental results show that the proposed method outperforms the state-of-the-art approaches in both matching accuracy and correct matches.

A comparison of feature extraction methods in image stitching

The usage of feature detectors for image stitching has become a popular research area in computer vision. Various feature extraction algorithms can be used in the process of image stitching process, but they perform varyingly when handling different images and no single algorithm could outperform all others. This paper focuses on the comparison of feature extraction algorithms used for panoramic image stitching. The research utilizes the SIFT, ORB, AKAZE, and BRISK to conduct feature points and match feature points on a group of image sets. The RANSAC algorithm is then used to filter out the outliers and calculate the homograph matrix. Completes the panoramic with image splicing and smoothing through the matrix transformation. Derived from the comparison of the matching and stitching results, the AKAZE detector is found to be the fastest feature point detection and extraction algorithm, while the SIFT detector will provide more feature points to make more accurate matches possible. These findings have implications for the development of efficient and effective computer vision technologies for various applications.

ALGD-ORB: An improved image feature extraction algorithm with adaptive threshold and local gray difference.

Simultaneous Localization and Mapping (SLAM) technology is crucial for achieving spatial localization and autonomous navigation. Finding image features that are representative presents a key challenge in visual SLAM systems. The widely used ORB (Oriented FAST and Rotating BRIEF) algorithm achieves rapid image feature extraction. However, traditional ORB algorithms face issues such as dense, overlapping feature points, and imbalanced distribution, resulting in mismatches and redundancies. This paper introduces an image feature extraction algorithm called Adaptive Threshold and Local Gray Difference-ORB(ALGD-ORB) to address these limitations. Specifically, an adaptive threshold is employed to enhance feature point detection, and an improved quadtree method is used to homogenize feature point distribution. This method combines feature descriptors generated from both gray size and gray difference to enhance feature descriptor distinctiveness. By fusing these descriptors, their effectiveness is improved. Experimental results demonstrate that the ALGD-ORB algorithm significantly enhances the uniformity of feature point distribution compared to other algorithms, while maintaining accuracy and real-time performance.

Effective camera calibration by using phase-shifting fringe patterns

The accuracy of conventional target-based camera calibration depends on the detection of feature points. To improve the calibration accuracy in the flat field, this paper proposes an innovative calibration method based on encoded phase-shifting fringe (PSF) patterns. By using arbitrary quadrilateral interpolation, the relationship between a point on the virtual phase plane and the corresponding four original phase points is determined, establishing a mapping between the simplified original phase field and the virtual phase field. This spatial relationship can be used for accurate camera calibration. The proposed research investigates the details of the PSF based camera calibration method and demonstrates its accuracy and effectiveness through experiments. The evaluation of this method includes the checkerboard edge length variance analysis and the linear distance analysis of the checkerboard fit, both of which serve as validation metrics. Camera distortion is described by the phase distribution, and the average phase error after correction is less than 0.001 pixels (px).

Application of SLAM in the biomedical field

With the continuous development and improvement of sensors, sensor technology is also constantly making breakthroughs. It is also widely used in various fields of medicine today. This article summarizes the practice of two types of SLAM (Simultaneous Localization and Mapping) in medicine. The detection and matching of human luminal feature points under SLAM were studied so as to reconstruct the human internal environment so that doctors can obtain three-dimensional reconstruction feedback within the surgical range and improve the safety of surgery. And for different surgical environments, the same SLAM is improved on the basis of innovation, which is more suitable for different scenarios. Three algorithms are described in this article for taking images of the inside of the abdominal cavity and collecting data based on the images using different algorithms. Based on the collected data, the internal abdominal cavity is reconstructed in 3D. Finally, a three-dimensional visualization system for the abdominal cavity was constructed. It has important research significance in the field of medical auxiliary research.