Feature Matching Process Research Articles

Comprehensive empirical evaluation of feature extractors in computer vision

Feature detection and matching are fundamental components in computer vision, underpinning a broad spectrum of applications. This study offers a comprehensive evaluation of traditional feature detections and descriptors, analyzing methods such as Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Binary Robust Independent Elementary Features (BRIEF), Oriented FAST and Rotated BRIEF (ORB), Binary Robust Invariant Scalable Keypoints (BRISK), KAZE, Accelerated KAZE (AKAZE), Fast Retina Keypoint (FREAK), Dense and Accurate Invariant Scalable descriptor for Yale (DAISY), Features from Accelerated Segment Test (FAST), and STAR. Each feature extractor was assessed based on its architectural design and complexity, focusing on how these factors influence computational efficiency and robustness under various transformations. Utilizing the Image Matching Challenge Photo Tourism 2020 dataset, which includes over 1.5 million images, the study identifies the FAST algorithm as the most efficient detector when paired with the ORB descriptor and Brute-Force (BF) matcher, offering the fastest feature extraction and matching process. ORB is notably effective on affine-transformed and brightened images, while AKAZE excels in conditions involving blurring, fisheye distortion, image rotation, and perspective distortions. Through more than 2 million comparisons, the study highlights the feature extractors that demonstrate superior resilience across various conditions, including rotation, scaling, blurring, brightening, affine transformations, perspective distortions, fisheye distortion, and salt-and-pepper noise.

Metasurface-enabled 3D imaging via local bright spot gray scale matching using the structured light dot array.

Three-dimensional (3D) imaging is widely utilized in various applications, such as light detection, autonomous vehicles, and machine vision. However, conventional 3D imaging systems often rely on bulky optical components. Metasurfaces, as next-generation optical devices, possess flexible wavefront modulation capabilities and excellent combination with computer vision algorithms. Here, we propose a large field-of-view (FOV) structured light dot array projection device based on a metasurface, covering a 2π-FOV, for projecting coded point clouds in Fourier space. We explore a local bright spot gray scale matching algorithm for depth extraction, enabling 3D imaging. This algorithm simplifies the data processing flow and optimizes depth extraction and feature matching processes through a customized region gray scale comparison. As a result, it effectively reduces computational complexity and enhances tolerance to image quality fluctuations. The proposed approach provides new possibilities for developing compact and high-performance planar 3D optical imaging devices, which will drive the advancement of fields such as computer vision and artificial intelligence.

D3L-SLAM: A Comprehensive Hybrid Simultaneous Location and Mapping System with Deep Keypoint, Deep Depth, Deep Pose, and Line Detection

Robust localization and mapping are crucial for autonomous systems, but traditional handcrafted feature-based visual SLAM often struggles in challenging, textureless environments. Additionally, monocular SLAM lacks scale-aware depth perception, making accurate scene scale estimation difficult. To address these issues, we propose D3L-SLAM, a novel monocular SLAM system that integrates deep keypoints, deep depth estimates, deep pose priors, and a line detector. By leveraging deep keypoints, which are more resilient to lighting variations, our system improves the robustness of visual SLAM. We further enhance perception in low-texture areas by incorporating line features in the front-end and mitigate scale degradation with learned depth estimates. Additionally, point-line feature constraints optimize pose estimation and mapping through a tightly coupled point-line bundle adjustment (BA). The learned pose estimates refine the feature matching process during tracking, leading to more accurate localization and mapping. Experimental results on public and self-collected datasets show that D3L-SLAM significantly outperforms both traditional and learning-based visual SLAM methods in localization accuracy.

Building Better Models: Benchmarking Feature Extraction and Matching for Structure from Motion at Construction Sites

The popularity of Structure from Motion (SfM) techniques has significantly advanced 3D reconstruction in various domains, including construction site mapping. Central to SfM, is the feature extraction and matching process, which identifies and correlates keypoints across images. Previous benchmarks have assessed traditional and learning-based methods for these tasks but have not specifically focused on construction sites, often evaluating isolated components of the SfM pipeline. This study provides a comprehensive evaluation of traditional methods (e.g., SIFT, AKAZE, ORB) and learning-based methods (e.g., D2-Net, DISK, R2D2, SuperPoint, SOSNet) within the SfM pipeline for construction site mapping. It also compares matching techniques, including SuperGlue and LightGlue, against traditional approaches such as nearest neighbor. Our findings demonstrate that deep learning-based methods such as DISK with LightGlue and SuperPoint with various matchers consistently outperform traditional methods like SIFT in both reconstruction quality and computational efficiency. Overall, the deep learning methods exhibited better adaptability to complex construction environments, leveraging modern hardware effectively, highlighting their potential for large-scale and real-time applications in construction site mapping. This benchmark aims to assist researchers in selecting the optimal combination of feature extraction and matching methods for SfM applications at construction sites.

Copy-move forgery detection using Regional Density Center clustering

Copy-move forgery detection is a common image tampering detection technology. In this paper, a novel copy-move forgery detection scheme is proposed. The proposed scheme is based on Regional Density Center (RDC) clustering and Refined Length Homogeneity Filtering (RLHF) policy. First, to obtain an adequate number of keypoints in smooth or small areas of the image, the proposed scheme employs scale normalization and adjustment of the contrast threshold of the input image. Subsequently, to speed up the feature matching process, a matching algorithm based on gray value grouping is used to match the keypoints. RLHF policy is applied to filter the mismatched pairs. To guarantee a good estimation of the affine transformation, the RDC clustering algorithm is proposed to group the matched pairs. Finally, the correlation coefficients are computed to precisely locate the tampered regions. The proposed copy-move forgery detection scheme based on RDC and RLHF can effectively identify duplicated regions of digital images. It demonstrates the effectiveness and robustness of the proposed scheme over many state-of-the-art schemes on public datasets.

Loop closure detection of visual SLAM based on variational autoencoder.

Loop closure detection is an important module for simultaneous localization and mapping (SLAM). Correct detection of loops can reduce the cumulative drift in positioning. Because traditional detection methods rely on handicraft features, false positive detections can occur when the environment changes, resulting in incorrect estimates and an inability to obtain accurate maps. In this research paper, a loop closure detection method based on a variational autoencoder (VAE) is proposed. It is intended to be used as a feature extractor to extract image features through neural networks to replace the handicraft features used in traditional methods. This method extracts a low-dimensional vector as the representation of the image. At the same time, the attention mechanism is added to the network and constraints are added to improve the loss function for better image representation. In the back-end feature matching process, geometric checking is used to filter out the wrong matching for the false positive problem. Finally, through numerical experiments, the proposed method is demonstrated to have a better precision-recall curve than the traditional method of the bag-of-words model and other deep learning methods and is highly robust to environmental changes. In addition, experiments on datasets from three different scenarios also demonstrate that the method can be applied in real-world scenarios and that it has a good performance.

A high-precision image registration method for multi-channel single molecule localization

Single-molecule localization technology has been widely used in single-particle tracking and super-resolution imaging of biological samples, as it can bypass the diffraction limit of optical systems. Multi-channel single-molecule localization uses multiple imaging channels to simultaneously track different targets or perform multi-color super-resolution imaging, and can also improve the axial depth of single-particle tracking or achieve higher localization precision and density for super-resolution imaging. However, the difference between images in each channel can affect collaborative localization or quantitative analysis, so image registration is a key step in its image data preprocessing. Moreover, due to the high precision of single-molecule localization, its requirements for multi-channel image registration accuracy are also high. Existing technologies generally use control point-based registration methods and often use complicated and precise methods to obtain fiducial images for locating control point pairs to achieve high-precision image registration, which involves high sample or experimental equipment requirements and is difficult to directly extend to other systems. Therefore, developed in this work, is a high-precision image registration method that can directly use randomly distributed fluorescent beads as fiducial samples based on local nonlinear transformation and elimination of mismatched points. By monitoring and iteratively filtering control points in the process of feature matching and transformation model parameter estimation to eliminate control point pairs that are not accurately matched due to inaccurate or poor precision of single-molecule localization, the adverse effects on accurate acquisition and precise matching of control points when using randomly distributed fluorescent beads as fiducial samples are eliminated. At the same time, a second-order polynomial fitting based on local weighted mean is used for estimating the transformation model parameter to better adapt to the existence of local nonlinear deformation between different channels. The results show that using this method only requires three iterations to find and eliminate control point pairs that are not accurately located and matched, thereby achieving more accurate transformation model parameter and improving the registration accuracy by an order of magnitude, achieving a registration accuracy of about 6 nm in a complex dual-channel single-molecule localization imaging system based on orthogonal astigmatism.

MATCHING FILTER-BASED VSLAM OPTIMIZATION IN INDOOR ENVIRONMENTS

Abstract. An important factor that reduced the accuracy of motion trajectories in existing VSLAM (Visual Simultaneous Localization and Mapping) systems is the poor estimation of the position pose of the vision odometer. The existing methods generate many incorrect matches during the feature matching process, resulting in low computational accuracy of rotations and translations between cameras, which further leads to a reduction in the robustness of the overall system. In addition, the sparse feature point maps do not provide a detailed description of the surrounding environment, which makes it difficult for the devices equipped with VSLAM systems to perform advanced tasks such as navigation, path planning and human-computer interaction. To address the accuracy problem, we select the set of matches from existing feature matching algorithms based on the motion consistency constraint and use a random sampling consistency algorithm to obtain the best quality matches from the selected samples for computing the geometric transformation model and estimating the current pose. To address the problem of sparse map points, we use the depth information from the RGB-D or Stereo camera to build a dense map module to ensure that information about the surrounding environment is recorded as a point cloud, which provides data support for the implementation of advanced tasks of the device.

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

For Synthetic Aperture Radar (SAR) image registration, successive processes following feature extraction are required by both the traditional feature-based method and the deep learning method. Among these processes, the feature matching process—whose time and space complexity are related to the number of feature points extracted from sensed and reference images, as well as the dimension of feature descriptors—proves to be particularly time consuming. Additionally, the successive processes introduce data sharing and memory occupancy issues, requiring an elaborate design to prevent memory leaks. To address these challenges, this paper introduces the OptionEM-based reinforcement learning framework to achieve end-to-end SAR image registration. This framework outputs registered images directly without requiring feature matching and the calculation of the transformation matrix, leading to significant processing time savings. The Transformer architecture is employed to learn image features, while a correlation network is introduced to learn the correlation and transformation matrix between image pairs. Reinforcement learning, as a decision process, can dynamically correct errors, making it more-efficient and -robust compared to supervised learning mechanisms such as deep learning. We present a hierarchical reinforcement learning framework combined with Episodic Memory to mitigate the inherent problem of invalid exploration in generalized reinforcement learning algorithms. This approach effectively combines coarse and fine registration, further enhancing training efficiency. Experiments conducted on three sets of SAR images, acquired by TerraSAR-X and Sentinel-1A, demonstrated that the proposed method’s average runtime is sub-second, achieving subpixel registration accuracy.

Fast multi-feature tracking method based on tightly coupled sensors

In contrast to conventional methods of optimizing feature-matching results at the data-processing level, we utilize the tightly coupled mode to fuse the information collected by the camera and inertial sensor at the data-gathering level, which contributed to better accuracy and robustness in the feature-matching process. Specifically, the proposed visual-inertial feature tracking method that combines inertial measurement unit (IMU) calibration, feature matching, and prediction algorithms. The approach includes a vision-aided multi-level IMU systemic calibration method and an inertial-aided image feature prediction algorithm, which effectively processes and utilizes information from multiple sensors. Our method addresses not only the influence of image distortion and blur resulting from illumination changes and fast camera motion but also the problem of measurement errors that can arise from the long-term operation of the inertial sensors. Extensive experiments have been conducted to demonstrate that the efficiency of it is superior to the state-of-the-art methods: With the accuracy rate remains the same level, the speed of feature matching is improved by 41.8%. Additionally, when it is applied to simultaneous localization and mapping systems, its localization performances are better than that of the VINS-mono method by 8.1%.

POEM: A prototype cross and emphasis network for few-shot semantic segmentation

To tackle the demand for large labeled datasets and poor unseen class generalization for segmentation tasks, few-shot segmentation (FSS) has received widespread attention, which utilizes prototypical learning to achieve significant progress. However, the prototypes generated by averaging the global object information still suffer from the appearance diversity within the target class. Motivated by the fact that humans only need part attributes rather than all attributions to recognize objects from a particular class, we propose an effective prototype cross and emphasis segmentation network (POEM) to provide more precise guidance in the feature matching process. Firstly, we present a regional crossed referencing module to improve the prototypical learning strategy, which mines the relevant attributes of the target objects in both support and query images and customizes the prototypes for the query images. Secondly, we develop a foreground-emphasized reasoning module to reduce the negative influence caused by complex background clutter, in which the foreground and background attributions are regarded as fundamental information and auxiliary information, respectively. Extensive experiments on two standard datasets have demonstrated the superiority of POEM over the state-of-the-art methods.

Deteksi Kartu Tanda Mahasiswa UIN Pada Gambar dengan Feature Matching

UIN Walisongo's student identity card (KTM) does not have much function other than just for student identification. Even if the function is increased, it can be used for absenteeism at lectures, borrowing books, or double as an automated teller card (ATM). Doing absences using KTM requires a feature matching method for matching the intended KTM image with the KTM that is searched for in the student database. The feature matching process is based on feature detection in images using various methods such as ORB and Scale Invariant Feature Transform (SIFT). We can perform the feature matching method using the Brute-Force method and the Fast Library Approximated Nearest Neighbor (FLANN) on Google Colab with Python. The results of feature matching on the FLANN method are more than the Brute-Force method. The validation of the two image feature matching was carried out using the Root Mean Square Error (RMSE) method, resulting in an average value of 10.424. The purpose of this article is to detect student ID cards with matching features in the image. The FLANN and Brute-Force feature matching methods can be used to detect KTM UIN in images.
 Keywords: feature matching, SIFT, FLANN, Brute-Force, RMSE

Real-Time Stereo Visual Odometry Based on an Improved KLT Method

Real-time stereo visual odometry (SVO) localization is a challenging problem, especially for a mobile platform without parallel computing capability. A possible solution is to reduce the computational complexity of SVO using a Kanade–Lucas–Tomasi (KLT) feature tracker. However, the standard KLT is susceptible to scale distortion and affine transformation. Therefore, this work presents a novel SVO algorithm yielding robust and real-time localization based on an improved KLT method. First, in order to improve real-time performance, feature inheritance is applied to avoid time-consuming feature detection and matching processes as much as possible. Furthermore, a joint adaptive function with respect to the average disparity, translation velocity, and yaw angle is proposed to determine a suitable window size for the adaptive KLT tracker. Then, combining the standard KLT method with an epipolar constraint, a simplified KLT matcher is introduced to substitute feature-based stereo matching. Additionally, an effective veer chain matching scheme is employed to reduce the drift error. Comparative experiments on the KITTI odometry benchmark show that the proposed method achieves significant improvement in terms of time performance than the state-of-the-art single-thread approaches and strikes a better trade-off between efficiency and accuracy than the parallel SVO or multi-threaded SLAM.

A Generic Image Processing Pipeline for Enhancing Accuracy and Robustness of Visual Odometry.

The accuracy of pose estimation from feature-based Visual Odometry (VO) algorithms is affected by several factors such as lighting conditions and outliers in the matched features. In this paper, a generic image processing pipeline is proposed to enhance the accuracy and robustness of feature-based VO algorithms. The pipeline consists of three stages, each addressing a problem that affects the performance of VO algorithms. The first stage tackles the lighting condition problem, where a filter called Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied to the images to overcome changes in lighting in the environment. The second stage uses the Suppression via Square Covering (SSC) algorithm to ensure the features are distributed properly over the images. The last stage proposes a novel outliers rejection approach called the Angle-based Outlier Rejection (AOR) algorithm to remove the outliers generated in the feature matching process. The proposed pipeline is generic and modular and can be integrated with any type of feature-based VO (monocular, RGB-D, or stereo). The efficiency of the proposed pipeline is validated using sequences from KITTI (for stereo VO) and TUM (for RGB-D VO) datasets, as well as experimental sequences using an omnidirectional mobile robot (for monocular VO). The obtained results showed the performance gained by enhancing the accuracy and robustness of the VO algorithms without compromising on the computational cost using the proposed pipeline. The results are substantially better as opposed to not using the pipeline.

Robust and Efficient Velocity Estimation for MAVs With an RGB-D Camera

Micro Aerial Vehicles (MAVs) have become increasingly popular due to their wide applications in many areas. For autonomous navigation and safe control of MAVs, it is essential to have accurate and reliable velocity and position estimation. However, due to limited computational power and payload, it is still challenging for autonomous operation of MAVs in complex environments. In this paper, we propose a robust and efficient velocity estimation framework for MAVs with a single downward-facing RGB-D camera, which is able to provide metric velocity estimation in three dimensions as well as yaw rate in real-time without the fusion of additional sensors. Unlike traditional algorithms, our method uses kernel cross-correlators (KCCs) to efficiently determine optical flow for motion estimation, which does not rely on costly feature extraction and matching process. Moreover, we utilize depth images to estimate the vertical velocity of MAVs without the assumption of a flat ground. Autonomous flight tests on a quadrotor in complex environments demonstrate the robustness and efficiency of our method.

Obstacle Detection for Unmanned Aerial Vehicle (UAV)

This study aims to develop an obstacle detection system for unmanned aerial vehicles utilisingthe ORB feature extraction. In the past, small unmanned aerial vehicles(UAV)were typically equipped with vision-based or range-based sensors. Each sensor in the sensor-based technique possesses different advantages and disadvantages. As a result, the small unmanned aerial vehicle is unable to determine the obstacle's distance or bearing precisely. Due to physical size restrictions and payload capacity, the lightweight Pi Camera and TF Luna LiDAR sensor were selected as the most suitable sensors for integration. In algorithmdevelopmentandfiltration isused to improve the accuracy of the feature matching process, which is required for classifying the obstacle region and free region of any texture obstacle. The experimentwas under the environment of OpenCV and Spyder. In real-time experiment, the success rate for good texture (40%), poor texture (55%) and texture-less (45%) The findings indicate that the recommended method works well for detectingtextures-less obstacle even though the success rate is only 40% becauseout of 10 test only one test is fail on detecting free region. The sensor calibration and constructing convex hullfor obstacle detectionis recommended in future works to improve the efficiency of the obstacle detection system and classified the free region and obstacle region to create safe avoidance path.

OG-SLAM: A real-time and high-accurate monocular visual SLAM framework

The challenge of improving the accuracy of monocular Simultaneous Localization and Mapping (SLAM) is considered, which widely appears in computer vision, autonomous robotics, and remote sensing. A new framework (ORB-GMS-SLAM (or OG-SLAM)) is proposed, which introduces the region-based motion smoothness into a typical Visual SLAM (V-SLAM) system. The region-based motion smoothness is implemented by integrating the Oriented Fast and Rotated Brief (ORB) features and the Grid-based Motion Statistics (GMS) algorithm into the feature matching process. The OG-SLAM significantly reduces the absolute trajectory error (ATE) on the key-frame trajectory estimation without compromising the real-time performance. This study compares the proposed OG-SLAM to an advanced V-SLAM system (ORB-SLAM2). The results indicate the highest accuracy improvement of almost 75% on a typical RGB-D SLAM benchmark. Compared with other ORB-SLAM2 settings (1800 key points), the OG-SLAM improves the accuracy by around 20% without losing performance in real-time. The OG-SLAM framework has a significant advantage over the ORB-SLAM2 system in that it is more robust for rotation, loop-free, and long ground-truth length scenarios. Furthermore, as far as the authors are aware, this framework is the first attempt to integrate the GMS algorithm into the V-SLAM.

RTSDM: A Real-Time Semantic Dense Mapping System for UAVs

Intelligent drones or flying robots play a significant role in serving our society in applications such as rescue, inspection, agriculture, etc. Understanding the scene of the surroundings is an essential capability for further autonomous tasks. Intuitively, knowing the self-location of the UAV and creating a semantic 3D map is significant for fully autonomous tasks. However, integrating simultaneous localization, 3D reconstruction, and semantic segmentation together is a huge challenge for power-limited systems such as UAVs. To address this, we propose a real-time semantic mapping system that can help a power-limited UAV system to understand its location and surroundings. The proposed approach includes a modified visual SLAM with the direct method to accelerate the computationally intensive feature matching process and a real-time semantic segmentation module at the back end. The semantic module runs a lightweight network, BiSeNetV2, and performs segmentation only at key frames from the front-end SLAM task. Considering fast navigation and the on-board memory resources, we provide a real-time dense-map-building module to generate an OctoMap with the segmented semantic map. The proposed system is verified in real-time experiments on a UAV platform with a Jetson TX2 as the computation unit. A frame rate of around 12 Hz, with a semantic segmentation accuracy of around 89% demonstrates that our proposed system is computationally efficient while providing sufficient information for fully autonomous tasks such as rescue, inspection, etc.

Vision-based aerial image mosaicking algorithm with object detection

Aerial image sequence mosaicking is one of the challenging research fields in computer vision. To obtain large-scale orthophoto maps with object detection information, we propose a vision-based image mosaicking algorithm without any extra location data. According to object detection results, we define a complexity factor to describe the importance of each input Image and dynamically optimize the feature extraction process. The feature points extraction and matching processes are mainly guided by the speeded-up robust features (SURF) and the grid motion statistic (GMS) algorithm respectively. A robust reference frame selection method is proposed to eliminate the transformation distortion by searching for the center area based on overlaps. Besides, the sparse Levenberg-Marquardt (LM) algorithm and the heavy occluded frames removal method are Applied to reduce accumulated errors and further improve the mosaicking performance. The proposed algorithm is performed by using multithreading and graphics processing unit (GPU) acceleration on several aerial image datasets. Extensive experiment results demonstrate that our algorithm outperforms most of the existing aerial image mosaicking methods in visual quality while guaranteeing a high calculation speed.

Trilateral Filterative Hermitian feature transformed deep perceptive fuzzy neural network for finger vein verification

Automated person recognition is an essential field to preserve individual privacy in biometrics. A conventional system performs the recognition, but inaccurate detection of finger-vein lines causes lesser verification accuracy. To address this problem, a novel Trilateral Filterative Hermitian Feature Transformation based Deep Perceptive Fuzzy Neural Network (TFHFT-DPFNN) model is introduced for the capability of learning biometric features that offers robust and accurate vein image verification with minimum time. The proposed TFHFT-DPFNN model performs four different processing stages in the network layers. In the first stage, multiple finger vein images are captured from the database. Then the input images are taken to perform the denoising in the second stage by applying a guided trilateral filter to smooth the input image for accurate matching with minimum time consumption. Followed by, Hermitian Hat wavelet transformation is applied to perform decomposition and feature extraction from the input image. Then, the extracted features are transferred to the next hidden layer where the feature matching process is carried by applying the Jaccard similarity index. Based on the similarity value, the finger vein verification is performed through the fuzzy triangular membership function at the output layer. Finally, the verified results are obtained at the output layer with minimum error. Experimental evaluation is carried out with training images and testing images on different factors such as peak signal-to-noise ratio, finger vein verification accuracy, false-positive rate, and computation time with the number of finger vein images. The proposed TFHFT-DPFNN model offers promising outcomes in terms of higher verification accuracy and lesser false-positive rate with minimum time consumption than the conventional methods.