Feature Matching Method Research Articles

Dynamic Soft Sensor Model for Endpoint Carbon Content and Temperature in BOF Steelmaking Based on Adaptive Feature Matching Variational Autoencoder

The key to endpoint control in basic oxygen furnace (BOF) steelmaking lies in accurately predicting the endpoint carbon content and temperature. However, BOF steelmaking data are complex and change distribution due to variations in raw material batches, process adjustments, and equipment conditions, leading to concept drift and affecting model performance. In order to resolve these problems, this paper proposes a dynamic soft sensor model based on an adaptive feature matching variational autoencoder (VAE-AFM). Firstly, this paper innovatively proposes an adaptive feature matching (AFM) method. This method utilizes the maximum mean discrepancy to calculate the values of the marginal and conditional distributions. Based on the discrepancy between these two values, a dynamic adjustment algorithm is designed to adaptively assign different weights to the two distributions. This approach dynamically and quantitatively evaluates and adjusts the relative importance of different distributions in the domain adaptation process, thereby enhancing the effectiveness of cross-domain data alignment. Secondly, a variational autoencoder (VAE) is employed to process the data, as the VAE model can capture the complex data structures and latent features in the steelmaking process. Finally, the features extracted by the VAE are processed with the adaptive feature matching method, thereby constructing the VAE-AFM dynamic soft sensor model. Experimental studies on actual BOF steelmaking data validate the efficacy of the offered approach, offering a reliable solution to the challenges of high complexity and concept drift in BOF steelmaking data.

A novel 3D reconstruction method of blast furnace burden surface based on virtual camera array

Accurate three-dimensional (3D) blast furnace burden surface topography is crucial for optimizing material distribution and furnace operation. However, harsh conditions within the furnace pose challenges to this 3D reconstruction, such as difficulties in matching feature points and sparse 3D point clouds. To overcome these challenges, we propose a 3D reconstruction method for the blast furnace burden surface using virtual camera array. Firstly, an ORB feature extraction and matching method based on bidirectional optical flow tracking is proposed to solve the impact of illumination changes and noise. Then, the virtual camera array is constructed to generate a sparse 3D point cloud from an image sequence. Finally, dense surface reconstruction of the burden surface is achieved based on the improved PMVS and the Poisson surface reconstruction method. Experimental results demonstrate that the proposed method can extract accurate feature points and reconstruct the accurate 3D topography of the burden surface.

Building Better Models: Benchmarking Feature Extraction and Matching for Structure from Motion at Construction Sites

The popularity of Structure from Motion (SfM) techniques has significantly advanced 3D reconstruction in various domains, including construction site mapping. Central to SfM, is the feature extraction and matching process, which identifies and correlates keypoints across images. Previous benchmarks have assessed traditional and learning-based methods for these tasks but have not specifically focused on construction sites, often evaluating isolated components of the SfM pipeline. This study provides a comprehensive evaluation of traditional methods (e.g., SIFT, AKAZE, ORB) and learning-based methods (e.g., D2-Net, DISK, R2D2, SuperPoint, SOSNet) within the SfM pipeline for construction site mapping. It also compares matching techniques, including SuperGlue and LightGlue, against traditional approaches such as nearest neighbor. Our findings demonstrate that deep learning-based methods such as DISK with LightGlue and SuperPoint with various matchers consistently outperform traditional methods like SIFT in both reconstruction quality and computational efficiency. Overall, the deep learning methods exhibited better adaptability to complex construction environments, leveraging modern hardware effectively, highlighting their potential for large-scale and real-time applications in construction site mapping. This benchmark aims to assist researchers in selecting the optimal combination of feature extraction and matching methods for SfM applications at construction sites.

Mix-VIO: A Visual Inertial Odometry Based on a Hybrid Tracking Strategy.

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur.

A virtual-reality spatial matching algorithm and its application on equipment maintenance support: System design and user study

Equipment maintenance support is an important technical measure to maintain the equipment’s expected performance. However, the current maintenance supports are mainly completed by maintainers under the guidance of technical manual or additional experts, which may be insufficient for some advanced equipment with rapid update rate and complex inner structure. The rising technology of augmented reality (AR) provides a new solution for equipment maintenance support, while one of the key issues limiting the practical application of AR in maintenance field is the spatial matching issue between virtual space and reality space. In this paper, a virtual-reality spatial matching algorithm is designed to accurately superimpose the virtual information to the corresponding actual scene on the AR glasses. In this algorithm, two methods are proposed to help achieve the stable matching of virtual space and reality space. In detail, to obtain the saliency map with less background interference and improved saliency detection accuracy, a saliency detection method is designed based on the super-pixel segmentation. To deal with the problems of uneven distribution on the feature points and weak robustness to the light changes, a feature extraction and matching method is proposed for acquiring the feature point matching set with the utilization of the obtained saliency map. Finally, an immersive equipment maintenance support system (IEMSS) is developed based on this spatial matching algorithm, which provides the maintainers with immediate and immersive guidance to improve the efficiency and safety in the maintenance task, as well as offers maintenance training for inexperienced maintainers with expanded virtual information in case of limited experts. Several comparative experiments are implemented to verify the effectiveness of proposed methods, and a user study of real system application is carried out to further evaluate the superiority of these methods when applied in the IEMSS.

Low-light image enhancement based on cell vibration energy model and lightness difference

Low-light image enhancement algorithms play a crucial role in revealing details obscured by darkness in images and substantially improving overall image quality. However, existing methods often suffer from issues like color or lightness distortion and possess limited scalability. In response to these challenges, we introduce a novel low-light image enhancement algorithm leveraging a cell vibration energy model and lightness difference. Initially, a new low-light image enhancement framework is proposed, building upon a comprehensive understanding and analysis of the cell vibration energy model and its statistical properties. Subsequently, to achieve pixel-level multi-lightness difference adjustment and exert control over the lightness level of each pixel independently, a lightness difference adjustment strategy is introduced utilizing Weibull distribution and linear mapping. Furthermore, to expand the adaptive range of the algorithm, we consider the disparities between HSV space and RGB space. Two enhanced image output modes are designed, accompanied by a thorough analysis and deduction of the relevant image layer mapping formulas. Finally, to enhance the reliability of experimental results, certain image faults in the SICE database are rectified using the feature matching method. Experimental results showcase the superiority of the proposed algorithm over twelve state-of-the-art algorithms. The resource code of this article will be released at https://github.com/leixiaozhou/CDEGmethod.

Improvement methods for feature matching in Basalt SLAM

Abstract Visual simultaneous localization and mapping (SLAM) is widely used to provide localization information for mobile robots, but most visual SLAM algorithms lack effective data association methods, resulting in unstable position tracking results. In this paper, relevant improvement schemes for global feature matching in the Basalt SLAM algorithm are proposed to enhance the effectiveness of its data association method. Firstly, the more efficient box average difference (BAD) descriptor is used instead of the ORB descriptor. Secondly, secondary matching is performed by narrowing the matching range with the epipolar geometric constraint to expand matching pairs. Finally, grid-based motion statistics (GMS) and histogram-based rotation consistency constraints are introduced to eliminate false matches. Experiments on the EuRoC dataset demonstrate that the proposed methods significantly enhance the effectiveness and stability of visual data association and improve the localization accuracy of the Basalt SLAM system.

A robust and accurate feature matching method for multi-modal geographic images spatial registration

ABSTRACT While the current research has achieved satisfactory results for the registration of single mode data, there has always been a significant challenge in the registration of multi-modal images due to the obvious nonlinear radiation differences caused by different imaging mechanisms and imaging time. For example, multi-temporal visible, visible-synthetic aperture radar, visible-near infrared, near infrared-short wave infrared, visible-MAP, etc. To address this problem, we propose a Robust and Accurate Feature Matching Method for Multi-modal Geographic Images Spatial Registration (RAMMR) to fully extract common key points between images, weaken the radiation difference between data, and finally accurately match more inliers to realize multi-modal image registration. Considering the influence of noise and edge information on key point extraction, RAMMR first constructs a new scale space by introducing the Side Window Filter (SWF); Then, we improve Harris algorithm to extract key points based on the SWF scale space; After that, we propose an enhanced log-polar descriptor based on the gradient angles and gradient amplitudes of the scale space, which effectively improves the quality of the descriptor and avoids the mismatch of key points; Based on the standard Euclidean distance, we design a re-match strategy to obtain the initial matching results, and Random Sample Consensus (RANSAC) is used to eliminate outliers. Finally, the affine transformation parameters are calculated based on inliers, and multi-modal image registration is realized. RAMMR is evaluated on different multi-modal datasets and compared with some state-of-art methods. The experimental results show that RAMMR accurately registers multi-modal geographic images and obtains comparative results compared with benchmark methods. Our source datasets are publicly available at https://github.com/RSmfmr/multimodal-dataset.

A feature matching and compensation method based on importance weighting for occluded human pose estimation

Human pose estimation is a hot research in the field of computer vision. Existing algorithms perform well in normal scenarios but struggle in occlusion environments. In order to address occlusion problems, we firstly design a multi-scale feature reinforcement backbone (MFRI) to obtain accurate features, which contains three sub-modules. The first sub-module reinforces detailed keypoint features and eliminates redundant background, The second sub-module assigns different weights to different scale information based on importance. The fourth sub-module integrates different scale feature information using a cascaded method. In addition, we design an occlusion handling method. To begin with, we design a multi-scale feature extractor (MSFE) to construct human topological feature, and propose a feature similarity matching method between keypoint detail features and topological features to identify occluded keypoints and eliminate obstacle features. Based on similarity, we design a feature compensation method to extract common attributes among all keypoints and unique features of each keypoint from multiple aspects to compensate for occluded features. Finally, we propose an adjacency matrix improvement method to enhance its ability for describing node relationships. We conduct comparative experiments on the COCO2017 dataset, COCO-Wholebody dataset, and CrowdPose dataset, achieving accuracy of 78.8%, 66.3%, and 77.8%, respectively. Additionally, we design a series of ablation experiments and visualizations. The results show that our method has better performance in handling occlusions.

Efficient Feature Matching for Large-scale Images based on Cascade Hash and Local Geometric Constraint

Abstract. Feature matching plays a crucial role in 3D reconstruction to provide correspondences between overlapped images. The accuracy and efficiency of feature matching significantly impact the performance of 3D reconstruction. The widely used framework with the exhaustive nearest neighbor searching (NNS) between descriptors and RANSAC-based geometric estimation is, however, low-efficient and unreliable for large-scale UAV images. Inspired by indexing-based NNS, this paper implements an efficient feature matching method for large-scale images based on Cascade Hashing and local geometric constraints. Our proposed method improves upon traditional feature matching approaches by introducing a combination of image retrieval, data scheduling, and GPU-accelerated Cascade Hashing. Besides, it utilizes a local geometric constraint to filter matching results within a matching framework. On the one hand, the GPU-accelerated Cascade Hashing technique generates compact and discriminative hash codes based on image features, facilitating the rapid completion of the initial matching process, and significantly reducing the search space and time complexity. On the other hand, after the initial matching is completed, the method employs a local geometric constraint to filter the initial matching results, enhancing the accuracy of the matching results. This forms a three-tier framework based on data scheduling, GPU-accelerated Cascade Hashing, and local geometric constraints. We conducted experiments using two sets of large-scale UAV image data, comparing our method with SIFTGPU to evaluate its performance in initial matching, outlier rejection, and 3D reconstruction. The results demonstrate that our method achieves a feature matching speed 2.0 times that of SIFTGPU while maintaining matching accuracy and producing comparable reconstruction results. This suggests that our method holds promise for efficiently addressing large-scale image matching.

Research on moving object tracking with a large number of outliers based on TRESAC++ algorithm

In contrast to the method based on motion equations, image registration proves effective for tracking moving objects in missions where establishing a motion model is unfeasible. However, in cases where the object is captured at varying imaging angles or against a similar background, a large number of outliers may arise during the process of image feature extraction. Consequently, when confronted with a large number of outliers, image registration methods exhibit a drawback in accurate outlier filtering, especially when spatial constraints on feature matching pairs are constrained. Ultimately, these factors can render object tracking inaccurate or even unattainable. In light of the aforementioned circumstances, this paper proposes TRESAC++, which based on a triplet relation feature matching method to effectively filter outliers. To enhance the operational efficiency of the algorithm, Principal Component Analysis is employed for dimensionality reduction, aiming to decrease the incidence of triple matches and subsequently mitigate execution time. To fortify the algorithm's robustness, a triplet relation linked with an initial data subset selection strategy is additionally proposed. Experimental results on five challenging experiments show that TRESAC++ rivals current state-of-the-art techniques in terms of computational cost, while the implementation exhibits better outlier filtering. Additionally, Experiment 4 and Experiment 5 use moving object datasets to confirm TRESAC++'s superior capability in precise object tracking.

CattNIS: Novel identification system of cattle with retinal images based on feature matching method

Small and large farm animals are frequently identified using techniques like microchipping or tagging. However, these techniques may have a detrimental impact on animal welfare and may result in theft and fraud. Modern and sophisticated identification procedures using biometrics, including retina, face, and iris recognition, are available and have less of a detrimental effect on animal welfare. Because biometric markers are unchangeable and unaffected, it is feasible to stop situations of theft and fraud. This study uses retina images to identify animals based on each animal's different and specific retinal patterns. To the best of our knowledge, there isn't a publicly accessible dataset for images of animal retinas. The three main goals of this study are to: 1) produce a novel dataset of Cattle retinal images and make it publicly available; 2) create a successful image processing-based system for retinal identification and recognition; and 3) create a graphical user interface (GUI) that enables querying the database to ascertain whether a specific Cattle retina is present. To accomplish these goals, a dataset comprising 2430 images from the left and right eyes of 300 cattle was produced. After the dataset was created, the animal retinal images were segmented using image processing methods such as scaling, color transformations, image sharpening, contrast enhancements, noise filtering, and histogram equalization. By using the segmented images with the SURF, FAST, BRISK, and HARRIS techniques, distinctive retinal features were found. During the identification phase, the animal whose retinal characteristic matched the sought animal's retina the closest was recognized. Within the developed CattNIS system, the SURF approach has the highest accuracy rate of 92.25 percent. The results of this study show that the identification of cattle using retinal vascular patterns is highly successful. The availability of the collected cattle retinal images from this study is expected to significantly contribute to the advancement of future research in this field, both in terms of quantity and quality.

Research on Full-Field Dynamic Deflection Measurement of Beams Based on Dense Feature Matching and Mismatch Removal Method

To solve the problems of measurement errors led by mismatches of dense feature matching in machine vision structural deflection measurement, this paper proposes a dense feature extraction, matching, and dual-step mismatch-removal-based full-field structural dynamic deflection measurement method. First, the of dense feature detection and matching theory is introduced to extract the SIFT feature points on a structural surface in an image sequence and matched by FLANN to trace the structure movement, and the mechanisms and causes of mismatches are analyzed. Then, a dual-step mismatch removal method combining RANSAC and Structural Displacement Continuity Restriction (SDCR) is introduced to achieve full-field dynamic beam deflection measurement. The proposed method is validated through indoor cantilever beam experiments, and results show that the method can effectively eliminate a large number of SIFT feature mismatches (accounting for approximately 55% of the total matched feature points). The full-field dynamic displacement field of the beam can be measured with the correctly matched dense feature points by converting dense feature point displacements into continuous and uniform spatiotemporal deflections of the structure. A comparison with the GOM Correlate Professional DIC measurement system was conducted, and the maximum measurement error of the cantilever beam dynamic displacement of the proposed method is between 0.024 and 0.053 mm, the root mean squared error of displacement is approximately 0.01 mm, and the correlation coefficient between two deflection–time curves reaches 0.9964. The proposed algorithm is proven to be effective in full-field displacement measurement and has great potential in future structural health monitoring of bridges.

Dual-Perspective Knowledge Enrichment for Semi-supervised 3D Object Detection

Semi-supervised 3D object detection is a promising yet under-explored direction to reduce data annotation costs, especially for cluttered indoor scenes. A few prior works, such as SESS and 3DIoUMatch, attempt to solve this task by utilizing a teacher model to generate pseudo-labels for unlabeled samples. However, the availability of unlabeled samples in the 3D domain is relatively limited compared to its 2D counterpart due to the greater effort required to collect 3D data. Moreover, the loose consistency regularization in SESS and restricted pseudo-label selection strategy in 3DIoUMatch lead to either low-quality supervision or a limited amount of pseudo labels. To address these issues, we present a novel Dual-Perspective Knowledge Enrichment approach named DPKE for semi-supervised 3D object detection. Our DPKE enriches the knowledge of limited training data, particularly unlabeled data, from two perspectives: data-perspective and feature-perspective. Specifically, from the data-perspective, we propose a class-probabilistic data augmentation method that augments the input data with additional instances based on the varying distribution of class probabilities. Our DPKE achieves feature-perspective knowledge enrichment by designing a geometry-aware feature matching method that regularizes feature-level similarity between object proposals from the student and teacher models. Extensive experiments on the two benchmark datasets demonstrate that our DPKE achieves superior performance over existing state-of-the-art approaches under various label ratio conditions. The source code and models will be made available to the public.

Local feature matching using deep learning: A survey

Local feature matching enjoys wide-ranging applications in the realm of computer vision, encompassing domains such as image retrieval, 3D reconstruction, and object recognition. However, challenges persist in improving the accuracy and robustness of matching due to factors like viewpoint and lighting variations. In recent years, the introduction of deep learning models has sparked widespread exploration into local feature matching techniques. The objective of this endeavor is to furnish a comprehensive overview of local feature matching methods. These methods are categorized into two key segments based on the presence of detectors. The Detector-based category encompasses models inclusive of Detect-then-Describe, Joint Detection and Description, Describe-then-Detect, as well as Graph Based techniques. In contrast, the Detector-free category comprises CNN Based, Transformer Based, and Patch Based methods. Our study extends beyond methodological analysis, incorporating evaluations of prevalent datasets and metrics to facilitate a quantitative comparison of state-of-the-art techniques. The paper also explores the practical application of local feature matching in diverse domains such as Structure from Motion, Remote Sensing Image Registration, and Medical Image Registration, underscoring its versatility and significance across various fields. Ultimately, we endeavor to outline the current challenges faced in this domain and furnish future research directions, thereby serving as a reference for researchers involved in local feature matching and its interconnected domains. A comprehensive list of studies in this survey is available at https://github.com/vignywang/Awesome-Local-Feature-Matching.

A survey of feature matching methods

AbstractFeature matching plays a crucial role in computer vision, with applications in visual localization, simultaneous localization and mapping (SLAM), image stitching, and more. It establishes correspondences between sets of feature points from multiple images, enabling various tasks. Over the years, feature matching has witnessed significant development, with an increasing number of methods being applied. However, different methods exhibit different degrees of applicability in different scenarios and requirements due to their different rationales. To cope with these issues, a comprehensive analysis and comparison of matching methods are essential. Existing reviews often lack coverage of deep learning models and focus more on feature detection and description, neglecting the matching process. This survey investigates feature detection, description, and matching techniques within the feature‐based image‐matching pipeline. Representative methods, their mechanisms, and application scenarios are also briefly introduced. In addition, comprehensive evaluations of classical and state‐of‐the‐art methods are conducted through extensive experiments on representative datasets. Particularly, matching‐based applications are compared to fully demonstrate the advantages of the methods. Lastly, this survey highlights current problems and development directions in matching methods, serving as a reference for researchers in the field.

GPU Accelerated Processing Method for Feature Point Extraction and Matching in Satellite SAR Images

This paper addresses the challenge of extracting feature points and image matching in Synthetic Aperture Radar (SAR) satellite images, particularly focusing on large-scale embedding. The widely used Scale Invariant Transform (SIFT) algorithm, successful in computer vision and optical satellite image matching, faces challenges when applied to satellite SAR images due to the presence of speckle noise, leading to increased matching errors. The SAR–SIFT method is explored and analyzed in-depth, considering the unique characteristics of satellite SAR images. To enhance the efficiency of matching identical feature points in two satellite SAR images, the paper proposes a Graphics Processing Unit (GPU) mapping implementation based on the SAR–SIFT algorithm. The paper introduces a multi-GPU collaborative acceleration strategy for SAR image matching. This strategy addresses the challenge of matching feature points in the region and embedding multiple SAR images in large areas. The goal is to achieve efficient matching processing of multiple SAR images in extensive geographical regions. The proposed multi-GPU collaborative acceleration algorithm is validated through experiments involving feature point extraction and matching using 21 GF-3 SAR images. The results demonstrate the feasibility and efficiency of the algorithm in enhancing the processing speed of matching feature points in large-scale satellite SAR images. Overall, the paper contributes to the advancement of SAR image processing techniques, specifically in feature point extraction and matching in large-scale applications.

Source depth estimation with feature matching using convolutional neural networks in shallow water

A feature matching method based on the convolutional neural network (named FM-CNN), inspired from matched-field processing (MFP), is proposed to estimate source depth in shallow water. The FM-CNN, trained on the acoustic field replicas of a single source generated by an acoustic propagation model in a range-independent environment, is used to estimate single and multiple source depths in range-independent and mildly range-dependent environments. The performance of the FM-CNN is compared to the conventional MFP method. Sensitivity analysis for the two methods is performed to study the impact of different environmental mismatches (i.e., bottom parameters, water column sound speed profile, and topography) on depth estimation performance in the East China Sea environment. Simulation results demonstrate that the FM-CNN is more robust to the environmental mismatch in both single and multiple source depth estimation than the conventional MFP. The proposed FM-CNN is validated by real data collected from four tracks in the East China Sea experiment. Experimental results demonstrate that the FM-CNN is capable of reliably estimating single and multiple source depths in complex environments, while MFP has a large failure probability due to the presence of strong sidelobes and wide mainlobes.

Local geometric edge features based registration for textureless object in augmented reality assisted assembly

Image-based methods have been widely used in augmented reality (AR) assistant assembly systems. However, due to the lack of sufficient texture information on the surface of assembly part, traditional image feature matching methods still face challenges. This paper proposes a coarse-to-fine AR registration method for textureless assembly part. In the first stage, a new feature matching method which is called line neighborhood edge descriptor (LNED) is presented to find the coarse camera pose from textureless image. The LNED take the contour line of assembly part as the description object, and use local geometric edge of assembly part to describe the contour line. During the image matching, the binary encoding is used to reduce the computational consumption for LNED. In the second stage, spatial points in the CAD model of assembly part are reverse projected to the textureless image based on the coarse camera pose. And the bundle adjustment method based on the edge distance of the textureless image is adopted to iteratively calculate the precise camera pose. In the experimental evaluation, the proposed registration method shows high accuracy and fast speed in comparison with conventional registration methods, which demonstrates that our method can effectively solve the problem of AR registration for textureless assembly part.

Local feature matching and tracking optimization of machine VSLAM based on AG Sinkhorn

SummaryAt present, the visual Simultaneous localization and mapping method has a wide range of applications in mobile robots and unmanned driving. However, when the environment changes, the accuracy of the system drops sharply. Aiming at the low matching accuracy of local feature matching and tracking optimization in current visual simultaneous localization and mapping methods, the attention map neural network and Sinkhorn algorithm are used to extract and match the image features. Then, it improves the real‐time performance of the visual simultaneous localization and mapping system by local optimization of feature point matching, and proposes an optimization method for visual simultaneous localization and mapping in view of an end‐to‐end algorithm. The experiment illustrates that in the experimental comparison of HPatches View dataset, the Homography estimation was improved by 0.092, the matching accuracy was improved by 0.087, and the matching recall was improved by 0.038. In the experimental comparison of HPatches Illum dataset, the Homography estimation was improved by 0.015, the matching accuracy is improved by 0.076, and the matching recall was improved by 0.036. The local feature matching and tracking optimization method of the visual Simultaneous localization and mapping system through the attention map neural network and Sinkhorn algorithm can enhance the matching accuracy of local feature matching and tracking optimization in dynamic environment and under changing lighting conditions.