Feature Matching Research Articles

TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching.

This study tackles image matching in difficult scenarios, such as scenes with significant variations or limited texture, with a strong emphasis on computational efficiency. Previous studies have attempted to address this challenge by encoding global scene contexts using Transformers. However, these approaches have high computational costs and may not capture sufficient high-level contextual information, such as spatial structures or semantic shapes. To overcome these limitations, we propose a novel image-matching method that leverages a topic-modeling strategy to capture high-level contexts in images. Our method represents each image as a multinomial distribution over topics, where each topic represents semantic structures. By incorporating these topics, we can effectively capture comprehensive context information and obtain discriminative and high-quality features. Notably, our coarse-level matching network enhances efficiency by employing attention layers only to fixed-sized topics and small-sized features. Finally, we design a dynamic feature refinement network for precise results at a finer matching stage. Through extensive experiments, we have demonstrated the superiority of our method in challenging scenarios. Specifically, our method ranks in the top 9% in the Image Matching Challenge 2023 without using ensemble techniques. Additionally, we achieve an approximately 50% reduction in computational costs compared to other Transformer-based methods. Code is available at https://github.com/TruongKhang/TopicFM.

Lightweight image denoising method for feature-based visual SLAM in γ radiation environments

In the γ radiation environments, high-energy photons impact image sensor pixels and induce speckle noise in the scene image, which influences the accuracy of feature detection and matching in the visual Simultaneous Localization and Mapping (SLAM) algorithms. To mitigate these effects, this paper presents a lightweight image denoising method based on super-neighborhood down-sampling. Firstly, the super-neighborhood down-sampling strategy is used to split the speckle noise in the camera images into isolated point noises in the sub-image sets. Secondly, these isolated flickering point noises are removed by the detection-based median filter. Finally, the denoised scene image is reconstructed by the super-neighborhood up-sampling. Extensive experiments were performed on the radiation environment datasets that were captured by the camera inside the Cobalt-60 hot cell. The results show that the proposed method can effectively filter out the speckle radiation noise induced by γ rays, significantly improve the quality of features and feature matching rate, and improve the performance of ORB-SLAM3, the state-of-the-art feature-based SLAM algorithm, in radiation environments.

Design, Development and Testing of Electronic Class Attendance using Fingerprint

Electronic Class Attendance System (e-CAS) is an easy way to keep track of attendance of students and lecturers in School. It will cover the requirements of the personnel department in terms of day-to-day monitoring of students' and lecturers activities. Hence, students’ attendance is an important issue every institution must take into consideration to be productive. Hence, this research aims to design, develop and test using fingerprints. The methodologies used were miniature techniques and image processing techniques for design and development. This target was mainly decomposed into image pre-processing, feature extraction and feature match. For each sub-task, some classical and up-to-date methods in literature were analysed. Based on the analysis, an integrated solution for electronic class attendance was developed for demonstration. However, Poison probability process, Biometrics as well as NP-chart of Quality control were adopted for classical statistical analysis of quality and functionality of the e-Class Attendance System (e-CAS). The project on electronic class attendance system was coded by PHP and the hardware was developed with ATMEGA series of microcontrollers. The result when experimented achieved the objective of improving the performance and consistency of the electronic class attendance system based on template matching techniques as well as class attendance for both students and lecturers.

MCM-Vb\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$V_b$$\\end{document}F: dance hand gestures recognition with vision based features

To digitize and preserve the cultural heritage in the form of Indian classical dance become apparent area of research. Sattriya classical dance of North-East India (Assam) is one of the eight Indian classical dance forms that requires immediate preservation. Sattriya classical dance consists of 29 Asamyukta hastas (single-hand gestures) and 14 Samyukta hastas (double-hand gestures). Moreover, the foundation of Samyukta hasta depends on understanding Asamyukta hasta. Therefore, the paper aims to classify single-hand gestures of Sattriya classical dance only. Although, a solution based on two level classification method to classify the Sattriya classical dance is available in recent literature, but it requires a trial and error method to select the optimized features. Since, Asamyukta hastas can appear closely similar to each other and therefore misclassification chances are very high. Thus, accuracy rate obtained for the two level classification method was only 75.45%. So, to address this issues in this paper, a Multilevel Classification Model with Vision based Features (MCM-Vb\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$V_b$$\\end{document}F) has been proposed to classify the Asamyukta hastas of Sattriya classical dance. This model uses two types of feature matching, viz., high-level feature matching and low-level feature matching. To extract the high-level features and low-level features different algorithm has been proposed. In this model, features are automatically selected. This proposed MCM-Vb\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$V_b$$\\end{document}F model is also tested on Asamyukta hasta mudras of Bharatanatyam classical dance of South India (Tamil Nadu). This model obtain an accuracy 94.12%, 87.14% for Sattriya classical dance Single-Hand Gestures (SSHG) dataset and Bharatnatyam classical dance Single-Hand Gestures (BHSG) dataset respectively. This paper also provides the comparative study of the proposed model MCM-Vb\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$V_b$$\\end{document}F with traditional bench-mark classifier model such as Naive Bayes, Decision Tree and Support Vector Classifier (SVM) etc.

NCMNet: Neighbor Consistency Mining Network for Two-View Correspondence Pruning.

Correspondence pruning plays a crucial role in a variety of feature matching based tasks, which aims at identifying correct correspondences (inliers) from initial ones. Seeking consistent k-nearest neighbors in both coordinate and feature spaces is a prevalent strategy employed in previous approaches. However, the vicinity of an inlier contains numerous irregular false correspondences (outliers), which leads them to mistakenly become neighbors according to the similarity constraint of nearest neighbors. To tackle this issue, we propose a global-graph space to seek consistent neighbors with similar graph structures. This is achieved by using a global connected graph to explicitly render the affinity relationship between correspondences based on the spatial and feature consistency. Furthermore, to enhance the robustness of method for various matching scenes, we develop a neighbor consistency block to adequately leverage the potential of three types of neighbors. The consistency can be progressively mined by sequentially extracting intra-neighbor context and exploring inter-neighbor interactions. Ultimately, we present a Neighbor Consistency Mining Network (NCMNet) to estimate the parametric models and remove outliers. Extensive experimental results demonstrate that the proposed method outperforms other state-of-the-art methods on various benchmarks for two-view geometry estimation. Meanwhile, four extended tasks, including remote sensing image registration, point cloud registration, 3D reconstruction, and visual localization, are conducted to test the generalization ability.

Quantifying Uncertainty in Atmospheric Winds Retrieved from Optical Flow: Dependence on Weather Regime

Abstract This study explores the performance of a dense optical flow method in comparison to pattern-matching techniques for retrieving atmospheric motion vectors (AMVs) from water vapor images. Using high-resolution simulated datasets that represent various weather phenomena, we assess the performance of these methods across different weather regimes, time intervals, and pressure levels and quantify the uncertainties associated with retrieved winds. The optical flow algorithm consistently outperforms the feature matching approach. Notably, it produces wind speeds and AMVs that closely resemble the wind fields from the simulations, and unlike the feature matching algorithm, the optical flow algorithm exhibits consistent performance across different time intervals. In contrast, the feature matching approach yields vector fields that exhibit oversmoothing in certain areas and erratic behavior in others, while also producing less detailed, regionally constant speed maps. Furthermore, unlike feature matching, the optical flow method effectively calculates AMV near regions with missing data, generating a dense AMV field for every pixel in a pair of images. This superior performance and flexibility significantly influence the planning for future satellite missions aimed at retrieving atmospheric winds. As such, our work plays a critical role in determining the mission architecture and projected instrument performance for future atmospheric wind retrieval satellite missions. The study underscores the potential of the optical flow algorithm as a robust and efficient approach for atmospheric motion retrieval, thus contributing to advances in climate research and weather prediction. Significance Statement This research investigates the efficacy of two methods, optical flow and feature matching, for detecting atmospheric winds, referred to as atmospheric motion vectors, from satellite images of water vapor. Employing detailed simulated datasets that replicate real-world weather patterns, we found that optical flow consistently outperforms feature matching in various aspects. Notably, the optical flow method is not only more precise but also maintains its accuracy across different scenarios. These insights are critical for the design of future satellite missions focused on advancing our understanding of the atmosphere and enhancing weather predictions. This study contributes to advancements in climate research and supports improved weather forecasting, benefiting both scientific and societal needs.

Research on 3D laser SLAM algorithm based on graph optimization

Abstract Aiming at the problems that the positioning accuracy of laser Simultaneous Localization and Mapping (SLAM) is affected by the misclassification of close feature points and incorrect feature matching in the environment with redundant features, a 3D laser SLAM algorithm based on graph optimization was proposed. Firstly, to reduce the feature extraction error problem, the distance threshold is used to filter the close point cloud and improve the accuracy of feature classification, enhanced reliability of optoelectronic measurement data. Secondly, to improve the consistency of feature matching, based on the rough matching based on k-dimensional tree (KD-Tree), a two-stage fine matching based on geometric consistency is introduced to improved laser point alignment accuracy. The proposed method is compared with the mainstream localization algorithms based on KITTI dataset. Compared with Lidar-IMU Odometry with Sparse Adaptive Mapping and Fast LiDAR-Inertial Odometry, the positioning error of the proposed method is reduced by 21.88%and 24.43%on average, which showsthe superiority of the present algorithm in the application of optoelectronics and optical technology, and provides new ideas and methods for the development of high-precision laser navigation technology.

Image Matching for Multi spectral Image Satellite SIFT and Features

Abstract The Matching and registration of the satellite imagery play an essential role in many remote sensing and image processing applications. Certain steps, such as analysis of broad regions of interest and for the remotely detection. Pixel values in two or more images were play an important role in matching and detection algorithms between two or more images. In this work an image registration process (detection, extract and matching features) were utilized through using the sift algorithm to registration two image that have different resolutions (Landsat (30m) and Sentinel (10m)) utilize MATLAB program. A geometric correction was applied in this work for detect a correct points between these images. The results shows a corresponding between these two images. A preprocessing were utilized in good identifying features and accuracy of extracting information from satellite images.

FilterGNN: Image feature matching with cascaded outlier filters and linear attention

The cross-view matching of local image features is a fundamental task in visual localization and 3D reconstruction. This study proposes FilterGNN, a transformer-based graph neural network (GNN), aiming to improve the matching efficiency and accuracy of visual descriptors. Based on high matching sparseness and coarse-to-fine covisible area detection, FilterGNN utilizes cascaded optimal graph-matching filter modules to dynamically reject outlier matches. Moreover, we successfully adapted linear attention in FilterGNN with post-instance normalization support, which significantly reduces the complexity of complete graph learning from O(N2) to O(N). Experiments show that FilterGNN requires only 6% of the time cost and 33.3% of the memory cost compared with SuperGlue under a large-scale input size and achieves a competitive performance in various tasks, such as pose estimation, visual localization, and sparse 3D reconstruction.

Multi-scale feature reconstruction network for industrial anomaly detection

Unsupervised anomaly detection techniques, which operate without prior knowledge of anomalies, have garnered significant attention in industrial inspection due to their adaptability and generalization. Therefore, knowledge-based computer vision techniques have been broadly applied to identify unusual image patterns. However, real-time industrial applications present challenges such as limited anomalous samples, inadequate defect knowledge, and complex background textures. These factors lead to difficulties in accurately identifying defect regions, and conventional auto-encoder networks often struggle to overcome these issues.To address these limitations, we propose a multi-scale feature reconstruction (MSFR) network specifically designed for domain shift scenarios. Our approach employs a pyramidal vision transformer network (PVTN) to reconstruct multi-scale feature maps, capturing discriminative features at various scales. Additionally, a pre-trained module extracts multi-level features at the same scale, and a dedicated feature matching module enhances accuracy by improving the alignment probability between features. The MSFR strategy surpasses conventional auto-encoders by filtering pixel-level information at multiple depths.Empirical evaluations were conducted using benchmark datasets such as MVTec AD and AeBAD-S. Furthermore, an extensive ablation study demonstrates the effectiveness and viability of the proposed MSFR approach for industrial anomaly detection tasks. The experimental results show that the proposed model significantly outperforms recent approaches, making it highly suitable for real-world industrial applications, particularly in manufacturing.

Instant pose extraction based on mask transformer for occluded person re-identification

Re-Identification (Re-ID) of obscured pedestrians is a daunting task, primarily due to the frequent occlusion caused by various obstacles like buildings, vehicles, and even other pedestrians. To address this challenge, we propose a novel approach named Instant Pose Extraction based on Mask Transformer (MTIPE), tailored specifically for occluded person Re-ID. MTIPE consists of several new modules: a Mask Aware Module (MAM) for alignment between the overall prototype and the occluded image; a Multi-headed Attention Constraint Module (MACM) to enrich the feature representation; a Pose Aggregation Module (PAM) to separate useful human information from the occlusion noise; a Feature Matching Module (FMM) in matching non-occluded parts; introduction of learnable local prototypes in the defined local prototype-based transformer decoder; a Pooling Attention Module (PAM) instead of traditional self-attention module to better extract and propagate local contextual information; and Pose Key-points Loss to better match non-occluded body parts. Through comprehensive experimental evaluations and comparisons, MTIPE demonstrates encouraging performance improvements in both occluded and holistic person Re-ID tasks. Its results surpass or at least match those of current state-of-the-art methods in various aspects, highlighting its potential advantages and promising application prospects.

Multi-attention associate prediction network for visual tracking

Classification-regression prediction networks have realized impressive success in several modern deep trackers. However, there is an inherent difference between classification and regression tasks, so they have diverse even opposite demands for feature matching. Existed models always ignore the key issue and only employ a unified matching block in two task branches, decaying the decision quality. Besides, these models also struggle with decision misalignment situation. In this paper, we propose a multi-attention associate prediction network (MAPNet) to tackle the above problems. Concretely, two novel matchers, i.e., category-aware matcher and spatial-aware matcher, are first designed for feature comparison by integrating self, cross, channel or spatial attentions organically. They are capable of fully capturing the category-related semantics for classification and the local spatial contexts for regression, respectively. Then, we present a dual alignment module to enhance the correspondences between two branches, which is useful to find the optimal tracking solution. Finally, we describe a Siamese tracker built upon the proposed prediction network, which achieves the leading performance on five tracking benchmarks, consisting of LaSOT, TrackingNet, GOT-10k, TNL2k and UAV123, and surpasses other state-of-the-art approaches.

Robust affine point matching via quadratic assignment on Grassmannians

Robust Affine Matching with Grassmannians (RoAM) is a new algorithm to perform affine registration of point clouds. The algorithm is based on minimizing the Frobenius distance between two elements of the Grassmannian. For this purpose, an indefinite relaxation of the Quadratic Assignment Problem (QAP) is used, and several approaches to affine feature matching are studied and compared. Experiments demonstrate that RoAM is more robust to noise and point discrepancy than previous methods.

Feature Matching Data Synthesis for Non-IID Federated Learning

Feature Matching Data Synthesis for Non-IID Federated Learning

Hierarchical Degradation-aware Network for Full-Reference Image Quality Assessment

Full-Reference Image Quality Assessment (FR-IQA) algorithms excel in evaluating perceptual distortions by comparing reference and distorted images. However, as the severity and quantity of distortions in datasets increase, existing FR-IQA methods struggle to capture complex nonlinear perceptual features. This limitation results in reduced adaptability and inaccurate assessments for images with more severe or multiple distortions. Recognizing the importance of understanding image degradation mechanisms, we propose a novel hierarchical degradation-aware network (HDaN) method. First, by exploring the degradation mechanisms from the reference image to the distorted image, our degradation network matches distortions that align more closely with the human visual system (HVS). Next, we design a convertor to project the matched features into multiple spaces, creating multidimensional feature representations that more comprehensively capture the complexity of image distortions rather than being confined to a single feature space. Then, we calculate a similarity matrix between the distorted and mapped features, selecting the most similar (top-k) features for merging. Finally, a regression network maps the merged features to quality scores, providing the final quality prediction. The experimental results demonstrate that our proposed HDaN method outperforms traditional deep learning-based FR-IQA methods. Specifically, the HDaN shows higher PLCC and SROCC metrics on benchmark datasets, significantly improving over existing methods. Moreover, the method exhibits better adaptability to images with varying degrees and types of distortions, thereby greatly enhancing the overall performance of IQA.

Innovative multi-stage matching for counting anything

Few-shot counting (FSC) is the task of counting the number of objects in an image that belong to the same category, by using a provided exemplar pattern. By replacing the exemplar, we can effectively count anything, even in cases where we have no prior knowledge of that category’s exemplar. However, due to the variations within the same category and the impact of inter-class similarity, it is challenging to achieve accurate intra-class similarity matching using conventional similarity comparison methods. To tackle these issues, we propose a novel few-shot counting method called Multi-stage Exemplar Attention Match Network (MEAMNet), which increases the accuracy of matching, reduces the impact of noise, and enhances similarity feature matching. Specifically, we propose a multi-stage matching strategy to obtain more stable and effective matching results by acquiring similar feature in different feature spaces. In addition, we propose a novel feature matching module called Exemplar Attention Match (EAM). With this module, the intra-class similarity representation in each stage will be enhanced to achieve a better matching of the key feature. Experimental results indicate that our method not only significantly surpasses the state-of-the-art (SOTA) methods in most evaluation metrics on the FSC-147 dataset but also achieves comprehensive superiority on the CARPK dataset. This highlights the outstanding accuracy and stability of our matching performance, as well as its exceptional transferability. We will release the code at https://github.com/hzg0505/MEAMNet.

Multi-Temporal Snow-Covered Remote Sensing Image Matching via Image Transformation and Multi-Level Feature Extraction

To address the challenge of image matching posed by significant modal differences in remote sensing images influenced by snow cover, this paper proposes an innovative image transformation-based matching method. Initially, the Pix2Pix-GAN conversion network is employed to transform remote sensing images with snow cover into images without snow cover, reducing the feature disparity between the images. This conversion facilitates the extraction of more discernible features for matching by transforming the problem from snow-covered to snow-free images. Subsequently, a multi-level feature extraction network is utilized to extract multi-level feature descriptors from the transformed images. Keypoints are derived from these descriptors, enabling effective feature matching. Finally, the matching results are mapped back onto the original snow-covered remote sensing images. The proposed method was compared to well-established techniques such as SIFT, RIFT2, R2D2, and ReDFeat and demonstrated outstanding performance. In terms of NCM, MP, Rep, Recall, and F1-measure, our method outperformed the state of the art by 177, 0.29, 0.22, 0.21, and 0.25, respectively. In addition, the algorithm shows robustness over a range of image rotation angles from −40° to 40°. This innovative approach offers a new perspective on the task of matching multi-temporal snow-covered remote sensing images.

Enhanced feature matching in single-cell proteomics characterizes IFN-γ response and co-existence of cell states

Proteome analysis by data-independent acquisition (DIA) has become a powerful approach to obtain deep proteome coverage, and has gained recent traction for label-free analysis of single cells. However, optimal experimental design for DIA-based single-cell proteomics has not been fully explored, and performance metrics of subsequent data analysis tools remain to be evaluated. Therefore, we here formalize and comprehensively evaluate a DIA data analysis strategy that exploits the co-analysis of low-input samples with a so-called matching enhancer (ME) of higher input, to increase sensitivity, proteome coverage, and data completeness. We assess the matching specificity of DIA-ME by a two-proteome model, and demonstrate that false discovery and false transfer are maintained at low levels when using DIA-NN software, while preserving quantification accuracy. We apply DIA-ME to investigate the proteome response of U-2 OS cells to interferon gamma (IFN-γ) in single cells, and recapitulate the time-resolved induction of IFN-γ response proteins as observed in bulk material. Moreover, we uncover co- and anti-correlating patterns of protein expression within the same cell, indicating mutually exclusive protein modules and the co-existence of different cell states. Collectively our data show that DIA-ME is a powerful, scalable, and easy-to-implement strategy for single-cell proteomics.

Emotional Working Memory Training Treatment for Young Adult Problem Online Sports Bettors: A Preliminary Randomized Controlled Trial.

One of the key features of gambling disorder (GD) is impairment in cognitive-emotional control. Considering the negative consequences of GD, the present study investigated the effectiveness of emotional working memory training (eWMT) in improving cognitive control, attention, working memory capacity, and cognitive emotion regulation strategies (CERS) among young adults with GD compared to a placebo group. Following the initial assessment in the pre-test phase, eligible participants were randomly assigned to one of two groups: experimental (n = 34) and placebo (n = 30). These groups completed eWMT and a feature matching task for 20 sessions respectively. The post-test and follow-up measures indicated that eWMT significantly improved cognitive control, attention, working memory capacity, and the use of maladaptive cognitive emotion regulation strategies, but it had no significant effect on adaptive CERS. The promising results of the present study suggest the use of eWMT as a new intervention to improve cognitive-emotional control among individuals with online gambling problems.

A unified feature-motion consistency framework for robust image matching

Establishing reliable feature matches between a pair of images in various scenarios is a long-standing open problem in photogrammetry. Attention-based detector-free matching with coarse-to-fine architecture has been a typical pipeline to build matches, but the cross-attention module with global receptive field may compromise the structural local consistency by introducing irrelevant regions (outliers). Motion field can maintain structural local consistency under the assumption that matches for adjacent features should be spatially proximate. However, motion field can only estimate local displacements between consecutive images and struggle with long-range displacements estimation in large-scale variation scenarios without spatial correlation priors. Moreover, large-scale variations may also disrupt the geometric consistency with the application of mutual nearest neighbor criterion in patch-level matching, making it difficult to recover accurate matches. In this paper, we propose a unified feature-motion consistency framework for robust image matching (MOMA), to maintain structural consistency at both global and local granularity in scale-discrepancy scenarios. MOMA devises a motion consistency-guided dependency range strategy (MDR) in cross attention, aggregating highly relevant regions within the motion consensus-restricted neighborhood to favor true matchable regions. Meanwhile, a unified framework with hierarchical attention structure is established to couple local motion field with global feature correspondence. The motion field provides local consistency constraints in feature aggregation, while feature correspondence provides spatial context prior to improve motion field estimation. To alleviate geometric inconsistency caused by hard nearest neighbor criterion, we propose an adaptive neighbor search (soft) strategy to address scale discrepancy. Extensive experiments on three datasets demonstrate that our method outperforms solid baselines, with AUC improvements of 4.73/4.02/3.34 in two-view pose estimation task at thresholds of 5°/10°/20° on Megadepth test, and 5.94% increase of accuracy at threshold of 1px in homography task on HPatches datasets. Furthermore, in the downstream tasks such as 3D mapping, the 3D models reconstructed using our method on the self-collected SYSU UAV datasets exhibit significant improvement in structural completeness and detail richness, manifesting its high applicability in wide downstream tasks. The code is publicly available at https://github.com/BunnyanChou/MOMA.