Keypoint Matching Method Research Articles

Point of Interest Recognition and Tracking in Aerial Video during Live Cycling Broadcasts

Road cycling races, such as the Tour de France, captivate millions of viewers globally, combining competitive sportsmanship with the promotion of regional landmarks. Traditionally, points of interest (POIs) are highlighted during broadcasts using manually created static overlays, a process that is both outdated and labor-intensive. This paper presents a novel, fully automated methodology for detecting and tracking POIs in live helicopter video streams, aiming to streamline the visualization workflow and enhance viewer engagement. Our approach integrates a saliency and Segment Anything-based technique to propose potential POI regions, which are then recognized using a keypoint matching method that requires only a few reference images. This system supports both automatic and semi-automatic operations, allowing video editors to intervene when necessary, thereby balancing automation with manual control. The proposed pipeline demonstrated high effectiveness, achieving over 75% precision and recall in POI detection, and offers two tracking solutions: a traditional MedianFlow tracker and an advanced SAM 2 tracker. While the former provides speed and simplicity, the latter delivers superior segmentation tracking, albeit with higher computational demands. Our findings suggest that this methodology significantly reduces manual workload and opens new possibilities for interactive visualizations, enhancing the live viewing experience of cycling races.

ANALYSIS OF IMAGE KEY POINT DETECTION, DESCRIPTION AND MATCHING METHODS

ANALYSIS OF IMAGE KEY POINT DETECTION, DESCRIPTION AND MATCHING METHODS

A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES

Abstract. Currently, multimodal remote sensing images have complex geometric and radiometric distortions, which are beyond the reach of classical hand-crafted feature-based matching. Although keypoint matching methods have been developed in recent decades, most manual and deep learning-based techniques cannot effectively extract highly repeatable keypoints. To address that, we design a Siamese network with self-supervised training to generate similar keypoint feature maps between multimodal images, and detect highly repeatable keypoints by computing local spatial- and channel-domain peaks of the feature maps. We exploit the confidence level of keypoints to enable the detection network to evaluate potential keypoints with end-to-end trainability. Unlike most trainable detectors, it does not require the generation of pseudo-ground truth points. In the experiments, the proposed method is evaluated using various SAR and optical images covering different scenes. The results prove its superior keypoint detection performance compared with current state-of-art matching methods based on keypoints.

Research on Pose Estimation Method of Medicine Box Based on Secondary Key Point Matching

Aiming at the difficulty and low efficiency of pose estimation in the process of automatic drug picking, a pose estimation method combining plane segmentation and key point matching is proposed. First, a point cloud hierarchical clustering algorithm is used to segment the plane area, and the improved secondary key point matching method is used to identify and locate the pill box in each thread, which is convenient for multi-thread processing. Then the point cloud plane normal vector estimation method based on least squares is used to realize the posture reconstruction of the pill box. 600 images with viewpoint change, rotation transformation and scale change were collected for testing, the experimental results show that the pose estimation speed of the medicine box can reach 11 frames/s, the positioning deviation is less than 5 mm, and the pose estimation deviation is within 2°. So the method is efficient and can accurately identify the pose of the medicine box.

Fast ORB-SLAM Without Keypoint Descriptors.

Indirect methods for visual SLAM are gaining popularity due to their robustness to environmental variations. ORB-SLAM2 (Mur-Artal and Tardós, 2017) is a benchmark method in this domain, however, it consumes significant time for computing descriptors that never get reused unless a frame is selected as a keyframe. To overcome these problems, we present FastORB-SLAM which is light-weight and efficient as it tracks keypoints between adjacent frames without computing descriptors. To achieve this, a two stage descriptor-independent keypoint matching method is proposed based on sparse optical flow. In the first stage, we predict initial keypoint correspondences via a simple but effective motion model and then robustly establish the correspondences via pyramid-based sparse optical flow tracking. In the second stage, we leverage the constraints of the motion smoothness and epipolar geometry to refine the correspondences. In particular, our method computes descriptors only for keyframes. We test FastORB-SLAM on TUM and ICL-NUIM RGB-D datasets and compare its accuracy and efficiency to nine existing RGB-D SLAM methods. Qualitative and quantitative results show that our method achieves state-of-the-art accuracy and is about twice as fast as the ORB-SLAM2.

An anchor-free object detector with novel corner matching method

Recently, remarkable object detection performance has been made by keypoint-based detectors with different keypoint matching strategies. However, the factors considered in most of keypoint matching strategies are not comprehensive, which greatly affects the detection results. In this paper, based on CornerNet we propose a novel anchor-free detector with a brand new keypoint matching method. In our approach, instead of using embedding of each corner to achieve keypoint matching like CornerNet, we predict a matching degree score for each predicted bounding box formed by corners. In order to train model to get accurate matching degree prediction results, we fully consider the location and geometric information of targets, calculating the distance between centers of predicted and ground truth bounding boxes as well as IoU value to decide which pair of top-left and bottom-right corners can form the right bounding box. Besides that, to relieve the problem that categorical predictions associated with corners are not always reliable, we set an additional classification score for each predicted bounding boxes to further promote final detection results. We apply our model on MS-COCO test−dev set for evaluation and the result shows that our method achieves an AP of 46.3 % at single-scale and 47.9% at multi-scale, which are competitive with other state-of-the-art models such as CentripetalNet and Corner Proposal Net (CPN).

New Keypoint Matching Method Using Local Convolutional Features for Power Transmission Line Icing Monitoring.

Power transmission line icing (PTLI) problems, which cause tremendous damage to the power grids, has drawn much attention. Existing three-dimensional measurement methods based on binocular stereo vision was recently introduced to measure the ice thickness in PTLI, but failed to meet requirements of practical applications due to inefficient keypoint matching in the complex PTLI scene. In this paper, a new keypoint matching method is proposed based on the local multi-layer convolutional neural network (CNN) features, termed Local Convolutional Features (LCFs). LCFs are deployed to extract more discriminative features than the conventional CNNs. Particularly in LCFs, a multi-layer features fusion scheme is exploited to boost the matching performance. Together with a location constraint method, the correspondence of neighboring keypoints is further refined. Our approach achieves 1.5%, 5.3%, 13.1%, 27.3% improvement in the average matching precision compared with SIFT, SURF, ORB and MatchNet on the public Middlebury dataset, and the measurement accuracy of ice thickness can reach 90.9% compared with manual measurement on the collected PTLI dataset.

SCALE INVARIANT FEATURE TRANSFORM PLUS HUE FEATURE

Abstract. This paper presents an enhanced method for extracting invariant features from images based on Scale Invariant Feature Transform (SIFT). Although SIFT features are invariant to image scale and rotation, additive noise, and changes in illumination but we think this algorithm suffers from excess keypoints. Besides, by adding the hue feature, which is extracted from combination of hue and illumination values in HSI colour space version of the target image, the proposed algorithm can speed up the matching phase. Therefore, we proposed the Scale Invariant Feature Transform plus Hue (SIFTH) that can remove the excess keypoints based on their Euclidean distances and adding hue to feature vector to speed up the matching process which is the aim of feature extraction. In this paper we use the difference of hue features and the Mean Square Error (MSE) of orientation histograms to find the most similar keypoint to the under processing keypoint. The keypoint matching method can identify correct keypoint among clutter and occlusion robustly while achieving real-time performance and it will result a similarity factor of two keypoints. Moreover removing excess keypoint by SIFTH algorithm helps the matching algorithm to achieve this goal.

2D Matching Using Repetitive and Salient Features in Architectural Images

Matching and aligning architectural imagery is an important step for many applications but can be a difficult task due to repetitive elements often present in buildings. Many keypoint descriptor and matching methods will fail to produce distinctive descriptors for each region of man-made structures, which causes ambiguity when attempting to match areas between images. In this paper, we outline a technique for reducing the search space for matching by taking a two-step approach, aligning pairs one dimension at a time and by abstracting images that originally contain many repetitive elements into a set of distinct, representative patches. We also present a simple, but very effective method for computing the intra-image saliency for a single image that allows us to directly identify unique areas in an image without machine learning. We use this information to find distinctive keypoint matches across image pairs. We show that our pipeline is able to overcome many of the pitfalls encountered when using traditional keypoint and regional matching techniques on commonly encountered images of urban scenes.

Reconstruction of Banknote Fragments Based on Keypoint Matching Method.

Banknotes may be shredded by a scrap machine, ripped up by hand, or damaged in accidents. This study proposes an image registration method for reconstruction of multiple sheets of banknotes. The proposed method first constructs different scale spaces to identify keypoints in the underlying banknote fragments. Next, the features of those keypoints are extracted to represent their local patterns around keypoints. Then, similarity is computed to find the keypoint pairs between the fragment and the reference banknote. The banknote fragments can determine the coordinate and amend the orientation. Finally, an assembly strategy is proposed to piece multiple sheets of banknote fragments together. Experimental results show that the proposed method causes, on average, a deviation of 0.12457±0.12810° for each fragment while the SIFT method deviates 1.16893±2.35254° on average. The proposed method not only reconstructs the banknotes but also decreases the computing cost. Furthermore, the proposed method can estimate relatively precisely the orientation of the banknote fragments to assemble.

Keypoint Recognition with Two-Stage Randomized Trees

This paper proposes a high-precision, high-speed keypoint matching method using two-stage randomized trees (RTs). The keypoint classification uses conventional RTs for high-precision, real-time keypoint matching. However, the wide variety of view transformations for templates expressed by RTs make it diffidult to achieve high-precision classification for all transformations with a single RTs. To solve this problem, the proposed method classifies the template view transformations in the first stage and then, in the second stage, classifies the keypoints using the RTs that corresponds to each of the view transformations classified in the first stage. Testing demonstrated that the proposed method is 88.5% more precise than SIFT, and 63.5% more precise than using conventional RTs for images in which the viewpoint of the object is rotated by 70 degrees. We have also shown that the proposed method supports real-time keypoint matching at 12fps.