Feature Point Detection Research Articles

Adaptive video stabilization based on feature point detection and full-reference stability assessment

Adaptive video stabilization based on feature point detection and full-reference stability assessment

RTV-SIFT: Harnessing Structure Information for Robust Optical and SAR Image Registration

Registration of optical and synthetic aperture radar (SAR) images is challenging because extracting located identically and unique features on both images are tricky. This paper proposes a novel optical and SAR image registration method based on relative total variation (RTV) and scale-invariant feature transform (SIFT), named RTV-SIFT, to extract feature points on the edges of structures and construct structural edge descriptors to improve the registration accuracy. First, a novel RTV-Harris feature point detection method by combining the RTV and the multiscale Harris algorithm is proposed to extract feature points on both images’ significant structures. This ensures a high repetition rate of the feature points. Second, the feature point descriptors are constructed on enhanced phase congruency edge (EPCE), which combines the Sobel operator and maximum moment of phase congruency (PC) to extract edges from structured images that enhance robustness to nonlinear intensity differences and speckle noise. Finally, after coarse registration, the position and orientation Euclidean distance (POED) between feature points is utilized to achieve fine feature point matching to improve the registration accuracy. The experimental results demonstrate the superiority of the proposed RTV-SIFT method in different scenes and image capture conditions, indicating its robustness and effectiveness in optical and SAR image registration.

A Straightforward Bifurcation Pattern-Based Fundus Image Registration Method.

Fundus image registration is crucial in eye disease examination, as it enables the alignment of overlapping fundus images, facilitating a comprehensive assessment of conditions like diabetic retinopathy, where a single image's limited field of view might be insufficient. By combining multiple images, the field of view for retinal analysis is extended, and resolution is enhanced through super-resolution imaging. Moreover, this method facilitates patient follow-up through longitudinal studies. This paper proposes a straightforward method for fundus image registration based on bifurcations, which serve as prominent landmarks. The approach aims to establish a baseline for fundus image registration using these landmarks as feature points, addressing the current challenge of validation in this field. The proposed approach involves the use of a robust vascular tree segmentation method to detect feature points within a specified range. The method involves coarse vessel segmentation to analyze patterns in the skeleton of the segmentation foreground, followed by feature description based on the generation of a histogram of oriented gradients and determination of image relation through a transformation matrix. Image blending produces a seamless registered image. Evaluation on the FIRE dataset using registration error as the key parameter for accuracy demonstrates the method's effectiveness. The results show the superior performance of the proposed method compared to other techniques using vessel-based feature extraction or partially based on SURF, achieving an area under the curve of 0.526 for the entire FIRE dataset.

Registration of Large Optical and SAR Images with Non-Flat Terrain by Investigating Reliable Sparse Correspondences

Optical and SAR image registration is the primary procedure to exploit the complementary information from the two different image modal types. Although extensive research has been conducted to narrow down the vast radiometric and geometric gaps so as to extract homogeneous characters for feature point matching, few works have considered the registration issue for non-flat terrains, which will bring in more difficulties for not only sparse feature point matching but also outlier removal and geometric relationship estimation. This article addresses these issues with a novel and effective optical-SAR image registration framework. Firstly, sparse feature points are detected based on the phase congruency moment map of the textureless SAR image (SAR-PC-Moment), which helps to identify salient local regions. Then a template matching process using very large local image patches is conducted, which increases the matching accuracy by a significant margin. Secondly, a mutual verification-based initial outlier removal method is proposed, which takes advantage of the different mechanisms of sparse and dense matching and requires no geometric consistency assumption within the inliers. These two procedures will produce a putative correspondence feature point (CP) set with a low outlier ratio and high reliability. In the third step, the putative CPs are used to segment the large input image of non-flat terrain into dozens of locally flat areas using a recursive random sample consensus (RANSAC) method, with each locally flat area co-registered using an affine transformation. As for the mountainous areas with sharp elevation variations, anchor CPs are first identified, and then optical flow-based pixelwise dense matching is conducted. In the experimental section, ablation studies using four precisely co-registered optical-SAR image pairs of flat terrain quantitatively verify the effectiveness of the proposed SAR-PC-Moment-based feature point detector, big template matching strategy, and mutual verification-based outlier removal method. Registration results on four 1 m-resolution non-flat image pairs prove that the proposed framework is able to produce robust and quite accurate registration results.

UAV image matching of mountainous terrain using the LoFTR deep learning model

In natural terrain scene UAV image matching, traditional feature point-based methods often have problems such as an unstable number of extracted feature points, difficulty in detecting feature points in weak texture areas, uneven distribution, and low robustness. Deep learning-based image matching methods can produce larger and more reasonably distributed matching pairs, so this research paper tries to perform UAV image matching based on a deep learning LoFTR algorithm for natural terrain scenes. The critical technical process was: first, the LoFTR algorithm was used to generate dense feature matching, and then the epipolar line constraints were used to purify the interior points, specifically, this study used the MAGSAC++ method to estimate the fundamental matrix, eliminate the wrong matching pairs, and finally get reliable matching results. In this research paper, six sets of visible images taken by different UAVs equipped with different sensors in the field were selected as experimental data to test the method and were compared and analyzed with the traditional classical SIFT, ASIFT, and AKAZE algorithms and the KeyNet-AdaLAM deep learning method. The experimental results show that the method in this study obtains a dense number of robust matching pairs with uniform spatial distribution in the UAV image matching of natural scenes mainly in mountainous areas, and the comprehensive performance is higher and more advantageous than the comparison methods.

Deep‐sea image stitching: Using multi‐channel fusion and improved AKAZE

AbstractDeep‐sea image is of great significance for exploring seabed resources. However, the information of a single image is limited. Besides, deep‐sea image with low contrast and colour distortion further restricts useful feature extraction. To address the issues above, this paper presents a multi‐channel fusion and accelerated‐KAZE (AKAZE) feature detection algorithm for deep‐sea image stitching. First, the authors restore deep‐sea image in LAB colour space and RGB colour space, respectively; in LAB space, the authors use homomorphic filtering in L colour channel, and in RGB space, the authors adopt multi‐scale Retinex with chromaticity preservation algorithm to adjust the colour information. Then, the authors blend two pre‐processed images with dark channel prior weighted coefficient. After that, the authors detect feature points with the AKAZE algorithm and obtain feature descriptors with Boosted Efficient Binary Local Image Descriptor. Finally, the authors match the feature points and warp deep‐sea images to obtain the stitched image. Experimental results demonstrate that the authors’ method generates high‐quality stitched image with minimized seam. Compared with state‐of‐the‐art algorithms, the proposed method has better quantitative evaluation, visual stitching results, and robustness.

An Improved Visual SLAM Method with Adaptive Feature Extraction

The feature point method is the mainstream method to accomplish inter-frame estimation in visual Simultaneous Localization and Mapping (SLAM) methods, among which the Oriented FAST and Rotated BRIEF (ORB) feature-based method provides an equilibrium of accuracy as well as efficiency. However, the ORB algorithm is prone to clustering phenomena, and its unequal distribution of extracted feature points is not conducive to the subsequent camera tracking. To solve the above problems, this paper suggests an adaptive feature extraction algorithm that first constructs multiple-scale images using an adaptive Gaussian pyramid algorithm, calculates adaptive thresholds, and uses an adaptive meshing method for regional feature point detection to adapt to different scenes. The method uses Adaptive and Generic Accelerated Segment Test (AGAST) to speed up feature detection and the non-maximum suppression method to filter feature points. The feature points are then divided equally by a quadtree technique, and the orientation of those points is determined by an intensity centroid approach. Experiments were conducted on publicly available datasets, and the outcomes demonstrate the algorithm has good adaptivity and solves the problem of a large number of corner point clusters that may result from using manually set detection thresholds. The RMSE of the absolute trajectory error of SLAM applying this method on four sequences of TUM RGB-D datasets is decreased by 13.88% when compared with ORB-SLAM3. It is demonstrated that the algorithm provides high-quality feature points for subsequent image alignment, and the application to SLAM improves the reliability and accuracy of SLAM.

Unmanned aerial vehicle orthogonal laser localization by Gaussian mixture model‐based map representation

AbstractLocalization is a core problem in mobile robot navigation. Simultaneous localization and mapping (SLAM) costs much for an unmanned aerial vehicle (UAV). This research aims to design an orthogonal laser scan device for localization and to save computation costs. Based on disturbance analysis, residual influences on sensor state are quantitative, and they are related to uncertainty and sensitivity. This research applied the residual selection method to a UAV. The feature point detection utilises multi‐scale and Gaussian model fitting techniques to guarantee true positives. The map is represented by Gaussian Mixture Models (GMM) with lower memory costs. The orthogonal laser scan device is composed and placed on a UAV for real‐time three‐dimensional localization, whose errors are at the centimeter level.

Guided Local Feature Matching with Transformer

GLFNet is proposed to be utilized for the detection and matching of local features among remote-sensing images, with existing sparse feature points being leveraged as guided points. Local feature matching is a crucial step in remote-sensing applications and 3D reconstruction. However, existing methods that detect feature points in image pairs and match them separately may fail to establish correct matches among images with significant differences in lighting or perspectives. To address this issue, the problem is reformulated as the extraction of corresponding features in the target image, given guided points from the source image as explicit guidance. The approach is designed to encourage the sharing of landmarks by searching for regions in the target image with features similar to the guided points in the source image. For this purpose, GLFNet is developed as a feature extraction and search network. The main challenge lies in efficiently searching for accurate matches, considering the massive number of guided points. To tackle this problem, the search network is divided into a coarse-level match network-based guided point transformer that narrows the search space and a fine-level regression network that produces accurate matches. The experimental results on challenging datasets demonstrate that the proposed method provides robust matching and benefits various applications, including remote-sensing image registration, optical flow estimation, visual localization, and reconstruction registration. Overall, a promising solution is offered by this approach to the problem of local feature matching in remote-sensing applications.

A Blind Image Steganography Algorithm Based on Knight Tour Algorithm and QR Codes

Internet proliferation and technological progress have made multimedia information quickly accessible, but they have also posed a threat to privacy and security. Researchers have been interested in digital images due to their capacity to store large amounts of data due to the possibility of protecting sensitive information through digital steganography. Despite their visual imperceptibility, robustness, and ability to embed information, existing image steganography techniques face several challenges. To overcome these challenges, a novel image steganography approach based on a blind model strategy has been proposed for hiding covert messages. The model consists of two stages: embedding and extracting. In the embedding stage, a suitable cover image is selected using the FAST feature point detector. A text message is then converted to a QR code and embedded in the feature points' neighbors using knight tour steps in a chess game. The result is a stego image that appears identical to the original cover image but contains the secret message in its feature points' neighbors. The extracting stage involves finding the feature points and extracting the QR code to obtain the original text message. Since the feature points were not altered during the embedding process, the proposed model is known as a blind model. This approach eliminates the need for the original cover image during the extracting stage. The proposed model was evaluated using several metrics, including the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The results demonstrate that the proposed algorithm can effectively embed and extract secret messages with high accuracy while maintaining the visual quality of the cover image with 100% of (SSIM), and 73.48 as an average of (PSNR).

Image Intensity Variation Information for Interest Point Detection.

Interest point detection methods are gaining more attention and are widely applied in computer vision tasks such as image retrieval and 3D reconstruction. However, there still exist two main problems to be solved: (1) from the perspective of mathematical representations, the differences among edges, corners, and blobs have not been convincingly explained and the relationships among the amplitude response, scale factor, and filtering orientation for interest points have not been thoroughly explained; (2) the existing design mechanism for interest point detection does not show how to accurately obtain intensity variation information on corners and blobs. In this paper, the first- and second-order Gaussian directional derivative representations of a step edge, four common genres of corners, an anisotropic-type blob, and an isotropic-type blob are analyzed and derived. Multiple interest point characteristics are discovered. The characteristics for interest points that we obtained help us describe the differences among edges, corners, and blobs, explain why the existing interest point detection methods with multiple scales cannot properly obtain interest points from images, and present novel corner and blob detection methods. Extensive experiments demonstrate the superiority of our proposed methods in terms of detection performance, robustness to affine transformations, noise, image matching, and 3D reconstruction.

A Fast Image Matching Method Based on Improved SURF

In order to solve the problems of low matching accuracy, slow speed and high system overhead in image matching methods, a rotation binary descriptor construction method based on Speed Up Robust Features (SURF) feature point detection is designed by using different Fast Library for Approximate Nearest Neighbors (FLANN) parameters and the filtering mechanism to screen out wrong matches according to the types of feature descriptors constructed in different feature extraction algorithms. This method ensures scale and rotation invariant while simplifying the representation of feature descriptors and speeding up the calculation speed in the initial stage of matching by combining the binary characteristics of descriptors. Finally, the Hamming distance is used as the filtering mechanism to improve the success rate of the final matching. The experimental results show that the accuracy of image matching is improved by 1.5% and the matching time is improved by 0.116s, while the robustness of the image to noise and rotation is ensured.

Feature point detection in HDR images based on coefficient of variation

Feature point detection in HDR images based on coefficient of variation

Point clouds segmentation of rapeseed siliques based on sparse-dense point clouds mapping.

In this study, we propose a high-throughput and low-cost automatic detection method based on deep learning to replace the inefficient manual counting of rapeseed siliques. First, a video is captured with a smartphone around the rapeseed plants in the silique stage. Feature point detection and matching based on SIFT operators are applied to the extracted video frames, and sparse point clouds are recovered using epipolar geometry and triangulation principles. The depth map is obtained by calculating the disparity of the matched images, and the dense point cloud is fused. The plant model of the whole rapeseed plant in the silique stage is reconstructed based on the structure-from-motion (SfM) algorithm, and the background is removed by using the passthrough filter. The downsampled 3D point cloud data is processed by the DGCNN network, and the point cloud is divided into two categories: sparse rapeseed canopy siliques and rapeseed stems. The sparse canopy siliques are then segmented from the original whole rapeseed siliques point cloud using the sparse-dense point cloud mapping method, which can effectively save running time and improve efficiency. Finally, Euclidean clustering segmentation is performed on the rapeseed canopy siliques, and the RANSAC algorithm is used to perform line segmentation on the connected siliques after clustering, obtaining the three-dimensional spatial position of each silique and counting the number of siliques. The proposed method was applied to identify 1457 siliques from 12 rapeseed plants, and the experimental results showed a recognition accuracy greater than 97.80%. The proposed method achieved good results in rapeseed silique recognition and provided a useful example for the application of deep learning networks in dense 3D point cloud segmentation.

Measurement of displacement and top beam attitude angle of advanced hydraulic support based on visual detection

We constructed an advanced hydraulic support test bench and proposed a visual detection-based approach for measuring the displacement and attitude angle of the advanced hydraulic support. Firstly, our methodology involved using a binocular camera for image acquisition and stereo rectification of the advanced hydraulic support. Then, we employed the ORB algorithm to detect feature points and the SGBM matching algorithm to determine the disparity value. Finally, the attitude angle solution algorithm and the triangulation method are written to calculate the support displacement and the top beam attitude angle. Results showed that the advanced hydraulic support had an average absolute error of 8 mm in displacement measurement and 0.4° in attitude angle measurement, meeting the accuracy requirements for fully mechanized mining face operations.

A Coupled 3d Morphological Reconstruction Approach for Surface Microcrack in Si3 n4 Ceramic Bearing Roller Based on Adaptive Nano Feature Extraction & Multiscale Depth Fusion.

To extract the fuzzy contour features, tiny depth features of surface microcracks in the Si3 N4 ceramic bearings roller. An adaptive nano feature extraction and multiscale deep fusion coupling method is proposed, to sufficiently reconstruct the three-dimensional morphology characteristics of surface microcracks. Construct an adaptive nano feature extraction method, form the surface microcrack image scale space and the Gaussian difference pyramid function equation, realize the detection and matching of global feature points. The sparse point cloud is obtained. Through polar-line correction, depth estimation, and fusion of feature points on the surface microcracks image, a multiscale depth fusion matching cost pixel function is established to realize a dense point cloud reconstruction of surface microcracks. The reconstruction results show that the highest value of the local convex surface reconstructed by the dense point cloud reaches 1183nm, and the lowest local concave surface is accurate to 296nm. Compared with the measurement results of the confocal platform, the relative error of the reconstruction result is 24.6%. The overall feature-matching rate of the reconstruction reaches 93.3%. It provides a theoretical basis for the study of surface microcrack propagation mechanism and the prediction of bearing life.

YOLO-Weld: A Modified YOLOv5-Based Weld Feature Detection Network for Extreme Weld Noise

Weld feature point detection is a key technology for welding trajectory planning and tracking. Existing two-stage detection methods and conventional convolutional neural network (CNN)-based approaches encounter performance bottlenecks under extreme welding noise conditions. To better obtain accurate weld feature point locations in high-noise environments, we propose a feature point detection network, YOLO-Weld, based on an improved You Only Look Once version 5 (YOLOv5). By introducing the reparameterized convolutional neural network (RepVGG) module, the network structure is optimized, enhancing detection speed. The utilization of a normalization-based attention module (NAM) in the network enhances the network’s perception of feature points. A lightweight decoupled head, RD-Head, is designed to improve classification and regression accuracy. Furthermore, a welding noise generation method is proposed, increasing the model’s robustness in extreme noise environments. Finally, the model is tested on a custom dataset of five weld types, demonstrating better performance than two-stage detection methods and conventional CNN approaches. The proposed model can accurately detect feature points in high-noise environments while meeting real-time welding requirements. In terms of the model’s performance, the average error of detecting feature points in images is 2.100 pixels, while the average error in the world coordinate system is 0.114 mm, sufficiently meeting the accuracy needs of various practical welding tasks.

Time-series NARX feedback neural network for forecasting impedance cardiography ICG missing points: a predictive model.

One of the crucial steps in assessing hemodynamic parameters using impedance cardiography (ICG) is the detection of the characteristic points in the dZ/dt ICG complex, especially the X point. The most often estimated parameters from the ICG complex are stroke volume and cardiac output, for which is required the left ventricular pre-ejection time. Unfortunately, for beat-to-beat calculations, the accuracy of detection is affected by the variability of the ICG complex subtypes. Thus, in this work, we aim to create a predictive model that can predict the missing points and decrease the previous work percentages of missing points to support the detection of ICG characteristic points and the extraction of hemodynamic parameters according to several existing subtypes. Thus, a time-series non-linear autoregressive model with exogenous inputs (NARX) feedback neural network approach was implemented to forecast the missing ICG points according to the different existing subtypes. The NARX was trained on two different datasets with an open-loop mode to ensure that the network is fed with correct feedback inputs. Once the training is satisfactory, the loop can be closed for multi-step prediction tests and simulation. The results show that we can predict the missing characteristic points in all the complexes with a success rate ranging between 75% and 88% in the evaluated datasets. Previously, without the NARX predictive model, the successful detection rate was 21%-30% for the same datasets. Thus, this work indicates a promising method and an accuracy increase in the detection of X, Y, O, and Z points for both datasets.

UW Deep SLAM-CNN Assisted Underwater SLAM

Abstract Underwater simultaneous localization and mapping (SLAM) poses significant challenges for modern visual SLAM systems. The integration of deep learning networks within computer vision offers promising potential for addressing these difficulties. Our research draws inspiration from deep learning approaches applied to interest point detection and matching, single image depth prediction and underwater image enhancement. In response, we propose 3D-Net, a deep learning-assisted network designed to tackle these three tasks simultaneously. The network consists of three branches, each serving a distinct purpose: interest point detection, descriptor generation, and depth prediction. The interest point detector and descriptor generator can effectively serve as a front end for a classical SLAM system. The predicted depth information is akin to a virtual depth camera, opening up possibilities for various applications. We provide quantitative and qualitative evaluations to illustrate some of these potential uses. The network was trained in in several steps, using in-air datasets and followed by generated underwater datasets. Further, the network is integrated into feature-based SALM systems ORBSLAM2 and ORBSSLAM3, providing a comprehensive assessment of its effectiveness for underwater navigation.

Research on the algorithm of image feature detection and matching

In-depth research on feature detection technology affects people's modern life. Modern artificial intelligence can act as the eyes of human beings and efficiently filter out effective information from complex pictures. Corner detection has now evolved into a tool for efficient image scanning. People's increasingly stringent requirements for image processing continue to promote the birth of new technologies. Corner detection methods have been improved and perfected, and have experienced detectors such as Harris, FAST, Scalriant Feature Transform (SIFT), Speeded Up Robust Feature (SURF), Binary Robust Invariant Scalable Keypoints (BRISK), KAZE, AKAZE and Oriented FAST and Rotated BRIEF (ORB). In the face of many mainstream detectors, the main research purpose of this paper is to rely on Python and Computer vision to study the detection efficiency of various detectors in different environments. In this experiment, thirteen groups of pictures were selected, and after being flipped, complicated, and blurred respectively, they were detected by different detectors to obtain the results. Finally, by comparing the feature detection points, detection time and other factors, this study found that the ORB detector can be competent in most situations and is currently the fastest and stable feature point detection and extraction algorithm. On the other hand, the BRISK detector can handle highly blurred images more efficiently.