Feature Point Detection Research Articles

Improved SURF in Color Difference Scale Space for Color Image Matching

This paper presents an improved SURF (Speeded Up Robust Features) for image matching which considers color information. Firstly, a new color difference scale space is constructed based on color information to detect feature point. Then we extracted a 192-dimensional vector to describe feature point, which includes a 64-dimensional vector representing the brightness information and a 128-dimensional vector representing the color information in a color image. Finally, in the process images matching, a new weighted Murkovski distance is used to measure the distance between two descriptors. From the experiment results, we can know that, compared the other methods, the feature points detection method proposed is more robust. The matching scores and precision of our method are dominant among different methods of color image matching. Compared with SURF, the number of feature points detected by the proposed method increases by 163%, the average matching scores and matching precision increase by 16% and 15.81% respectively.

An Efficient Real-Time FPGA-Based ORB Feature Extraction for an UHD Video Stream for Embedded Visual SLAM

The detection and description of feature points are important components of many computer vision systems. For example, in the field of autonomous unmanned aerial vehicles (UAV), these methods form the basis of so-called Visual Odometry (VO) and Simultaneous Localisation and Mapping (SLAM) algorithms. In this paper, we present a hardware feature points detection system able to process a 4K video stream in real-time. We use the ORB algorithm—Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features)—to detect and describe feature points in the images. We make numerous modifications to the original ORB algorithm (among others, we use the RS-BRIEF instead of classic R-BRIEF) to adapt it to the high video resolution, make it computationally efficient, reduce the resource utilisation and achieve lower power consumption. Our hardware implementation supports a 4 ppc (pixels per clock) format (with simple adaptation to 2 ppc, 8 ppc, and more) and real-time processing of a 4K video stream (UHD—Ultra High Definition, 3840×2160 pixels) @ 60 frames per second (150 MHz clock). We verify our system using simulations in the Vivado IDE and implement it in hardware on the ZCU 104 evaluation board with the AMD Xilinx Zynq UltraScale+ MPSoC device. The proposed design consumes only 5 watts.

Application of Sparse-Channel Lidar Sensors on Viewpoint-Invariant Loop Closing Task

Loop closing is an essential task for SLAM system, which can reduce the pose drifts and map distortion. However, challenges from viewpoint changes and sparse-channel lidar are still open problems. In this paper, a novel efficient and viewpoint-invariant loop closing method is presented, which well fits sparse-channel lidars. To keep the metric information of 3D point clouds, elevation maps are explored to represent sparse point clouds, based on which a novel and viewpoint-invariant key point feature detection method is proposed. To improve efficiency, Bag of Words technique is applied to accelerate the process of scene description and matching. In addition, a strict geometric consistency check of matched key point pairs is proposed to reject false positive detections, which makes our method robust to scenes with similar layouts in urban environments. Extensive experiments on challenging public datasets verify the validity and superiority of the proposed method.

Color Image Feature Matching Method Based on the Improved Firework Algorithm

An improved ORB feature point purification method has been proposed to address the problem of large feature point matching error and low image registration rate in the oriented FAST and rotated BRIEF (ORB) feature point color image matching algorithm. Pure virtual quaternions were used to represent the color image pixels in this method, and an improved FAST algorithm was used to detect the color feature points at first. The firework algorithm has been used to divide the detected feature points into key areas, auxiliary areas, and small influence areas, depending on the degree of attachment. The feature points of the key area and the subsidiary area are the required feature points. In order to further improve the efficiency of matching the color image feature points, the firework explosion radius formula and the explosion number formula in the firework algorithm have been improved, and an improved firework algorithm is proposed. This improved algorithm purifies the quaternion-represented color image feature points. For feature matching, the hamming distance was used. Experiments show that, when compared to the traditional ORB algorithm, the improved algorithm retains the key feature points of the color image with high accuracy, removes a large number of irrelevant feature points and noise points, and provides significantly higher accuracy and efficiency of color image matching.

A Two-Stage Feature Point Detection and Marking Approach Based on the Labeled Multi-Bernoulli Filter.

In recent years, various algorithms using random finite sets (RFS) to solve the issue of simultaneous localization and mapping (SLAM) have been proposed. Compared with the traditional method, the advantage of the RFS method is that it can avoid data association, landmark appearance and disappearance, missed detections, and false alarms in Bayesian recursion. There are many problems in the existing robot SLAM methods, such as low estimation accuracy, poor back-end optimization, etc. On the basis of previous studies, this paper presents a labeled random finite set (L-RFS) SLAM method. We describe a scene where the sensor moves along a given path and avoids obstacles based on the L-RFS framework. Then, we use the labeled multi-Bernoulli filter (LMB) to estimate the state of the sensor and feature points. At the same time, the B-spline curve is used to smooth the obstacle avoidance path of the sensor. The effectiveness of the algorithm is verified in the final simulation.

Semantic Segmentation for Free Space and Lane Based on Grid-Based Interest Point Detection

An increasing number of tasks have been developed for autonomous driving and advanced driver assistance systems. However, this gives rise to the problem of incorporating plural functionalities to be ported into a power-constrained computing device. Therefore, the objective of this work is to alleviate the complex learning procedure of the pixel-wise approach for driving scene understanding. In this paper, we go beyond the pixel-wise detection of the semantic segmentation task as a point detection task and implement it to detect free space and lane. Instead of pixel-wise learning, we trained a single deep convolution neural network for point of interest detection in a grid-based level and followed with a computer vision (CV) based post-processing of end branches corresponding to the characteristic of target classes. To achieve the corresponding final result of pixel-wise detection of semantic segmentation and parametric description of lanes, we propose a CV-based post-processing to decode points of output from the neural network. The final results showed that the network could learn the spatial relationship for point of interest, including the representative points on the contour of the free space segmented region and the representative points along the center of the road lane. We verify our method on two publicly available datasets, which achieved 98.2% mIoU on the KITTI dataset for the evaluation of free space and 97.8% accuracy on the TuSimple dataset (with the field of view below the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$y=320$ </tex-math></inline-formula> axis) for the evaluation of the lane.

Moving Object Detection based on Reverse Projection Error Model

A novel moving feature point detection method is proposed in this paper in order to solve the problems in existing methods. Firstly, the coordinates of feature points are normalized by using spherical coordinates in order to solve the imaging distortion of the fish-eye camera. Secondly, based on the spherical normalization coordinates of feature points the reverse projection error model of the fish-eye camera is deduced and a moving feature point’s discriminant is derived to improve the detection integrity of moving object. Finally, according to the law of object motion, the detected potential moving feature points are clustered to further remove outliers. Experimental results show the proposed moving feature point detection method has better detection results compared with the commonly used methods.

Image registration method using representative feature detection and iterative coherent spatial mapping for infrared medical images with flat regions

In the registration of medical images, nonrigid registration targets, images with large displacement caused by different postures of the human body, and frequent variations in image intensity due to physiological phenomena are substantial problems that make medical images less suitable for intensity-based image registration modes. These problems also greatly increase the difficulty and complexity of feature detection and matching for feature-based image registration modes. This research introduces an automatic image registration algorithm for infrared medical images that offers the following benefits: effective detection of feature points in flat regions (cold patterns) that appear due to changes in the human body’s thermal patterns, improved mismatch removal through coherent spatial mapping for improved feature point matching, and large-displacement optical flow for optimal transformation. This method was compared with various classical gold standard image registration methods to evaluate its performance. The models were compared for the three key steps of the registration process—feature detection, feature point matching, and image transformation—and the results are presented visually and quantitatively. The results demonstrate that the proposed method outperforms existing methods in all tasks, including in terms of the features detected, uniformity of feature points, matching accuracy, and control point sparsity, and achieves optimal image transformation. The performance of the proposed method with four common image types was also evaluated, and the results verify that the proposed method has a high degree of stability and can effectively register medical images under a variety of conditions.

A night pavement crack detection method based on image‐to‐image translation

AbstractDeep learning provides an efficient automated method for pavement condition surveys, but the datasets used for this model are usually images taken in good lighting conditions. If images are taken at night, this model cannot work effectively. This paper proposes a method for normalizing pavement images at night, which includes three main steps. First, the image feature point detection and matching method is used to process images taken during the day and night. Then, paired images of pavement during the day and night are obtained. Second, with the help of the image‐to‐image translation model, those paired images are used for training, and the best model for converting night images into day images is selected. Third, a convolutional neural network (CNN) based on VGGNet is constructed, and pavement images taken during the day are used for training. After that, six types of images are used and tested separately, namely, those taken during the day and the night, converted by the proposed method and converted by traditional methods. As evaluated by various evaluation indices and visualization methods, the detection performance of the CNN model can be significantly improved by using the proposed method of converted night‐to‐day images.

A robust multimodal remote sensing image registration method and system using steerable filters with first- and second-order gradients

Co-registration of multimodal remote sensing (RS) images (e.g., optical, infrared, LiDAR, and SAR) is still an ongoing challenge because of nonlinear radiometric differences (NRD) and significant geometric distortions (e.g., scale and rotation changes) between these images. In this paper, a robust matching method based on the Steerable filters is proposed consisting of two critical steps. First, to address severe NRD, a novel structural descriptor named the Steerable Filters of first- and second-Order Channels (SFOC) is constructed, which combines the first- and second-order gradient information by using the steerable filters with a multi-scale strategy to depict more discriminative structure features of images. Then, a fast similarity measure is established called Fast Normalized Cross-Correlation (Fast-NCCSFOC), which employs the Fast Fourier Transform (FFT) technique and the integral image to improve the matching efficiency. Furthermore, to achieve reliable registration performance, a coarse-to-fine multimodal registration system is designed consisting of two pivotal modules. The local coarse registration is first conducted by involving both detection of interest points (IPs) and local geometric correction, which effectively utilizes the prior georeferencing information of RS images to address global geometric distortions. In the fine registration stage, the proposed SFOC is used to resist significant NRD, and to detect control points (CPs) between multimodal images by a template matching scheme. The performance of the proposed matching method has been evaluated with many different kinds of multimodal RS images. The results show its superior matching performance compared with the state-of-the-art methods. Moreover, the designed registration system also outperforms the popular commercial software (e.g., ENVI, ERDAS, and PCI) in both registration accuracy and computational efficiency. Our system is available at https://github.com/yeyuanxin110/SFOC-Multimodal_Remote_Sensing_Image_Registration_System.

A watermarking scheme for dual‐color images based on URV decomposition and image correction

In view of the fact that many watermarking schemes cannot well adapt to the occasion of image copyright protection with a large amount of information, and the copyright protection of color image is facing severe challenges, a dual-color image watermarking scheme based on URV decomposition (URVD) and image correction is designed in this paper. The two-dimensional Logistic-adjusted-Sine map is first used to generate two chaotic sequences for pixel-level scrambling and bit-level diffusion of the watermark image, and URVD is applied to watermarking scheme. After fully considering the relationship between image channels, the watermark is embedded into the maximum URVD coefficient by using quantization index modulation based on variable quantization steps. The greatest contribution of this paper is to propose a new image correction algorithm as a pretreatment of watermark extraction, which mainly includes fast detection of feature points, image correction, and sawtooth processing, and it solves some problems existing in image correction algorithms based on geometric characteristics. Experimental results show that the proposed scheme has good invisibility, strong robustness, and high security. Compared with existing schemes, the proposed scheme has better performance under the same actual embedding rate, especially stronger and more effective robustness against some common geometric attacks.

SGLBP: Subgraph‐based Local Binary Patterns for Feature Extraction on Point Clouds

AbstractExtraction for points that can outline the shape of a point cloud is an important task for point cloud processing in various applications. The topology information of the neighbourhood of a point usually contains sufficient information for detecting features, which is fully considered in this study. Therefore, a novel method for extracting feature points based on the topology information is proposed. First, an improved ‐shape technique is introduced, generating two graphs for potential feature detection and neighbourhood description, respectively. Local binary pattern (LBP) is then applied to the subgraphs, thus subgraph‐based local binary patterns (SGLBPs) are generated for encoding the topology of the neighbourhoods of points, which helps to remove non‐feature points from potential feature points. The proposed method can directly process raw point clouds and needs no prior surface reconstruction or geometric invariants computation; furthermore, the proposed method detects feature points by analysing the topologies of the neighbourhoods of points, consequently promoting the effectiveness for tiny features and the robustness to noises and non‐uniformly sampling patterns. The experimental results demonstrate that the proposed method is robust and achieves state‐of‐the‐art performance.

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Methods on Visual Positioning Based on Basketball Shooting Direction Standardisation

The existing basketball training shooting direction correction method has the problems of low correction accuracy and poor self-adaptability, and proposes a basketball training shooting direction correction method based on visual perception. A visual localisation algorithm for tracking feature points of object targets is used as the basis for the process of visual robot localisation and its effects, from camera calibration, template matching, background modelling and foreground target separation to feature point extraction, motion estimation, and Kalman filtering. An in-depth understanding and analysis of the traditional corner point detection algorithm is presented, on the basis of which improvements are proposed. An accurate tracking method based on improved Harris corner point extraction is introduced, which builds on the traditional Harris feature point detection by using the changing relationship between the gradient of the grey value of the pixels near the corner point, using simple operations and analysis to exclude some pseudocorner points and noncorner points, and further processing the retained points to derive the correct feature points. The code of this algorithm is written to finally achieve its detection effect, and compared with the traditional algorithm, it is concluded that this algorithm can then extract more accurate corner points in a shorter time, which lays the foundation for the next step of accurate basketball tracking, reflecting the practicality of this algorithm.

Feature Set for Classification of Man-Made Underwater Objects in Optical and SAS Data

This paper proposes a feature characterization method that can discriminate between man-made and natural objects on the seabed using a pair of SAS and optical images. Feature matching methods, like scale-invariant feature transform (SIFT), usually utilize gradient information to detect feature points, whereas contour-based features often use geometrical descriptors such as roughness, circularity, and solidity. However, due to the different characteristics of sonar and optical imagery, current descriptors cannot be directly applied over sonar-optical image pairs. The clarity of the water (muddy water, concentration of particles that scatter light) for optical images, and the stability of the platform, and reflections from seabed for SAS images are important parameters that distort the two images and make it hard to match between the two image sources. This paper proposes a feature characterization method that can identify man-made objects in underwater optic-SAS image pairs. Two new contour-based feature descriptors are introduced. The first is the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">entropy angle feature</i> , which aims to reflect the entropy of the angles that originate from points on the contour. The second feature is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local curve fitting</i> , which characterizes the radial-polar space of the local curve polynomial fitting. The descriptors are designed to characterize man-made objects whose contour is expected to be smoother than natural objects such as rocks. Experimental results on a real, large dataset comprised of 1,519 optic-SAS image pairs, are shown to verify that the proposed method has a high level of classification accuracy and good discrimination between man-made and natural objects.

DAN-SuperPoint: Self-Supervised Feature Point Detection Algorithm with Dual Attention Network.

In view of the poor performance of traditional feature point detection methods in low-texture situations, we design a new self-supervised feature extraction network that can be applied to the visual odometer (VO) front-end feature extraction module based on the deep learning method. First, the network uses the feature pyramid structure to perform multi-scale feature fusion to obtain a feature map containing multi-scale information. Then, the feature map is passed through the position attention module and the channel attention module to obtain the feature dependency relationship of the spatial dimension and the channel dimension, respectively, and the weighted spatial feature map and the channel feature map are added element by element to enhance the feature representation. Finally, the weighted feature maps are trained for detectors and descriptors respectively. In addition, in order to improve the prediction accuracy of feature point locations and speed up the network convergence, we add a confidence loss term and a tolerance loss term to the loss functions of the detector and descriptor, respectively. The experiments show that our network achieves satisfactory performance under the Hpatches dataset and KITTI dataset, indicating the reliability of the network.

The Identification of ECG Signals Using WT-UKF and IPSO-SVM.

The biometric identification method is a current research hotspot in the pattern recognition field. Due to the advantages of electrocardiogram (ECG) signals, which are difficult to replicate and easy to obtain, ECG-based identity identification has become a new direction in biometric recognition research. In order to improve the accuracy of ECG signal identification, this paper proposes an ECG identification method based on a multi-scale wavelet transform combined with the unscented Kalman filter (WT-UKF) algorithm and the improved particle swarm optimization-support vector machine (IPSO-SVM). First, the WT-UKF algorithm can effectively eliminate the noise components and preserve the characteristics of ECG signals when denoising the ECG data. Then, the wavelet positioning method is used to detect the feature points of the denoised signals, and the obtained feature points are combined with multiple feature vectors to characterize the ECG signals, thus reducing the data dimension in identity identification. Finally, SVM is used for ECG signal identification, and the improved particle swarm optimization (IPSO) algorithm is used for parameter optimization in SVM. According to the analysis of simulation experiments, compared with the traditional WT denoising, the WT-UKF method proposed in this paper improves the accuracy of feature point detection and increases the final recognition rate by 1.5%. The highest recognition accuracy of a single individual in the entire ECG identification system achieves 100%, and the average recognition accuracy can reach 95.17%.

Complex shearlets and rotary phase congruence tensor for corner detection

Corner detection algorithms based on multi-scale analysis attract more attention due to their promising performance. However, they only consider amplitude information, neglect phase information and partially utilize multi-scale decomposition coefficients to detect corners. This limits their detection accuracy, repeatability and localization ability. This paper describes a new multi-scale analysis based corner detector. To overcome the problems of bilateral margin responses, edge extension and lack of phase information in traditional shearlets, a novel complex shearlet transform is proposed to better localize distributed discontinuities and especially to extract phase information from geometrical features. Moreover, a new rotary phase congruence tensor is proposed to utilize all amplitude and phase information for corner detection. Its tolerances to noise and ability for corner localization are improved further by screening and normalizing the amplitude information. Experimental results demonstrate that the localization ability and detection accuracy of the proposed method are superior to current detectors, and its repeatability is generally higher than current detectors and recent machine learning based interest point detectors.

A Robust Checkerboard Corner Detection Method for Camera Calibration Based on Improved YOLOX

Camera calibration plays an important role in various optical measurement and computer vision applications. Accurate calibration parameters of a camera can give a better performance. The key step to camera calibration is to robustly detect feature points (typically in the form of checkerboard corners) in the images captured by the camera. This paper proposes a robust checkerboard corner detection method for camera calibration based on improved YOLOX deep learning network and Harris algorithm. To get high checkerboard corner detection robustness against the images with poor quality (i.e., degradation, including focal blur, heavy noise, extreme poses, and large lens distortions), we first detect the corner candidate areas through the improved YOLOX network which attention mechanism is added. Then, the Harris algorithm is performed on these areas to detect sub-pixel corner points. The proposed method is not only more accurate than the existing methods, but also robust against the types of degradation. The experimental results on different datasets demonstrate its superior robustness, accuracy, and wide effectiveness.

Strain accuracy enhancement of stereo digital image correlation for object deformation with large rotations

We propose a full-field stereo digital image correlation (DIC) strain measurement method to overcome the poor accuracy while measuring the deformation under large rotations. Such a drawback comes from the failure to consider rotation movements of the deformed objects when calculating their strain values. To address this, we first used a DIC matching algorithm combined with a rotated subset and feature point detection to obtain displacement fields. By employing a singular value decomposition method, we can then calculate rotation matrices of the strain windows before and after deformations. Finally, to eliminate the strain errors caused by rotation, we introduced the rotation matrices into the classical pointwise least squares DIC strain calculation method. Both numerical simulations and experiments are performed, and the accuracy and effectiveness of the proposed method are confirmed by the experimental results.