Robust Feature Descriptor Research Articles

DINO-Mix enhancing visual place recognition with foundational vision model and feature mixing

Using visual place recognition (VPR) technology to ascertain the geographical location of publicly available images is a pressing issue. Although most current VPR methods achieve favorable results under ideal conditions, their performance in complex environments, characterized by lighting variations, seasonal changes, and occlusions, is generally unsatisfactory. Therefore, obtaining efficient and robust image feature descriptors in complex environments is a pressing issue. In this study, we utilized the DINOv2 model as the backbone for trimming and fine-tuning to extract robust image features and employed a feature mix module to aggregate image features, resulting in globally robust and generalizable descriptors that enable high-precision VPR. We experimentally demonstrated that the proposed DINO-Mix outperforms the current state-of-the-art (SOTA) methods. Using test sets having lighting variations, seasonal changes, and occlusions such as Tokyo24/7, Nordland, and SF-XL-Testv1, our proposed architecture achieved Top-1 accuracy rates of 91.75%, 80.18%, and 82%, respectively, and exhibited an average accuracy improvement of 5.14%. In addition, we compared it with other SOTA methods using representative image retrieval case studies, and our architecture outperformed its competitors in terms of VPR performance. Furthermore, we visualized the attention maps of DINO-Mix and other methods to provide a more intuitive understanding of their respective strengths. These visualizations serve as compelling evidence of the superiority of the DINO-Mix framework in this domain.

Extraction of complex pipeline features from incomplete point clouds

Abstract Control point detection in industrial pipelines, characterized by flanges
and multiple passes, is critical for accurate virtual simulations and quality assessments
in manufacturing. This paper introduces an innovative method for detecting control
points in complex pipelines using incomplete point clouds, significantly streamlining
the process. Our approach uniquely requires only straight sections and end-plane
localizations as inputs, markedly reducing both data acquisition and processing times.
We develop a robust feature descriptor to align the CAD model with incomplete
point clouds, facilitating semantic automatic segmentation despite the lack of explicit
semantic information. Following this, geometric primitives are fitted to the segmented
clouds, and a cylindrical fitting algorithm tailored for incomplete data is introduced.
The control points are computed based on the relative positions and geometric
parameters of these primitives. Our method has been validated through experiments
on several real-world industrial complex pipelines. The results confirm that our
approach achieves a high measurement accuracy of 0.067 mm, even with point cloud
incompleteness of up to 50%. These findings highlight the effectiveness of our method in
accurately determining the geometric parameters of complex pipelines and suggesting its
considerable potential for practical applications.

FApSH: An effective and robust local feature descriptor for 3D registration and object recognition

Three-dimensional (3D) local feature descriptor plays an important role in 3D computer vision because it is widely used to build point-to-point correspondences in many 3D vision applications. However, existing descriptors are difficult to have both high descriptiveness and strong robustness to various nuisances (e.g., noise and occlusion). To address this problem, a descriptor named Fully Attribute-pairs Statistical Histogram (FApSH) is proposed. FApSH is constructed on a local reference axis (LRA), and fully encodes the relevancy information of five attributes at each neighbor point by ten attribute-pair statistics. In this process, a priori distribution-based partition strategy is proposed for evenly distributing the attribute values of all neighbor points, and a radial distance-based histogram assignment method is proposed to improve the robustness to noise and outliers. The proposed methods are rigorously evaluated on six benchmark datasets with different application scenarios and nuisances. The results show that FApSH has high descriptiveness and strong robustness. It obviously outperforms the existing handcrafted descriptors, and is comparable to some superior learning-based descriptors. The results also show that the proposed priori distribution-based partition strategy significantly reduces the length and also improves the descriptiveness of FApSH, and the proposed radial distance-based histogram assignment method improves the robustness of FApSH on various datasets.

An elliptical sampling based fast and robust feature descriptor for image matching

An elliptical sampling based fast and robust feature descriptor for image matching

OD-Net: Orthogonal descriptor network for multiview image keypoint matching

Learning robust feature descriptors from multiple views plays an important role in image matching and its downstream tasks. Due to various uncertainty factors from illumination to viewpoint, robust feature matching remains a challenging task; the generalization property of learned feature descriptors has also remained poorly understood. Our intuition is that ensemble learning can assemble a collection of feature descriptors by mining richer information for each pixel, which can lead to improved robustness and generalization properties. In this paper, we propose a new image feature description method named Orthogonal Descriptor Network (OD-Net), which describes a pixel by a multi-branch structure and then fuses them. To avoid the model being trapped into an ill-posed solution and encourage the network to mine complementary information, we have designed an orthogonality constraint and modeled it as a novel loss function. It is worth noting that the idea of orthogonal feature extraction is general and can be easily plugged into many existing frameworks. Extensive experiments have been carried out to show that our OD-Net can produce better results than the current state-of-the-art on a variety of image-matching tasks and evaluation metrics.

Optimal multi-kernel SVM classifier with rotation, illumination and scale invariant hybrid DWT-Shearlet based GLCM feature descriptor and its application to face recognition

In computer vision, we must handle with the various structural aspects of image or video data. The texture is one of the most important aspects of this type of data, which is utilised to identify objects or regions of interest in an image. As imaging conditions change, textures inside actual images significantly change in brightness, contrast, size, and skew. To recognise textures in real-world images, a similarity measure that is invariant to these features must be used. Existing recognition techniques did not perform well due to issues such as illumination, scale, and subject rotation. To address this issue, invariant feature representation methods are being developed to generate features that are insensitive to such variations. In this paper, we proposed a robust hybrid feature descriptor and predicted the faces under illumination, scale, and pose variations using an optimum multi-kernel support vector machine. Additionally, the suggested robust hybrid feature descriptor is enhanced by combining a hybrid transform composed of discrete wavelet and discrete shearlet transforms with some image statistical textural data. The proposed face recognition system is implemented in MATLAB, and analysed using various parameters to show proposed methods improved performance as compared to the state of the art methods.

Identifying License Plates in Distorted Vehicle Images: Detecting Distorted Vehicle Licence Plates Using a Novel Preprocessing Methods With Hybrid Feature Descriptors

Intelligent transportation systems (ITSs) play a key role in many people’s lives with different aspects. Most of the ITS applications include a system for detecting license plates (LPs) in transportation vehicles. A new framework for such detection systems is introduced in this article. It includes a novel technique for preprocessing, extraction, and detection stages to identify LPs from distorted vehicle images. An efficient preprocessing method is developed in this study. An enhanced, contrast-limited adaptive histogram equalization method for filtering the unwanted LP features is proposed. At the features-extraction stage, strong hybrid features from extended local binary patterns with a median robust pattern descriptor and speeded-up robust feature descriptor are applied to extract complicated features from LP areas. Those hybrid features can enhance the useful information, and the detection system performed well under difficult scenarios. In the detection stage, the trained model by an extreme learning machine (ELM) classifier, with mean-shift algorithm, is used as a detector to decide output results. Performances of the proposed framework were compared with other classifiers using true- and false-positive rates. The system’s performance improvements by the proposed method were also compared with our previous methods and the existing detection methods in the literature. The experiments on an English car LP’s database and other language vehicle images showed that the proposed method made significant improvements to the accuracy and runtime speed for the detection system under difficult image conditions.

Texture-Mapping Error Removal Based on the BRIEF Operator in Image-Based Three-Dimensional Reconstruction

In image-based three-dimensional (3D) reconstruction, texture-mapping techniques can give the model realistic textures. When the geometric surface in some regions is not reconstructed, such as for moving cars, powerlines, and telegraph poles, the textures in the corresponding image are textured to other regions, resulting in errors. To solve this problem, this letter proposes an image consistency detection method based on the Binary Robust Independent Elementary Features (BRIEF) descriptor. The method is composed of two parts. First, each triangle in the mesh and its neighboring triangles are sampled uniformly to obtain sampling points. Then, these sampled points are projected into the visible image of the triangle, and the corresponding sampled points and their RGB color values are obtained on the corresponding image. Based on the sampled points on these images, a BRIEF descriptor is calculated for each image corresponding to that triangle. In the second step, the Hamming distance between these BRIEF descriptors is calculated, outliers are removed according to the method, and noisy images are also removed. In addition, we propose adding semantic information to Markov energy optimization to reduce errors further. The two methods effectively reduced errors in texture mapping caused by objects not reconstructed, improving the texture quality of 3D models.

Potential leukaemia classification using ISURF-DLCNN from blood cell image

There will be a total of 412,000 persons across the world who are diagnosed with leukaemia, with acute lymphoblastic leukaemia accounting for around 12% of all cases. As a consequence of this, leukaemia detection at an earlier stage has the potential to save the lives of millions of individuals. The identification of leukaemia using deep learning algorithms is the primary emphasis of this paper, along with blood cell counts. The photos are preprocessed using median filters, and then the K-means clustering (KMC) algorithm is used to split the data into its constituent parts. After that, the gathered features are fed into a deep learning convolutional neural network (DLCNN) in order to perform classification utilising an upgraded, speeded-up, and more robust feature descriptor (ISURF). The proposed technique achieved an accuracy rate of 99 percent while requiring a very low amount of effort, and it outperformed conventional approaches in terms of overall performance.

Learning a Task-Specific Descriptor for Robust Matching of 3D Point Clouds

Existing learning-based point feature descriptors are usually task-agnostic, which pursue describing the individual 3D point clouds as accurate as possible. However, the matching task aims at describing the corresponding points consistently across different 3D point clouds. Therefore these too accurate features may play a counterproductive role due to the inconsistent point feature representations of correspondences caused by the unpredictable noise, partiality, deformation, \etc, in the local geometry. In this paper, we propose to learn a robust task-specific feature descriptor to consistently describe the correct point correspondence under interference. Born with an Encoder and a Dynamic Fusion module, our method EDFNet develops from two aspects. First, we augment the matchability of correspondences by utilizing their repetitive local structure. To this end, a special encoder is designed to exploit two input point clouds jointly for each point descriptor. It not only captures the local geometry of each point in the current point cloud by convolution, but also exploits the repetitive structure from paired point cloud by Transformer. Second, we propose a dynamical fusion module to jointly use different scale features. There is an inevitable struggle between robustness and discriminativeness of the single scale feature. Specifically, the small scale feature is robust since little interference exists in this small receptive field. But it is not sufficiently discriminative as there are many repetitive local structures within a point cloud. Thus the resultant descriptors will lead to many incorrect matches. In contrast, the large scale feature is more discriminative by integrating more neighborhood information. ...

Registration of multi-modal images under a complex background combining multiscale features extraction and semantic segmentation.

Multi-modal imaging technology has a very broad application value in target recognition and other fields, and image registration is one of its key technologies. In this paper, a multi-modal image registration algorithm that combines multiscale features extraction and semantic segmentation is proposed to achieve accurate registration of polarized images and near-infrared images under complex backgrounds. A classical convolutional neural network ResNet is employed to capture the robust feature descriptors, and a convolutional neural network with an attention mechanism is trained to filter out the irrelevant feature points. Further, the two multi-modal images can be further registered. The experimental results show the feasibility and effectiveness of the proposed method.

RI-MFM: A Novel Infrared and Visible Image Registration with Rotation Invariance and Multilevel Feature Matching

In the past ten years, multimodal image registration technology has been continuously developed, and a large number of researchers have paid attention to the problem of infrared and visible image registration. Due to the differences in grayscale distribution, resolution and viewpoint between two images, most of the existing infrared and visible image registration methods are still insufficient in accuracy. To solve such problems, we propose a new robust and accurate infrared and visible image registration method. For the purpose of generating more robust feature descriptors, we propose to generate feature descriptors using a concentric-circle-based feature-description algorithm. The method enhances the description of the main direction of feature points by introducing centroids, and, at the same time, uses concentric circles to ensure the rotation invariance of feature descriptors. To match feature points quickly and accurately, we propose a multi-level feature-matching algorithm using improved offset consistency for matching feature points. We redesigned the matching algorithm based on the offset consistency principle. The comparison experiments with several other state-of-the-art registration methods in CVC and homemade datasets show that our proposed method has significant advantages in both feature-point localization accuracy and correct matching rate.

Automatic Detection of Microaneurysms in OCT Images Using Bag of Features.

Diabetic retinopathy (DR) caused by diabetes occurs as a result of changes in the retinal vessels and causes visual impairment. Microaneurysms (MAs) are the early clinical signs of DR, whose timely diagnosis can help detecting DR in the early stages of its development. It has been observed that MAs are more common in the inner retinal layers compared to the outer retinal layers in eyes suffering from DR. Optical coherence tomography (OCT) is a noninvasive imaging technique that provides a cross-sectional view of the retina, and it has been used in recent years to diagnose many eye diseases. As a result, this paper attempts to identify areas with MA from normal areas of the retina using OCT images. This work is done using the dataset collected from FA and OCT images of 20 patients with DR. In this regard, firstly fluorescein angiography (FA) and OCT images were registered. Then, the MA and normal areas were separated, and the features of each of these areas were extracted using the Bag of Features (BOF) approach with the Speeded-Up Robust Feature (SURF) descriptor. Finally, the classification process was performed using a multilayer perceptron network. For each of the criteria of accuracy, sensitivity, specificity, and precision, the obtained results were 96.33%, 97.33%, 95.4%, and 95.28%, respectively. Utilizing OCT images to detect MAs automatically is a new idea, and the results obtained as preliminary research in this field are promising.

Online product sentiment analysis using random evolutionary whale optimization algorithm and deep belief network

• Image and text normalization techniques are used to improve the data quality. • Used MLBP , SURF, TF-IDF and LSA techniques for feature extraction. • Used RE-WOA to reduce the dimension of the extracted feature vectors. • Used DBN classifier to classify the user's sentiments of the online products. In the recent decades, the online product sentiment analysis is an emerging research topic that assists the customers to take better decisions on purchasing the products and to achieve better sales of the products. Recently, several machine learning techniques are experimented on many datasets for analyzing the customer's sentiments through online portals. Still, the customers are struggling to obtain the aspect sentiments expressed by other customers, particularly in the amazon websites. Therefore, a novel automated model is proposed in this manuscript for an effective online product sentiment analysis. After collecting the multimodal data from the amazon websites, the image and data normalization techniques are employed for better understanding of the collected data. Further, the feature extraction is performed by utilizing Latent Semantic Analysis (LSA), Term Frequency- Inverse Document Frequency (TF-IDF), Modified Local Binary Pattern (MLBP), and Speeded Up Robust Features (SURF) descriptors for extracting the textual and visual feature vectors from the preprocessed data. Finally, the Random Evolutionary Whale Optimization Algorithm (REWOA) and Deep Belief Network (DBN) classifier are integrated for feature vector optimization and sentiment classification. By using feature optimization, the system complexity and running time of the classifier is improved. The experimental investigation states that the developed REWOA-DBN model achieved 96.86% of classification accuracy, which is better compared to other optimizers and classifiers

Multi-scale super-pixels-based passive forensics for copy-move forgery using local characteristics

Copy-move forgery is the most common kind of tampering technique for digital images. This paper presents a novel hybrid approach, which uses the Speeded-Up Robust Features (SURF) point and characteristics of Local Feature Regions (LFRs) matching. First, the multi-scale super-pixels algorithm adaptively divides the suspect image into irregular blocks according to the texture level of host images. Then, the improved SURF detector is adopted in extracting feature points from each super-pixel and the feature point threshold is related to the entropy of each super-pixel block. Next, LFRs are defined, and a robust feature descriptor is extracted from each LFR as a vector field. Last, the matching LFRs are found by using Euclidean locality sensitive hashing; the removal of falsely matched pairs is realised by using the Random Sample Consensus (RANSAC) algorithm. Comparing with the leading-edge block-matching methods and point-based methods, our method can produce far better detection results.

You Only Train Once: Learning General and Distinctive 3D Local Descriptors.

Extracting distinctive, robust, and general 3D local features is essential to downstream tasks such as point cloud registration. However, existing methods either rely on noise-sensitive handcrafted features, or depend on rotation-variant neural architectures. It remains challenging to learn robust and general local feature descriptors for surface matching. In this paper, we propose a new, simple yet effective neural network, termed SpinNet, to extract local surface descriptors which are rotation-invariant whilst sufficiently distinctive and general. A Spatial Point Transformer is first introduced to embed the input local surface into an elaborate cylindrical representation (SO(2) rotation-equivariant), further enabling end-to-end optimization of the entire framework. A Neural Feature Extractor, composed of point-based and 3D cylindrical convolutional layers, is then presented to learn representative and general geometric patterns. An invariant layer is finally used to generate rotation-invariant feature descriptors. Extensive experiments on both indoor and outdoor datasets demonstrate that SpinNet outperforms existing state-of-the-art techniques by a large margin. More critically, it has the best generalization ability across unseen scenarios with different sensor modalities.

KAZE-SAR: SAR Image Registration Using KAZE Detector and Modified SURF Descriptor for Tackling Speckle Noise

Synthetic aperture radar (SAR) image registration is still a challenging task in remote sensing. The scale-invariant feature transform (SIFT) method and its extensions are most widely used feature detectors in SAR images. However, due to the presence of nonlinear speckle noise, some wrong features are chosen which directly influence the feature matching process. In this article, for the first time, the KAZE algorithm with a modified version of speeded-up robust features (SURF) descriptor is used for SAR image registration. It uses discrete stochastic second order nonlinear partial differential equations (PDEs) to model the SAR image’s edge structure. KAZE uses nonlinear diffusion filtering to build up the scale levels of the SIFT descriptor. It preserves the edges while simultaneously decreasing the speckle noise in smoothing the image. Therefore, it captures the exact location of features in down-sampling and filtering stages of SIFT. Experimental results show that our proposed method has increased the localization accuracy and distinctiveness of feature extraction through the use of KAZE detector. Our method is also able to handle challenging cases in different image sources, different view angles, different image times, complex affine mapping, presence of blurring, and noise.

Binocular Vision Positioning Method for Safety Monitoring of Solitary Elderly

In nowadays society, the safety of the elderly population is becoming a pressing concern, especially for those who live alone. There might be daily risks such as accidental falling or treatment attack on them. Aiming at these problems, indoor positioning could be a critical way to monitor their states. With the rapidly development of the imaging techniques, wearable and portable cameras are very popular, which could be set on human individual. And in view of the advantages of the visual positioning, the authors propose a binocular visual positioning algorithm to real-timely locate the elderly indoor. In this paper, the imaging model has been established with the corrected image data from the binocular camera; then feature extraction has been completed to provide reference to adjacent image matching based on the binary robust independent elementary feature (BRIEF) descriptor, finally the camera movement and the states of the elderly have been estimated to distinguish their falling risk. In the experiments, the real-sense D435i sensors were adopted as the binocular cameras to obtain indoor images, and three experimental scenarios have been carried out to test the proposed method. The results show that the proposed algorithm can effectively locate the elderly indoor and improve the real-time monitoring capability.

Multi-Level Feature Aggregation-Based Joint Keypoint Detection and Description

Image keypoint detection and description is a popular method to find pixel-level connections between images, which is a basic and critical step in many computer vision tasks. The existing methods are far from optimal in terms of keypoint positioning accuracy and generation of robust and discriminative descriptors. This paper proposes a new end-to-end self-supervised training deep learning network. The network uses a backbone feature encoder to extract multi-level feature maps, then performs joint image keypoint detection and description in a forward pass. On the one hand, in order to enhance the localization accuracy of keypoints and restore the local shape structure, the detector detects keypoints on feature maps of the same resolution as the original image. On the other hand, in order to enhance the ability to percept local shape details, the network utilizes multi-level features to generate robust feature descriptors with rich local shape information. A detailed comparison with traditional feature-based methods Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF) and deep learning methods on HPatches proves the effectiveness and robustness of the method proposed in this paper.

A Text Tracking Method using Maximally Stable Extremal Regions and Speeded up Robust Features

Content-based image retrieval is an active research area because of vast applications of image and video collections. Text embedded in video can provide significant contribution in identifying the contents of the multimedia data and facilitate the process of video indexing, retrieval and analysis. Text tracking is a vital part of text extraction process. It can speed up the text extraction process and also improves the text localization accuracy for videos. This paper proposes a novel approach for text tracking in videos. This method experimentally proposes Maximally Stable Extremal Regions (MSER) and Speeded Up Robust Features (SURF) descriptors for text tracking in videos. MSER is used for interest point detection as Extremal regions are affine invariant, so it is a suitable option for text tracking which can undergo scale, rotation and translation changes. SURF is chosen as a feature descriptor because of its scale and rotation invariance, speed and robustness. Proposed technique is testified on two datasets; the first is designed to test the tracking methodology as part of this research and second is publicly available Youtube Video Text(YVT) dataset. Succeeding analysis on diverse datasets endures verification to the fact that the projected technique demonstrates visible improvements to text tracking in videos