Binary Feature Descriptors Research Articles

B3D-EAR: Binarized 3D descriptors for ear-based human recognition

Traditional 3D feature descriptors often utilize real-valued vectors, posing challenges in terms of computational complexity and space constraints during matching. This study introduces a novel approach for generating binary 3D feature descriptors using correntropy, an online estimate of Renyi’s quadratic entropy. By employing this technique, the efficiency of state-of-the-art 3D descriptors is significantly enhanced. The effectiveness of the proposed methodology is demonstrated through evaluations on two datasets, UND-J2 and an in-house dataset, comprising 3D scans of human ears. Efficient algorithms for matching keypoints and their binary descriptors are employed, addressing both space and time complexities. The matching process is assessed using distance metrics and iterative closest point (ICP) alignment. Experimental results reveal that the method achieves a recognition performance of 98.62% with an equal error rate (EER) of 1.54%, comparable to state-of-the-art techniques. This highlights the efficacy of the method in generating binary descriptors without compromising accuracy.

Learning modality-invariant binary descriptor for crossing palmprint to palm-vein recognition

Palmprint has shown promising potential for biometrics recognition due to its excellent contactless and hygienic properties. However, most existing methods focus only on feature learning of unimodal palmprint images leading to the challenge of palmprint recognition crossing different modalities. In this paper, we propose a modality-invariant binary features learning (MIBFL) method for crossing palmprint to palm-vein recognition, where palm images are captured under visible and invisible near-infrared illumination, respectively. We first map the multiple modal palm images into their high-dimensional alignment representation to reduce the impact of misalignment and noise between different image modalities. Then, we simultaneously learn the discriminative features from different modalities of alignment images via matrix factorization by enforcing the orthogonal and balanced constraints. Lastly, we jointly learn a pair of encoding functions to project multi-modal palm features into the common binary feature descriptor for crossing palmprint to palm-vein recognition. Experimental results on the widely used PolyU multi-spectral palmprint database are presented to demonstrate the effectiveness of the proposed MIBFL method.

Illumination-Insensitive Binary Descriptor for Visual Measurement Based on Local Inter-Patch Invariance

Binary feature descriptors have been widely used in various visual measurement tasks, particularly those with limited computing resources and storage capacities. Existing binary descriptors may not perform well for long-term visual measurement tasks due to their sensitivity to illumination variations. It can be observed that when image illumination changes dramatically, the relative relationship among local patches mostly remains intact. Based on the observation, consequently, this study presents an illumination-insensitive binary (IIB) descriptor by leveraging the local inter-patch invariance exhibited in multiple spatial granularities to deal with unfavorable illumination variations. By taking advantage of integral images for local patch feature computation, a highly efficient IIB descriptor is achieved. It can encode scalable features in multiple spatial granularities, thus facilitating a computationally efficient hierarchical matching from coarse to fine. Moreover, the IIB descriptor can also apply to other types of image data, such as depth maps and semantic segmentation results, when available in some applications. Numerical experiments on both natural and synthetic datasets reveal that the proposed IIB descriptor outperforms state-of-the-art binary descriptors and some testing float descriptors. The proposed IIB descriptor has also been successfully employed in a demo system for long-term visual localization. The code <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of the IIB descriptor will be publicly available.

Feature-Based Direct Tracking and Mapping for Real-Time Noise-Robust Outdoor 3D Reconstruction Using Quadcopters

In this work, we focus on real-time 3D reconstruction or localization and mapping for outdoor scene using an aerial vehicle called quadcopter. Quadcopter provides the advantages of high flexibility and wide view field in spatial movement. However, existing feature-based and direct methods (using dense or semi-dense approach) are not suitable for outdoor environment, in which multiple challenging scenarios arise such as lighting variance, jittering views, high-speed and non-smooth flight trajectory. The main reason is that the existing methods rely on the assumption of brightness constancy across multiple images and only raw pixel intensities are employed for direct image alignment. In order to tackle these scenarios, a novel method called Feature-based Direct Tracking and Mapping (FDTAM) is proposed, which i) incorporates an efficient binary feature descriptor into direct image alignment module to tackle the challenging scenarios, such as drifting issue under lighting variance problem; ii) applies semi-dense approach to obtain high reconstruction quality; iii) provides a framework with low computational complexity for real-time reconstruction. Compared to other state-of-the-art feature-based and direct methods, our proposed method is shown to tackle the challenging scenarios and improve the accuracy and robustness even in CPU (rather than GPU) platform.

An Improved ASIFT Image Feature Matching Algorithm Based on POS Information.

The affine scale-invariant feature transform (ASIFT) algorithm is a feature extraction algorithm with affinity and scale invariance, which is suitable for image feature matching using unmanned aerial vehicles (UAVs). However, there are many problems in the matching process, such as the low efficiency and mismatching. In order to improve the matching efficiency, this algorithm firstly simulates image distortion based on the position and orientation system (POS) information from real-time UAV measurements to reduce the number of simulated images. Then, the scale-invariant feature transform (SIFT) algorithm is used for feature point detection, and the extracted feature points are combined with the binary robust invariant scalable keypoints (BRISK) descriptor to generate the binary feature descriptor, which is matched using the Hamming distance. Finally, in order to improve the matching accuracy of the UAV images, based on the random sample consensus (RANSAC) a false matching eliminated algorithm is proposed. Through four groups of experiments, the proposed algorithm is compared with the SIFT and ASIFT. The results show that the algorithm can optimize the matching effect and improve the matching speed.

Point Cloud Data Registration Based on Binary Feature Descriptors

Point Cloud Data Registration Based on Binary Feature Descriptors

Binocular Vision Positioning Method for Safety Monitoring of Solitary Elderly

In nowadays society, the safety of the elderly population is becoming a pressing concern, especially for those who live alone. There might be daily risks such as accidental falling or treatment attack on them. Aiming at these problems, indoor positioning could be a critical way to monitor their states. With the rapidly development of the imaging techniques, wearable and portable cameras are very popular, which could be set on human individual. And in view of the advantages of the visual positioning, the authors propose a binocular visual positioning algorithm to real-timely locate the elderly indoor. In this paper, the imaging model has been established with the corrected image data from the binocular camera; then feature extraction has been completed to provide reference to adjacent image matching based on the binary robust independent elementary feature (BRIEF) descriptor, finally the camera movement and the states of the elderly have been estimated to distinguish their falling risk. In the experiments, the real-sense D435i sensors were adopted as the binocular cameras to obtain indoor images, and three experimental scenarios have been carried out to test the proposed method. The results show that the proposed algorithm can effectively locate the elderly indoor and improve the real-time monitoring capability.

QBB: Quantile-Based Binarization of 3D Point Cloud Descriptors

Local 3D point feature descriptors play an important role in many areas of computer vision, such as object recognition, registration, etc. There are many well-functioning feature descriptors, but they are typically real-valued and multidimensional vectors, leading to high computational complexity in nearest neighbor searches. To overcome this challenge, methods binarizing real-valued descriptors have emerged. In this paper, we first investigate the available binarization methods and standalone binary feature descriptors and show that existing binarization techniques cannot generally achieve good performance for arbitrary feature descriptors. To remedy this problem, we propose a new binarization method called quantile-based binarization (QBB) that can be applied to any real-valued feature descriptors. It analyses the distribution of feature descriptors that is then used to form meaningful groups along each dimension. To this end, QBB computes quantiles of the empirical distribution and the interval lengths (bin sizes) defined by quantile boundaries. Finally, it assigns a binary code to each group and concatenate them to get the final binary descriptor. QBB is able to adaptively compute the number of bits based on a capacity constraint, i.e., with the appropriate capacity setting, the resulting binary descriptor can be used on devices with lower computational power. We evaluate the descriptiveness of well-known descriptors binarized by QBB and compare them to state-of-the-art methods. According to our evaluation, QBB is able to create binary descriptors whose descriptiveness is closer to the real-valued descriptors than prior approaches. Finally, we also show that QBB can even compete with standalone binary feature descriptors.

High dynamic range 3D measurements based on space-time speckle correlation and color camera.

Structured light (SL) based three-dimensional (3D) measurement struggles to estimate high dynamic range (HDR) scenes, where both high and low reflectivity parts exist simultaneously. This paper proposes a method through the joint design and optimization of hardware and algorithms, in which only four frames are required to realize the 3D reconstruction of HDR scenes. The height information of each sub-area in the scene under test can be encoded effectively by temporally projecting two sets of complementary speckle patterns onto target surface. To decode the corresponding patterns captured by the cameras, we design a stereo matching strategy consisting of space-time binary feature (ST-BIF) descriptor preliminary screening and zero-mean normalized cross-correlation (ST-ZNCC) final retrieval. The ST-BIF descriptor based on neighborhood comparison is designed to describe the space-time relative intensity change of projected speckles. Besides the HDR adaptability, the ST-BIF descriptor can effectively improve the matching speed. In addition, the measurable dynamic range can be further improved by fusing all channel disparities as evaluated results, benefitting from the different response of R, G and B channels in color camera to monochromatic light. Experiments are conducted to demonstrate the feasibility of the proposed method. The results indicate that our method achieves the root mean square error 0.2516mm (vs. 1.0668 by commonly used ZNCC) and an average coverage rate up to 94.87% (vs. 93.35% by commonly used ZNCC). Furthermore, the experimental results show that the proposed method can achieve 3D reconstruction of HDR scenes including specular reflection region.

FSD-BRIEF: A Distorted BRIEF Descriptor for Fisheye Image Based on Spherical Perspective Model.

Fisheye images with a far larger Field of View (FOV) have severe radial distortion, with the result that the associated image feature matching process cannot achieve the best performance if the traditional feature descriptors are used. To address this challenge, this paper reports a novel distorted Binary Robust Independent Elementary Feature (BRIEF) descriptor for fisheye images based on a spherical perspective model. Firstly, the 3D gray centroid of feature points is designed, and the position and direction of the feature points on the spherical image are described by a constructed feature point attitude matrix. Then, based on the attitude matrix of feature points, the coordinate mapping relationship between the BRIEF descriptor template and the fisheye image is established to realize the computation associated with the distorted BRIEF descriptor. Four experiments are provided to test and verify the invariance and matching performance of the proposed descriptor for a fisheye image. The experimental results show that the proposed descriptor works well for distortion invariance and can significantly improve the matching performance in fisheye images.

An improved bioinspired cognitive map-building system based on episodic memory recognition

Before the cognitive map is generated through the fire of the rodent hippocampal spatial cells, mammals can obtain the outside knowledge through the visual information, which comes from the eyeball to the brain. The information is encoded and transferred to the two regions of the brain based on the fact of biophysiological research, which are known as “what” loop and “where” loop. In this article, we simulate an episodic memory recognition unit consisting of the integration of two-loop information, which is applied to building the accurate bioinspired spatial cognitive map of real environments. We employ the visual bag of word algorithm based on oriented Feature from Accelerated Segment Test and rotated Binary Robust Independent Elementary Features feature to build the “what” loop and the hippocampal spatial cells cognitive model, which comes from the front-end visual information input system to build the “where” loop. At the same time, the environmental cognitive map is a topological map containing information about place cell competition firing rate, oriented Feature from Accelerated Segment Test and rotated Binary Robust Independent Elementary Features feature descriptor, similarity of image retrieval, and relative location of cognitive map nodes. The simulation experiments and physical experiments in a mobile robot platform have been done to verify the environmental adaptability and robustness of the algorithm. This proposing algorithm would provide a foundation for further research on bioinspired navigation of robots.

Optimization and Implementation of Synthetic Basis Feature Descriptor on FPGA

Feature detection, description, and matching are crucial steps for many computer vision algorithms. These steps rely on feature descriptors to match image features across sets of images. Previous work has shown that our SYnthetic BAsis (SYBA) feature descriptor can offer superior performance to other binary descriptors. This paper focused on various optimizations and hardware implementation of the newer and optimized version. The hardware implementation on a field-programmable gate array (FPGA) is a high-throughput low-latency solution which is critical for applications such as high-speed object detection and tracking, stereo vision, visual odometry, structure from motion, and optical flow. We compared our solution to other hardware designs of binary descriptors. We demonstrated that our implementation of SYBA as a feature descriptor in hardware offered superior image feature matching performance and used fewer resources than most binary feature descriptor implementations.

Real-Time Image Stabilization Method Based on Optical Flow and Binary Point Feature Matching

The strap-down missile-borne image guidance system can be easily affected by the unwanted jitters of the motion of the camera, and the subsequent recognition and tracking functions are also influenced, thus severely affecting the navigation accuracy of the image guidance system. So, a real-time image stabilization technology is needed to help improve the image quality of the image guidance system. To satisfy the real-time and accuracy requirements of image stabilization in the strap-down missile-borne image guidance system, an image stabilization method based on optical flow and image matching with binary feature descriptors is proposed. The global motion of consecutive frames is estimated by the pyramid Lucas-Kanade (LK) optical flow algorithm, and the interval frames image matching based on fast retina keypoint (FREAK) algorithm is used to reduce the cumulative trajectory error. A Kalman filter is designed to smooth the trajectory, which is conducive to fitting to the main motion of the guidance system. Simulations have been carried out, and the results show that the proposed algorithm improves the accuracy and real-time performance simultaneously compared to the state-of-art algorithms.

Binary GAN based Approach for Unsupervised Loop Closure Detection in Autonomous Unmanned Systems

Abstract Inspired by generative adversarial network (GAN), we propose a novel unsupervised approach for loop closure detection in autonomous unmanned systems. A binary GAN model dedicated to mobile application scenarios is designed to obtain binary feature descriptors, which are further incorporated into the most commonly used Bag of Visual Words (BoVW) model for loop closure detection. Compared with those hand-crafted features like SIFT and ORB, the performance of loop closure detection in complex environments with strong viewpoint and condition changes can be greatly improved. Compared with existing supervised approach based on convolutional neural network like AlexNet and AMOSNet, the cost-expensive task of supervised data annotation is totally avoided, which make the proposed approach more practical.

EMCM: A Novel Binary Edge-Feature-Based Maximum Clique Framework for Multispectral Image Matching

Seeking reliable correspondence between multispectral images is a fundamental and important task in computer vision. To overcome the nonlinearity problem occurring in multispectral image matching, a novel, edge-feature-based maximum clique-matching frame (EMCM) is proposed, which contains three main parts: (1) a novel strong edge binary feature descriptor, (2) a new correspondence-ranking algorithm based on keypoint distinctiveness analysis algorithms in the feature space of the graph, and (3) a false match removal algorithm based on maximum clique searching in the correspondence space of the graph considering both position and angle consistency. Extensive experiments are conducted on two standard multispectral image datasets with respect to the three parts. The feature-matching experiments suggest that the proposed feature descriptor is of high descriptiveness, robustness, and efficiency. The correspondence-ranking experiments validate the superiority of our correspondences-ranking algorithm over the nearest neighbor algorithm, and the coarse registration experiments show the robustness of EMCM with varied interferences.

Hardware Friendly Robust Synthetic Basis Feature Descriptor

Finding corresponding image features between two images is often the first step for many computer vision algorithms. This paper introduces an improved synthetic basis feature descriptor algorithm that describes and compares image features in an efficient and discrete manner with rotation and scale invariance. It works by performing a number of similarity tests between the feature region surrounding the feature point and a predetermined number of synthetic basis images to generate a feature descriptor that uniquely describes the feature region. Features in two images are matched by comparing their descriptors. By only storing the similarity of the feature region to each synthetic basis image, the overall storage size is greatly reduced. In short, this new binary feature descriptor is designed to provide high feature matching accuracy with computational simplicity, relatively low resource usage, and a hardware friendly design for real-time vision applications. Experimental results show that our algorithm produces higher precision rates and larger number of correct matches than the original version and other mainstream algorithms and is a good alternative for common computer vision applications. Two applications that often have to cope with scaling and rotation variations are included in this work to demonstrate its performance.

Spatiotemporal local compact binary pattern for background subtraction in complex scenes

A variety of binary feature descriptors such as local binary pattern (LBP) and its variations have recently attracted considerable attention for modelling backgrounds, due to their robustness and strong discriminatory power. However, most existing binary feature descriptors fail to model complex scenes due to their sensitivity to noise. In this paper, we propose an effective local compact binary descriptor for background modelling. For each image, local compact binary patterns (LCBPs) are first extracted by computing a number of low-dimensional pixel difference vectors (PDVs). Then, the LCBP is extended to the spatiotemporal domain taking into account the temporal persistence of pixels, and a novel local compact binary descriptor, STLCBP, is proposed. Multiple color spaces are also considered in order to separate foreground from background pixels accurately. Finally, a joint domain-range adaptive kernel density estimate (KDE) model is used to estimate the background and foreground scores by combining texture features with color features. Experimental results on two well-known datasets, I2R and CDnet2014, demonstrate that the proposed approach significantly outperforms many state-of-the-art methods and works effectively on a wide range of complex videos.

Compact and Low-Complexity Binary Feature Descriptor and Fisher Vectors for Video Analytics.

In this paper, we propose a compact and low-complexity binary feature descriptor for video analytics. Our binary descriptor encodes the motion information of a spatio-temporal support region into a low-dimensional binary string. The descriptor is based on a binning strategy and a construction that binarizes separately the horizontal and vertical motion components of the spatio-temporal support region. We pair our descriptor with a novel Fisher Vector (FV) scheme for binary data to project a set of binary features into a fixed length vector in order to evaluate the similarity between feature sets. We test the effectiveness of our binary feature descriptor with FVs for action recognition, which is one of the most challenging tasks in computer vision, as well as gait recognition and animal behavior clustering. Several experiments on the KTH, UCF50, UCF101, CASIA-B, and TIGdog datasets show that the proposed binary feature descriptor outperforms the state-of-the-art feature descriptors in terms of computational time and memory and storage requirements. When paired with FVs, the proposed feature descriptor attains a very competitive performance, outperforming several state-of-the-art feature descriptors and some methods based on convolutional neural networks.

Single Sample Face Recognition Using Multicross Pattern and Learning Discriminative Binary Features

Binary feature descriptors, require considerable amount of information to be applicable in wide appearance variations, which contradicts the single sample per person (SSPP) problem. To address this challenge, a novel binary feature learning method called discriminative binary feature mapping is presented. Then, based on a number of precisely selected objectives, a feature mapping is learned by projecting all of the extracted vectors to a lower-dimensional feature space. The resulting feature vectors are then used to obtain a holistic face representation based on dictionary learning. Extensive experimental results show that the proposed method is able to obtain superior performance.

Random Binary Search Trees for approximate nearest neighbour search in binary spaces

Approximate nearest neighbour (ANN) search is one of the most important problems in numerous computer science applications, including data mining, machine learning and computer vision. In this paper, we address the problem of ANN for high-dimensional binary vectors and we propose a simple yet powerful search method that is based on Random Binary Search Trees (RBST). Our method is validated on a dataset of 1.25M binary local feature descriptors obtained from a real-life image-based localisation system provided by Google as a part of Project Tango (now known as ARCore). The results of an extensive evaluation of our method performed on this dataset against the state-of-the-art ANN algorithms, such as Locality Sensitive Hashing, Uniform LSH and Multi-probe LSH, show the superiority of our method in terms of retrieval precision with a performance boost of over 20%.