Binary Descriptors Research Articles

RLDB: Robust Local Difference Binary Descriptor with Integrated Learning-based Optimization

RLDB: Robust Local Difference Binary Descriptor with Integrated Learning-based Optimization

Combining multi-channel color space with local binary co-occurrence feature descriptors for accurate smoke detection from surveillance videos

Fire is one of the most devastating hazards that can cause serious damage to human life, health and property. As smoke is often an initial sign of fire, smoke detection using surveillance cameras is key to providing early alarm in open space environments. In this paper, we propose a new feature extraction method that combines local binary patterns with co-occurrence of texture features in RGB color space to characterize the diverse manifestations of smoke. The proposed RGB color based Local Binary Co-occurrence Patterns (RGB_LBCoP) extracts smoke features from candidate smoke regions which are extracted by Fuzzy C-Means (FCM) algorithm. Subsequently, Support Vector Machine (SVM) is used for training and classification based on these features. The major benefit of the proposed feature descriptor is the ability to incorporate local and global texture properties of smoke along with color information. This property enables the detection of smoke in complex environments and provides insensitivity to illumination changes. For validation, performance of the proposed method is compared with other LBP variants and Gray-level co-occurrence matrix (GLCM). Experimental analysis on publicly available smoke video datasets demonstrates that the proposed algorithm outperforms the other methods by achieving an average True Positive Rate (TPR) of 92.02%.

GMD: geometric model-based local binary descriptor

Many researchers focused on binary descriptor due to its low computational power. It is an adequate solution for embedded systems. The previous works on the elaboration of binary descriptor did not exploit all information brought by a patch. The authors used just the pixel intensities to generate a binary test, so these descriptors lack efficiency on patch description. In this paper, we design a new descriptor named Geometric Model Descriptor (GMD) based on intensity and geometric coordinate of the pixel. We divide the patch into a grid cell (sub-patches); then each sub-patch is augmented in dimension (being 3D) by considering the intensity as the third dimension rather than its two dimensions (2D) form. Based on geometric modeling (polynomial interpolation and Bezier curve), we interpolate the cloud of points to find an approximation of the geometric model for each sub-patch. A binary test will be performed between the coefficients of sub-patch models to generate the patch description. The deep evaluation of GMD on a benchmark data set shows a high discrimination in the presence of similarity. The GMD exhibits robustness and reliability against severe changes. A computation costing is reported in the results section by presenting a compromise between accuracy and real-time constraint.

Learning Deep Binary Descriptor with Multi-Quantization.

In this paper, we propose an unsupervised feature learning method called deep binary descriptor with multi-quantization (DBD-MQ) for visual analysis. Existing learning-based binary descriptors such as compact binary face descriptor (CBFD) and DeepBit utilize the rigid sign function for binarization despite of data distributions, which usually suffer from severe quantization loss. In order to address the limitation, we propose a deep multi-quantization network to learn a data-dependent binarization in an unsupervised manner. More specifically, we design a K-Autoencoders (KAEs) network to jointly learn the parameters of feature extractor and the binarization functions under a deep learning framework, so that discriminative binary descriptors can be obtained with a fine-grained multi-quantization. As DBD-MQ simply allocates the same number of quantizers to each real-valued feature dimension ignoring the elementwise diversity of informativeness, we further propose a deep competitive binary descriptor with multi-quantization (DCBD-MQ) method to learn optimal allocation of bits with the fixed binary length in a competitive manner, where informative dimensions gain more bits for complete representation. Moreover, we present a similarity-aware binary encoding strategy based on the earth mover's distance of Autoencoders, so that elements that are quantized into similar Autoencoders will have smaller Hamming distances. Extensive experimental results on six widely-used datasets show that our DBD-MQ and DCBD-MQ outperform most state-of-the-art unsupervised binary descriptors.

Hierarchical registration of unordered TLS point clouds based on binary shape context descriptor

Automatic registration of unordered point clouds collected by the terrestrial laser scanner (TLS) is the prerequisite for many applications including 3D model reconstruction, cultural heritage management, forest structure assessment, landslide monitoring, and solar energy analysis. However, most of the existing point cloud registration methods still suffer from some limitations. On one hand, most of them are considerable time-consuming and high computational complexity due to the exhaustive pairwise search for recovering the underlying overlaps, which makes them infeasible for the registration of large-scale point clouds. On the other hand, most of them only leverage pairwise overlaps and rarely use the overlaps between multiple point clouds, resulting in difficulty dealing with point clouds with limited overlaps. To overcome these limitations, this paper presents a Hierarchical Merging based Multiview Registration (HMMR) algorithm to align unordered point clouds from various scenes. First, the multi-level descriptors (i.e., local descriptor: Binary Shape Context (BSC) and global descriptor: Vector of Locally Aggregated Descriptor (VLAD)) are calculated. Second, the point clouds overlapping (adjacent) graph is efficiently constructed by leveraging the similarity between their corresponding VLAD vectors. Finally, the proposed method hierarchically registers multiple point clouds by iteratively performing optimal registration point clouds calculation, BSC descriptor based pairwise registration and point cloud groups overlapping (adjacent) graph update, until all the point clouds are aligned into a common coordinate reference. Comprehensive experiments demonstrate that the proposed algorithm obtains good performance in terms of successful registration rate, rotation error, translation error, and runtime, and outperformed the state-of-the-art approaches.

RWBD: Learning Robust Weighted Binary Descriptor for Image Matching

Binary descriptors are drawing increasing interest as they enable faster processing in a lower memory footprint, which are suitable to large-scale and real-time vision applications. However, the performance of most existing binary descriptors tends to be inferior to those of the competing floating-point descriptors. In this paper, by considering each bit might contribute differently to the robustness and distinctiveness, a novel binary descriptor based on binary weights learning is proposed, referred to as a robust weighted binary descriptor (RWBD). Technically, each bit of RWBD is built upon difference tests of pairwise complementary information sampling with varying Gaussian kernels, leading to a more informative and robust description. Furthermore, RWBD imposes different emphases on all bits utilizing the learned binary weights, which can significantly improve the discriminative power and preserve the fast matching advantage. Extensive experiments on the publicly challenging benchmarks, including image matching and object recognition data sets, clearly demonstrate the effectiveness and efficiency of the proposed approach.

Video stitching using interacting multiple model based feature tracking

In this paper, we propose a novel video stitching algorithm for videos from multiple cameras using interacting multiple model feature tracking to maintain spatial-temporal consistency. Apart from image alignment challenges while stitching a video, inter frame consistency, video jitter due to moving object and camera movement also need to be addressed. To address these challenges, feature point detected in the initial frame is tracked in the subsequent frames to maintain spatial-temporal consistency and reduce computation complexity in feature point detection. Firstly, the feature points are detected using Features from Accelerated Segment Test algorithm. Secondly, using Binary Robust Invariant Scalable Keypoints descriptor values are obtained from detected feature points and matched using hamming distance. The outliers are removed by Random Sample Consensus Algorithm. Once, the first frame is stitched, feature points detected from first frame are tracked using kalman filter with interacting multiple model. The tracked feature points are descripted and homography between the frames are found. This will maintain the spatio-temporal consistency by reducing jitter effect between frames after stitching, and since the frames are neglected from feature point detection, computation complexity is reduced. From the experimental results, we observed that the execution time of the proposed method is less and the performance of structural similarity is better than the existing methods.

Aerial image matching method based on HSI hash learning

Specific to characteristics of aerial images like large size, high resolution and rich edges and texture, HSI hash learning matching algorithm for aerial image is proposed. Firstly, the original RGB component image is converted to HSI component image, then HSI-based local multi-feature descriptor is constructed. Secondly, in order to decrease the encoding mapping error in the traditional hash learning during model parameter optimization, objective function for each component image is constructed by the hash degree and the similarity information. And then, the projection matrix and bias threshold are determined. Finally, in the Hamming matching stage, the minimized loss distance function is defined and the spatial similarity between the Euclidean space and the Hamming space is preserved. Thus, the quantization loss in hash learning is reduced significantly. The proposed HSI hash learning algorithm evaluated on the standard dataset and actual aerial images demonstrates that it may significantly outperform the traditional algorithms in terms of efficiency and performance. As shown by experimental results, the matching accuracy of the proposed method is at least 20% higher than that of other binary descriptors, with desired computation time consumption.

Human Epithelial Type-2 Cell Image Classification Using an Artificial Neural Network with Hybrid Descriptors

ABSTRACTAntinuclear antibody (ANA) testing is best performed using the indirect immunofluorescence (IIF) method with human epithelial type-2 (HEp-2) cells as the substrate. IIF is a subjective procedure in which HEp-2 patterns are analyzed manually from the microscope. Therefore, ANA test results greatly rely on the experience and expertise of pathologists. Hence, complete automation of the ANA test is required to avoid incorrect diagnoses. This paper represents an algorithm for the complex HEp-2 cell classification problem. The proposed algorithm used a small hybrid feature set that characterizes the texture and morphology of the HEp-2 cells along with artificial neural network (ANN). The hybrid features were extracted by breaking up the image into eight binary images. The proposed hybrid descriptors were more efficient than the popular co-occurrence matrix descriptor and local binary pattern descriptors for texture analysis. The proposed algorithm was evaluated on the ICPR 2016 IIF HEp-2 cell image dataset. The results indicated that the hybrid descriptor with an ANN approach achieved improved performance, with “96.8%” mean class accuracy.

Learning discriminative compact binary face descriptor for kinship verification

In this paper, we propose a weakly-supervised feature learning method called discriminative compact binary face descriptor (D-CBFD) for facial kinship verification. Unlike most existing kinship verification methods where hand-crafted features are used for face representation, our D-CBFD learns discriminative face representation from a set of weakly-labeled samples. Given a face image, we first compute pixel difference vectors (PDVs) at local patches. Then, we learn a discriminative projection to map each PDV into a low-dimensional binary feature space, where the energy information of the PDV can be well preserved and the distance between the positive pair is smaller and that between the negative pair is larger. Lastly, we pool these binary feature vectors within each face into a histogram feature as the final representation. Experimental results on three kinship datasets are presented to show the effectiveness of the proposed method.

Unsupervised Deep Learning of Compact Binary Descriptors.

Binary descriptors have been widely used for efficient image matching and retrieval. However, most existing binary descriptors are designed with hand-craft sampling patterns or learned with label annotation provided by datasets. In this paper, we propose a new unsupervised deep learning approach, called DeepBit, to learn compact binary descriptor for efficient visual object matching. We enforce three criteria on binary descriptors which are learned at the top layer of the deep neural network: 1) minimal quantization loss, 2) evenly distributed codes and 3) transformation invariant bit. Then, we estimate the parameters of the network through the optimization of the proposed objectives with a back-propagation technique. Extensive experimental results on various visual recognition tasks demonstrate the effectiveness of the proposed approach. We further demonstrate our proposed approach can be realized on the simplified deep neural network, and enables efficient image matching and retrieval speed with very competitive accuracies.

LOOP Descriptor: Local Optimal-Oriented Pattern

This letter introduces the LOOP binary descriptor (local optimal oriented pattern) that encodes rotation invariance into the main formulation itself. This makes any post processing stage for rotation invariance redundant and improves on both accuracy and time complexity. We consider fine-grained lepidoptera (moth/butterfly) species recognition as the representative problem since it involves repetition of localized patterns and textures that may be exploited for discrimination. We evaluate the performance of LOOP against its predecessors as well as few other popular descriptors. Besides experiments on standard benchmarks, we also introduce a new small image dataset on NZ Lepidoptera. Loop performs as well or better on all datasets evaluated compared to previous binary descriptors. The new dataset and demo code of the proposed method are to be made available through the lead author's academic webpage and GitHub.

Context-Aware Local Binary Feature Learning for Face Recognition.

In this paper, we propose a context-aware local binary feature learning (CA-LBFL) method for face recognition. Unlike existing learning-based local face descriptors such as discriminant face descriptor (DFD) and compact binary face descriptor (CBFD) which learn each feature code individually, our CA-LBFL exploits the contextual information of adjacent bits by constraining the number of shifts from different binary bits, so that more robust information can be exploited for face representation. Given a face image, we first extract pixel difference vectors (PDV) in local patches, and learn a discriminative mapping in an unsupervised manner to project each pixel difference vector into a context-aware binary vector. Then, we perform clustering on the learned binary codes to construct a codebook, and extract a histogram feature for each face image with the learned codebook as the final representation. In order to exploit local information from different scales, we propose a context-aware local binary multi-scale feature learning (CA-LBMFL) method to jointly learn multiple projection matrices for face representation. To make the proposed methods applicable for heterogeneous face recognition, we present a coupled CA-LBFL (C-CA-LBFL) method and a coupled CA-LBMFL (C-CA-LBMFL) method to reduce the modality gap of corresponding heterogeneous faces in the feature level, respectively. Extensive experimental results on four widely used face datasets clearly show that our methods outperform most state-of-the-art face descriptors.

Robust and efficient method for matching features in omnidirectional images

Binary descriptors have been widely used in many real-time applications due to their efficiency. These descriptors are commonly designed for perspective images but perform poorly on omnidirectional images, which are severely distorted. To address this issue, this paper proposes tangent plane BRIEF (TPBRIEF) and adapted log polar grid-based motion statistics (ALPGMS). TPBRIEF projects keypoints to a unit sphere and applies the fixed test set in BRIEF descriptor on the tangent plane of the unit sphere. The fixed test set is then backprojected onto the original distorted images to construct the distortion invariant descriptor. TPBRIEF directly enables keypoint detecting and feature describing on original distorted images, whereas other approaches correct the distortion through image resampling, which introduces artifacts and adds time cost. With ALPGMS, omnidirectional images are divided into circular arches named adapted log polar grids. Whether a match is true or false is then determined by simply thresholding the match numbers in a grid pair where the two matched points located. Experiments show that TPBRIEF greatly improves the feature matching accuracy and ALPGMS robustly removes wrong matches. Our proposed method outperforms the state-of-the-art methods.

Representing local shape geometry from multi-view silhouette perspective: A distinctive and robust binary 3D feature

Local geometry description is of central importance in 3D computer vision and robotics, while the design of a distinctive and robust binary descriptor still remains a challenge at present. This paper tackles this problem by proposing to utilize the silhouette cue from multiple viewpoints to represent local shape geometry, forming a new binary feature called rotational silhouette maps (RSM). Key to the RSM descriptor are the leverage of multi-view information for comprehensive characterization paired with the silhouette method for binary and robust feature encoding. Specifically, an RSM is computed for the radius neighbors of a keypoint, at which a local reference frame (LRF) is first constructed to attain rotation invariance. Then, the local shape is rotated in the LRF multiple times to capture the multi-view information. For each view, a silhouette map is generated via projection. By concatenating all the silhouette maps, the RSM feature is computed. Extensive experiments are deployed on three standard datasets, where descriptiveness and robustness with respect to Gaussian noise, shot noise, varying mesh resolutions, the number of keypoints, keypoint localization errors, clutter and occlusion are assessed based on a comparison with nine state-of-the-art descriptors. The results reveal the overall superiority of the proposed descriptor in terms of distinctiveness and robustness.

The 2016 G. Paul Moore Lecture: Lessons in Voice Rehabilitation: Journal of Voice and Clinical Practice

This Paul Moore Lecture honors the contributions made by authors to the Journal of Voice during a period of 30 years, from 1987. Fifty articles were selected and included under the following five topics: (1) normalcy of the larynx and voice; (2) the clinical speech-language pathologist's evaluation; (3) the patient's perspective; (4) the core of vocal rehabilitation; and (5) behavioral versus organic dysphonias. The analysis reflects a vivid landscape of the specific area and significant advances in knowledge. It also shows the valuable interdependence between science and clinical practice. The topics highlight the following information: (1) The physical appearance of a healthy larynx varies across individuals with normal voices. (2) The voice is not a binary descriptor (normal versus abnormal) but a variable measure, with many cultural influences on the perceptual auditory analysis of a voice. (3) The clinical speech-language pathologist assessment is multidimensional and multiparametric, with both subjective and objective analyses. The patients' opinion about the impact of a voice problem on his or her quality of life is significant when proposing a treatment. Therefore, it is also included in the initial assessment. (4) Vocal rehabilitation is a nonlinear process that combines direct and indirect approaches. Evidence of the positive effect of voice therapy is now well established. (5) Behavioral dysphonias may be linked to self-regulation of the use of voice and this needs to be taken into consideration. Although organic dysphonias are not necessarily the result of harmful vocal behaviors, they too can benefit from vocal rehabilitation.

Local voxelized structure for 3D binary feature representation and robust registration of point clouds from low-cost sensors

Local feature-based 3D point cloud registration is a central issue in the fields of 3D computer vision and robotics, and most previously proposed 3D local features are real-valued. In this paper (1) a novel binary descriptor named local voxelized structure (LoVS) for 3D local shape description and (2) a LoVS-based registration algorithm for low-quality, e.g., Kinect-captured, point clouds are proposed. LoVS simply encodes the local shape structure represented by point clouds into bit string using point spatial locations without computing complex geometric attributes, such as normals and curvature, at the feature representation stage. Specifically, the LoVS descriptor is extracted within a local cubic volume around the keypoint. The orientation of the cubic volume is determined by a local reference frame (LRF) to achieve rotation invariance. Then, the cubic volume is uniformly split into a set of voxels. A voxel is labeled 1 if it contains points; otherwise, 0. All the labels are integrated into the LoVS descriptor. Based on the LoVS descriptor, a robust and accurate point cloud registration algorithm was developed, which effectively handles the challenges presented by low-cost sensors, e.g., noise and varying data resolutions. Experiments and extensive comparisons on three descriptor-matching benchmarks and a large-scale Kinect point cloud registration dataset show the effectiveness and the over-all superiority of our proposed LoVS descriptor and LoVS-based point cloud registration algorithm.

Medical image rigid registration using a novel binary feature descriptor and modified affine transform

Robust and reliable features with noise immunity, rotation-invariance, and low-dimensionality are the challenging aspects of pattern recognition. In this study, the authors presented a novel low-dimensional binary feature descriptor local diagonal Laplacian pattern (LDLP) for medical image registration. LDLP method is developed by defining the local relationship between a centre pixel and its diagonal neighbours and encoding it to a binary feature vector. The idea of centre-diagonal pixel correlation has drastically reduced the length of the feature vector without compromising the quality of local texture analysis. In the proposed work, first, the LDLP feature histograms of computed tomography (CT), magnetic resonance (MR), and ultrasound images are obtained. Further, these LDLP features of individual medical images are considered as target/fixed objects while their corresponding rotated and noisy features are considered as moving/floating objects to perform mono-modal rigid registration using an improved Procrustes analysis-based affine transform. The registration quality is examined by calculating the squared intensity error and the results are compared with the existing binary patterns such as local binary patterns, local tetra patterns, and local diagonal extrema patterns. The proposed LDLP feature descriptor-based rigid registration has attained relatively better performance in terms of registration accuracy and computational complexity.

Biologically Inspired Blind Quality Assessment of Tone-Mapped Images

Currently, many tone mapping operators (TMOs) have been provided to compress high dynamic range images to low dynamic range (LDR) images for visualizing them on the common displays. Since quality degradation is inevitably induced by compression, how to evaluate the obtained LDR images is indeed a headache problem. Until now, only a few full reference (FR) image quality assessment metrics have been proposed. However, they highly depend on reference image and neglect human visual system characteristics, hindering the practical applications. In this paper, we propose an effective blind quality assessment method of tone-mapped image without access to reference image. Inspired by that the performance of existing TMOs largely depend on the brightness and chromatic and structural properties of a scene, we evaluate the perceptual quality from the perspective of color information processing in the brain. Specifically, motivated by the physiological and psychological evidence, we simulate the responses of single-opponent (SO) and double-opponent (DO) cells, which play an important role in the processing of the color information. To represent the textural information, we extract three features from gray-level co-occurrence matrix (GLCM) calculated from SO responses. Meanwhile, both GLCM and local binary pattern descriptor are employed to extract texture and structure in the responses of DO cells. All these extracted features and associated subjective ratings are learned to reveal the connection between feature space and human opinion score. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art blind quality assessment methods and is comparable with the popular FR methods on two recently published tone-mapped image databases.

Local multi-feature hashing based fast matching for aerial images

Despite their superior accuracy, existing floating-point feature descriptor based image matching algorithms are too computationally intensive for real world aerial imaging applications, where target images are often of extremely high resolutions. To improve the efficiency of these algorithms, this paper presents a novel low-complexity image matching approach, namely local multi-feature hashing (LMFH). Similar to conventional techniques, LMFH employs a floating-point local multi-feature descriptor for accurate matching, however, the proposed method does not compare descriptors directly. Instead, LMFH projects each high-dimensional descriptor to a compact binary code in Hamming space using a trained hash function and then performs the feature comparison in the mapped domain. Experimental results show that, in terms of the computational cost, LMFH is on pair with the most efficient modern binary descriptors. Furthermore, LMFH has a performance gain which is significantly higher than the state-of-the-art binary descriptors and is comparable with conventional floating-point descriptors in terms of accuracy.