Discriminative Local Descriptors Research Articles

Learning more discriminative local descriptors with parameter-free weighted attention for few-shot learning

Learning more discriminative local descriptors with parameter-free weighted attention for few-shot learning

CSPFormer: A cross-spatial pyramid transformer for visual place recognition

Recently, the Vision Transformer (ViT), which applied the Transformer structure to various visual detection tasks, has outperformed convolutional neural networks (CNNs). Nonetheless, due to the lack of scale representation ability of the Transformer, how to extract the local features of the scene to effectively form a global descriptor is still a challenging problem. In the paper, we propose a Cross-Spatial Pyramid Transformer (CSPFormer) to learn the discriminative global descriptors from multi-scale visual features for efficient visual place recognition. Specifically, we first develop a pyramid CNN module that can extract multi-scale visual feature representations. Then, the extracted feature representations of multi-scales are input to multiple connected spatial pyramid Transformer modules that adaptively learn the spatial relationship of the different scale descriptors, where the multiple self-attention is applied to learn a global descriptor from discriminative local descriptors. CNN pyramid features and Transformer multi-scale features are mutually weighted to perform cross-spatial feature representation. The multiple self-attention enhances the long-term dependencies of multi-scale visual descriptors and reduces the computational cost. To obtain the final place-matching result accurately, the cosine function is used to calculate the spatial similarity between the two scenes. Experimental results on public place datasets show that the proposed method achieves state-of-the-art on large-scale visual place recognition tasks. Our model has achieved 94.7%, 92.8%, 91.3%, and 95.7% average recall based on the top 1% candidate scenario on KITTI, Nordland, VPRICE, and EuRoc datasets, respectively.

Image retrieval based on aggregated deep features weighted by regional significance and channel sensitivity

Deep convolutional neural networks (CNN) have demonstrated a very powerful approach for extracting discriminative local descriptors for image description. Many related works suggest that an effective aggregation representation for deep convolutional features is particularly important in forming robust and compact image representations. In this paper, a new robust global descriptor for image retrieval is proposed by creating an effective method for aggregating local deep convolutional features weighted by regional significance and channel sensitivity through sum-pooling on multiple regions. The proposed aggregation method effectively takes advantage of multiple scales, and considers both the varied significance of regional visual content and the sparsity and intensity of response values in the channel. This can improve the ability of deep features to be described and discerned. The experimental results on six benchmark datasets demonstrate that our method can achieve retrieval results comparable to those of some popular approaches for deep feature aggregation but without fine-tuning strategies and multiple image inputs.

Learning local shape descriptors for computing non-rigid dense correspondence

A discriminative local shape descriptor plays an important role in various applications. In this paper, we present a novel deep learning framework that derives discriminative local descriptors for deformable 3D shapes. We use local “geometry images” to encode the multi-scale local features of a point, via an intrinsic parameterization method based on geodesic polar coordinates. This new parameterization provides robust geometry images even for badly-shaped triangular meshes. Then a triplet network with shared architecture and parameters is used to perform deep metric learning; its aim is to distinguish between similar and dissimilar pairs of points. Additionally, a newly designed triplet loss function is minimized for improved, accurate training of the triplet network. To solve the dense correspondence problem, an efficient sampling approach is utilized to achieve a good compromise between training performance and descriptor quality. During testing, given a geometry image of a point of interest, our network outputs a discriminative local descriptor for it. Extensive testing of non-rigid dense shape matching on a variety of benchmarks demonstrates the superiority of the proposed descriptors over the state-of-the-art alternatives.

Real-time hand posture recognition using hand geometric features and Fisher Vector

Hand posture recognition (HPR), one of the most effective and intuitive human computer interfaces, has been extensively studied and widely adopted in various multimedia applications. Shape descriptors extracted from a hand contour or silhouette have been proved effective in representing a hand posture. However, it is difficult for these shape-based methods to achieve good balance between accuracy and efficiency. To this end, in this paper, we propose a novel hand shape descriptor based on a set of geometric features (SoGF) and Fisher Vector (FV), for effective and efficient HPR. Three types of geometric features, including distances, angles and curvatures, are extracted from a hand silhouette to form a discriminative local descriptor, and FV is adopted to encode the set of local descriptors for compact hand shape representation. To recognize hand postures, we construct a classifier using a multi-class Support Vector Machine (SVM) with FVs as input. The experimental results on four public HPR datasets show that the proposed method can achieve the mean accuracy of the state-of-the-art methods in real time.

From Selective Deep Convolutional Features to Compact Binary Representations for Image Retrieval

In the large-scale image retrieval task, the two most important requirements are the discriminability of image representations and the efficiency in computation and storage of representations. Regarding the former requirement, Convolutional Neural Network is proven to be a very powerful tool to extract highly discriminative local descriptors for effective image search. Additionally, to further improve the discriminative power of the descriptors, recent works adopt fine-tuned strategies. In this article, taking a different approach, we propose a novel, computationally efficient, and competitive framework. Specifically, we first propose various strategies to compute masks, namely, SIFT-masks , SUM-mask , and MAX-mask , to select a representative subset of local convolutional features and eliminate redundant features. Our in-depth analyses demonstrate that proposed masking schemes are effective to address the burstiness drawback and improve retrieval accuracy. Second, we propose to employ recent embedding and aggregating methods that can significantly boost the feature discriminability. Regarding the computation and storage efficiency, we include a hashing module to produce very compact binary image representations. Extensive experiments on six image retrieval benchmarks demonstrate that our proposed framework achieves the state-of-the-art retrieval performances.

CDbin: Compact Discriminative Binary Descriptor Learned With Efficient Neural Network

As an important computer vision task, image matching requires efficient and discriminative local descriptors. Most of the existing descriptors like SIFT and ORB are hand-crafted; therefore it is necessary to study more optimized descriptors through end-to-end learning. This paper proposes the compact binary descriptors learned with a lightweight Convolutional Neural Network (CNN), which is efficient for training and testing. Specifically, we propose a CNN with no larger than five layers for descriptor learning. The resulting descriptors, i.e. , Compact Discriminative binary descriptors (CDbin) are optimized with four complementary loss functions, i.e. , 1) triplet loss to ensure the discriminative power; 2) quantization loss to decrease the quantization error; 3) correlation loss to ensure the feature compactness; and 4) even-distribution loss to enrich the embedded information. The extensive experiments on two image patch datasets and three image retrieval datasets show that the CDbin exhibits competitive performance compared with the existing descriptors. For example, the 64-bit CDbin substantially outperforms the 256-bit ORB and 1024-bit SIFT on Hpatches dataset. Although generated by a shallow CNN, CDbin also outperforms several recent deep descriptors.