Sparsity-guided Discriminative Feature Encoding for Robust Keypoint Detection

Abstract

Existing handcrafted keypoint detectors typically focus on designing specific local structures manually while ignoring whether they have enough flexibility to explore diverse visual patterns in an image. Despite the advancement of learning-based approaches in the past few years, most of them still rely on the availability of the outputs of handcrafted detectors as a part of training. In fact, such dependence limits their ability to discover various visual information. Recently, semi-handcrafted methods based on sparse coding have emerged as a promising paradigm to alleviate the above issue. However, the visual relationships between feature points have not been considered in the encoding stage, which may weaken the discriminative capability of feature representations for keypoint recognition. To tackle this problem, we propose a novel sparsity-guided discriminative feature representation (SDFR) method that attempts to explore the intrinsic correlations of keypoint candidates, thus ensuring the validity of characterizing distinctive and diverse structural information. Specifically, we first incorporate an affinity constraint into the feature representation objective, which jointly encodes all the patches in an image while highlighting the similarities and differences between them. Meanwhile, a smoother sparsity regularization with the Frobenius norm is leveraged to further preserve the similarity relationships of patch representations. Due to the differentiable property of this sparsity, SDFR is computationally feasible and effective for representing dense patches. Finally, we treat the SDFR model as multiple optimization sub-problems and introduce an iterative solver. During comprehensive evaluations on five challenging benchmarks, the proposed method achieves favorable performances compared with the state of the art in the literature.