Abstract
This work aims to establish visual correspondences between a pair of images depicting objects of the same semantic category. It encounters many challenges such as non-overlapping of scenes or objects, background clutter, and large intra-class variation. Existing methods handle this task with handcrafted features, which cannot effectively fit the correlations between non-overlapping images. Besides, additional training or information may be implemented into the learned features. In this paper, we propose a novel approach for semantic correspondence, which is based on deep feature representation, geometric and semantic associations between intra-class objects, and hierarchical matching selection according to the convolutional feature pyramid. Firstly, we construct the initial correspondence by developing a sparse feature matching model on the coarsest feature level, which enforces the nearest-neighbor searching under semantic and geometric consistency constraints. Further, a narrowing strategy is proposed and employed from the coarsest to the finest feature level, which hierarchically refine and optimize the correspondence. The results illustrate that this approach achieves competitive performance on the public datasets for semantic correspondence.
Highlights
Establishing correspondences between images is one of the fundamental problems in computer vision and graphics
They assume that the input image pair shows a proportion of the same scenes or objects from different viewpoints, and correspondences are obtained by using the handcrafted feature descriptors, e.g., Scale-Invariant Feature Transform (SIFT) [10] and Speeded Up Robust Features (SURF) [11]
To propagate the patch-level correspondence to pixel-level, we develop a novel hierarchical mapping scheme to gradually mapping the initial correspondences from the top layer to the bottom layer
Summary
Establishing correspondences between images is one of the fundamental problems in computer vision and graphics. The key issue is how to utilize the finite associated information to select enough salient features for matching Traditional handcrafted features, such as SIFT [10] and SURF [11], work well on matching the overlapping images. They are not suitable for the category-level correspondence since they are mainly designed for the same objects or scenes. We select as many representational and salient features as possible and establish inter-image correspondences based on semantic and geometric consistencies via the nearest-neighbor searching. We introduce a simple yet effective narrowing strategy, i.e., the imitation foreground detection method, for feature selection to improve the exclusivity among candidates It can reduce the searching scope at the lower pyramid layer and mitigate the error accumulation during hierarchical optimization. Experiments illustrate that the proposed approach obtains competitive performance on standard benchmarks for semantic correspondence
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
