Keypoint descriptor fusion with Dempster–Shafer theory

Abstract

Keypoint matching is the task of accurately finding the location of a scene point in two images. Many keypoint descriptors have been proposed in the literature aiming at providing robustness against scale, translation and rotation transformations, each having advantages and disadvantages. This paper proposes a novel approach to fuse the information from multiple keypoint descriptors using Dempster–Shafer theory of evidence [1] , which has proven particularly efficient in combining sources of information providing incomplete, imprecise, biased, and conflictive knowledge. The matching results of each descriptor are transformed into an evidence distribution on which a confidence factor is computed making use of its entropy. Then, the evidence distributions are fused using Dempster–Shafer Theory (DST), considering its confidence. As a result of the fusion, a new evidence distribution that improves the result of the best descriptor is obtained. Our method has been tested with SIFT, SURF, ORB, BRISK and FREAK descriptors using all possible their combinations. Results on the Oxford keypoint dataset [2] show that the proposed approach obtains an improvement of up to 10 % compared to the best one (FREAK). • Novel approach fusing information from multiple keypoint descriptors using the Dempster–Shafer Theory of evidence. • Descriptors matches are transformed in evidence distributions assigning a confidence factor using Shannon's entropy. • Results on the Oxford keypoint dataset show improvements of up to 10% compared to the best keypoint descriptor.