Light field multi-scale blocked local binary pattern feature in the spatial-angular domain

Abstract

Image feature detection and matching are still important to many applications in computer vision. Light field imaging captures structured multi-view data, which provides a potential capability for solving the polysemy problem of feature matching. In this paper, we propose a light field multi-scale blocked LBP feature extraction and matching algorithm based on the light field spatial-angular domain. The proposed algorithm is on the basis of the classic local binary pattern feature, which can be enhanced by adding the description of the variations of different angular views. First, the Harris feature detection is employed to select feature candidates from the light field central sub-aperture images. Then, an orientation selection is introduced to calculate the most invariant angular neighbors. We suggest extracting the local binary pattern features of the selected angular views and stitching them together, to form a novel light field multi-scale blocked LBP feature. Finally, the distance between different features is measured using the vector cosine similarity. The proposed algorithm achieves an average matching precision of 93% in both virtual and real scenes on the paired light fields matching dataset. The proposed algorithm outperforms the classical SIFT feature and the state-of-the-art light field feature. The proposed local binary features are also significantly better than SIFT features and LiFF features in terms of descriptor length.