Stereo Matching Based on Features of Image Patch

Shi Zhou,Zhen Li,Miaomiao Zhu,Lifeng Zhang,He Li,Mitsunori Mizumachi

doi:10.12792/jiiae.9.91

Abstract

Stereo matching is a branch of 3D vision and has a wide range of applications in 3D reconstructionand autonomous driving. Recently, stereo matching methods leverage all the information of the stereo image to calculate disparity map. However, these methods still have difficulties in texture-less areas and occlusion areas, and post-processing improves accuracy. Therefore, there is a high computational cost in feature extraction and post-processing. In this paper, we propose a stereo matching method based on features of image patches to predict the disparity of non-occlusion areas instead of full image features. And aggregation methods are performed to modify all kinds of mismatching pixels based on the correct disparity in the non-occlusion areas. Furthermore, we evaluated the proposed method on the Middlebury dataset. The result shows that the proposed method performs well in all areas.

Highlights

Stereo matching [1] is a fundamental problem in 3D computer vision, and its purpose is to predict the disparity map for stereo images
Calculating the matching cost with high accuracy is the main problem of the stereo matching method
We proposed a novel stereo matching method based on features extracted from image patches to achieve low computational cost disparity prediction

Summary

Introduction

Stereo matching [1] is a fundamental problem in 3D computer vision, and its purpose is to predict the disparity map for stereo images. As a popular research topic for decades, traditional stereo matching methods comprise four steps: calculation of matching cost, aggregation of matching cost, optimization, and disparity refinement [3] These methods usually calculate the similarity between pixels by designing an energy function. We proposed a novel stereo matching method based on features extracted from image patches to achieve low computational cost disparity prediction These features are extracted from the neural network, which can describe the image patch from different light condition well, so that the proposed method has strong robustness. According to these features, the similarity between image patches can be calculated. After extracting features from the neural network and performing post-processing for different mismatching pixels, the output disparity map of the proposed method has a good performance on the Middlebury dataset [6]

Proposed method

Experiments

Conclusions