Recently, a pair of RGB and near-infrared (NIR) cameras is applied to stereo vision systems for all-day vision applications. The images captured by the RGB-NIR stereo vision system have spectral ranges that differ significantly. Hence, the NIR image displays richer image information during nighttime. By contrast, during daytime, the RGB image generally provides abundant information. Therefore, these images can complement each other’s disadvantages in all-day environments. However, from the perspective of image matching, it is difficult to search for correspondences between two images because of their different spectral ranges. Although various methods for translating RGB images into NIR images have been proposed to solve this problem, hight-quality conversion results have not been achieved. Incomplete conversion results cause the inaccurate estimation of disparity during stereo matching. Therefore, we propose a warping-based spectral translation network (WASTNet) to enhance the training performance of a disparity estimation network by improving the performance of image translation.The WASTNet involves the following three steps: (1) initial disparity estimation for warping an NIR image to the viewpoint of RGB image; (2) post-processing for enhancing the initial disparity map; (3) image warping and spectral translation. Furthermore, we propose a confidence-based smoothness term that is applied to a loss function to mitigate disparity errors in the disparity map. The WASTNet and the proposed smoothness term are used concurrently with a conventional spectral translation network, which improves the performance of an unsupervised stereo matching network.
Read full abstract