Learning Stereo High Dynamic Range Imaging From A Pair of Cameras With Different Exposure Parameters

Abstract

It is possible to generate stereo high dynamic range (HDR) images/videos by using a pair of cameras with different exposure parameters. In this article, a learning-based stereo HDR imaging (SHDRI) method with three modules is proposed. In the proposed method, we construct three convolutional neural network (CNN) modules that perform specific tasks, including exposure calibration CNN (EC-CNN) module, hole-filling CNN (HF-CNN) module and HDR fusion CNN (HDRF-CNN) module, to combine with traditional image processing methods to model SHDRI pipeline. To avoid ambiguity, we assume that the left-view image is under-exposed and the right-view image is over-exposed. Specifically, the EC-CNN module is first constructed to convert stereo multi-exposure images into the same exposure to facilitate subsequent stereo matching. Then, based on the estimated disparity map, the right-view image is forward-warped to generate the initial left-view over-exposure image. After that, extra exposure information is utilized to guide hole-filling. Finally, the HDRF-CNN module is constructed and employed to extract fusion features to fuse the hole-filled left-view over-exposure image with the original left-view under-exposure image into the left-view HDR image. Right-view HDR images can be generated in the same way. In addition, we propose an effective two-phase training strategy to overcome the lack of a sufficient large stereo multi-exposure dataset. The experimental results demonstrate that the proposed method can generate stereo HDR images with high visual quality. Furthermore, the proposed method achieves better performance in comparison with the latest SHDRI method.