Dark Photo Reconstruction by Event Camera

Abstract

Imaging in low light environment is a challenging problem due to the limiting number of the photons gathered by the traditional camera. To tackle the problem, this paper introduces a novel deep learning method for intensity images reconstruction in low light with event cameras. Event cameras are biologically-inspired sensors that capture brightness changes in the form of asynchronous “events” instead of intensity frames. They have significant advantages over conventional cameras: high temporal resolution, high dynamic range, and no motion blur. This method exploits the high dynamic range characteristic of event cameras, and bridges the gap between the intensity image and event data stream. The main challenges in our approach are that it is very hard to build paired low/high exposure event/intensity data for training and event data captured in dark are very noisy and sparse. Even if we have paired event-intensity data captured in the daytime, the models trained on it cannot generalize well in low light condition. In this paper, we combine paired and unpaired data and propose a novel GAN-based hybrid learning framework to get over the difficulty and improve the quality of reconstructed images. Experimental results on both synthetic and real data demonstrate the superiority of our model method in comparison to the state-of-the-art.