EAGAN: Event‐based attention generative adversarial networks for optical flow and depth estimation

Abstract

Event camera is a new vision sensor that produces independent asynchronous responses to each pixel's change of illumination intensity. The unique principle of event camera has many advantages over traditional cameras, such as low latency, high temporal resolution, and high dynamic range (HDR). These advantages make event camera ideal for dealing with high speed, HDR visual tasks, especially in automatic driving scenes. In this study, we propose an image generation network named Event-based attention generative adversarial networks (EAGAN), which simultaneously deals with optical flow and depth estimation of monocular event camera data. In addition to the innovative network architecture and loss function suitable for depth estimation, we are also the first to process incomplete training data to obtain more dense and uniform prediction results. Experiments on the multi-vehicle stereo event camera dataset show that our EAGAN is competitive on the depth estimation task and achieves the state-of-the-art effect in the optical flow estimation task.