Mono-camera based pitch rate estimation in nighttime scenarios

Abstract

The camera relative pose is essential information for driver assistance systems in general, and especially so for systems that aim to visualize obstacles or other relevant objects for the driver. In order to display objects that were not detected in the vision frame of the camera (e.g. road information of a digital map), the exact transformation from camera to the world has to be known for every time frame. The horizontal movement of the camera in the real world is well known due to sufficiently accurate inertial sensors and wheel speed sensors available in the car. In contrast, the vertical movement is both unknown and highly dynamic. Relative pose estimation is a widely used technique that determines a vehicle's movement and rotation from frame to frame. These techniques rely primarily on point correspondences computed beforehand. In the nighttime case, these methods fail because of the poor image quality and low number of correspondences. This paper presents a method that fuses vehicle movement information with the features extracted from a mono camera. By restricting the problem to pitch estimation only, this method allows robust and accurate estimation of the vehicle's pitch movements even in nighttime scenarios. A highly accurate inertial navigation system is used to evaluate the results obtained in challenging real world nighttime video sequences. Experiments show the feasibility and robustness of the proposed approach.