Single Frame Based Video Geo-Localisation Using Structure Projection

Abstract

Community image and video platforms like FlickR and Youtube offer large image collections from different perspectives. However, the majority of publicly available imagery from online communities lack a reasonable exact location and orientation information, which is important for many geo-spatial applications like object geo-referencing, knowledge transfer or augmented reality. In this work we exploit publicly available drone videos in order to bridge the gap between ground and aerial imagery. We propose a framework for the fast determination of full 6-D georeferenced motion trajectories of online community drone video footage using geo-localized map data. Our method requires the registration of a single video frame from a video sequence in order to exactly geo-reference complete motion trajectories w.r.t. to existing geo-referenced map data. The method relies on SfM and SLAM techniques in combination with a simple, yet efficient appearance and structure matching based on rendered map data (e.g. LiDAR) in order to generate geo-registered 3D feature maps. These maps enable a simple and fast global appearance based geo-registration of visually overlapping community videos and images. We evaluate our method on a large set of community drone videos. Our method produces drift free geo-data overlays at an average speed of 29,7 frames per second with an average positional error of 0,4m. In addition we release a large scale processed LiDAR dataset and geo-registered feature maps as an extension to the converging perspectives dataset. This data may provide visual links from ground based sensors to aerial imagery. Possible applications are numerous and include autonomous navigation, map updating/extension, image and video dehazing, object localisation or augmented reality.