Abstract

The idea of virtual time machines in digital environments like hand-held virtual reality or four-dimensional (4D) geographic information systems requires an accurate positioning and orientation of urban historical images. The browsing of large repositories to retrieve historical images and their subsequent precise pose estimation is still a manual and time-consuming process in the field of Cultural Heritage. This contribution presents an end-to-end pipeline from finding relevant images with utilization of content-based image retrieval to photogrammetric pose estimation of large historical terrestrial image datasets. Image retrieval as well as pose estimation are challenging tasks and are subjects of current research. Thereby, research has a strong focus on contemporary images but the methods are not considered for a use on historical image material. The first part of the pipeline comprises the precise selection of many relevant historical images based on a few example images (so called query images) by using content-based image retrieval. Therefore, two different retrieval approaches based on convolutional neural networks (CNN) are tested, evaluated, and compared with conventional metadata search in repositories. Results show that image retrieval approaches outperform the metadata search and are a valuable strategy for finding images of interest. The second part of the pipeline uses techniques of photogrammetry to derive the camera position and orientation of the historical images identified by the image retrieval. Multiple feature matching methods are used on four different datasets, the scene is reconstructed in the Structure-from-Motion software COLMAP, and all experiments are evaluated on a newly generated historical benchmark dataset. A large number of oriented images, as well as low error measures for most of the datasets, show that the workflow can be successfully applied. Finally, the combination of a CNN-based image retrieval and the feature matching methods SuperGlue and DISK show very promising results to realize a fully automated workflow. Such an automated workflow of selection and pose estimation of historical terrestrial images enables the creation of large-scale 4D models.

Highlights

  • Licensee MDPI, Basel, Switzerland.The idea of virtual time machines in digital environments like hand-held virtual reality (VR) or four-dimensional (4D) geographic information systems (GIS) requires an accurate positioning and orientation (=pose) of historical images

  • For each result list from the MD search an image retrieval (IR) is performed for improving the order of the list. These image retrievals are based on 3 different query images for every object of interest (OoI), leading to 3 result lists

  • Image retrieval applied to historical image works can provide assistance to the researcher for automation within a hand-crafted working process

Read more

Summary

Introduction

Licensee MDPI, Basel, Switzerland.The idea of virtual time machines in digital environments like hand-held virtual reality (VR) or four-dimensional (4D) geographic information systems (GIS) requires an accurate positioning and orientation (=pose) of historical images. Often the repositories show varying metadata and usability quality [2,3], and the usage of historical data material for a photogrammetric reconstruction of buildings and structures relies heavily on archive browsing and manually selecting appropriate sources [4,5,6,7]. This contribution tackles this problem focusing on a completely automated retrieval and pose estimation workflow for historical terrestrial images using deep learning methods

Methods
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call