Big Multimodal Visual Data Registration for Digital Media Production

Abstract

Modern digital media production relies on various heterogeneous source of supporting data (snapshots, LiDAR, HDR and depth images) as well as videos from cameras. Recent developments of camera and sensing technology have led to huge amounts of digital media data. The management and process of this heterogeneous data consumes enormous resources. In this chapter, we present a multimodal visual data registration framework. A new feature description and matching method for multimodal data is introduced, considering local/semi-global geometry and colour information in the scene for more robust registration. Combined 2D/3D visualisation of this registered data allows an integrated overview of the entire dataset. The proposed framework is tested on multimodal dataset of film and broadcast production which are made publicly available. The resulting automated registration of multimodal datasets supports more efficient creative decision making in media production enabling data visualisation, search and verification across a wide variety of assets.