Ying Zhang

Abstract

In recent years, we have witnessed an overwhelming number of user-generated videos being captured on a daily basis. An essential reason is the rapid development in camera technology and hence videos are easily recorded on multiple portable devices, especially mobile smartphones. Such flexibility encourages the modern videos to be tagged with additional various sensor properties. In this thesis, we are interested in geo-referenced videos whose metadata is closely tied to geographic identifications. These videos have great appeal for prospective travelers and visitors who are unfamiliar with a region, an area or a city. For example, before someone visits a place, a georeferenced video search engine can quickly retrieve a list of videos that are captured in this place so the visitors could obtain an overall visual impression, conveniently and quickly. However, users face the prospect of an ever increasing viewing burden if the size of these video repositories keeps increasing and as a result more videos are relevant to a search query. To manage these video retrievals and provide viewers with an efficient way to browse, we introduce a novel solution to automatically generate a summarization from multiple user generated videos and present their salience to viewers in an enjoyable manner. In this thesis, we investigate how to formulate, display and improve a multi-video summarization with the following contributions. The first three works propose solutions to detect video salience among multiple videos according to their geographic properties and convert the summarization to a graph-analysis problem with a dynamic programming-based solution to achieve the optimized informativeness, quality and coherency. The fourth work proposes an interactive and dynamic video exploration system where people can conduct personalized summary queries through direct map-based manipulations. Lastly, we investigate whether external crowdsourcing databases contribute to improving the summary quality by recommending a list of photography spots which are of potential to capture appealing photos for a landmark.