Vox Populi : generating video documentaries from semantically annotated media repositories

Abstract

The scenario in which the present research takes place is that of one or more online video repositories containing several hours of documentary footage and users possibly interested only in particular topics of that material. In such a setting it is not possible to craft a single version containing all possible topics the user might like to see, unless including all the material, which is clearly not feasible. The main motivation for this research is, therefore, to enable an alternative authoring process for film makers to make all their material dynamically available to users, without having to edit a static final cut that would select out possible informative footage. We developed a methodology to automatically organize video material in an edited video sequence with a rhetorical structure. This was enabled by defining an annotation schema for the material and a genera- tion process with the following two requirements: • the data repository used by the generation process could be extended by simply adding annotated material to it • the final resulting structure of the video generation process would seem familiar to a video literate user The first requirement was satisfied by developing an annotation schema that explic- itly identifies rhetorical elements in the video material, and a generation process that assembles longer sequences of video by manipulating the annotations in a bottom-up fashion. The second requirement was satisfied by modelling the generation process accord- ing to documentary making and general film theory techniques, in particular making the role of rhetoric in video documentaries explicit. A specific case study was carried out using video material for video documentaries. These used an interview structure, where people are asked to make statements about subjective matters. This category is characterized by rich information encoded in the audio track and by the controversiality of the different opinions expressed in the inter- views. The approach was tested by implementing a system called Vox Populi that real- izes a user-driven generation of rhetoric-based video sequences. Using the annotation schema, Vox Populi can be used to generate the story space and to allow the user to select and browse such a space. The user can specify the topic but also the characters of the rhetorical dialogue and the rhetoric form of the presentation. Presenting controversial topics can introduce some bias: Vox Populi tries to con- trol that by modelling some rhetoric and film theory editing techniques that influence