A video engine supported by social buzz to automatically create TV summaries

Abstract

Viewers post a lot of TV program-related information on social networks while they are watching TV, especially during its key moments. Therefore, this social buzz has the potential to be used as an automatic editorial criterion. Having this premise in consideration, this paper reports on the nowUP solution, a service developed with the main goal of automatically creating TV summaries of popular television programs (like football matches, talent or reality shows) based on the Twitter activity and integrating a part of that activity in the TV show summary. A data-mining engine continuously processes the activity of this social network looking for tweets associated with the TV shows. Based on the program metadata it indexes the twitter activity; correlates tweets; and creates clusters of peaks, being the relevant clusters associated with the highlights of the TV show. With this, the video engine automatically creates a full video summary (an edited sequence of TV highlights) and publishes it in an online platform and on a Catch-up TV service. The paper reports on the nowUP development and on the results of its evaluation, namely comparing its outputs with official editorial/professional video summaries. The results show that the solution was very successful in achieving the project main goal and users want to have access to this type of social buzz-based video summaries. The nowUP solution also promises potential gains in the value chain of TV producers and broadcasters.