Semantic photo books: leveraging blogs and social media for photo book creation

Abstract

Recently, we observed a substantial increase in the users' interest in sharing their photos online in travel blogs, social communities and photo sharing websites. An interesting aspect of these web platforms is their high level of user-media interaction and thus a high-quality source of semantic annotations: Users comment on the photos of each others, add external links to their travel blogs, tag each other in the social communities and add captions and descriptions to their photos. However, while those media assets are shared online, many users still highly appreciate the representation of these media in appealing physical photo books where the semantics are represented in form of descriptive text, maps, and external elements in addition to their related photos. Thus, in this paper we aim at fulfilling this need and provide an approach for creating photo books from Web 2.0 resources. We concentrate on two kinds of online shared media as resources for printable photo books: (a) Blogs especially travel blogs (b) Social community websites like Facebook which witness a rapidly growing number of shared media elements including photos. We introduce an approach to select media elements including photos, geographical maps and texts from both blogs and social networks semi-automatically, and then use these elements to create a printable photo book with an appealing layout. Because the selected media elements can be too many for the resulting book, we choose the most proper ones by exploiting content based, social based, and interactive based criteria. Additionally we add external media elements such as geographical maps, texts and externally hosted photos from linked resources. Having selected the important media, our approach uses a genetic algorithm to create an appealing layout using aesthetical rules, such as positioning the photo with the related text or map in a way that respects the golden ratio and symmetry. Distributing the media over the pages is done by optimizing the distribution according to several rules such that no pages with purely textual elements without photos are produced. For the page layout appropriate photos are chosen for the background based on their salience. Other media assets, such as texts, photos and geographical maps are positioned in the foreground by a dynamic page layout algorithm respecting both the content of the photos and the background, and common rules for visual layout. The result of our system is a photo book in a printable format. We implemented our approach as web services that analyze the media elements, enrich them, and create the layout in order to finally publish a photo book. The connection to those services is implemented in two interfaces. The first is a tool to select entries from personal blogs, and the second is a Facebook application that allows the user to select photos from his albums.