VSTAR: Visual Semantic Thumbnails and tAgs Revitalization

Abstract

Nowadays, video-sharing portals’ popularity has entailed massive growth in data uploads over the Internet. For several applications (e.g., browsing, retrieval, or recommendation of videos), dealing with vast data volumes has become a critical issue. In a video-sharing scenario, the devising of tools and infrastructures able to completely satisfy users’ interests and requests is becoming increasingly crucial to influence their online experiences. On the one hand, annotating a video with meaningful human-friendly words (i.e., tags) plays an essential role in matching users’ interests. On the other hand, providing a condensed and straightforward preview of the video content (i.e., thumbnails) is crucial to capture the user’s attention immediately. In this context, we propose VSTAR (Visual Semantic Thumbnails and tAgs Revitalization), a novel approach in video optimization aimed at generating both suitable tags and thumbnails from a different perspective than classical approaches. The novelty lies in: (i) exploiting image captioning to extract visual and semantic information for generating both tags and thumbnails; (ii) identifying semantically related popular search queries (i.e., trends) to be suggested as new tags; (iii) giving the final user the control on a trade-off between quality and quantity of the generated items (tags and thumbnails); (iv) creating a proper dataset and making it publicly available. Experiments demonstrate the viability of our proposal.