A unified model for egocentric video summarization: an instance-based approach

Abstract

Video summarization generates compact representations of videos in the form of summaries. The proposed framework is a unified model for instance-driven egocentric video summarization addressing generic and query-based summarization along with multi-video summarization. The model employs deep learning for object detection and semantic web technologies in the form of ontologies for query inferences. Combining user preferences in the form of object queries has aided in producing summaries that are subjective in nature. Quantitative evaluations performed on two novel datasets namely, ‘vehicle expo’ and ‘academic inspection’ prove that the proposed framework produces remarkable results with the employment of instance-driven modules for summarization. Additional experimental analysis for shot boundary detection have been conducted based on proposed method and conventional methods establishing the significance of the instance-based model. Moreover, qualitative evaluations further ensure that the summaries are concise, representative, diverse and semantically relevant further substantiating the need for instance-driven models in video summarization.