Domain adaptation assisted automatic real-time human-based video summarization

Abstract

The video summarization plays a momentous role for real-time surveillance for detection of suspicious human activities in public scenes. However, efficient, and accurate human-centric video summarization in real-time videos is still a challenging issue due to several inconsistencies such as noise, illumination effects, scale, or rotational variance, etc. Though, several prominent research studies are available related to human activities detection but a robust human-centric video summarization approach is a major issue. In this research, we expound an efficient video summary generation model (ReHuSum) that creates a short summary with human. Our approach employs the conception of domain adaptation for efficient summarization via transferring the knowledge of a potent pre-trained benchmark Inception-v3 model. Additionally, a frame differencing redundancy removal technique is adopted that aids in generating a precise and accurate video summary with human. Our model generates efficient short summaries from real-time videos with an overall human detection accuracy of 99.97%. Besides, we present two novel qualitative metrics namely; summarization rate (Summrate) and summarization loss (Summloss) for assessing the efficacy of overall summarization. The approach generates a real-time human-based summary of 46s with a (Summrate) of 98.7% and (Summloss) of 0.013 for an outdoor CCTV with a duration of 25.2 min. Similarly, an indoor CCTV of 40.10 min duration is summarized to 44s at a (Summrate) of 96.5% and (Summloss) of 0.035. The model can be utilized in real-time security applications (i.e., border or home surveillance, critical infrastructure, smart cities, etc.) to generate accurate video summaries with human.