Smart video summarization using mealy machine-based trajectory modelling for surveillance applications

Abstract

In this paper, we propose a smart video summarization technique that compiles a synopsis of event(s)-of-interest occurring within a segment of image frames in a video. The proposed solution space consists of extracting appropriate features that represent the dynamics of targets in surveillance environments using their motion trajectories combined with a finite state automaton model for analyzing state changes of such features to detect and localize event(s)-of-interest. We introduce the cumulative moving average (CMA) and the preceding segment average (PSA) statistical metric as features that indicate gradual and sudden changes in the instantaneous velocity of moving targets. In order to support both on-line and off-line summarization, a finite state machine, that is often referred to as Mealy Machine, has been proposed to model the trajectory of a moving target and used for detecting transitions that represents a change from one state to another when initiated by a triggering event or condition. We conduct several systematic experiments on different scenario-specific in-house videos and other publicly available datasets to demonstrate the effectiveness of our proposed approach and benchmark its performance against chosen baseline strategies. The results of our experiments highlight the superiority of our proposed method in accurately localizing the start and end of event(s)-of-interest in videos within the chosen dataset.