Dense Video Captioning Using Video-Audio Features and Topic Modeling Based on Caption

Abstract

Videos are composed of multiple tasks. Dense video captioning entails captioning of different events in the video. A textual description is generated based on visual, speech and audio cues from a video and then topic modeling is performed on the generated caption. Uncertainty modeling technique is applied for finding temporal event proposals where timestamps for each event in the video are produced and also uses Transformer which inputs multi-modal features to identify captions effectively and to make it more precise. Topic modeling tasks include highlighted keywords in the captions generated and topic generation i.e., category under which the whole caption belongs to. The proposed model generates a textual description based on the dynamic and static visual features and audio cues from a video and then topic modeling is performed on the generated caption.