Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders

Eftychios Protopapadakis,Nikolaos Doulamis,Anastasios Doulamis,Athanasios Voulodimos,Ioannis Rallis

doi:10.3390/app10228226

Eftychios Protopapadakis, Nikolaos Doulamis + Show 3 more

Open Access

https://doi.org/10.3390/app10228226

Copy DOI

Abstract

In this paper, a deep stacked auto-encoder (SAE) scheme followed by a hierarchical Sparse Modeling for Representative Selection (SMRS) algorithm is proposed to summarize dance video sequences, recorded using the VICON Motion capturing system. SAE’s main task is to reduce the redundant information embedding in the raw data and, thus, to improve summarization performance. This becomes apparent when two dancers are performing simultaneously and severe errors are encountered in the humans’ point joints, due to dancers’ occlusions in the 3D space. Four summarization algorithms are applied to extract the key frames; density based, Kennard Stone, conventional SMRS and its hierarchical scheme called H-SMRS. Experimental results have been carried out on real-life dance sequences of Greek traditional dances while the results have been compared against ground truth data selected by dance experts. The results indicate that H-SMRS being applied after the SAE information reduction module extracts key frames which are deviated in time less than 0.3 s to the ones selected by the experts and with a standard deviation of 0.18 s. Thus, the proposed scheme can effectively represent the content of the dance sequence.

Highlights

One interesting procedure for video visual analysis is video content summarization, a technique which has received wide research interest in recent years due to its wide application spectrum.The scope of a video summarization algorithm is to find out a set of the most representative key-frames of a video sequence, taking into consideration salient events and actions on video content so as to form a short but meaningful synopsis [1]
We present several experiments to demonstrate the performance of the proposed unsupervised 3D motion summarization framework based on a stacked auto-encoder used to reduce the redundant information
The proposed stacked auto-encoder scheme is evaluated over three different dance sequences

Summary

Introduction

One interesting procedure for video visual analysis is video content summarization, a technique which has received wide research interest in recent years due to its wide application spectrum. The scope of a video summarization algorithm is to find out a set of the most representative key-frames of a video sequence, taking into consideration salient events and actions on video content so as to form a short but meaningful synopsis [1]. The existing video summarization techniques abstract the input data using three different approaches [2]. The first is the so-called representative key-frame selection that creates video summaries through a collection of representative key frames [3]. The key subshot-oriented approach selects the representative subshots of key-frames to form the video synopsis [4]. The key object detection method decomposes the whole video sequence into several single frames, each revealing representative objects in a given video sequence [5]

Methods

Results

Conclusion