Discovery of Sparse Contextual Relationships for Activity Recognition in Continuous Videos

Abstract

Recent trends in activity recognition have shown that structured learning methods are useful in modeling spatio-temporal relationships, often termed as context in continuous videos. However, learning an efficient set of relationships or an ideal structure for representation of context in these videos still remains a challenging problem. While a continuous video consists of several activities, the contextual relationships between these activities are relatively sparse. We propose a method which aims to discover these sparse relationships using an L1-regularization based automatic structure discovery of a graphical model representing the video. We propose a 2-layered graphical model, where the lower level models feature information and the higher level models spatiotemporal context. The features are modeled using space-time interest points, while the relationships between activities are modeled using an undirected graphical model. Sparsity is imposed on the edges of the graph so as to model a sparse set of relationships. Experiments have been conducted on complex activity recognition datasets to demonstrate the superior performance of our approach in classification of continuous activities.