A role‐entity based human activity recognition using inter‐body features and temporal sequence memory

Abstract

Recognizing entities and their corresponding roles are important in human activity recognition. In light of recent advancements, the primary emphasis is recognizing the abstract activities involving person-person interaction. The contribution of this work is proposing an architecture, which utilizes the knowledge of the human body parts coordinates in role detection of each individual. The network preprocesses the coordinates to build intra-body and inter-body features. The extracted features build the relationship between the interacting bodies and learn the temporal relation corresponding to each role using the human memory-inspired hierarchical temporal memory. The model is tested on vague samples of mutual actions in the experimental work. The model is found robust in action and role recognition tasks and performed well per expectations.