Human Interaction Anticipation by Combining Deep Features and Transformed Optical Flow Components

Shafina Bibi,Naeem Ramzan,Tehmina Amjad,Graeme McRobbie,Nadeem Anjum

doi:10.1109/access.2020.3012557

Abstract

The anticipation of ongoing human interactions is not only highly dynamic and challenging problem but extremely crucial in applications such as remote monitoring, video surveillance, human-robot interaction, anti-terrorists and anti-crime securities. In this work, we address the problem of anticipating the interactions between people monitored by single as well as multiple camera views. To this end, we propose a novel approach that integrates Deep Features with novel hand-crafted features, namely Transformed Optical Flow Components (TOFCs). In order to validate the performance of the proposed approach, we have tested the proposed approach in real outdoor environments, captured using single as well as multiple cameras, having shadow and illumination variations as well as cluttered backgrounds. The results of the proposed approach are also compared with the state-of-the-art approaches. The experimental results show that the proposed approach is promising to anticipate real human interactions.

Highlights

The aim of human interaction anticipation is to recognize an interaction before its complete execution [1]
We propose to combine deep features with handcrafted features to reduce the effects of shadows and illumination variations to provide novel anticipation method
Experiments are performed on multi-view datasets (MUInteraction1 and MU-Interaction2) and on publicly available UT-Interaction dataset [33] and results are compared with state-of-the-art approaches

Summary

Introduction

The aim of human interaction anticipation is to recognize an interaction before its complete execution [1]. Preliminary studies have been attempted to recognize the actions of a person from single frame and from a few frames [2], [3]. This leads to the concept of anticipating an action from partially observed videos. The anticipation of human interactions and activities is becoming an active area of research. It has grabbed the attentions of research community due to its importance in several applications such as (a) in video surveillance

Objectives

Methods

Results

Conclusion