Improved Action Recognition with Separable Spatio-Temporal Attention Using Alternative Skeletal and Video Pre-Processing.

Pau Climent-Pérez,Francisco Florez-Revuelta

doi:10.3390/s21031005

Pau Climent-Pérez, Francisco Florez-Revuelta

Open Access

https://doi.org/10.3390/s21031005

Copy DOI

Journal: Sensors (Basel, Switzerland)	Publication Date: Feb 2, 2021
Citations: 9	License type: CC BY 4.0

Affiliation: University of Alicante

Abstract

The potential benefits of recognising activities of daily living from video for active and assisted living have yet to be fully untapped. These technologies can be used for behaviour understanding, and lifelogging for caregivers and end users alike. The recent publication of realistic datasets for this purpose, such as the Toyota Smarthomes dataset, calls for pushing forward the efforts to improve action recognition. Using the separable spatio-temporal attention network proposed in the literature, this paper introduces a view-invariant normalisation of skeletal pose data and full activity crops for RGB data, which improve the baseline results by 9.5% (on the cross-subject experiments), outperforming state-of-the-art techniques in this field when using the original unmodified skeletal data in dataset. Our code and data are available online.

Highlights

Societies of countries in the organisation for economic co-operation and development (OECD) are faced with the challenge of increasing older population [1] as reported by multiple agencies [2,3,4]
Active and assisted living (AAL) technologies aim at ameliorating the situation by providing tools to older people, their caregivers, and health practitioners with the goal of supporting end users to stay independent for longer using information and communication technologies (ICTs)
The Video-Pose Network (VPN) model of Das et al [31] focuses on a shortfall of separable STA, which is that the pose and RGB information are not coupled: The LSTM

Summary

Introduction

Societies of countries in the organisation for economic co-operation and development (OECD) are faced with the challenge of increasing older population [1] as reported by multiple agencies [2,3,4]. This increase brings associated fears: how to keep welfare and provide care and health services for such a large population of older people, with evershrinking workforce. The European Union and other governmental bodies have recognised the importance of this field by funding specific calls for research into the development of related technologies, as noted by Calvaresi et al [5]

Methods

Results

Conclusion