Fall-Perceived Action Recognition of Persons With Neurological Disorders Using Semantic Supervision

Abstract

Frequent uncertain falls is one of the common cause of injury among elderly adults and persons suffering from the neurological disorder. It will be costlier to go through <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$24\times 7$ </tex-math></inline-formula> medical monitoring if we monitor a person suffering from the early stage of the neurological disorder. An “uncertain” action classification model can be a less costly and easily scalable. It can help to regularly monitor a person suffering from neurological declines and how frequent it relapse. In this article, we propose a video-based action recognition with fall detection architecture, FallNet, which learns the features of uncertain actions related to day-to-day activities. FallNet first incorporates semantic supervision using the per-class weight of uncertain action through class-wise weighted focal loss. It addresses both the class imbalance problem and the weak interclass separability issue. We design a joint training model to train the overall architecture efficiently in an end-to-end manner. We utilize benchmark data sets, OOPS, HMDB51, and Kinetics-600, for experimentation that has less falling action videos. Therefore, we have collected videos to create a data set, denoted by FallAction, that consists of different 15 falling action classes with an average of 100 videos per class. The proposed network gain an accuracy of 13.2% in OOPs, 2% in HMDB51, and 0.2% in Kinetics-600 data set.