Abnormal Infant Movements Classification With Deep Learning on Pose-Based Features

Kevin D. McCay,Hubert P. H. Shum,Claire Marcroft,Gerhard Fehringer,Edmond S. L. Ho,Nicholas D. Embleton

doi:10.1109/access.2020.2980269

Abstract

The pursuit of early diagnosis of cerebral palsy has been an active research area with some very promising results using tools such as the General Movements Assessment (GMA). In our previous work, we explored the feasibility of extracting pose-based features from video sequences to automatically classify infant body movement into two categories, normal and abnormal. The classification was based upon the GMA, which was carried out on the video data by an independent expert reviewer. In this paper we extend our previous work by extracting the normalised pose-based feature sets, Histograms of Joint Orientation 2D (HOJO2D) and Histograms of Joint Displacement 2D (HOJD2D), for use in new deep learning architectures. We explore the viability of using these pose-based feature sets for automated classification within a deep learning framework by carrying out extensive experiments on five new deep learning architectures. Experimental results show that the proposed fully connected neural network FCNet performed robustly across different feature sets. Furthermore, the proposed convolutional neural network architectures demonstrated excellent performance in handling features in higher dimensionality. We make the code, extracted features and associated GMA labels publicly available.

Highlights

Automated human action recognition has been an active area of research for a number of years [2]
Our experiments examine the effectiveness of 3 separate types of neural network architecture in classification of the extracted pose-based feature sets
EXPERIMENTAL RESULTS we present the experimental results in this study to evaluate the performance of the proposed motion classification framework with different deep neural network architectures

Summary

Introduction

Automated human action recognition has been an active area of research for a number of years [2]. It is a condition that primarily affects movement, posture and coordination, it can manifest in a range of other complications, such as swallowing difficulties, speech. The associate editor coordinating the review of this manuscript and approving it for publication was Wei Wei. problems, vision problems and learning disabilities. The severity of these symptoms can vary quite significantly, with some individuals presenting very minor symptoms, whilst others may be severely disabled. Whilst the continual development and enhancement of neonatal care has provided a significant decline in infant mortality rates, studies suggest that this has contributed towards an increase in the incidence and associated severity of cerebral palsy [29]

Methods

Results

Conclusion