Automatic Posture and Movement Tracking of Infants with Wearable Movement Sensors

Manu Airaksinen,Elina Ilén,Sampsa Vanhatalo,Leena Haataja,Sonja Blom,Anni Varhe,Nico Kaartinen,Okko Räsänen,Anna Kivi,Taru Häyrinen,Viviana Marchi,Anastasia Gallen

doi:10.1038/s41598-019-56862-5

Abstract

Infants’ spontaneous and voluntary movements mirror developmental integrity of brain networks since they require coordinated activation of multiple sites in the central nervous system. Accordingly, early detection of infants with atypical motor development holds promise for recognizing those infants who are at risk for a wide range of neurodevelopmental disorders (e.g., cerebral palsy, autism spectrum disorders). Previously, novel wearable technology has shown promise for offering efficient, scalable and automated methods for movement assessment in adults. Here, we describe the development of an infant wearable, a multi-sensor smart jumpsuit that allows mobile accelerometer and gyroscope data collection during movements. Using this suit, we first recorded play sessions of 22 typically developing infants of approximately 7 months of age. These data were manually annotated for infant posture and movement based on video recordings of the sessions, and using a novel annotation scheme specifically designed to assess the overall movement pattern of infants in the given age group. A machine learning algorithm, based on deep convolutional neural networks (CNNs) was then trained for automatic detection of posture and movement classes using the data and annotations. Our experiments show that the setup can be used for quantitative tracking of infant movement activities with a human equivalent accuracy, i.e., it meets the human inter-rater agreement levels in infant posture and movement classification. We also quantify the ambiguity of human observers in analyzing infant movements, and propose a method for utilizing this uncertainty for performance improvements in training of the automated classifier. Comparison of different sensor configurations also shows that four-limb recording leads to the best performance in posture and movement classification.

Highlights

A key global healthcare challenge is the early recognition of infants that eventually develop lifelong neurocognitive disabilities
We explored the capabilities of the smart jumpsuit in measuring the proposed posture and movement categories in conjunction with two different classifier architectures: First, a support vector machine (SVM) classifier based on established signal-level features from the human activity detection literature[19,20], and second, a new end-to-end convolutional neural network (CNN) architecture designed for the task at hand
The result comparisons for SVM and CNN classifiers are shown in Fig. 4 for all recordings in terms of their category-specific F-scores

Summary

Introduction

A key global healthcare challenge is the early recognition of infants that eventually develop lifelong neurocognitive disabilities. Characterization of infants’ typical pattern of variation in different postures and movement activity over longer time could be used as a tool for early screening of infants at neurodevelopmental risks. Such a system would consist of an easy-to-use recording setup applicable to home environments, followed by an automated analysis pipeline for objective and quantitative assessment. The overall workflow in our study included (i) the development of a wearable garment and mobile data collection system for infant recordings (Fig. 1), (ii) the development and implementation of the visual analysis scheme to obtain a human benchmark, and iii) the development, training, and performance testing of the machine learning methods for an automated quantitative analysis of infants’ movement activity (see Fig. 2)

Methods

Results

Conclusion