Wearable Sensor Data Research Articles

Deep Learning Models Predict Dynamic Oxygen Uptake Responses from Wearable Sensor Data during Moderate‐ and Heavy‐Intensity Exercise

Wearable technologies and artificial intelligence have enabled continuous and non-intrusive cardiorespiratory monitoring. Machine learning techniques, such as random forest (RF) and long short-term memory (LSTM) models, have been used to predict the oxygen uptake (V̇O2) response to exercise, but their abilities to track V̇O2 during changes in work rate lack precision. Here, we propose using a sequential deep learning model based on temporal convolutional networks (TCN) to estimate V̇O2 from wearable sensor data. Twenty-two healthy adults (9 females, age: 26±5 yr, peak V̇O2: 42±6 ml·min-1·kg-1) completed a 25 W·min-1 ramp cycling test to exhaustion, and a combination of 3 different pseudorandom binary sequence (PRBS) cycling tests to simulate non-constant work rate exercise ranging from low to moderate, low to heavy, and ventilatory threshold to heavy-intensity exercise, respectively. Breath-by-breath V̇O2 was measured using a portable metabolic device, and wearable sensor data were simultaneously collected using a Hexoskin® sensor shirt. Work rate, heart rate, percent heart rate reserve, estimated minute ventilation, and breathing rate were used as model inputs to predict instantaneous V̇O2. Participant data were split into 3 groups to train (n=10), validate (n=7), and test (n=5) the newly proposed TCN model and previously reported RF and LSTM models. Repeated measures Bland-Altman analysis (bias; 95% limits of agreement) revealed that the TCN model (-1 ml·min-1; -237 to 235 ml·min-1) was more accurate at estimating the dynamic V̇O2 responses to ramp and PRBS exercise than the LSTM (-10 ml·min-1; -321 to 301 ml·min-1) and RF models (62 ml·min-1; -259 to 383 ml·min-1), despite containing fewer trainable parameters (63% and 97% reduction vs. LSTM and RF, respectively). These results suggest that the TCN model is more accurate and efficient at estimating V̇O2 than other previously used machine learning models across a wide range of exercise intensities. Our findings are an important step in the development of a framework to non-intrusively assess aerobic fitness levels and energy expenditure during real-life situations outside of the laboratory without cumbersome equipment.

Wearables and Deep Learning Classify Fall Risk From Gait in Multiple Sclerosis.

Falls are a significant problem for persons with multiple sclerosis (PwMS). Yet fall prevention interventions are not often prescribed until after a fall has been reported to a healthcare provider. While still nascent, objective fall risk assessments could help in prescribing preventative interventions. To this end, retrospective fall status classification commonly serves as an intermediate step in developing prospective fall risk assessments. Previous research has identified measures of gait biomechanics that differ between PwMS who have fallen and those who have not, but these biomechanical indices have not yet been leveraged to detect PwMS who have fallen. Moreover, they require the use of laboratory-based measurement technologies, which prevent clinical deployment. Here we demonstrate that a bidirectional long short-term (BiLSTM) memory deep neural network was able to identify PwMS who have recently fallen with good performance (AUC of 0.88) based on accelerometer data recorded from two wearable sensors during a one-minute walking task. These results provide substantial improvements over machine learning models trained on spatiotemporal gait parameters (21% improvement in AUC), statistical features from the wearable sensor data (16%), and patient-reported (19%) and neurologist-administered (24%) measures in this sample. The success and simplicity (two wearable sensors, only one-minute of walking) of this approach indicates the promise of inexpensive wearable sensors for capturing fall risk in PwMS.

Extending the dimensions of personal exposure assessment: A methodological discussion on perceived and measured noise and air pollution in traffic

BackgroundThe exposure to air pollution and noise is severely impacting people's health and is especially high alongside urban road- and rail-traffic. In traditional exposure research, air pollution and noise are monitored with stationary measurement devices or based on models. During the last years, mobile measurement techniques with GPS-tracking have increased. Moreover, studies have investigated another dimension of personal exposure: the perceived exposure. AimMost of these studies make use of quantitative methods such as surveys, complemented by stationary or wearable sensors. Little research exist that applies qualitative methods to examine how people experience and perceive exposure on-the-move, contrasting it to actual measurements. The aim of this paper is to discuss the potential of a novel method, which extends the dimensions of personal exposure by including the situational context of exposure perception. Methods/case studyFirstly, different methods for exposure research are presented. Secondly, we introduce a novel mixed-method approach, exploring cyclists and pedestrians perceived and measured exposure on-the-move by combining mobile interviews (Go-/Ride-Alongs) and wearable sensors. We will present the methodological findings using a case study and have a quantitative method (smartphone questionnaire, wearable sensors) as a reference. Results and discussionThe differences of perceived and measured exposure, proven through the reference method, are a result of different situational contexts as shown by the mobile interviews (knowledge, embodied experience, life situation, activities). The methodological findings show, that mobile methods complemented by wearables introduce new dimensions of personal exposure: they shed light on the situational contexts that affect exposure perception during commute. ConclusionWe argue that both, perceived and measured exposure to air pollution and noise, need to be considered simultaneously. Complementing mobile interviews or surveys with wearable sensor data improves the understanding of urban dwellers requirements for healthier mobility. Potentials of these methods should be investigated further, both in research and for supporting urban transport planning decisions adapted to people's needs.

A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data

Night shift workers are often associated with circadian misalignment and physical discomfort, which may lead to burnout and decreased work performance. Moreover, the irregular work hours can lead to significant negative health outcomes such as poor eating habits, smoking, and being sedentary more often. This paper uses commercial wearable sensors to explore correlates and differences in the level of physical activity, sleep, and circadian misalignment indicators among day shift nurses and night shift nurses. We identify which self-reported assessments of affect, life satisfaction, and sleep quality, are associated with physiological and behavioral signals captured by wearable sensors. The results using data collected from 113 nurses in a large hospital setting, over a period of 10 weeks, indicate that night shift nurses are more sedentary, and report lower levels of life satisfaction than day-shift nurses. Moreover, night shift nurses report poorer sleep quality, which may be correlated with challenges in their attempts to fall asleep on off-days.

Dilated causal convolution with multi-head self attention for sensor human activity recognition

Systems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.

MODELING REALITY: REVISITING CALVERT'S FITNESS SIMULATION.

Wearable sensors have gained mainstream acceptance for health and fitness monitoring despite the absence of clinically validated analytic models for clinical decision support. Individual sensors measuring, say, EKG signal and heart rate can provide insight on cardiovascular response, but a more complete picture of health and fitness requires a more complete portfolio of sensors and data. This paper outlines the research underway to revisit and reconfigure the 1976 Calvert systems model of the effect of training on physical performance. Specifically, we use wearable sensor data from clinical trials to supplement a hybrid model created by nesting Perl's Performance-Potential model within Calvert's transfer function approach to system simulation. Contemporary simulation tools combined with wearables clinical trial data is the foundation for a more agile platform for simulation of fitness and exploration of causality between training and physical performance. This platform offers the opportunity to strategically integrate data from various wearable sensors in a fashion enabling improved support for post-injury and return to sport decision-making.

Heterogeneous Network Approach to Predict Individuals’ Mental Health

Depression and anxiety are critical public health issues affecting millions of people around the world. To identify individuals who are vulnerable to depression and anxiety, predictive models have been built that typically utilize data from one source. Unlike these traditional models, in this study, we leverage a rich heterogeneous dataset from the University of Notre Dame’s NetHealth study that collected individuals’ (student participants’) social interaction data via smartphones, health-related behavioral data via wearables (Fitbit), and trait data from surveys. To integrate the different types of information, we model the NetHealth data as a heterogeneous information network (HIN). Then, we redefine the problem of predicting individuals’ mental health conditions (depression or anxiety) in a novel manner, as applying to our HIN a popular paradigm of a recommender system (RS), which is typically used to predict the preference that a person would give to an item (e.g., a movie or book). In our case, the items are the individuals’ different mental health states. We evaluate four state-of-the-art RS approaches. Also, we model the prediction of individuals’ mental health as another problem type—that of node classification (NC) in our HIN, evaluating in the process four node features under logistic regression as a proof-of-concept classifier. We find that our RS and NC network methods produce more accurate predictions than a logistic regression model using the same NetHealth data in the traditional non-network fashion as well as a random-approach. Also, we find that the best of the considered RS approaches outperforms all considered NC approaches. This is the first study to integrate smartphone, wearable sensor, and survey data in a HIN manner and use RS or NC on the HIN to predict individuals’ mental health conditions.

Wearable Sensor Data Based Human Behavior Recognition: a Method of Data Feature Extraction

<p indent=0mm>With the development of wearable devices, it is great research value to conduct human behavior detection based on wearable sensor data. At present, most human behavior recognition work is based on images. However, there are two challenges in using computer vision technology for human behavior recognition. Firstly, it is difficult to make users involved in the data collection in the nature under the true state of motion data, before starting data collection, it is often necessary to train the personnel involved in data collection and strictly regulate their collection actions. Data that collected in that way will be a departure from the real life. Its research value will be discounted. Secondly, the privacy protection of data collectors is involved in the process of data collection. To this end, a data feature extraction algorithm is proposed based on deep learning. Firstly, introduces the branch structure of neural network on the basis of flexible convolution kernel setting to extract the depth characteristics of the original data at multiple scales. After that, the data features obtained by each branch are fused and used as the input experiment of the next convolution layer. The experimental results show that compared with the current mainstream algorithm, proposed algorithm has achieved better results in accuracy and recall rates on the three standard data sets of MHEALTH, WHARF and USCHAD. In addition, the method has been verified on two newer data sets, Stanford-ECM Dataset and DATAEGO Dataset, and the results show that the method has good generalization ability.

Translating Videos into Synthetic Training Data for Wearable Sensor-Based Activity Recognition Systems Using Residual Deep Convolutional Networks

Human activity recognition (HAR) using wearable sensors has benefited much less from recent advances in Deep Learning than fields such as computer vision and natural language processing. This is, to a large extent, due to the lack of large scale (as compared to computer vision) repositories of labeled training data for sensor-based HAR tasks. Thus, for example, ImageNet has images for around 100,000 categories (based on WordNet) with on average 1000 images per category (therefore up to 100,000,000 samples). The Kinetics-700 video activity data set has 650,000 video clips covering 700 different human activities (in total over 1800 h). By contrast, the total length of all sensor-based HAR data sets in the popular UCI machine learning repository is less than 63 h, with around 38 of those consisting of simple mode of locomotion activities like walking, standing or cycling. In our research we aim to facilitate the use of online videos, which exist in ample quantities for most activities and are much easier to label than sensor data, to simulate labeled wearable motion sensor data. In previous work we already demonstrated some preliminary results in this direction, focusing on very simple, activity specific simulation models and a single sensor modality (acceleration norm). In this paper, we show how we can train a regression model on generic motions for both accelerometer and gyro signals and then apply it to videos of the target activities to generate synthetic Inertial Measurement Units (IMU) data (acceleration and gyro norms) that can be used to train and/or improve HAR models. We demonstrate that systems trained on simulated data generated by our regression model can come to within around 10% of the mean F1 score of a system trained on real sensor data. Furthermore, we show that by either including a small amount of real sensor data for model calibration or simply leveraging the fact that (in general) we can easily generate much more simulated data from video than we can collect its real version, the advantage of the latter can eventually be equalized.

Deep longitudinal phenotyping of wearable sensor data reveals independent markers of longevity, stress, and resilience.

Biological age acceleration (BAA) models based on blood tests or DNA methylation emerge as a de facto standard for quantitative characterizations of the aging process. We demonstrate that deep neural networks trained to predict morbidity risk from wearable sensor data can provide a high-quality and cheap alternative for BAA determination. The GeroSense BAA model was trained and validated using steps per minute recordings from 103,830 one-week long and 2,599 of up to 2 years-long longitudinal samples and exhibited a superior association with life-expectancy over the average number of steps per day in, e.g., groups stratified by professional occupations. The association between the BAA and effects of lifestyles, the prevalence of future incidence of diseases was comparable to that of BAA from models based on blood test results. Wearable sensors let sampling of BAA fluctuations at time scales corresponding to days and weeks and revealed the divergence of organism state recovery time (resilience) as a function of chronological age. The number of individuals suffering from the lack of resilience increased exponentially with age at a rate compatible with Gompertz mortality law. We speculate that due to the stochastic character of BAA fluctuations, its mean and auto-correlation properties together comprise the minimum set of biomarkers of aging in humans.

EmoCure: Utilising Social Media Data and Smartphones to Predict and Cure Depression

Depression is a common and recurring illness. It is a serious medical condition which might lead to self-harm. Various factors like unawareness about depression, shortage of medical health professionals and social stigmas make proper treatment for depression inaccessible. With the growing importance of technology in every sector of life, healthcare systems have also started using technology to provide better treatment. Various studies have shown that the widespread use of smartphones can be useful in predicting as well as treating depression by recommending activities. Smartphones can be very useful in the continuous monitoring of a patient which in turn helps to keep track of the activities of the patient. Social media data can also be used to find out the mental state of a patient. We propose EmoCure, a smartphone application that uses social media data, wearable sensor data, patient history, and smartphone usage patterns to predict, monitor, and treat depression using emotion regulating activities. We use machine learning models for finding out the sentiments in the social media posts. To predict depression, we use ensemble learning. We then recommend personalized emotion regulating activities whichever the user prefers.

A comprehensive, open-source dataset of lower limb biomechanics in multiple conditions of stairs, ramps, and level-ground ambulation and transitions

We introduce a novel dataset containing 3-dimensional biomechanical and wearable sensor data from 22 able-bodied adults for multiple locomotion modes (level-ground/treadmill walking, stair ascent/descent, and ramp ascent/descent) and multiple terrain conditions of each mode (walking speed, stair height, and ramp inclination). In this paper, we present the data collection methods, explain the structure of the open dataset, and report the sensor data along with the kinematic and kinetic profiles of joint biomechanics as a function of the gait phase. This dataset offers a comprehensive source of locomotion information for the same set of subjects to motivate applications in locomotion recognition, developments in robotic assistive devices, and improvement of biomimetic controllers that better adapt to terrain conditions. With such a dataset, models for these applications can be either subject-dependent or subject-independent, allowing greater flexibility for researchers to advance the field.

Limb and trunk accelerometer data collected with wearable sensors from subjects with Parkinson\u2019s disease

Parkinson’s disease (PD) is a neurodegenerative disorder characterized by motor and non-motor symptoms. Dyskinesia and motor fluctuations are complications of PD medications. An objective measure of on/off time with/without dyskinesia has been sought for some time because it would facilitate the titration of medications. The objective of the dataset herein presented is to assess if wearable sensor data can be used to generate accurate estimates of limb-specific symptom severity. Nineteen subjects with PD experiencing motor fluctuations were asked to wear a total of five wearable sensors on both forearms and shanks, as well as on the lower back. Accelerometer data was collected for four days, including two laboratory visits lasting 3 to 4 hours each while the remainder of the time was spent at home and in the community. During the laboratory visits, subjects performed a battery of motor tasks while clinicians rated limb-specific symptom severity. At home, subjects were instructed to use a smartphone app that guided the periodic performance of a set of motor tasks.

Accelerometer data collected with a minimum set of wearable sensors from subjects with Parkinson\u2019s disease

Parkinson’s disease (PD) is a neurodegenerative disorder associated with motor and non-motor symptoms. Current treatments primarily focus on managing motor symptom severity such as tremor, bradykinesia, and rigidity. However, as the disease progresses, treatment side-effects can emerge such as on/off periods and dyskinesia. The objective of the Levodopa Response Study was to identify whether wearable sensor data can be used to objectively quantify symptom severity in individuals with PD exhibiting motor fluctuations. Thirty-one subjects with PD were recruited from 2 sites to participate in a 4-day study. Data was collected using 2 wrist-worn accelerometers and a waist-worn smartphone. During Days 1 and 4, a portion of the data was collected in the laboratory while subjects performed a battery of motor tasks as clinicians rated symptom severity. The remaining of the recordings were performed in the home and community settings. To our knowledge, this is the first dataset collected using wearable accelerometers with specific focus on individuals with PD experiencing motor fluctuations that is made available via an open data repository.

Direct Lightweight Temporal Compression for Wearable Sensor Data

Emerging technologies enable massive deployment of wireless sensor networks across many industries. Internet of Things (IoT) devices are often deployed in critical infrastructure or health monitoring and require fast reaction time, reasonable accuracy, and high energy efficiency. In this letter, we introduce a lossy compression method for time-series data, named direct lightweight temporal compression (DLTC), enabling energy-efficient data transfer for power-restricted devices. Our method is based on the lightweight temporal compression method, targeting further reconstruction error minimization and complexity reduction. This letter highlights the key advantages of the proposed method and evaluates the method's performance on several sensor-based, time-series data types. We prove that DLTC outperforms the considered benchmark methods in compression efficiency at the same reconstruction error level.

Digital Phenotype for Childhood Internalizing Disorders: Less Positive Play and Promise for a Brief Assessment Battery.

Childhood internalizing disorders, like anxiety and depression, are common, impairing, and difficult to detect. Universal childhood mental health screening has been recommended, but new technologies are needed to provide objective detection. Instrumented mood induction tasks, designed to press children for specific behavioral responses, have emerged as means for detecting childhood internalizing psychopathology. In our previous work, we leveraged machine learning to identify digital phenotypes of childhood internalizing psychopathology from movement and voice data collected during negative valence tasks (pressing for anxiety and fear). In this work, we develop a digital phenotype for childhood internalizing disorders based on wearable inertial sensor data recorded from a Positive Valence task during which a child plays with bubbles. We find that a phenotype derived from features that capture reward responsiveness is able to accurately detect children with underlying internalizing psychopathology (AUC = 0.81). In so doing, we explore the impact of a variety of feature sets computed from wearable sensors deployed to two body locations on phenotype performance across two phases of the task. We further consider this novel digital phenotype in the context of our previous Negative Valence digital phenotypes and find that each task brings unique information to the problem of detecting childhood internalizing psychopathology, capturing different problems and disorder subtypes. Collectively, these results provide preliminary evidence for a mood induction task battery to develop a novel diagnostic for childhood internalizing disorders.

Wearable Technology and Analytics as a Complementary Toolkit to Optimize Workload and to Reduce Injury Burden.

Wearable sensors enable the real-time and non-invasive monitoring of biomechanical, physiological, or biochemical parameters pertinent to the performance of athletes. Sports medicine researchers compile datasets involving a multitude of parameters that can often be time consuming to analyze in order to create value in an expeditious and accurate manner. Machine learning and artificial intelligence models may aid in the clinical decision-making process for sports scientists, team physicians, and athletic trainers in translating the data acquired from wearable sensors to accurately and efficiently make decisions regarding the health, safety, and performance of athletes. This narrative review discusses the application of commercial sensors utilized by sports teams today and the emergence of descriptive analytics to monitor the internal and external workload, hydration status, sleep, cardiovascular health, and return-to-sport status of athletes. This review is written for those who are interested in the application of wearable sensor data and data science to enhance performance and reduce injury burden in athletes of all ages.

CDP-UA: Cognitive Data Processing Method Wearable Sensor Data Uncertainty Analysis in the Internet of Things Assisted Smart Medical Healthcare Systems.

Internet of Medical Things (IoMT) platform serves as an interoperable medium for healthcare applications by connecting wearable sensors, end-users, and clinical diagnosis centers. This interoperable medium provides solutions for disease diagnosis; predicting and monitoring end-user health using physiological vital signs sensed wearable sensor data. The communicating and data exchanging internet of things (IoT) platform imposes latency and overloading uncertainties in the heterogeneous environment. This article introduces cognitive data processing for uncertainty analysis (CDP-UA) to improve WS data management's efficiency. CDP-UA addresses uncertainties in two levels namely aggregation and dissemination of WS data. The uncertainties in synchronizing aggregation and dissemination slot mapping are addressed using classification learning. In the dissemination process overloaded intervals are identified and segregated using regression learning and conditional sigmoid function analysis. The joint learning process helps to classify overloaded and latency-centric dissemination and aggregation instances to improve WS data delivery in the clinical/medical analysis center. The experimental analysis shows that the proposed method is reliable in achieving less uncertainty factor, latency, and overloaded intervals for varying disseminations and sensing intervals.

RETRACTED ARTICLE: Error-less data fusion for posture detection using smart healthcare systems and wearable sensors for patient monitoring

Smart healthcare applications rely on frequent monitoring information from the patient’s body using wearable sensors (WSs). Healthcare systems rely on multi-level information for identifying the diseases and hence providing a proper diagnosis. The challenging task is the acquisition of WS information and assimilation of the same for diagnosis purposes. This proposal introduces an error-less data fusion (EDF) technique for improving the accuracy of posture identification. In a smart healthcare system, WS data fusion requires intense care for providing sensitive analysis of the diagnosed disease. Therefore, it depends on the WS data and collectively analyzes the same frequency for improving diagnosis efficiency. Errors due to the sensors’ failure, invariant time factor, aggregation, and analysis result in failed or inappropriate recommendations. This proposal addresses this issue using EDF relevant for posture detection of patients through healthcare monitoring systems. The WS data features are intensively analyzed using active and repeated learning for identifying the errors in different fusion processes. This technique is keen on improving the precision of posture detection, time of analysis, and error rate, irrespective of the users’ movement actions.

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. These networks automate the process of feature extraction relying completely on data. However, various noises in time series data with complex inter-modal relationships among sensors make this process more complicated. In this article, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process to make accurate recognition of actions by combining varieties of features extracted from diverse perspectives. Initially, instead of using single type of transformation, numerous transformations are employed on time series data to obtain variegated representations of the features encoded in raw data. An efficient deep CNN architecture is proposed that can be individually trained to extract features from different transformed spaces. Later, these CNN feature extractors are merged into an optimal architecture finely tuned for optimizing diversified extracted features through a combined training stage or multiple sequential training stages. This approach offers the opportunity to explore the encoded features in raw sensor data utilizing multifarious observation windows with immense scope for efficient selection of features for final convergence. Extensive experimentations have been carried out in three publicly available datasets that provide outstanding performance consistently with average five-fold cross-validation accuracy of 99.29% on UCI HAR database, 99.02% on USC HAR database, and 97.21% on SKODA database outperforming other state-of-the-art approaches.