Body Motion Data Research Articles

A contact sensor-free framework for ground reaction force observation in heavy-legged robots considering unknown loads

As a direct response to robot movements and load distributions, the ground reaction force (GRF) is pivotal for heavy-legged robot (HLR) applications. This study presents a technical framework for GRF monitoring in an electric cylinder-driven HLR, eliminating the need for information on body motion, load distributions, and measured servo outputs. Traditional joint space-based dynamics are extended to include servo currents, compensating for the influence of unknown servo outputs. An essential contribution is incorporating the impact of floating bases and unknown loads into a virtual spatial force (VSF) applied to the HLR hip joint. The VSF is obtained through the nonlinear disturbance observer when the HLR is in a stable contact phase. Subsequently, a high-order GRF observer (HOGO), compensated with VSF, enables GRF observations without the pre-acquired body movement and load distribution data. In contrast to conventional GRF observations, the proposed framework could determine virtual payloads acting on the hip joint while ensuring precise GRF monitoring without requiring supplementary sensors. The GRF observations of the proposed framework are experimentally superior to those of the conventional methods with unknown HLR body motion and load information.

A multi-modality and multi-dyad approach to measuring flexibility in psychotherapy

Introduction Flexibility, the ability of an individual to adapt to environmental changes in ways that facilitate goal attainment, has been proposed as a potential mechanism underlying psychopathology and psychotherapy. In psychotherapy, most findings are based on self-report measures that have important limitations. We propose a multimodal, multi-dyad approach based on a nonlinear dynamical systems framework to capture the complexity of this concept. Method A new research paradigm was designed to explore the validity of the proposed conceptual model. The paradigm includes a psychotherapy-like social interaction, during which body movement and facial expressiveness data were collected. We analyzed the data using Hankel Alternative View of Koopmann analysis to reconstruct attractors of the observed behaviors and compare them. Results The patterns of behavior in the two cases differ, and differences in the reconstructed attractors correspond with differences in self-report measures and behavior in the interactions. Conclusions The case studies show that information provided by a single modality is not enough to provide the full picture, and multiple modalities are needed. These observations can serve as an initial support for our claims that a multi-modal and multi-dyad approach to flexibility can address some of the issues of measurement in the field.

Cooperative Learning for Personalized Context-aware Pain Assessment from Wearable Data.

Despite the promising performance of automated pain assessment methods, current methods suffer from performance generalization due to the lack of relatively large, diverse, and annotated pain datasets. Further, the majority of current methods do not allow responsible interaction between the model and user, and do not take different internal and external factors into consideration during the model's design and development. This paper aims to provide an efficient cooperative learning framework for the lack of annotated data while facilitating responsible user communication and taking individual differences into consideration during the development of pain assessment models. Our results using body and muscle movement data, collected from wearable devices, demonstrate that the proposed framework is effective in leveraging both the human and the machine to efficiently learn and predict pain.

Deep learning-based sleep stage classification with cardiorespiratory and body movement activities in individuals with suspected sleep disorders

Deep learning methods have gained significant attention in sleep science. This study aimed to assess the performance of a deep learning-based sleep stage classification model constructed using fewer physiological parameters derived from cardiorespiratory and body movement data. Overnight polysomnography (PSG) data from 123 participants (age: 19–82 years) with suspected sleep disorders were analyzed. Multivariate time series data, including heart rate, respiratory rate, cardiorespiratory coupling, and body movement frequency, were input into a bidirectional long short-term memory (biLSTM) network model to train and predict five-class sleep stages. The trained model's performance was evaluated using balanced accuracy, Cohen's κ coefficient, and F1 scores on an epoch-per-epoch basis and compared with the ground truth using the leave-one-out cross-validation scheme. The model achieved an accuracy of 71.2 ± 5.8%, Cohen's κ of 0.425 ± 0.115, and an F1 score of 0.650 ± 0.083 across all sleep stages, and all metrics were negatively correlated with the apnea–hypopnea index, as well as age, but positively correlated with sleep efficiency. Moreover, the model performance varied for each sleep stage, with the highest F1 score observed for N2 and the lowest for N3. Regression and Bland–Altman analyses between sleep parameters of interest derived from deep learning and PSG showed substantial correlations (r = 0.33–0.60) with low bias. The findings demonstrate the efficacy of the biLSTM deep learning model in accurately classifying sleep stages and in estimating sleep parameters for sleep structure analysis using a reduced set of physiological parameters. The current model without using EEG information may expand the application of unobtrusive in-home monitoring to clinically assess the prevalence of sleep disorders outside of a sleep laboratory.

An Integrated Eye-Tracking and Motion Capture System in Synchronized Gaze and Movement Analysis.

Integrating mobile eye-tracking and motion capture emerges as a promising approach in studying visual-motor coordination, due to its capability of expressing gaze data within the same laboratory-centered coordinate system as body movement data. In this paper, we proposed an integrated eye-tracking and motion capture system, which can record and analyze temporally and spatially synchronized gaze and motion data during dynamic movement. The accuracy of gaze measurement were evaluated on five participants while they were instructed to view fixed vision targets at different distances while standing still or walking towards the targets. Similar accuracy could be achieved in both static and dynamic conditions. To demonstrate the usability of the integrated system, several walking tasks were performed in three different pathways. Results revealed that participants tended to focus their gaze on the upcoming path, especially on the downward path, possibly for better navigation and planning. In a more complex pathway, coupled with more gaze time on the pathway, participants were also found having the longest step time and shortest step length, which led to the lowest walking speed. It was believed that the integration of eye-tracking and motion capture is a feasible and promising methodology quantifying visual-motor coordination in locomotion.

Learning data science methods through a mobile device and full body motion data

This study presents a mobile app that facilitates undergraduate students to learn data science through their own full body motions. Leveraging the built-in camera of a mobile device, the proposed app captures the user and feeds their images into an open-source computer-vision algorithm that localizes the key joint points of human body. As students can participate in the entire data collection process, the obtained motion data is context-rich and personally relevant to them. The app utilizes the collected motion data to explain various concepts and methods in data science under the context of human movements. The app also visualizes the geometric interpretation of data through various visual aids, such as interactive graphs and figures. In this study, we use principal component analysis, a commonly used dimensionality reduction method, as an example to demonstrate the proposed learning framework. Strategies to encompass other learning modules are also discussed for further improvement.

Development of a real‐time work‐related postural risk assessment system of farm workers using a sensor‐based artificial intelligence approach

AbstractIn recent years, the promotion of farm mechanization has been directed toward reducing the human discomfort and fatigue associated with various agricultural work‐related activities. During these activities, many factors (like force, awkward posture, vibration, repetition, etc.) play a significant role in causing musculoskeletal disorders. Second, ergonomic risk assessment of physical work is conventionally conducted through observation and direct/indirect physiological measurements. However, these methods are time‐consuming and require human subjects to perform the motion to obtain detailed body movement data. In the present study, a semiautomatic rapid entire body assessment (REBA) evaluation tool is developed for real‐time assessment of agricultural work‐related musculoskeletal disorders risk of farm workers using Kinect V2 sensor‐based artificial intelligence approach. It allows the investigator speedy detect of awkward postures leading to critical conditions and to reduce subjective bias. It is useful to analyze online as well as offline posture analysis, it detects the critical areas of the body posture, which may lead to the musculoskeletal disorders of agricultural workers, and suggest aptly to correct the posture. The Kinect V2 REBA assessment score was found with a factual significant match with the reference expert evaluation as reflected by the Landis and Koch scale k = 0.673 (p < 0.001), 95% confidence interval (CI) for the left side, and k = 0.644 (p < 0.001), 95% CI for the right side of the body respectively.

An Exploration of Prediction Accuracy of Developmental Degrees of Music Expression in Early Childhood Based on A Simultaneous Analysis of Both Eye Movement and Full-Body Movement

This article explores validity to introduce eye movement of children in association with body movement as feature quantities of machine learning, in particular, to assess children’s maturity level of musical expression. There is a growing awareness of applying machine learning technique to yield practical benefit on predicting developmental degrees of music education by observing body movements with motion capture technology. From professional teaching perspective, it is believed only experienced teachers can evaluate children’s music achievement because coordinated functioning of body movements need carefully be observed in connection with various music factor such as rhythm, beat strength, tones, etc. If new teachers can take advantage of having objective results of evaluation of children’s expression level by machines, it will enhance educational efficiency to certain levels which experts could attain. The author aims to improve classification accuracy of machine learning by focusing on eye movements data simultaneously recorded with body movements. In this study, children (n=43) at two child facilities participated in the data capture of both eye movement and body movement during musical expression. Feature quantities were extracted from the results of a three-way of ANOVA, and applied to machine learning to improve the classification accuracy of developmental degree in musical expression. Specifically, when using several classifiers such as NN (Neural Network model), the classification accuracy of developmental degree of musical development was more precisely in the case of both body movement data and eye movement data included in feature quantities than in the case of only body movement data as feature quantities.

Non-contact determination of sleep/wake state in residential environments by neural network learning of microwave radar and electroencephalogram–electrooculogram measurements

This study developed and validated a non-contact, integrated sleep measurement method using microwave radar for daily sleep measurement in a house. In order to replicate the participant's normal sleep as much as possible, we employed a portable electroencephalogram–electrooculogram (EEG–EOG) recording system, which has the lowest burden and established reliability in the bedroom environment. A total of 38 nights of sleep for 15 participants were simultaneously recorded using the portable EEG–EOG recording system, actigraph, and microwave radar in the bedroom of typical houses. The sleep/wake state determination obtained by applying EEG–EOG data to a deep learning-based sleep staging program was used as the target variable. A sleep/wake state determination prediction model was obtained by learning body movement data from microwave radar using a neural network. The inter-method reliability was evaluated using sleep parameters, including sleep onset latency (SOL), wake after sleep onset (WASO), total sleep time (TST), and sleep efficiency (SE), by using paired t-test. Based on this prediction model, the average accuracy, sensitivity, specificity, F1-score*100, and Cohen's kappa*100 for all participants' sleep/wake state determination using microwave radar were 95.27%, 70.11%, 97.05%, 65.95, and 63.46, respectively. Compared with the portable EEG–EOG recording system, microwave radar showed no statistically significant differences in SOL, TST, and SE in evaluating sleep parameters. As the main conclusion, in the sleep/wake state determination, microwave radar showed performance comparable with or better than actigraph. It serves for continuous non-contact sleep measurement in residential environments.

Analysis of Human Joint Mechanics Based on High-Precision Unmarked Motion Capture System

The analysis of human joint mechanics is of great value in rehabilitation assessment and competitive sports training. There are many limitations in the application of the existing motion capture system, which makes the usage scenarios relatively limited. To solve this problem, this paper proposes a multi-camera 3D motion capture system based on reprojection optimization, and combines the foot mechanics data to obtain the human joint torque. First, build a multi-camera-based motion capture system to obtain human body motion data, reconstruct key points to obtain 3D data, and then propose an accurate key point algorithm based on weight-based reprojection optimization to obtain precise coordinates of key points. Finally, the coordinates of the key points combined with the mechanical data of the foot are calculated by OpenSim to obtain the mechanical data of the human joints. The experimental results show that the capture accuracy of key points is 5% higher than the average accuracy of the traditional least square method, and the analysis of joint mechanics data conforms to the laws of motion, which can be used to guide rehabilitation evaluation and competitive sports.

Advancing biomedical engineering through a multi-modal sensor fusion system for enhanced physical training

<abstract> <p>In this paper, we introduce a multi-modal sensor fusion system designed for biomedical engineering, specifically geared toward optimizing physical training by collecting detailed body movement data. This system employs inertial measurement units, flex sensors, electromyography sensors, and Microsoft's Kinect V2 to generate an in-depth analysis of an individual's physical performance. We incorporate a gated recurrent unit- recurrent neural network algorithm to achieve highly accurate body and hand motion estimation, thus surpassing the performance of traditional machine learning algorithms in terms of accuracy, precision, recall, and F1 score. The system's integration with the PICO 4 VR environment creates a rich, interactive experience for physical training. Unlike conventional motion capture systems, our sensor fusion system is not limited to a fixed workspace, allowing users to engage in exercise within a flexible, free-form environment.</p> </abstract>

Arm motion symmetry in conversation

Data-driven synthesis of human motion during conversational speech is an active research area with applications that include character animation, computer gaming and conversational agents. Natural looking motion is key to both perceived realism and understanding of any synthesised animation. Multi-modal speech and body-motion data is scarce and limited, so it is common to augment real motion data by mirroring the body pose to double the number of training samples. This augmentation is based on the assumption that a person’s gesturing is not affected by handedness and that the reflected pose is plausible. In this study, we explore the validity of this assumption by evaluating the reflective symmetry of a speaker’s arms during conversational exchanges. We analyse the left and right arm motion of 36 subjects during dyadic conversation and present the per-frame symmetry of the arm gestures. To identify temporal offsets caused by the presence of a leading hand, we compute the time lag between movements of the left and right arms. We perform a nearest neighbour search to test the validity of any mirrored pose. We also consider information theory to examine the information gain from mirroring the data. We implement a speech-to-gesture generative model to determine the efficacy of lateral mirroring techniques for data augmentation. Our findings suggest that both positional symmetry and left–right motion offsets vary from speaker to speaker. We conclude that data augmentation by mirroring is valid in certain cases when considering the mirrored pose as a new virtual identity, but that it should be carefully considered as a generic approach if the gesturing style and handedness of the original speaker is to be maintained. • Review the motion symmetry of multiple speakers during dyadic conversation. • Analyse positional, temporal and informational symmetry of arm motion. • Discuss the efficacy of lateral mirroring of the human body for data augmentation. • Conclude lateral mirroring is not suited as a generic approach. • Suggest lateral mirroring as a new identity is a suitable data augmentation method. • Propose our statistical analysis for evaluating speech-driven conversational agents.

The Function of the Proprioceptive, Vestibular and Visual Systems Following Fatigue in Individuals With and Without Chronic Ankle Instability.

To maintain postural balance, the proprioceptive, vestibular, and visual systems continuously provide body position and movement data to the central nervous system. In this study, our main aim was to examine, for the first time, the influence of anaerobically or aerobically induced fatigue on these separate functions in persons with and without chronic ankle instability (CAI). We obtained assessments pre- and post-fatigue protocols from 60 physical education students (Mage = 24.3, SD = 3.4) Twenty-seven students had CAI, and 33 students did not have CAI). To measure proprioception, we used the AMEDA device; for vision, we used near point of convergence (NPC); and, for vestibular function, we used subjective visual vertical (SVV). We found a pre-post proprioception (AMEDA) effect in the aerobic group (p < .001), and a visual (NPC) effect in both anaerobic and aerobic participant groups (both p < .001). There were no visual system (NPC) fatigue effect differences among aerobic or anerobic participants who had or did not have CAI (p = .047); there was a significant aerobic fatigue effect on proprioception (AMEDA) (p = .010) that favored participants without CAI. There was a significant interaction effect between time of testing and CAI for visual (NPC) (p = .003) in the aerobic group only. In both the anaerobic and aerobic groups, post-fatigue vestibular function (AMEDA) was significantly lower for those with than those without CAI (anaerobic: p = .030; and aerobic: p =.016). Thus, post-fatigue, participants with CAI showed worse proprioceptive, visual, and vestibular function than those without CAI. Future investigators should further examine each movement sense system in individuals with CAI.

User Identification Utilizing Minimal Eye-Gaze Features in Virtual Reality Applications

Emerging Virtual Reality (VR) displays with embedded eye trackers are currently becoming a commodity hardware (e.g., HTC Vive Pro Eye). Eye-tracking data can be utilized for several purposes, including gaze monitoring, privacy protection, and user authentication/identification. Identifying users is an integral part of many applications due to security and privacy concerns. In this paper, we explore methods and eye-tracking features that can be used to identify users. Prior VR researchers explored machine learning on motion-based data (such as body motion, head tracking, eye tracking, and hand tracking data) to identify users. Such systems usually require an explicit VR task and many features to train the machine learning model for user identification. We propose a system to identify users utilizing minimal eye-gaze-based features without designing any identification-specific tasks. We collected gaze data from an educational VR application and tested our system with two machine learning (ML) models, random forest (RF) and k-nearest-neighbors (kNN), and two deep learning (DL) models: convolutional neural networks (CNN) and long short-term memory (LSTM). Our results show that ML and DL models could identify users with over 98% accuracy with only six simple eye-gaze features. We discuss our results, their implications on security and privacy, and the limitations of our work.

A Recognition Method of Athletes’ Mental State in Sports Training Based on Support Vector Machine Model

Athletes participate in competitive competitions, and the ultimate goal is to better display their personal competitive level in the competition so as to achieve the goal of defeating their opponents and winning the competition. In all types of competitions, most matches are instantaneous, and opportunities are fleeting. The instantaneous nature and fierce competition of sports competitions require athletes who participate in sports competitions to have a high psychological quality. It can be seen that the quality of the mental state directly determines the performance of the athletes in usual training and competition. In the process of sports, if athletes can obtain real-time changes in their mental states when they encounter various situations, they can formulate more targeted and effective training or competition strategies according to the athletes’ states. For the opponent, by analyzing the opponent’s psychological state during exercise, the game strategy can be adjusted in real time in a targeted manner, and the probability of winning the game can be provided. Based on this background, this paper proposes to use support vector machine (SVM) to identify the mental state of athletes during exercise. This paper first collects the data of body movements and facial expressions of athletes during training or competition. Use multimodal data to train an SVM model. Output the emotional state of athletes at different stages based on test data. In order to verify the applicability of the method in this paper to the athlete subjects, several comparative models were used in the experiment to verify the performance of the used models. The experimental results show that the accuracy rate of emotion recognition obtained by this method is more than 80%. This shows that the research in this paper has certain application value.

Aided Evaluation of Motion Action Based on Attitude Recognition

For athletes who are eager for success, it is difficult to obtain their own movement data due to field equipment, artificial errors, and other factors, which means that they cannot get professional movement guidance and posture correction from sports coaches, which is a disastrous problem. To solve this big problem, combined with the latest research results of deep learning in the field of computer technology, based on the related technology of human posture recognition, this paper uses convolution neural network and video processing technology to create an auxiliary evaluation system of sports movements, which can obtain accurate data and help people interact with each other, so as to help athletes better understand their body posture and movement data. The research results show that: (1) using OpenPose open-source library for pose recognition, joint angle data can be obtained through joint coordinates, and the key points of video human posture can be identified and calculated for easy analysis. (2) The movements of the human body in the video are evaluated. In this way, it is judged whether the action amplitude of the detected target conforms to the standard action data. (3) According to the standard motion database created in this paper, a formal motion auxiliary evaluation system is established; compared with the standard action, the smaller the Euclidean distance is, the more standard it is. The action with an Euclidean distance of 4.79583 is the best action of the tested person. (4) The efficiency of traditional methods is very low, and the correct recognition rate of the method based on BP neural network can be as high as 96.4%; the correct recognition rate of the attitude recognition method based on this paper can be as high as 98.7%, which is 2.3% higher than the previous method. Therefore, the method in this paper has great advantages. The research results of the sports action assistant evaluation system in this paper are good, which effectively solves the difficult problems that plague athletes and can be considered to have achieved certain success; the follow-up system test and operation work need further optimization and research by researchers.

Prediction of Parkinson's Disease Using Deep Learning in TensorFlow

One of the most common neurodegenerative disorders of the present age is Parkinson’s Disease or Parkinsonism. To estimate its advancement in the patient, huge amounts of data are being collected and studied to draw out inferences. The types of data generally studied towards that end are vocal data, body movement data, eye movement data, handwriting and drawing patterns, etc. In this work, the use of a Deep Neural Network has been proposed which can predict the Unified Parkinson's Disease Rating Scale (UPDRS) both motor and total by studying vocal data from UCI Machine Learning Repository. Both 2 layered as well as 3 layered networks were studied and it was found that the performance of 3-layer Deep Neural Network having 10, 20, 10 neurons in different layers was found to be the best with an accuracy of 97% and 99.62% for motor UPDRS and total UPDRS respectively. The other three parameters MSE, MAE and RMSE also showed improvement in the 3 layered model as compared to the 2 layered model.

A New Approach to Estimate Concentration Levels with Filtered Neural Nets for Online Learning

The COVID‐19 pandemic heavily influenced human life by constricting human social activity. Following the spread of the pandemic, humans did not have a choice but to change their lifestyles. There has been much change in the field of education, which has led to schools hosting online classes as an alternative to face‐to‐face classes. However, the concentration level is lowered in the online learning class, and the student’s learning rate decreases. We devise a framework for recognizing and estimating students’ concentration levels to help lecturers. Previous studies have a limitation in that they classified attention levels using only discrete states. Due to the partial information from discrete states, the concentration levels could not be recognized well. This research aims to estimate more subtle levels as specified states by using a minimum amount of body movement data. The deep neural network is used to continuously recognize the human concentration model, and the concentration levels can be predicted and estimated by the Kalman filter. Using our framework, we successfully extracted the concentration levels, which can aid lecturers and can be expanded to other areas. To implement the framework, we recruited participants to take online classes. Data were collected and preprocessed using pose points, and an accuracy of 90.62 % was calculated by predicting the concentration level using the framework. Furthermore, the concentration level was approximated based on the Kalman filter. We found that webcams can be used to quantitatively measure student concentration when conducting online classes. Our framework is a great help for instructors to measure concentration levels, which can increase the learning efficiency. As a future work of this study, if emotion data and skin thermal data are comprehensively considered, a student’s concentration level can be measured more precisely.

Classification of human body motions using an ultra-wideband pulse radar.

BACKGROUND: The motion or gestures of a person are primarily recognized by detecting a specific object and the change in its position from image information obtained via an image sensor. However, the use of such systems is limited due to privacy concerns.OBJECTIVE: To overcome these concerns, this study proposes a radar-based motion recognition method.METHODS: Detailed human body movement data were generated using ultra-wideband (UWB) radar pulses, which provide precise spatial resolution. The pulses reflected from the body were stacked to reveal the body’s movements and these movements were expressed in detail in the micro-range components. The collected radar data with emphasized micro-ranges were converted into an image. Convolutional neural networks (CNN) trained on radar images for various motions were used to classify specific motions. Instead of training the CNNs from scratch, transfer learning is performed by importing pretrained CNNs and fine-tuning their parameters with the radar images. Three pretrained CNNs, Resnet18, Resnet101, and Inception-Resnet-V2, were retrained under various training conditions and their performance was experimentally verified.RESULTS: As a result of various experiments, we conclude that detailed motions of subjects can be accurately classified by utilizing CNNs that were retrained with images obtained from the UWB pulse radar.

Effects of Pubertal Maturation on ACL Forces During a Landing Task in Females

Background: Rates of anterior cruciate ligament (ACL) rupture in young people have increased by >70% over the past two decades. Adolescent and young adult females are at higher risk of ACL injury as compared with their prepubertal counterparts. Purpose: To determine ACL loading during a standardized drop-land-lateral jump in females at different stages of pubertal maturation. Study Design: Controlled laboratory study. Methods: On the basis of the Tanner classification system, 19 pre-, 19 early-/mid-, and 24 late-/postpubertal females performed a standardized drop-land-lateral jump while 3-dimensional body motion, ground-reaction forces, and surface electromyography data were acquired. These data were used to model external biomechanics, lower limb muscle forces, and knee contact forces, which were subsequently used in a validated computational model to estimate ACL loading. Statistical parametric mapping analysis of variance was used to compare ACL force and its causal contributors among the 3 pubertal maturation groups during stance phase of the task. Results: When compared with pre- and early-/midpubertal females, late-/postpubertal females had significantly higher ACL force with mean differences of 471 and 356 N during the first 30% and 48% to 85% of stance, and 343 and 274 N during the first 24% and 59% to 81% of stance, respectively, which overlapped peaks in ACL force. At the point of peak ACL force, contributions from sagittal and transverse plane loading mechanisms to ACL force were higher in late-/postpubertal compared with pre- and early-/midpubertal groups (medium effect sizes from 0.44 to 0.77). No differences were found between pre- and early-/midpubertal groups in ACL force or its contributors. Conclusion: The highest ACL forces were observed in late-/postpubertal females, consistent with recently reported rises of ACL injury rates in females aged 15 to 19 years. It is important to quantify ACL force and its contributors during dynamic tasks to advance our understanding of the loading mechanism and thereby provide guidance to injury prevention. Clinical Relevance: Growth of ACL volume plateaus around 10 years of age, before pubertal maturation, meaning that a late-/postpubertal female could have an ACL of similar size to their less mature counterparts. However, late-/postpubertal females have higher body mass requiring higher muscle forces to accelerate the body during dynamic tasks, which may increase ACL loading. Thus, if greater forces develop in these females, in part because of their increased body mass, these higher forces will be applied to an ACL that is not proportionally larger. This may partially explain the higher rates of ACL injury in late-/postpubertal females.