Human Motion Data Research Articles

Activity triangles: a new approach to measure activity spaces

There is an on-going challenge to describe, analyse and visualise the actual and potential extent of human spatial behaviour. The concept of an activity space has been used to examine how people interact with their environment and how the actual or potential spatial extent of individual spatial behaviour can be defined. In this paper, we introduce a new method for measuring activity spaces. We first focus on the definitions and the applications of activity space measures, identifying their respective limitations. We then present our new method, which is based on the theoretical concept of significant locations, that is, places where people spent most of their time. We identify locations of significant places from GPS trajectories and define the activity space of an individual as a set of the first three significant places forming a so-called “activity triangle”. Our new method links the distances travelled for different activities to whether or not people group their activities, which is not possible using existing methods of measuring activity spaces. We test our method on data from a GPS-based travel survey across three towns is Scotland and look at the variations in size and shape of the designed activity triangle among people of different age and gender. We also compare our activity triangle with five other activity spaces and conclude by providing possible routes for improvement of activity space measures when using real human movement data (GPS data).

Influence of built environment and user experience on the waterfront vitality of historical urban areas: A case study of the Qinhuai River in Nanjing, China

Urban waterfronts are important gateways that reflect a city's image and characteristics. Evaluating waterfront vitality and its influencing factors is critical for guiding urban waterfront planning and redevelopment. Hourly human movement data, provided by the Baidu Heatmap, were used to explore the weekday and weekend urban vitality spatial distribution characteristics of the waterfronts of the Qinhuai River in Nanjing. Global (ordinary least squares) and local (multiscale geographically weighted regression) models revealed the influence of physical–environmental characteristics (objective) and spatial experience evaluation factors (subjective) on urban vitality. (1) The Qinhuai River waterfront urban vitality agglomeration characteristics were similar between weekdays and weekends, and the core vitality areas were distributed in the dense tourism, commercial, and residential areas along the river. (2) The evaluation of catering experience had the strongest positive correlation with vitality, whereas the waterfront distance had the strongest negative correlation. (3) The influence of each factor on waterfront vitality in the study area exhibited considerable spatial differences, with attenuation trends observed from the east-to-west river sections. This study shows the urban vitality distribution characteristics of the Qinhuai River waterfront spaces. Exploring the influence of physical–environmental and spatial evaluation factors on the vitality distribution can provide a scientific basis and reference for urban waterfront planning and redevelopment focusing on cultivating vitality.

Modelling the impact of interventions on imported, introduced and indigenous malaria infections in Zanzibar, Tanzania

Malaria cases can be classified as imported, introduced or indigenous cases. The World Health Organization’s definition of malaria elimination requires an area to demonstrate that no new indigenous cases have occurred in the last three years. Here, we present a stochastic metapopulation model of malaria transmission that distinguishes between imported, introduced and indigenous cases, and can be used to test the impact of new interventions in a setting with low transmission and ongoing case importation. We use human movement and malaria prevalence data from Zanzibar, Tanzania, to parameterise the model. We test increasing the coverage of interventions such as reactive case detection; implementing new interventions including reactive drug administration and treatment of infected travellers; and consider the potential impact of a reduction in transmission on Zanzibar and mainland Tanzania. We find that the majority of new cases on both major islands of Zanzibar are indigenous cases, despite high case importation rates. Combinations of interventions that increase the number of infections treated through reactive case detection or reactive drug administration can lead to substantial decreases in malaria incidence, but for elimination within the next 40 years, transmission reduction in both Zanzibar and mainland Tanzania is necessary.

Where do people meet? Time-series clustering for social interaction levels in daily-life spaces during the COVID-19 pandemic

The persistence of spatial segregation with respect to income and race is well documented. However, assessment of spatial segregation in daily activities is challenging due to the limited availability of human movement data. With the ubiquitous availability of mobile phones and location-based service applications, human movement data has become widely available. It is now possible to explore spatial interactions and assess the extent of social segregation in daily activity spaces. Using Los Angeles County as our case study, we perform a temporal analysis by conducting K-means time-series clustering using mobile phone data to examine social interaction levels among various sociodemographic groups during the COVID-19 pandemic. Selected sociodemographic variables are assessed among the identified time-series clusters. We find a strong association between sociodemographic characteristics and social interaction levels, potentially leading to disparate exposures to the risk from COVID-19. Socially disadvantaged populations tend to be more segregated from other groups in daily activities, and the COVID-19 pandemic increases the disparity. Low-income and ethnic minority populations became more isolated from Whites and the more economically affluent during the COVID-19 pandemic. Policies that aim to encourage social interactions and mitigate segregation effectively should further consider people's sociodemographic variables and relevant neighborhood characteristics.

Skeleton-Based Fall Detection with Multiple Inertial Sensors Using Spatial-Temporal Graph Convolutional Networks.

The application of wearable devices for fall detection has been the focus of much research over the past few years. One of the most common problems in established fall detection systems is the large number of false positives in the recognition schemes. In this paper, to make full use of the dependence between human joints and improve the accuracy and reliability of fall detection, a fall-recognition method based on the skeleton and spatial-temporal graph convolutional networks (ST-GCN) was proposed, using the human motion data of body joints acquired by inertial measurement units (IMUs). Firstly, the motion data of five inertial sensors were extracted from the UP-Fall dataset and a human skeleton model for fall detection was established through the natural connection relationship of body joints; after that, the ST-GCN-based fall-detection model was established to extract the motion features of human falls and the activities of daily living (ADLs) at the spatial and temporal scales for fall detection; then, the influence of two hyperparameters and window size on the algorithm performance was discussed; finally, the recognition results of ST-GCN were also compared with those of MLP, CNN, RNN, LSTM, TCN, TST, and MiniRocket. The experimental results showed that the ST-GCN fall-detection model outperformed the other seven algorithms in terms of accuracy, precision, recall, and F1-score. This study provides a new method for IMU-based fall detection, which has the reference significance for improving the accuracy and robustness of fall detection.

Investigating the Characteristics of Urban Comprehensive Hospitals from a Supply–Demand Balance Perspective: A Case Study of Three Districts in Shenzhen Based on Multi-Source Data

As an important public facility in cities, hospitals provide services that are vital to people’s lives. Big data is representative of geographic data from numerous sources. It has the advantages of large amounts of data, high timeliness, and abundant information, which may compensate for the inadequacy of traditional planning and design, which is overly subjective. Big data can be used to obtain large-scale human movement data to study hospital characteristics from a supply–demand balance perspective. Here, 182 comprehensive hospitals in three districts of Shenzhen were picked as an example, and multi-source data were introduced as a quantitative assessment tool to analyze them. First, the mobile public participation geographic information system (PPGIS) was researched and developed independently and used to collect volunteered geographic information (VGI) data, which was adopted to quantify hospital service supply. Second, the population’s demand for hospitals was characterized by the quantified point of interest (POI) data. Finally, the location entropy method was utilized to quantify the levels of supply and demand for comprehensive hospital services from a supply–demand balance perspective, which was used to guide the classification of the comprehensive hospitals. The result shows that: (1) the service scopes and the service pressure are different for the same type of comprehensive hospitals classified by the traditional method; (2) the spatial distribution of patients is related to land function; (3) the overlap area between the part with a high service pressure level in the comprehensive hospital and the part with high medical demand of the patient is prone to service blindness. This study paves a new way to optimize the spatial allocation of medical institutions or other types of public facilities, rationalize resource allocation, and alleviate urban transportation congestion.

A Music-Driven Deep Generative Adversarial Model for Guzheng Playing Animation.

To date relatively few efforts have been made on the automatic generation of musical instrument playing animations. This problem is challenging due to the intrinsically complex, temporal relationship between music and human motion as well as the lacking of high quality music-playing motion datasets. In this article, we propose a fully automatic, deep learning based framework to synthesize realistic upper body animations based on novel guzheng music input. Specifically, based on a recorded audiovisual motion capture dataset, we delicately design a generative adversarial network (GAN) based approach to capture the temporal relationship between the music and the human motion data. In this process, data augmentation is employed to improve the generalization of our approach to handle a variety of guzheng music inputs. Through extensive objective and subjective experiments, we show that our method can generate visually plausible guzheng-playing animations that are well synchronized with the input guzheng music, and it can significantly outperform the state-of-the-art methods. In addition, through an ablation study, we validate the contributions of the carefully-designed modules in our framework.

Skill learning framework for human–robot interaction and manipulation tasks

In this article, a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed. The learning framework makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way. This approach offers human tutors the ability to control all joints of the robotic manipulator in real-time and able to achieve complex manipulation. The robotic manipulator is controlled remotely with our low-cost wearable devices for easy calibration and continuous motion mapping. We believe that our approach might lead to improving the human-robot skill learning, adaptability, and sensitivity of the proposed human-robot interaction for flexible task execution and thereby giving room for skill transfer and repeatability without complex coding skills. • This article presents a system that enables remote teleoperation based on wearable sensors for 6-Degree of freedom robotic manipulation control appication. • A skill learning framework is proposed based on dynamic movement primitives model from human manipulability to robot with the capability of replicating human demonstration.

Omnidirectional Spectrogram Generation for Radar-Based Omnidirectional Human Activity Recognition

Micro-Doppler-based human activity recognition has been extensively researched in remote sensing. However, a well-performing classifier requires sufficient radar data of omnidirectional human movements due to the “angle sensitivity”, resulting in high costs for radar data acquisition. To address this issue, this study defines for the first time the task of “omnidirectional spectrogram generation” and proposes a method to obtain enough omnidirectional spectrograms based on the spectrograms of human movements in one direction. It significantly reduces the dependence on radar measurements with omnidirectional setups. Our method is founded upon an image translation framework that is enhanced by incorporating the concept of information disentanglement and a proposed feature-level unbiased domain translation strategy. They enable us to generate high-quality omnidirectional spectrograms at various aspect angles. The generated spectrograms are then used as training support of omnidirectional micro-Doppler-based classifiers. Subsequently, we conduct an in-depth analysis of the metric correlation between the quality of generated spectrograms and the performance of these classifiers. Finally, we introduce a method for evaluating this correlation by proposed criterion. Our method is evaluated based on a radar simulation dataset, and the results show that it significantly exceeds the compared methods, demonstrating its great potential for the task of omnidirectional recognition of human activities. Besides, we find that there is a significant correlation between classification accuracy and several image quality assessment metrics, and we believe that this investigation will serve as the foundation for future research on assessing the contribution of data generation methods to downstream tasks using quantitative measures.

Analysis of Human Joint Mechanics Based on High-Precision Unmarked Motion Capture System

The analysis of human joint mechanics is of great value in rehabilitation assessment and competitive sports training. There are many limitations in the application of the existing motion capture system, which makes the usage scenarios relatively limited. To solve this problem, this paper proposes a multi-camera 3D motion capture system based on reprojection optimization, and combines the foot mechanics data to obtain the human joint torque. First, build a multi-camera-based motion capture system to obtain human body motion data, reconstruct key points to obtain 3D data, and then propose an accurate key point algorithm based on weight-based reprojection optimization to obtain precise coordinates of key points. Finally, the coordinates of the key points combined with the mechanical data of the foot are calculated by OpenSim to obtain the mechanical data of the human joints. The experimental results show that the capture accuracy of key points is 5% higher than the average accuracy of the traditional least square method, and the analysis of joint mechanics data conforms to the laws of motion, which can be used to guide rehabilitation evaluation and competitive sports.

Deep Transfer Learning-Enabled Activity Identification and Fall Detection for Disabled People

The human motion data collected using wearables like smartwatches can be used for activity recognition and emergency event detection. This is especially applicable in the case of elderly or disabled people who live self-reliantly in their homes. These sensors produce a huge volume of physical activity data that necessitates real-time recognition, especially during emergencies. Falling is one of the most important problems confronted by older people and people with movement disabilities. Numerous previous techniques were introduced and a few used webcam to monitor the activity of elderly or disabled people. But, the costs incurred upon installation and operation are high, whereas the technology is relevant only for indoor environments. Currently, commercial wearables use a wireless emergency transmitter that produces a number of false alarms and restricts a user’s movements. Against this background, the current study develops an Improved Whale Optimization with Deep Learning-Enabled Fall Detection for Disabled People (IWODL-FDDP) model. The presented IWODL-FDDP model aims to identify the fall events to assist disabled people. The presented IWODL-FDDP model applies an image filtering approach to pre-process the image. Besides, the EfficientNet-B0 model is utilized to generate valuable feature vector sets. Next, the Bidirectional Long Short Term Memory (BiLSTM) model is used for the recognition and classification of fall events. Finally, the IWO method is leveraged to fine-tune the hyperparameters related to the BiLSTM method, which shows the novelty of the work. The experimental analysis outcomes established the superior performance of the proposed IWODL-FDDP method with a maximum accuracy of 97.02%.

RESTEP Into the Future: Relational Spatio-Temporal Learning for Multi-Person Action Forecasting

Multi-person action forecasting is an emerging topic in the computer vision field, and it is a pivotal step toward video understanding at a semantic level. This task is difficult due to the complexity of spatial and temporal dependencies. Yet, the state- of-the-art literature does not seem to be adequately responsive to this challenge. Hence, how to better foresee the forthcoming actions per actor has to be further pursued. Toward this end, we put forth a novel RElational Spatio-TEmPoral learning approach (RESTEP) for multi-person action forecasting. Our RESTEP explores the key that inherently characterizes actions from a perspective of incorporating the spatial and temporal information in a single pass (spatio-temporal dependencies) by extending rela- tional reasoning. As a result, the RESTEP enables simultaneously predicting the actions of all actors in the scene. Our proposal significantly differs from mainstream works that heavily rely on independently processing the spatial and temporal dependencies. The proposed RESTEP first perceives a graph building upon the historical observations, then reasons the relational spatio- temporal context to extrapolate future actions. In order to augment the comprehension of individual actions that might vary over time, we further delve deeper into the essence behind this point the evolution of spatio-temporal dependencies via optimizing the corresponding mutual information. We assess the RESTEP method on the large-scale Atomic Visual Actions (AVA) dataset, Activities in Extended Videos (ActEV/VIRAT) dataset and Joint-annotated Human Motion Data Base (J-HMDB). The experimental outcomes reveal that RESTEP can introduce con- siderable improvements with respect to recent leading studies.

Contact Part Detection From 3D Human Motion Data Using Manually Labeled Contact Data and Deep Learning

Contact Part Detection From 3D Human Motion Data Using Manually Labeled Contact Data and Deep Learning

Accelerometers in Our Pocket: Does Smartphone Accelerometer Technology Provide Accurate Data?

This study evaluates accelerometer performance of three new state of the art smartphones and focuses on accuracy. The motivating research question was whether accelerator accuracy obtained with these off-the-shelf modern smartphone accelerometers was or was not statistically different from that of a gold-standard reference system. We predicted that the accuracy of the three modern smartphone accelerometers in human movement data acquisition do not differ from that of the Vicon MX motion capture system. To test this prediction, we investigated the comparative performance of three different commercially available current generation smartphone accelerometers among themselves and to a gold-standard Vicon MX motion capture system. A single subject design was implemented for this study. Pearson's correlation coefficients® were calculated to verify the validity of the smartphones' accelerometer data against that of the Vicon MX motion capture system. The Intraclass Correlation Coefficient (ICC) was used to assess the smartphones' accelerometer performance reliability compared to that of the Vicon MX motion capture system. Results demonstrated that (a) the tested smartphone accelerometers are valid and reliable devices for estimating accelerations and (b) there were not significant differences among the three current generation smartphones and the Vicon MX motion capture system's mean acceleration data. This evidence indicates how well recent generation smartphone accelerometer sensors are capable of measuring human body motion. This study, which bridges a significant information gap between the accuracy of accelerometers measured close to production and their accuracy in actual smartphone research, should be interpreted within the confines of its scope, limitations and strengths. Further research is warranted to validate our arguments, suggestions, and results, since this is the first study on this topic.

Introduction to the Special Issue on Understanding the Spread of COVID-19, Part 2

Infectious diseases are transmitted between human hosts when in close contact over space and time. Recently, an unprecedented amount of spatial and spatiotemporal data have been made available that can be used to improve our understanding of the spread of COVID-19 and other infectious diseases. This understanding will be paramount to prepare for future pandemics through spatial algorithms and systems to collect, capture, curate and analyze complex, multi-scale human movement data to solve problems such as infectious diseases prediction, contact tracing, and risk assessment. In exploring and deepening the conversation around this topic, the five articles included in the second volume of this special issue employ diverse theoretical perspectives, methodologies, and frameworks, including but not limited to close contact modeling, infectious diseases spread prediction, mobility analysis, effective testing and intervention strategies. Rather than focusing on a narrow set of problems, these articles provide a glimpse into the diverse possibilities of leveraging spatial and spatiotemporal data for pandemic preparedness.

Human Motion Capture Data Retrieval and Segmentation Technology for Professional Sports Training

Human motion capturing is frequently used in sports research. The focus of this research is to help the user choose an appropriate motion capture system for their experimental setup for sports activities that addresses the challenges of linear size and acceptable fits with non-linear futures. In this paper, the eigenvalue combination was used to represent different motion postures, so as to construct an index space related to the motion sequence; and according to this index space, fast and accurate motion retrieval was done; at the same time, a motion data segmentation method based on MVU nonlinear dimensionality reduction was proposed. MVU can overcome the shortcomings that the linear size is reduced and difficult to deal with nonlinear features, and can better fit human motion data, and achieve higher accuracy.

Multimodal Human Motion Synchronization Dataset

Human motion dataset is an important foundation for researches such as motion data denoising, motion editing, motion synthesis, etc. In order to support more generic studies of multimodal motion data fusion, designing and collecting a public multimodal human motion data set is an urgent problem. First, the acquisition environment is designed for precise motion data collected by sensor-based motion capture devices, rough motion data collected by body sensing devices, and local inertial data collected by inertial measurement units (IMU). Then, temporal synchronization among equipment is applied based on network time protocol (NTP) and spatial synchronization is applied among multi modal data. A full body motion dataset named HFUT-MMD is captured, which contains 6 971 568 frames in 6 types from 12 actors/actresses. The experimental results on the HFUT-MMD dataset using the existing algorithm show that the low precision motion data can be optimized to obtain the motion data similar to the accurate motion data, which corroborates the consistency between the modal data.

Introduction to the Special Issue on Understanding the Spread of COVID-19, Part 1

Infectious diseases are transmitted between human hosts when in close contact over space and time. Recently, an unprecedented amount of spatial and spatiotemporal data have been made available that can be used to improve our understanding of the spread of COVID-19 and other infectious diseases. This understanding will be paramount to prepare for future pandemics through spatial algorithms and systems to collect, capture, curate, and analyze complex, multi-scale human movement data to solve problems such as infectious diseases prediction, contact tracing, and risk assessment. In exploring and deepening the conversation around this topic, the eight articles included in the first volume of this special issue employ diverse theoretical perspectives, methodologies, and frameworks, including but not limited to infectious diseases simulation, risk prediction, response policy design, mobility analysis, and case diagnosis. Rather than focusing on a narrow set of problems, these articles provide a glimpse into the diverse possibilities of leveraging spatial and spatiotemporal data for pandemic preparedness.

Estimating Infection-Related Human Mobility Networks Based on Time Series Data of COVID-19 Infection in Japan

Background: Comprehensive and evidence-based countermeasures against emerging infectious diseases have become increasingly important in recent years. COVID-19 and many other infectious diseases are spread by human movement and contact, but complex transportation networks in the 21st century make it difficult to predict disease spread in rapidly changing situations. It is especially challenging to estimate the network of infection transmission in countries where traffic and human movement data infrastructure is not yet developed. Methods: In this study, we devised a method utilizing an ordinary and partial differential equations-based mathematical model and a modified mathematical optimization method to estimate the network of transmission of COVID-19 from the time series data of its infection and applied it to determine its spread across areas in Japan. Furthermore, utilizing the estimated human mobility network, we predicted the spread of infection using the Tokyo Olympics as a model. Findings: We incorporated the effects of soft lockdowns, such as the declaration of a state of emergency, and changes in the infection network due to government-sponsored travel promotion, and revealed that the estimated effective distance captured human mobility changing dynamically in the different stages of the pandemic. The model predicted that the Tokyo Olympic and Paralympic Games would increase the number of infected cases in the host prefectures by up to 80%. Interpretation: The models used in this study are available online, and our data-driven infection network models are scalable, whether it be at the level of a city, town, country, or continent, and applicable anywhere in the world, as long as the time-series data of infections per region is available. These estimations of effective distance and the depiction of infectious disease networks based on actual infection data are expected to be useful in devising data-driven countermeasures against emerging infectious diseases worldwide.

Wearable Device-Based Data Collection and Feature Analysis Method for Outdoor Sports

In recent years, with the rapid popularization of smart phones and wearable smart devices, it is no longer difficult to obtain a large number of human motion data related to people's heart rate and geographical location, which has spawned a series of running fitness applications, leading to the national running wave and promoting the rapid development of the sports industry. Based on the long short-term memory cyclic neural network, this paper processes, identifies, and analyzes the motion data collected by wearable devices. Through massive data training, a set of accurate auxiliary models of outdoor sports is obtained to help optimize and improve the effect of outdoor sports. The results show that the method proposed in this paper has a higher degree of sports action and feature recognition and can better assist in the completion of outdoor sports.