Real-world Motion Research Articles

Task Offloading and Resource Allocation for Augmented Reality Applications in UAV-Based Networks Using a Dual Network Architecture

This paper proposes a novel UAV-based edge computing system for augmented reality (AR) applications, addressing the challenges posed by the limited resources in mobile devices. The system uses UAVs equipped with edge computing servers (UECs) specifically to enable efficient task offloading and resource allocation for AR tasks with dependent relationships. This work specifically focuses on the problem of dependent tasks in AR applications within UAV-based networks. This problem has not been thoroughly addressed in previous research. A dual network architecture-based task offloading (DNA-TO) algorithm is proposed, leveraging the DNA framework to enhance decision-making in reinforcement learning while mitigating noise. In addition, a Karush–Kuhn–Tucker-based resource allocation (KKT-RA) algorithm is proposed to optimize resource allocation. Various simulations using real-world movement data are conducted. The results indicate that our proposed algorithm outperforms existing approaches in terms of latency and energy efficiency.

Riot Like a Girl? Gender-Stereotypical Associations Boost Support for Feminist Online Campaigns

The current study examined whether congruence with gender stereotypes is associated with support for online feminist campaigns. In Study 1 (Ntweets = 510,000), we investigated how two real-world movements (i.e., #metoo and #sexstrike) were viewed in terms of adhering to the traditional feminine stereotype of high warmth/communion and low competence/agency, as per language used in the tweets posted within each campaign. We found that the #metoo movement was characterised by more communal and less agentic content than #sexstrike. In Study 2 (N = 195), when presented with descriptions of the movements modelled after #metoo and #sexstrike, participants associated the #metoo-like campaign with more feminine (e.g., community orientation) and less masculine concepts (e.g., rebelliousness), which, in turn, translated to greater declared support for the campaign. In Study 3 (N = 446), conducted in a more controlled and context-independent setting, we once again observed the link between the gender stereotypes attached to feminist movements and the support they received. We also found that this effect was independent of the effect of perceiving movements as violating social norms. Controlling for feminist identification, political conservatism, and gender system justification did not affect the pattern of results. Uncovering these stereotype-related effects on support for feminist movements furthers understanding of the strategies that might be used in the fight for gender equality.

Topic modelling for spatial insights: Uncovering space use from movement data

We present a novel approach to understanding space use by moving entities based on repeated patterns of place visits and transitions. Our approach represents trajectories as text documents consisting of sequences of place visits or transitions and applies topic modelling to the corpus of these documents. The resulting topics represent combinations of places or transitions, respectively, that repeatedly co-occur in trips. Visualisation of the results in the spatial context reveals the regions of place connectivity through movements and the major channels used to traverse the space. This enables understanding of the use of space as a medium for movement. We compare the possibilities provided by topic modelling to alternative approaches exploiting a numeric measure of pairwise connectedness. We have extensively explored the potential of utilising topic modelling by applying our approach to multiple real-world movement data sets with different data collection procedures and varying spatial and temporal properties: GPS road traffic of cars, unconstrained movement on a football pitch, and episodic movement data reflecting social media posting events. The approach successfully demonstrated the ability to uncover meaningful patterns and interesting insights. We thoroughly discuss different aspects of the approach and share the knowledge and experience we have gained with people who might be potentially interested in analysing movement data by means of topic modelling methods.

Classification of human walking context using a single-point accelerometer

Real-world walking data offers rich insights into a person’s mobility. Yet, daily life variations can alter these patterns, making the data challenging to interpret. As such, it is essential to integrate context for the extraction of meaningful information from real-world movement data. In this work, we leveraged the relationship between the characteristics of a walking bout and context to build a classification algorithm to distinguish between indoor and outdoor walks. We used data from 20 participants wearing an accelerometer on the thigh over a week. Their walking bouts were isolated and labeled using GPS and self-reporting data. We trained and validated two machine learning models, random forest and ensemble Support Vector Machine, using a leave-one-participant-out validation scheme on 15 subjects. The 5 remaining subjects were used as a testing set to choose a final model. The chosen model achieved an accuracy of 0.941, an F1-score of 0.963, and an AUROC of 0.931. This validated model was then used to label the walks from a different dataset with 15 participants wearing the same accelerometer. Finally, we characterized the differences between indoor and outdoor walks using the ensemble of the data. We found that participants walked significantly faster, longer, and more continuously when walking outdoors compared to indoors. These results demonstrate how movement data alone can be used to obtain accurate information on important contextual factors. These factors can then be leveraged to enhance our understanding and interpretation of real-world movement data, providing deeper insights into a person’s health.

Advancing Security Measures in Governmental Institutions: Integration of Facial Recognition and Movement Monitoring Technologies in the House of Representatives

With the aim of improving security measures in public and government buildings, this study presents a system to protect the House of Representatives using Vicon sensors and facial recognition technology to detect threats as soon as they occur and alert accordingly to take appropriate measures. A face recognition model was realized, which took advantage of a DenseNet169-based feature extractor and a dense layer-based classifier. The machine learning models and Vicon sensors used by the physical motion capture system allowed for highly accurate analysis of real-world movements. Using Decision Tree (DT) and K-Nearest Neighbors (KNN) algorithms, the system achieved optimal accuracy using a dataset that included ten categories of behaviors performed by ten employees. Thanks to the combination of motion capture and facial recognition technology, allowing for precise threat classification, the House of Representatives will have a robust security system. This research highlights the importance of technical improvements in defending public employees and facilities.

Investigating how prior knowledge influences perception and action in developmental coordination disorder.

Developmental coordination disorder (DCD) is characterised by a broad spectrum of difficulties in performing motor tasks. It has recently been proposed that a specific deficit in sensorimotor prediction and feedforward planning might underpin these motoric impairments. The purpose of this study was to use a naturalistic object lifting paradigm to examine whether deficits in sensorimotor prediction might underpin the broad spectrum of difficulties individuals with DCD face when interacting with objects in their environment. We recruited 60 children with probable DCD and 61 children without DCD and measured perceptions of heaviness and fingertip force rate application when interacting with objects which varied in their apparent weight. If deficits in sensorimotor prediction do underpin the broad-ranging motor difficulties seen in DCD, we would expect to see a reduced effect of visual size cues on fingertip force rates and illusory misperceptions of object heaviness. We found no evidence of differences in any metrics of sensorimotor prediction between children with (n = 46) and without DCD (n = 61). Furthermore, there was no correlation between any metrics of sensorimotor prediction and motor performance (as assessed by the standard diagnostic movement assessment battery). Illusory misperceptions of object weight also did not appear to differ between groups. These findings suggest that issues with sensorimotor prediction are unlikely to affect the performance of simple real-world movements in those with DCD.

Accurate Monitoring of 24-h Real-World Movement Behavior in People with Cerebral Palsy Is Possible Using Multiple Wearable Sensors and Deep Learning.

Monitoring and quantifying movement behavior is crucial for improving the health of individuals with cerebral palsy (CP). We have modeled and trained an image-based Convolutional Neural Network (CNN) to recognize specific movement classifiers relevant to individuals with CP. This study evaluates CNN's performance and determines the feasibility of 24-h recordings. Seven sensors provided accelerometer and gyroscope data from 14 typically developed adults during videotaped physical activity. The performance of the CNN was assessed against test data and human video annotation. For feasibility testing, one typically developed adult and one adult with CP wore sensors for 24 h. The CNN demonstrated exceptional performance against test data, with a mean accuracy of 99.7%. Its general true positives (TP) and true negatives (TN) were 1.00. Against human annotators, performance was high, with mean accuracy at 83.4%, TP 0.84, and TN 0.83. Twenty-four-hour recordings were successful without data loss or adverse events. Participants wore sensors for the full wear time, and the data output were credible. We conclude that monitoring real-world movement behavior in individuals with CP is possible with multiple wearable sensors and CNN. This is of great value for identifying functional decline and informing new interventions, leading to improved outcomes.

A Hybrid Human Activity Recognition Method Using an MLP Neural Network and Euler Angle Extraction Based on IMU Sensors

Inertial measurement unit (IMU) technology has gained popularity in human activity recognition (HAR) due to its ability to identify human activity by measuring acceleration, angular velocity, and magnetic flux in key body areas like the wrist and knee. It has propelled the extensive application of HAR across various domains. In the healthcare sector, HAR finds utility in monitoring and assessing movements during rehabilitation processes, while in the sports science field, it contributes to enhancing training outcomes and preventing exercise-related injuries. However, traditional sensor fusion algorithms often require intricate mathematical and statistical processing, resulting in higher algorithmic complexity. Additionally, in dynamic environments, sensor states may undergo changes, posing challenges for real-time adjustments within conventional fusion algorithms to cater to the requirements of prolonged observations. To address these limitations, we propose a novel hybrid human pose recognition method based on IMU sensors. The proposed method initially calculates Euler angles and subsequently refines them using magnetometer and gyroscope data to obtain the accurate attitude angle. Furthermore, the application of FFT (Fast Fourier Transform) feature extraction facilitates the transition of the signal from its time-based representation to its frequency-based representation, enhancing the practical significance of the data. To optimize feature fusion and information exchange, a group attention module is introduced, leveraging the capabilities of a Multi-Layer Perceptron which is called the Feature Fusion Enrichment Multi-Layer Perceptron (GAM-MLP) to effectively combine features and generate precise classification results. Experimental results demonstrated the superior performance of the proposed method, achieving an impressive accuracy rate of 96.13% across 19 different human pose recognition tasks. The proposed hybrid human pose recognition method is capable of meeting the demands of real-world motion monitoring and health assessment.

Quantifying Patient-Initiated Upper Extremity Movement After Surgical Reconstruction for Adult Pan-Brachial Plexus Injury.

Determining functional recovery in adult patients with traumatic pan-brachial plexus injury (pBPI) is hampered by the fact that most outcome measures are collected in the clinical setting and may not reflect arm use in the real world. This study's objectives were to demonstrate the feasibility of using wearable motion sensor technology to quantify spontaneous arm movement in adult patients with pBPI after surgical reconstruction and report the time and intensity with which the affected arm was used. Twenty-nine patients with pBPI who underwent surgical reconstruction at least 2 years prior were included in this study. Study participants wore an accelerometer on bilateral arms for 7 days. The vector time (VT) and magnitude with which each arm moved were collected and divided by the same values collected from the uninjured arm to generate a ratio (VT and vector magnitude [VM], respectively) to quantify differences between the arms. Correlations between VT, VM, and patient demographic and physician-elicited clinical measures were calculated. Patients were enrolled at Chang Gung Memorial Hospital, Linkou Medical Center, Taiwan, and data analysis was performed at the University of Michigan. Twelve patients had pan-avulsion injuries, and 17 patients had C5 rupture with C6-T1 avulsion injuries. All underwent nerve reconstruction with contralateral C7 or ipsilateral C5 nerve roots as donors. At mean 7.3 years after surgery, the mean VT ratio was 0.54 ± 0.13 and the mean VM ratio was 0.30 ± 0.13. Both VT and VM ratios were significantly correlated with patient employment and movements at the elbow and forearm. Wearable motion detection technology can capture spontaneous, real-world movements of the arm in patients who have undergone surgical reconstruction for pBPI. Despite severe injuries, these patients are able to use their affected arm 50% of the time and with 30% of the intensity of their unaffected arm, which is positively correlated with return to work after injury. These data support the use of surgical reconstruction for pBPI.

Error-independent effect of sensory uncertainty on motor learning when both feedforward and feedback control processes are engaged.

Integrating sensory information during movement and adapting motor plans over successive movements are both essential for accurate, flexible motor behaviour. When an ongoing movement is off target, feedback control mechanisms update the descending motor commands to counter the sensed error. Over longer timescales, errors induce adaptation in feedforward planning so that future movements become more accurate and require less online adjustment from feedback control processes. Both the degree to which sensory feedback is integrated into an ongoing movement and the degree to which movement errors drive adaptive changes in feedforward motor plans have been shown to scale inversely with sensory uncertainty. However, since these processes have only been studied in isolation from one another, little is known about how they are influenced by sensory uncertainty in real-world movement contexts where they co-occur. Here, we show that sensory uncertainty may impact feedforward adaptation of reaching movements differently when feedback integration is present versus when it is absent. In particular, participants gradually adjust their movements from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error reduction. Riding on top of this slow envelope, participants exhibit large and abrupt changes in their initial movement vectors that are strongly correlated with the degree of sensory uncertainty present on the previous trial. However, these abrupt changes are insensitive to the magnitude and direction of the sensed movement error. These results prompt important questions for current models of sensorimotor learning under uncertainty and open up new avenues for future exploration in the field.

Tracking Perceptual Depth with Eye Vergence Movements in Real World, Augmented Reality, and Virtual Reality Environments

Tracking Perceptual Depth with Eye Vergence Movements in Real World, Augmented Reality, and Virtual Reality Environments

Vertical Cities: Emergent Patterns of Movement and Space Use in Dense Vertically Integrated Urban Built Environments

In high-density, land-scarce cities like Singapore, the successful translation of ground-level urban qualities and benefits into vertical living is crucial for social, economic, environmental, and ecological sustainability. This research introduces a Network science-based spatial analysis framework to evaluate the connectivity and relationships of vertically integrated urban open spaces. Kampung Admiralty (KA), a unique development integrating housing for the elderly with various facilities, serves as a case study. The methodology combines static spatial network measures and real-world movement data to predict movement flows, accessibility, and connectivity. Lift lobbies and elevated garden connectors emerged as critical paths, effectively distributing pedestrian flows. Landscape spaces played a key role in visual and physical connectivity, offering high recreational and social value. Strategic placement of “social attractors” improved space utilization. The study highlights the importance of spatial design parameters in user-space interactions and provides insights into socio-spatial networks at both ground and elevated levels. It identifies key connectors that facilitate effective planning and design of vertically integrated public space networks, promoting social and spatial effectiveness.

Optoelectronic graded neurons for bioinspired in-sensor motion perception.

Motion processing has proven to be a computational challenge and demands considerable computational resources. Contrast this with the fact that flying insects can agilely perceive real-world motion with their tiny vision system. Here we show that phototransistor arrays can directly perceive different types of motion at sensory terminals, emulating the non-spiking graded neurons of insect vision systems. The charge dynamics of the shallow trapping centres in MoS2 phototransistors mimic the characteristics of graded neurons, showing an information transmission rate of 1,200 bit s-1 and effectively encoding temporal light information. We used a 20 × 20 photosensor array to detect trajectories in the visual field, allowing the efficient perception of the direction and vision saliency of moving objects and achieving 99.2% recognition accuracy with a four-layer neural network. By modulating the charge dynamics of the shallow trapping centres of MoS2, the sensor array can recognize motion with a temporal resolution ranging from 101 to 106 ms.

Parieto-Occipital Electrocortical Dynamics during Real-World Table Tennis.

Traditional human electroencephalography (EEG) experiments that study visuomotor processing use controlled laboratory conditions with limited ecological validity. In the real world, the brain integrates complex, dynamic, multimodal visuomotor cues to guide the execution of movement. The parietal and occipital cortices are especially important in the online control of goal-directed actions. Table tennis is a whole-body, responsive activity requiring rapid visuomotor integration that presents a myriad of unanswered neurocognitive questions about brain function during real-world movement. The aim of this study was to quantify the electrocortical dynamics of the parieto-occipital cortices while playing a sport with high-density electroencephalography. We included analysis of power spectral densities (PSDs), event-related spectral perturbations, intertrial phase coherences (ITPCs), event-related potentials (ERPs), and event-related phase coherences of parieto-occipital source-localized clusters while participants played table tennis with a ball machine and a human. We found significant spectral power fluctuations in the parieto-occipital cortices tied to hit events. Ball machine trials exhibited more fluctuations in θ power around hit events, an increase in intertrial phase coherence and deflection in the event-related potential, and higher event-related phase coherence between parieto-occipital clusters as compared with trials with a human. Our results suggest that sport training with a machine elicits fundamentally different brain dynamics than training with a human.

The effect of real-world and retinal motion on speed perception for motion in depth.

For motion in depth, even if the target moves at a constant speed in the real-world (physically), it would appear to be moving with acceleration on the retina. Therefore, the purpose of this study was to determine whether real-world and retinal motion affect speed perception in depth and to verify the influence of eye movements on both motion signals in judging speed in depth. We used a two-alternative forced-choice paradigm with two types of tasks. One stimulus moved at a constant speed in the real-world (world constant task) with three conditions: 80-60 cm (far), 60-40 cm (middle), and 40-20 cm (near) from the participant. The other stimulus moved at a constant speed on the retina (retinal constant task) with three conditions: 4-8 deg (far), 8-12 deg (middle), and 12-16 deg (near) as the vergence angle. The results showed that stimulus speed was perceived faster in the near condition than in the middle and far conditions for the world constant task, regardless of whether it was during fixation or convergence eye movements. In contrast, stimulus speed was perceived faster in the order of the far, middle, and near conditions for the retinal constant task. Our results indicate that speed perception of a visual target approaching the observer depends on real-world motion when the target position is relatively far from the observer. In contrast, retinal motion may influence speed perception when the target position is close to the observer. Our results also indicate that the effects of real-world and retinal motion on speed perception for motion in depth are similar with or without convergence eye movements. Therefore, it is suggested that when the visual target moves from far to near, the effects of real-world and retinal motion on speed perception are different depending on the initial target position.

The Continuity of Locomotion

Natural interaction between multiple users within a shared virtual environment (VE) relies on each other's awareness of the current position of the interaction partners. This, however, cannot be warranted when users employ noncontinuous locomotion techniques, such as teleportation, which may cause confusion among bystanders. In this paper, we pursue two approaches to create a pleasant experience for both the moving user and the bystanders observing that movement. First, we will introduce a Smart Avatar system that delivers continuous full-body human representations for noncontinuous locomotion in shared virtual reality (VR) spaces. Smart Avatars imitate their assigned user's real-world movements when close-by and autonomously navigate to their user when the distance between them exceeds a certain threshold, i.e., after the user teleports. As part of the Smart Avatar system, we implemented four avatar transition techniques and compared them to conventional avatar locomotion in a user study, revealing significant positive effects on the observers' spatial awareness, as well as pragmatic and hedonic quality scores. Second, we introduce the concept of Stuttered Locomotion , which can be applied to any continuous locomotion method. By converting a continuous movement into short-interval teleport steps, we provide the merits of non-continuous locomotion for the moving user while observers can easily keep track of their path. Thus, while the experience for observers is similarly positive as with continuous motion, a user study confirmed that Stuttered Locomotion can significantly reduce the occurrence of cybersickness symptoms for the moving user, making it an attractive choice for shared VEs. We will discuss the potential of Smart Avatars and Stuttered Locomotion for shared VR experiences, both when applied individually and in combination.

FusIon Data Tracking System (FITS)

The field of generating movement profiles of individuals is valuable in many real-world applications (e.g., controlling disease spread or evaluating marketing engagement). Existing solutions often rely on global positioning systems (GPS) or similar systems, primarily targeted at outdoor use cases. However, the indoor tracking capabilities of current solutions either lack precision or are available in closed buildings only. The literature proposes sensor fusion approaches, but many of those are based on specific sensors. These approaches do not reveal implementation details or data to allow for their independent evaluation. Therefore, this article presents FusIon Data Tracking System ( <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FITS</monospace> ) as an approach and proof-of-concept to facilitate the correlation of data from different indoor sensors to movement profiles of different individuals. Functionally, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FITS</monospace> does this by generating synthetic sensor measurement data based on real-world movement data and correlating objects tracked from distinct sensors by effectively solving clustering and position prediction tasks. This correlation is evaluated based on different metrics [multiple object tracker accuracy/precision (MOTA/MOTP)] in four different scenarios, for example, sparse data, high density of sensors, low density of sensors, and a base case. Finally, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FITS</monospace> ’s performance was evaluated by increasing the load test (dataset up to 100000 measurements and 1000 visitors) to assess whether near real-time processing is feasible under a high workload.

A Feasibility Study of Bilateral Wrist Sensors for Measuring Motor Traits in Children With Autism.

Direct, quantitative measures of hyperactivity and motor coordination, two motor characteristics associated with impairment in autism, are limited. Wearable sensors can objectively index real-world movement variables that may relate to these behaviors. Here, we explored the feasibility of bilateral wrist accelerometers for measuring upper limb activity in 3-10-year-olds with autism (n = 22; 19 boys, 3 girls; M age = 5.64, SD = 2.73years) and without autism (n = 26; 15 boys, 11 girls; M age = 6.26, SD = 2.47 years). We investigated the relationships between movement characteristics related to duration, intensity, complexity, and symmetry on the one hand and parent-reported hyperactivity and motor coordination on the other. Participants with and without autism wore the sensors for 12-hour periods. Sensor variables varied by age but not sex, with movement intensity and complexity moderately related to motor coordination. These findings lend preliminary support to wearable sensors as a means of providing ecologically-valid metrics of motor characteristics that impact adaptive function in children with autism.

Measuring Dependencies between Variables of a Dynamical System Using Fuzzy Affiliations

A statistical, data-driven method is presented that quantifies influences between variables of a dynamical system. The method is based on finding a suitable representation of points by fuzzy affiliations with respect to landmark points using the Scalable Probabilistic Approximation algorithm. This is followed by the construction of a linear mapping between these affiliations for different variables and forward in time. This linear mapping, or matrix, can be directly interpreted in light of unidirectional dependencies, and relevant properties of it are quantified. These quantifications, given by the sum of singular values and the average row variance of the matrix, then serve as measures for the influences between variables of the dynamics. The validity of the method is demonstrated with theoretical results and on several numerical examples, covering deterministic, stochastic, and delayed types of dynamics. Moreover, the method is applied to a non-classical example given by real-world basketball player movement, which exhibits highly random movement and comes without a physical intuition, contrary to many examples from, e.g., life sciences.

MST-RNN: A Multi-Dimension Spatiotemporal Recurrent Neural Networks for Recommending the Next Point of Interest

With the increasing popularity of location-aware Internet-of-Vehicle services, the next-Point-of-Interest (POI) recommendation has gained significant research interest, predicting where drivers will go next from their sequential movements. Many researchers have focused on this problem and proposed solutions. Machine learning-based methods (matrix factorization, Markov chain, and factorizing personalized Markov chain) focus on a POI sequential transition. However, they do not recommend the user’s position for the next few hours. Neural network-based methods can model user mobility behavior by learning the representations of the sequence data in the high-dimensional space. However, they just consider the influence from the spatiotemporal dimension and ignore many important influences, such as duration time at a POI (Point of Interest) and the semantic tags of the POIs. In this paper, we propose a novel method called multi-dimension spatial–temporal recurrent neural networks (MST-RNN), which extends the ST-RNN and exploits the duration time dimension and semantic tag dimension of POIs in each layer of neural networks. Experiments on real-world vehicle movement data show that the proposed MST-RNN is effective and clearly outperforms the state-of-the-art methods.