Motion Classification Research Articles

SHREC 2024: Recognition of dynamic hand motions molding clay

Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

Combining EEG and eye-tracking in virtual reality: Obtaining fixation-onset event-related potentials and event-related spectral perturbations.

Extensive research conducted in controlled laboratory settings has prompted an inquiry into how results can be generalized to real-world situations influenced by the subjects' actions. Virtual reality lends itself ideally to investigating complex situations but requires accurate classification of eye movements, especially when combining it with time-sensitive data such as EEG. We recorded eye-tracking data in virtual reality and classified it into gazes and saccades using a velocity-based classification algorithm, and we cut the continuous data into smaller segments to deal with varying noise levels, as introduced in the REMoDNav algorithm. Furthermore, we corrected for participants' translational movement in virtual reality. Various measures, including visual inspection, event durations, and the velocity and dispersion distributions before and after gaze onset, indicate that we can accurately classify the continuous, free-exploration data. Combining the classified eye-tracking with the EEG data, we generated fixation-onset event-related potentials (ERPs) and event-related spectral perturbations (ERSPs), providing further evidence for the quality of the eye-movement classification and timing of the onset of events. Finally, investigating the correlation between single trials and the average ERP and ERSP identified that fixation-onset ERSPs are less time sensitive, require fewer repetitions of the same behavior, and are potentially better suited to study EEG signatures in naturalistic settings. We modified, designed, and tested an algorithm that allows the combination of EEG and eye-tracking data recorded in virtual reality.

Tracking of cogging stiffness using the multi-bin SDFT

Due to the high power density and wide speed range, a permanent magnet synchronous motor (PMSM) is commonly used in industry. Efficient production requires optimized motion control of the mechanism driven by the PMSM. Unfortunately, designing the controller is usually based on a model of the mechanism without including parameters originating from the motor. Cogging stiffness originates from the magnetic forces between rotor magnets and stator teeth and is one of these forgotten parameters. The performance is sub-optimal when this stiffness is not considered during control design. The stiffness also changes over time due to changing load conditions such as temperature. Aiming for optimized motion control of high-speed mechanisms, this article proposes to expand the classic motion controller with an on-line stiffness tracker. The tracker is based on the sliding discrete Fourier transform (SDFT). The tracking technique is conceptually analyzed and experimentally validated on a PMSM-driven rod. The classic Welch technique having an update time of at least 100 s is used as a benchmark. With similar accuracy and a much faster update time of 1.25 ms, the stiffness is tracked on-line using SDFT. The developed stiffness tracker is implemented on the provided commercial motion controller, proving its computational efficiency.

Viewing preferences of ASD children on paintings

The eye movement patterns of children with autism spectrum disorder (ASD) based on high-precision, high-sampling-rate professional eye trackers have been widely studied. Still, the equipment used in these studies is expensive and requires skilled operators, and the stimuli are focused on pictures or videos created out of ASD group. We utilized a previously developed eye-tracking device using a tablet, and the double-column paintings with one column from children with ASD, and the other from typically developing (TD) children as stimuli, to investigate the preference of ASD children for the paintings created within their group. This study collected eye movement data from 82 children with ASD and 102 TD children, and an adaptive eye movement classification algorithm was applied to the data aligned by the sampling rate, followed by feature extraction and statistical analysis from the aspects of time, frequency, range, and clustering. Statistical tests indicate that apart from displaying more pronounced non-compliance during the experiment, resulting in a higher data loss rate, children with ASD did not show significant preferences in viewing the two types of paintings compared to TD children. Therefore, we tend to believe that there is no significant difference in preference for ASD and TD paintings showcasing as diptych from the two groups of children using our eye tracking device, and their feature values indicate that they do not have a viewing preference for the paintings.

Wireless Mouth Motion Recognition System Based on EEG-EMG Sensors for Severe Speech Impairments.

This study aims to demonstrate the feasibility of using a new wireless electroencephalography (EEG)-electromyography (EMG) wearable approach to generate characteristic EEG-EMG mixed patterns with mouth movements in order to detect distinct movement patterns for severe speech impairments. This paper describes a method for detecting mouth movement based on a new signal processing technology suitable for sensor integration and machine learning applications. This paper examines the relationship between the mouth motion and the brainwave in an effort to develop nonverbal interfacing for people who have lost the ability to communicate, such as people with paralysis. A set of experiments were conducted to assess the efficacy of the proposed method for feature selection. It was determined that the classification of mouth movements was meaningful. EEG-EMG signals were also collected during silent mouthing of phonemes. A few-shot neural network was trained to classify the phonemes from the EEG-EMG signals, yielding classification accuracy of 95%. This technique in data collection and processing bioelectrical signals for phoneme recognition proves a promising avenue for future communication aids.

Hand and Wrist Movements Classification Using Surface Electromyogram

Myoelectric is a biological signal produced from physiological variations in muscle fibers when they contract and relax. The study of muscle activity through the recording and analysis of myoelectric signals is called electromyography. Electromyography can provide a comprehensive view of the operation and performance of internal muscle groups and cells. The objective of this study is an investigation into hand movement classification using surface electromyography (sEMG) signals. For training and evaluating the proposed model, two data sets from the Ninapro project, a publicly available database for prosthetic hand control, were used: DB5 with low‐cost 16 channels and 200 Hz sampling rate setting and DB10 with 12 channels and 1926 kHz sampling rate setup. First, we divided the EMG data into segments using the windowing technique. These segments were then used to extract a set of time features. Finally, the retrieved feature information was loaded into a simple pattern recognition model: Artificial Neural Network. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Rotating solutions to the incompressible Euler–Poisson equation with external particle

We consider a two-dimensional, incompressible fluid body, together with self-induced interactions. The body is perturbed by an external particle with small mass. The whole configuration rotates uniformly around the common center of mass. We construct solutions, which are stationary in a rotating coordinate system, using perturbative methods. In addition, we consider a large class of internal motions of the fluid. The angular velocity is related to the position of the external particle and is chosen to satisfy a non-resonance condition.

Optimizing Artificial Neural Network Performance Using Gray Level Co-occurrence Matrix (GLCM) Feature Extraction for Monitoring Elderly Movement

Machine learning methods are used to detect elderly accidents while image processing is used to support machine learning performance so that detection performance can be better. This study used two methods used simultaneously, namely GLCM and ANN. The study consisted of preparation of human gesture datasets, preprocessing stage, application of GLCM, analysis of feature extraction results, classification using ANN and analysis of motion class detection results. Overall, the GLCM method with homogeneity parameters and ANN as a classifier obtained an accuracy of 24.32%. The GLCM method with contrast parameters and ANN as a classifier gets an accuracy of 99.84%. The GLCM method with mean parameters and ANN as the classifier gets an accuracy of 99.99%. The GLCM method with dissimilarity parameters and ANN as a classifier to classify hand movement images gets the best accuracy of 100%.

Deep learning classification of EEG-based BCI monitoring of the attempted arm and hand movements.

The primary objective of this research is to improve the average classification performance for specific movements in patients with cervical spinal cord injury (SCI). The study utilizes a low-frequency multi-class electroencephalography (EEG) dataset from Graz University of Technology. The research combines convolutional neural network (CNN) and long-short-term memory (LSTM) architectures to uncover neural correlations between temporal and spatial aspects of the EEG signals associated with attempted arm and hand movements. To achieve this, three different methods are used to select relevant features, and the proposed model's robustness against variations in the data is validated using 10-fold cross-validation (CV). The research also investigates subject-specific adaptation in an online paradigm, extending movement classification proof-of-concept. The combined CNN-LSTM model, enhanced by three feature selection methods, demonstrates robustness with a mean accuracy of 75.75 % and low standard deviation (+/- 0.74 %) in 10-fold cross-validation, confirming its reliability. In summary, this research aims to make valuable contributions to the field of neuro-technology by developing EEG-controlled assistive devices using a generalized brain-computer interface (BCI) and deep learning (DL) framework. The focus is on capturing high-level spatiotemporal features and latent dependencies to enhance the performance and usability of EEG-based assistive technologies.

Simultaneous Velocity and Texture Classification from a Neuromorphic Tactile Sensor Using Spiking Neural Networks

The neuroTac, a neuromorphic visuo-tactile sensor that leverages the high temporal resolution of event-based cameras, is ideally suited to applications in robotic manipulators and prosthetic devices. In this paper, we pair the neuroTac with Spiking Neural Networks (SNNs) to achieve a movement-invariant neuromorphic tactile sensing method for robust texture classification. Alongside this, we demonstrate the ability of this approach to extract movement profiles from purely tactile data. Our systems achieve accuracies of 95% and 83% across their respective tasks (texture and movement classification). We then seek to reduce the size and spiking activity of our networks with the aim of deployment to edge neuromorphic hardware. This multi-objective optimisation investigation using Pareto frontiers highlights several design trade-offs, where high activity and large network sizes can both be reduced by up to 68% and 94% at the cost of slight decreases in accuracy (8%).

Human movement detection and classification capabilities using passive Wi-Fi based radar

Human detection and classification via Wi-Fi transmission have received a lot of attention in recent years as crucial facilitators in security and human-computer interaction (HCI). The passive Wi-Fi radar (PWR) system used by previous researchers applied cross-ambiguity function (CAF) and CLEAN algorithms to process the detected signals. This paper explores the feasibility and viability of a PWR system in detecting and classifying human movements without utilizing CAF and CLEAN algorithms. The movements are performed by four participants but with comparable body sizes and heights. Three daily human movements are investigated namely walking, bending, and sitting, with each participant performing each movement 24 times, providing a total of 96 samples per activity. The system is evaluated based on the consistency of the signal pattern in a frequency domain and the percentage accuracy is assessed using an artificial neural network (ANN) classifier and trained using a leave-one-out cross-validation (LOOCV) method. The frequency domain results reveal that the signals are consistent, with no noticeable variations or changes in the voltage intensity or shape of the main lobe. The classification of the movements shows that the classifier has an overall accuracy of 97.6%.

Instruction transmission of combat robotic arm based on neck muscles surface electromyography signals

This paper introduces a novel method for instructing wearable robotic arms based on neck movements to enhance user control. By capturing head movements representing four aiming directions and analysing surface electromyography (sEMG) signals from specific muscles, the study employs the random forest algorithm for movement recognition. Results show that utilising sEMG signals from six muscles improve classification accuracy, and applying the variational mode decomposition (VMD) algorithm enhances feature extraction. Notably, right-side muscles, particularly the sternocleidomastoid muscles, significantly impact movement classification. These findings provide theoretical support for better recognition of neck movements.

Rett syndrome: a review of clinical manifestations and therapeutic approaches

Rett syndrome (RTT) is a severe X-linked dominant neurodevelopmental disorder predominantly affecting females and over 90% of these patients mutations linked to the methyl-CpG-binding protein 2 (MeCP2) gene. Although the syndrome is well noted for the classic repetitive hand motion with decline in speech, patients may have a wide range of cognitive and motor impairments. Typical comorbidities in RTT are characterized by poor growth, seizures, disrupted sleep, hyperventilation, breath holding spells, feeding difficulties, scoliosis, and behavioral issues. This paper aims to provide a brief overview of RTT, focusing on the clinical features of sleep and epilepsy, effects on childhood development, and available emerging treatment options. Sleep disturbances, epilepsy, and developmental regression can have profound effects on the quality of life in affected individuals. Current management strategies focus on a multidisciplinary approach to address symptoms and enhance overall wellbeing of individuals with RTT.

Personalized Path-Tracking Approach Based on Reference Vector Field for Four-Wheel Driving and Steering Wire-Controlled Chassis

It is essential and forward-thinking to investigate the personalized use of four-wheel driving and steering wire-controlled unmanned chassis. This paper introduces a personalized path-tracking approach designed to adapt the vehicle’s control system to human-like characteristics, enhancing the fit and maximizing the potential of the chassis’ multi-directional driving and steering capabilities. By modifying the classic vehicle motion controller design, this approach aligns with individual driving habits, significantly improving upon traditional path-tracking control methods that rely solely on reference vector fields. First, the classic reference vector field’s logic was expanded upon, and it is shown that a personalized upgrade is feasible. Then, driving behavior data from multiple drivers were collected using a driving simulator. The fuzzy c-means clustering method was used to categorize drivers based on typical states that match vehicle path-tracking performance. Additionally, the random forest algorithm was used as the method for recognizing driving style. Subsequently, a personalized path-tracking control strategy based on the reference vector field was developed and a distributed execution architecture for four-wheel driving and steering wire-controlled unmanned chassis was established. Finally, the proposed personalized path-tracking approach was validated using a driving simulator. The results of the experimental tests demonstrated that the personalized path-tracking control approach not only fits well with various driving styles but also delivers high accuracy in driving style identification, making it highly suitable for application in four-wheel driving and steering wire-controlled chassis.

Attempted movement classification of spinal cord injured patients using hybrid CNN‐LSTM networks

AbstractIndividuals who are suffering from the most severe of motor disabilities can improve their quality of life by controlling and directing mechanical and electronic devices. As for Spinal Cord Injured (SCI) patients', attempted hand movements can be classified using electroencephalography (EEG). The research aims to develop a hybrid CNN‐LSTM (Convolutional Neural Network—Long Short Term Memory) architecture for multichannel EEG signal classification. It is a challenging task to classify real‐world multichannel EEG data from SCI patients. The proposed research preprocessed the EEG data to improve the signal‐to‐noise ratio and arranged for them to extract additional information from the data. The preprocessing step includes filtering, downsampling, and artifact removal, while the postprocessing step includes time‐frequency representation and spatial information encoding. A hybrid CNN‐LSTM is used for feature extraction and classification. The proposed method has been implemented on a dataset consisting of 5 different classes of attempted hand movements from 10 SCI patients. The average classification accuracy of 92.36% is achieved for 5‐class classification. To check the global validity of the proposed network, the BCI competition IV data is classified by the proposed method and has found 92.70% overall accuracy.

Enabling temporal–spectral decoding in multi-class single-side upper limb classification

This manuscript presents a novel approach for decoding pre-movement patterns from brain signals using a two-stage-training temporal–spectral neural network (TTSNet). The TTSNet employs a combination of filter bank task-related component analysis (FBTRCA) and convolutional neural network (CNN) techniques to enhance the classification of single-upper limb movements in non-invasive brain–computer interfaces (BCIs).In our previous work, we introduced the FBTRCA method which utilized filter banks and spatial filters to handle spectral and spatial information, respectively. However, we observed limitations in the temporal decoding phase, where correlation features failed to effectively utilize temporal information because of misaligned onset and noisy spikes. To address this issue, our proposed method focuses on analyzing multi-channel signals in the temporal–spectral domain. The TTSNet first divides the signals into various filter banks, employing task-related component analysis to reduce dimensionality and eliminate noise, respectively. Subsequently, a CNN is employed to optimize the temporal characteristics of the signals and extract class-related features. Finally, the class-related features from all filter banks are concatenated and classified using the fully connected layer.To evaluate the effectiveness of our proposed method, we conducted experiments on two publicly available datasets. In binary classification tasks, the TTSNet achieved an improved accuracy of 0.7707 ± 0.1168, surpassing the performance of EEGNet (accuracy: 0.7340 ± 0.1246) and FBTRCA (accuracy: 0.7487 ± 0.1250). In multi-class tasks, TTSNet achieved an accuracy of 0.4588 ± 0.0724, exhibiting a 4.27% and 3.95% accuracy increase over EEGNet and FBTRCA, respectively.The findings of this study suggest that the proposed TTSNet method holds promise for detecting limb movements and assisting in the rehabilitation of stroke patients. The classification of single-side limb movements is expected to facilitate the interaction between patients and external environment by increasing the number of control commands in BCIs.

Classification of hand movements based on EMD-CCT feature extraction method through EMG using machine learning

Classification of hand movements based on EMD-CCT feature extraction method through EMG using machine learning

Deep learning based ankle–foot movement classification for prosthetic foot

The primary motivation behind this study is the aspiration to design a prosthetic foot that demonstrates enhanced functionality, enabling more active and prompt responses, particularly tailored for individuals with below-knee amputations. This goal underscores the intention to create a prosthetic foot with the capability to execute foot movements in a more natural and effective manner. A new 1D-ResCNN model has been proposed for the rapid and accurate classification of foot movements based on user intent in the context of a prosthetic limb. This research introduces an innovative approach by integrating inertial measurement units with deep learning algorithms to advance the development of more functional prosthetic feet, specifically tailored for below-knee amputees. Leveraging wearable technologies, this method allows for the prolonged monitoring of foot movements within the users’ natural environments. The dual benefits of cost reduction and enhanced user experience are achieved through this combination of advanced technologies, providing a promising avenue for the evolution of prosthetic foot design and usage. The results obtained with this model are satisfying both in terms of speed and accuracy with 99.8% compared to other methods in the literature.

Deep learning-based spatial-temporal graph neural networks for price movement classification in crude oil and precious metal markets

In this study, we adapt three spatial-temporal graph neural network models to the unique characteristics of crude oil, gold, and silver markets for forecasting purposes. It aims to be the first to (i) explore the potential of spatial-temporal graph neural networks family for price forecasting of these markets, (ii) examine the role of attention mechanism in improving forecasting accuracy, and (iii) integrate various sources of predictors for better performance. Specifically, we present three distinct models: Multivariate Time Series Graph Neural Networks with Temporal Attention and Learnable Adjacency matrix (MTGNN-TAttLA), Spatial Attention Graph with Temporal Convolutional Networks (SAG-TCN), and Attention-based Spatial-Temporal Graph Convolutional Networks (ASTGCN), to capture the intricate interplay of spatial and temporal dependencies within crude oil and precious metals markets. Moreover, the effectiveness of the attention mechanism in improving models' accuracies is shown. Our empirical results reveal remarkable prediction accuracy, with all three models outperforming conventional deep learning methods such as Temporal Convolutional Networks (TCN), long short-term memory networks (LSTM) and convolutional neural networks (CNN). The MTGNN-TAttLA model, enriched with a temporal attention mechanism, exhibits exceptional performance in predicting the direction of price movement in the WTI, Brent, and silver markets, while ASTGCN is the best-performing model for the gold market. Additionally, we observed that incorporating technical indicators from the crude oil and precious metal markets into the graph structure has improved the classification accuracy of spatial-temporal graph neural networks.

A Novel Method for Tracking Neck Motions Using a Skin-Conformable Wireless Accelerometer: A Pilot Study

Introduction: Cervical spine disease is a leading cause of pain and disability. Degenerative conditions of the spine can result in neurologic compression of the cervical spinal cord or nerve roots and may be surgically treated with an anterior cervical discectomy and fusion (ACDF) in up to 137,000 people per year in the United States. A common sequelae of ACDF is reduced cervical range of motion (CROM) with patient-based complaints of stiffness and neck pain. Currently, tools for assessment of CROM are manual, subjective, and only intermittently utilized during doctor or physical therapy visits. We propose a skin-mountable acousto-mechanic sensor (ADvanced Acousto-Mechanic sensor; ADAM) as a tool for continuous neck motion monitoring in postoperative ACDF patients. We have developed and validated a machine learning neck motion classification algorithm to differentiate between eight neck motions (right/left rotation, right/left lateral bending, flexion, extension, retraction, protraction) in healthy normal subjects and patients. Methods: Sensor data from 12 healthy normal subjects and 5 patients were used to develop and validate a Convolutional Neural Network (CNN). Results: An average algorithm accuracy of 80.0 ± 3.8% was obtained for healthy normal subjects (94% for right rotation, 98% for left rotation, 65% for right lateral bending, 87% for left lateral bending, 89% for flexion, 77% for extension, 50% for retraction, 84% for protraction). An average accuracy of 67.5 ± 5.8% was obtained for patients. Discussion: ADAM, with our algorithm, may serve as a rehabilitation tool for neck motion monitoring in postoperative ACDF patients. Sensor-captured vital signs and other events (extubation, vocalization, physical therapy, walking) are potential metrics to be incorporated into our algorithm to offer more holistic monitoring of patients after cervical spine surgery.