Kinematic Sensors Research Articles

Improving Biological Joint Moment Estimation During Real-World Tasks With EMG and Instrumented Insoles.

Real-time measurement of biological joint moment could enhance clinical assessments and generalize exoskeleton control. Accessing joint moments outside clinical and laboratory settings requires harnessing non-invasive wearable sensor data for indirect estimation. Previous approaches have been primarily validated during cyclic tasks, such as walking, but these methods are likely limited when translating to non-cyclic tasks where the mapping from kinematics to moments is not unique. We trained deep learning models to estimate hip and knee joint moments from kinematic sensors, electromyography (EMG), and simulated pressure insoles from a dataset including 10 cyclic and 18 non-cyclic activities. We assessed estimation error on combinations of sensor modalities during both activity types. Compared to the kinematics-only baseline, adding EMG reduced RMSE by 16.9% at the hip and 30.4% at the knee (p < 0.05) and adding insoles reduced RMSE by 21.7% at the hip and 33.9% at the knee (p < 0.05). Adding both modalities reduced RMSE by 32.5% at the hip and 41.2% at the knee (p < 0.05) which was significantly higher than either modality individually (p < 0.05). All sensor additions improved model performance on non-cyclic tasks more than cyclic tasks (p < 0.05). These results demonstrate that adding kinetic sensor information through EMG or insoles improves joint moment estimation both individually and jointly. These additional modalities are most important during non-cyclic tasks, tasks that reflect the variable and sporadic nature of the real-world. Improved joint moment estimation and task generalization is pivotal to developing wearable robotic systems capable of enhancing mobility in everyday life.

An Object-Centric Hierarchical Pose Estimation Method Using Semantic High-Definition Maps for General Autonomous Driving.

To achieve Level 4 and above autonomous driving, a robust and stable autonomous driving system is essential to adapt to various environmental changes. This paper aims to perform vehicle pose estimation, a crucial element in forming autonomous driving systems, more universally and robustly. The prevalent method for vehicle pose estimation in autonomous driving systems relies on Real-Time Kinematic (RTK) sensor data, ensuring accurate location acquisition. However, due to the characteristics of RTK sensors, precise positioning is challenging or impossible in indoor spaces or areas with signal interference, leading to inaccurate pose estimation and hindering autonomous driving in such scenarios. This paper proposes a method to overcome these challenges by leveraging objects registered in a high-precision map. The proposed approach involves creating a semantic high-definition (HD) map with added objects, forming object-centric features, recognizing locations using these features, and accurately estimating the vehicle's pose from the recognized location. This proposed method enhances the precision of vehicle pose estimation in environments where acquiring RTK sensor data is challenging, enabling more robust and stable autonomous driving. The paper demonstrates the proposed method's effectiveness through simulation and real-world experiments, showcasing its capability for more precise pose estimation.

Assessment of Surgeons' Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental Study.

Robot-Assisted Minimally Invasive Surgery (RAMIS) marks a paradigm shift in surgical procedures, enhancing precision and ergonomics. Concurrently it introduces complex stress dynamics and ergonomic challenges regarding the human-robot interface and interaction. This study explores the stress-related aspects of RAMIS, using the da Vinci XI Surgical System and the Sea Spikes model as a standard skill training phantom to establish a link between technological advancement and human factors in RAMIS environments. By employing different physiological and kinematic sensors for heart rate variability, hand movement tracking, and posture analysis, this research aims to develop a framework for quantifying the stress and ergonomic loads applied to surgeons. Preliminary findings reveal significant correlations between stress levels and several of the skill-related metrics measured by external sensors or the SURG-TLX questionnaire. Furthermore, early analysis of this preliminary dataset suggests the potential benefits of applying machine learning for surgeon skill classification and stress analysis. This paper presents the initial findings, identified correlations, and the lessons learned from the clinical setup, aiming to lay down the cornerstones for wider studies in the fields of clinical situation awareness and attention computing.

Exploring the applicability of haptic actuators in aquatic environments

The use of high-tech wearable devices and real-time biomechanical feedback (RTBF) is widespread in modern sports and physical rehabilitation. While the use of kinematic sensors for RTBF is well established, the question of the most efficient and appropriate way to present feedback information to the user remains largely unanswered. The haptic modality has only been used in a limited number of relatively simple studies and applications, and it has never been studied in an aquatic environment. In this study, the main motivation was to design, develop, implement, and test an RTBF system for sports and physical rehabilitation with haptic actuators suitable for use in aquatic environments. The developed miniature remote-controlled device with vibration motors as actuators was tested to determine how people perceive the haptic modality in and out of water. The results of the exploratory study with 34 participants suggest that the feedback device can be further developed for future studies. The results show that both simple and complex haptic stimuli can be understood by athletes both outdoors and in water, as well as while standing and performing simple physical activities. This study tested the use of both head and waist mounted haptic actuators, with both showing similar and promising results. The results of this study provide evidence that haptic feedback can be perceived in water, highlighting the potential for real-time haptic interventions in aquatic environments. In summary, this study represents a significant contribution to the evolving field of RTBF in modern sport training and physical rehabilitation. The development of a haptic feedback device that can be worn during aquatic and other activities is a significant advance in the field of feedback actuators. This study provides a foundation for future usability studies and the development of haptic RTBF applications in both aquatic and outdoor environments.

SELECTION OF WEARABLE SENSORS FOR HEALTH AND SAFETY USE IN THE CONSTRUCTION INDUSTRY

Construction industry workers; are exposed to serious safety and health risks, hazardous work environments, and intense physical work. This situation causes fatal and non-fatal accidents, reduces productivity, and causes a loss of money and time. Construction safety management can use wearable sensors to improve safety performance. Since there are many types of sensors and not all sensors can be used in construction applications, it is necessary to identify suitable and reliable sensors. This requirement causes a sensor selection problem. The study aims to determine the priority order of physiological and kinematic sensors in preventing risks in the construction industry. Within the scope of this purpose, five criteria and seven alternatives were determined in line with the literature research and expert opinions. The criteria weights were calculated with the AHP method, and the alternatives were ranked with PROMETHEE and AHP. Providing a proactive approach to the use of sensors in the construction industry will provide safer working conditions, identify workers at risk, and help identify and predict potential health and safety risks. It will contribute to the literature on improving construction health and safety management.

Using hand pose estimation to automate open surgery training feedback.

This research aims to facilitate the use of state-of-the-art computer vision algorithms for the automated training of surgeons and the analysis of surgical footage. By estimating 2D hand poses, we model the movement of the practitioner's hands, and their interaction with surgical instruments, to study their potential benefit for surgical training. We leverage pre-trained models on a publicly available hands dataset to create our own in-house dataset of 100 open surgery simulation videos with 2D hand poses. We also assess the ability of pose estimations to segment surgical videos into gestures and tool-usage segments and compare them to kinematic sensors and I3D features. Furthermore, we introduce 6 novel surgical dexterity proxies stemming from domain experts' training advice, all of which our framework can automatically detect given raw video footage. State-of-the-art gesture segmentation accuracy of 88.35% on the open surgery simulation dataset is achieved with the fusion of 2D poses and I3D features from multiple angles. The introduced surgical skill proxies presented significant differences for novices compared to experts and produced actionable feedback for improvement. This research demonstrates the benefit of pose estimations for open surgery by analyzing their effectiveness in gesture segmentation and skill assessment. Gesture segmentation using pose estimations achieved comparable results to physical sensors while being remote and markerless. Surgical dexterity proxies that rely on pose estimation proved they can be used to work toward automated training feedback. We hope our findings encourage additional collaboration on novel skill proxies to make surgical training more efficient.

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Online handwriting trajectory reconstruction from kinematic sensors using temporal convolutional network

Development of a Head Acceleration Event Classification Algorithm for Female Rugby Union

Instrumented mouthguards have been used to detect head accelerations and record kinematic data in numerous sports. Each recording requires validation through time-consuming video verification. Classification algorithms have been posed to automatically categorise head acceleration events and spurious events. However, classification algorithms must be designed and/or validated for each combination of sport, sex and mouthguard system. This study provides the first algorithm to classify head acceleration data from exclusively female rugby union players. Mouthguards instrumented with kinematic sensors were given to 25 participants for six competitive rugby union matches in an inter-university league. Across all instrumented players, 214 impacts were recorded from 460 match-minutes. Matches were video recorded to enable retrospective labelling of genuine and spurious events. Four machine learning algorithms were trained on five matches to predict these labels, then tested on the sixth match. Of the four classifiers, the support vector machine achieved the best results, with area under the receiver operator curve (AUROC) and area under the precision recall curve (AUPRC) scores of 0.92 and 0.85 respectively, on the test data. These findings represent an important development for head impact telemetry in female sport, contributing to the safer participation and improving the reliability of head impact data collection within female contact sport.

Real-Time Global Localization of a Mobile Robot by Exploiting RFID Technology

This work addresses the problem of the global localization of a mobile robot by exploiting RFID technology. The robot is equipped with kinematic sensors capable of measuring the incremental distances covered by each wheel and an RFID reader which measures the phase of the signal that is backscattered by reference RFID tags situated inside the environment at known positions. The motion of the robot with respect to the tags enables the collection of measurements from multiple antenna-locations, in a synthetic-aperture sense. Localization of the mobile robot over time is accomplished by processing short segments of trajectory during each iteration. The odometry-data are exploited to estimate each segment of the vehicle’s trajectory relative to an unknown initial position. The odometry-based estimation, the known positions of the reference tags, the unwrapped phase measurements and a phase-to-distance model represent the input to a data-fit problem, the solution of which reveals the unknown initial position and orientation of the trajectory segment in the absolute frame. Thanks to phase unwrapping, the data-fit problem features convexity-type properties and the solution is rapidly found by non-linear optimization techniques. The influence of different algorithmic features on the performance of the method is investigated in a simulative context. An experimental campaign that employs a prototype robotic agent equipped with two antennas reports a mean absolute localization error of less than 0.1m, while the execution time stays below the measurements’ collection-time such that the real-time capability of the method is preserved.

Reduced motor planning underlying inhibition of prepotent responses in children with ADHD

To flexibly regulate their behavior, children’s ability to inhibit prepotent responses arises from cognitive and motor mechanisms that have an intertwined developmental trajectory. Subtle differences in planning and control can contribute to impulsive behaviors, which are common in Attention Deficit and Hyperactivity Disorder (ADHD) and difficult to be assessed and trained. We adapted a Go/No-Go task and employed a portable, low-cost kinematic sensor to explore the different strategies used by children with ADHD or typical development to provide a prepotent response (dominant condition) or inhibit the prepotent and select an alternative one (non-dominant condition). Although no group difference emerged on accuracy levels, the kinematic analysis of correct responses revealed that, unlike neurotypical children, those with ADHD did not show increased motor planning in non-dominant compared to dominant trials. Future studies should investigate whether motor control could help children with ADHD compensate for planning difficulties. This strategy might make inhibition harder in naturalistic situations that involve complex actions. Combining cognitive and kinematic measures is a potential innovative method for assessment and intervention of subtle differences in executive processes such as inhibition, going deeper than is possible based on accuracy outcomes alone.

Cortico-Subthalamic Field Potentials Support Classification of the Natural Gait Cycle in Parkinson's Disease and Reveal Individualized Spectral Signatures.

The ability of humans to coordinate stereotyped, alternating movements between the two legs during bipedal walking is a complex motor behavior that requires precisely timed activities across multiple nodes of the supraspinal network. Understanding of the neural network dynamics that underlie natural walking in humans is limited. We investigated cortical and subthalamic neural activities during overground walking and evaluated spectral biomarkers to decode the gait cycle in three patients with Parkinson's disease without gait disturbances. Patients were implanted with chronic bilateral deep brain stimulation (DBS) leads in the subthalamic nucleus (STN) and electrocorticography paddles overlaying the primary motor and somatosensory cortices. Local field potentials were recorded from these areas while the participants performed overground walking and synchronized to external gait kinematic sensors. We found that the STN displays increased low-frequency (4-12 Hz) spectral power during the period before contralateral leg swing. Furthermore, STN shows increased theta frequency (4-8 Hz) coherence with the primary motor through the initiation and early phase of contralateral leg swing. Additional analysis revealed that each patient had specific frequency bands that could detect a significant difference between left and right initial leg swing. Our findings indicate that there are alternating spectral changes between the two hemispheres in accordance with the gait cycle. In addition, we identified patient-specific, gait-related biomarkers in both the STN and cortical areas at discrete frequency bands that may be used to drive adaptive DBS to improve gait dysfunction in patients with Parkinson's disease.

Specific Test Design for the In-Depth Technique Analysis of Elite Karate Competitors with the Application of Kinematic Sensors

Karate fighters are under constant pressure to find adequate scoring solutions in ever-changing combat conditions. Thus, technique improvement at high levels of mastery demands a novel approach to key data acquisition and in-depth analysis of more than just the impact phase in punch execution. With the aim of describing the kinematic and temporal structure of a reverse punch in the developmental phase, two wireless sensors were used for the acquisition of selected quantities in ten modalities performed by a continental and world medallist. The results show that the timeline of kinematic parameters may be a reliable factor regarding the efficiency of the reverse punch. The obtained hand results show a tendency towards maintaining greater levels of stability in comparison to the body. Additionally, the differences between parameters in relation to applied tests that replicated training and combat conditions were noted. The highest acceleration values were obtained in sliding motion preceding RP, with a partner holding chest punch pad, both static (7.35 ± 0.47 g0) and dynamic (6.99 ± 1.23 g0) tests. The same applies for velocity (8.39 ± 0.14 and 7.30 ± 1.28 m/s). The obtained results indicate the need for specific testing and an individual approach in the analysis of the techniques of elite competitors, along with the use of sensors in data acquisition. Such an approach may help improve the training and competition practice of karate fighters.

Research on the identification of DCT vehicle driver’s starting intention based on LSTM neural network and multi-sensor data fusion

Currently, the objective evaluation of the DCT vehicle drivability requires the accurate identification of the driver’s intention and vehicle state as well as the selection of the targeted evaluation indicators. The existing identification methods usually cannot divide the driver’s intentions in detail and make full use of the characteristics of time-series signals. Simultaneously, external kinematic sensors are more commonly used than the sensors of vehicle powertrain, which impacts the recognition effect. This paper proposes a new method for identifying the DCT vehicle driver’s starting intentions based on an LSTM neural network and multi-sensor data fusion. The DCT vehicle driver’s starting intentions are subdivided and defined based on human–vehicle interaction analysis and K-means clustering. The input of the model consists of 11-dimensional variables that include motion parameters of the vehicle collected by the external sensors and the powertrain parameters collected by onboard sensors. The method proposed in this paper first establishes a recognition window, which is utilized to extract the starting process samples from the DCT vehicle driving data. Second, the 11 variables of each sample are used as one set of multi-dimensional time-series signals, which are preprocessed through wavelet denoising. Finally, the LSTM network is used to identify the samples. The identification results indicate that the highest recognition accuracy of the proposed algorithm is 94.27%, which is approximately 5% higher than conventional methods, such as fully connected neural networks and support vector machines. Furthermore, the model with 11 input variables outperforms the model with fewer input variables. The effectiveness and superiority of the identification model have been demonstrated.

Use of Brain Biomechanical Models for Monitoring Impact Exposure in Contact Sports

Head acceleration measurement sensors are now widely deployed in the field to monitor head kinematic exposure in contact sports. The wealth of impact kinematics data provides valuable, yet challenging, opportunities to study the biomechanical basis of mild traumatic brain injury (mTBI) and subconcussive kinematic exposure. Head impact kinematics are translated into brain mechanical responses through physics-based computational simulations using validated brain models to study the mechanisms of injury. First, this article reviews representative legacy and contemporary brain biomechanical models primarily used for blunt impact simulation. Then, it summarizes perspectives regarding the development and validation of these models, and discusses how simulation results can be interpreted to facilitate injury risk assessment and head acceleration exposure monitoring in the context of contact sports. Recommendations and consensus statements are presented on the use of validated brain models in conjunction with kinematic sensor data to understand the biomechanics of mTBI and subconcussion. Mainly, there is general consensus that validated brain models have strong potential to improve injury prediction and interpretation of subconcussive kinematic exposure over global head kinematics alone. Nevertheless, a major roadblock to this capability is the lack of sufficient data encompassing different sports, sex, age and other factors. The authors recommend further integration of sensor data and simulations with modern data science techniques to generate large datasets of exposures and predicted brain responses along with associated clinical findings. These efforts are anticipated to help better understand the biomechanical basis of mTBI and improve the effectiveness in monitoring kinematic exposure in contact sports for risk and injury mitigation purposes.

Blockchained Federated Learning for Privacy and Security Preservation: Practical Example of Diagnosing Cerebellar Ataxia.

Cerebellar ataxia (CA) refers to the incoordination of movements of the eyes, speech, trunk, and limbs caused by cerebellar dysfunction. Conventional machine learning (ML) utilizes centralised databases to train a model of diagnosing CA. Despite the high accuracy, these approaches raise privacy concern as participants' data revealed in the data centre. Federated learning is an effective distributed solution to exchange only the ML model weight rather than the raw data. However, FL is also vulnerable to network attacks from malicious devices. In this study, we depict the concept of blockchained FL with individual's validators. We simulate the proposed approach with real-world dataset collected from kinematic sensors of CA individuals with four geographically separated clinics. Experimental results show the blockchained FL maintains competitive accuracy of 89.30%, while preserving both privacy and security.

Fusion of Wearable Kinetic and Kinematic Sensors to Estimate Triceps Surae Work during Outdoor Locomotion on Slopes.

Muscle–tendon power output is commonly assessed in the laboratory through the work loop, a paired analysis of muscle force and length during a cyclic task. Work-loop analysis of muscle–tendon function in out-of-lab conditions has been elusive due to methodological limitations. In this work, we combined kinetic and kinematic measures from shear wave tensiometry and inertial measurement units, respectively, to establish a wearable system for estimating work and power output from the soleus and gastrocnemius muscles during outdoor locomotion. Across 11 healthy young adults, we amassed 4777 strides of walking on slopes from −10° to +10°. Results showed that soleus work scales with incline, while gastrocnemius work is relatively insensitive to incline. These findings agree with previous results from laboratory-based studies while expanding technological capabilities by enabling wearable analysis of muscle–tendon kinetics. Applying this system in additional settings and activities could improve biomechanical knowledge and evaluation of protocols in scenarios such as rehabilitation, device design, athletics, and military training.

Design and Implementation of Home Training System based on Personal Fitness Using Kinect Sensor

In this paper, we implemented and developed a home training program using sensors. The implemented program grasps the user's athletic ability and suggests personalized exercise program. In modern society, the aging population is growing rapidly, and most of them are working and living in the interior. For this reason, health management of the aging population is becoming an important social issue, and it is recognized that muscle strength is an important factor. In this paper, we designed and developed a home training program for aged people to exercise in a comfortable environment like natural environment. In the developed program, we extract the skeletal feature points of the user using Microsoft's Kinect sensor and generate feature vectors to check the user's motion state. The proposed algorithm measures the user 's athletic ability and suggests a suitable exercise based on the information. As a result of applying kinematic sensor measurements to the proposed algorithm, it showed up to 97% accuracy. We measured the user's athletic ability through the program using the proposed algorithm and suggested the exercise suitable for the individual ability based on this information.

Diagnosis Cerebellar Ataxia using Deep Learning with Time Series Transformed Image.

Cerebellar ataxia (CA) is defined by disrupted coordination of movement suffering from disease of the cerebellum. It reflects fragmented movements of the eyes, vocal, upper limbs, balance, gait, and lower limbs. This study aims to use a motion sensor to form a simple yet effective CA quantitative assessment framework. We suggest a pendant device to use a single kinematic sensor attached to the wearer's chest to investigate the balance capability. Via a standard neurological test (Romberg's standing), the device may reveal an early symptom of Cerebellar Ataxia tailoring toward rehabilitation or therapeutic program. We adopt a transformed-image based approach to leverage the advantage of state-of-the-art deep learning models into diagnosis CA. Three transform techniques are employed including recurrence plot, melspectrogram, and Poincaré plot. Experiment results show that melspectrogram transform technique performs best in implementation with MobileNetV2 to diagnose CA with an average validation accuracy of 89.99%.

Vision-based human joint angular velocity estimation during squat and walking on a treadmill actions.

Elderly health monitoring, rehabilitation training, and sport supervision could benefit from continuous assessment of joint angle, and angular velocity to identify the joint movement patterns. However, most of the measurement systems are designed based on special kinematic sensors to estimate angular velocities. The study aims to measure the lower limb joint angular velocity based on a 2D vision camera system during squat and walking on treadmill action using deep convolution neural network (CNN) architecture. Experiments were conducted on 12 healthy adults, and six digital cameras were used to capture the videos of the participant actions in lateral and frontal view. The normalized cross-correlation (Ccnorm) analysis was performed to obtain a degree of symmetry of the ground truth and estimated angular velocity waveform patterns. Mean Ccnorm for angular velocity estimation by deep CNN model has higher than 0.90 in walking on the treadmill and 0.89 in squat action. Furthermore, joint-wise angular velocities at the hip, knee, and ankle joints were observed and compared. The proposed system gets higher estimation performance under the lateral view and the frontal view of the camera. This study potentially eliminates the requirement of wearable sensors and proves the applicability of using video-based system to measure joint angular velocities during squat and walking on a treadmill actions.

Joint angle estimation with wavelet neural networks

This paper presents a wavelet neural network (WNN) based method to reduce reliance on wearable kinematic sensors in gait analysis. Wearable kinematic sensors hinder real-time outdoor gait monitoring applications due to drawbacks caused by multiple sensor placements and sensor offset errors. The proposed WNN method uses vertical Ground Reaction Forces (vGRFs) measured from foot kinetic sensors as inputs to estimate ankle, knee, and hip joint angles. Salient vGRF inputs are extracted from primary gait event intervals. These selected gait inputs facilitate future integration with smart insoles for real-time outdoor gait studies. The proposed concept potentially reduces the number of body-mounted kinematics sensors used in gait analysis applications, hence leading to a simplified sensor placement and control circuitry without deteriorating the overall performance.