Body-worn Inertial Measurement Units Research Articles

Accurate Long-Term Multiple People Tracking using Video and Body-Worn IMUs.

Most modern approaches for video-based multiple people tracking rely on human appearance to exploit similarities between person detections. Consequently, tracking accuracy degrades if this kind of information is not discriminative or if people change apparel. In contrast, we present a method to fuse video information with additional motion signals from body-worn inertial measurement units (IMUs). In particular, we propose a neural network to relate person detections with IMU orientations, and formulate a graph labeling problem to obtain a tracking solution that is globally consistent with the video and inertial recordings. The fusion of visual and inertial cues provides several advantages. The association of detection boxes in the video and IMU devices is based on motion, which is independent of a person's outward appearance. Furthermore, inertial sensors provide motion information irrespective of visual occlusions. Hence, once detections in the video are associated with an IMU device, intermediate positions can be reconstructed from corresponding inertial sensor data, which would be unstable using video only. Since no dataset exists for this new setting, we release a dataset of challenging tracking sequences, containing video and IMU recordings together with ground-truth annotations. We evaluate our approach on our new dataset, achieving an average IDF1 score of 91.2%. The proposed method is applicable to any situation that allows one to equip people with inertial sensors.

Validity of Pendant-Based IMU Assessment of Postural Stability Under Varying Balance Conditions Compared to a Sensor Positioned on the Lower Back

Body-worn inertial measurement units (IMUs) have been widely used in postural stability and balance studies because of their low cost and high level of convenience. In most studies, single IMU sensors are put on the lower back attached to a belt, placing the sensor near the body’s center of mass (COM). For some populations, such as pregnant women, wearing the sensor on a belt over the lower back presents challenges in terms of fit and comfort. Thus, it may be necessary to identify a better location for the sensor and a more comfortable means for attaching the sensor to the body. This study aims to implement and test a novel pendant IMU sensor hanging from the subject’s neck and placed over the sternum. Three standing tasks (double-leg, tandem, and single-leg standing) were performed under open- and closed-eye conditions for preliminary assessments of the ability of the new sensor to discriminate between balance conditions. Standard deviations were analyzed in different conditions, along with ROC curves and ANOVA analysis. The results showed that the pendant sensor can detect the signals as good as the sensor on the waist.

Golf swing motion analysis using machine learning

Identification of runner’s performance level is critical to coaching, performance enhancement and injury prevention. Machine learning techniques have been developed to measure biomechanical parameters with body-worn inertial measurement unit (IMU) sensors. However, a robust method to classify runners is still unavailable. In this paper, we developed two models to classify running performance and predict biomechanical parameters of 30 subjects. We named the models RunNet-CNN and RunNet-MLP based on their architectures: convolutional neural network (CNN) and multilayer perceptron (MLP), respectively. In addition, we examined two validation approaches, subject-wise (leave-one-subject-out) and record-wise. RunNet-MLP classified runner’s performance levels with an overall accuracy of 97.1%. Our results also showed that RunNet-CNN outperformed RunNet-MLP and gradient boosting decision tree in predicting biomechanical parameters. RunNet-CNN showed good agreement (R2 > 0.9) with the ground-truth reference on biomechanical parameters. The prediction accuracy for the record-wise method was better than the subject-wise method regardless of biomechanical parameters or models. Our findings showed the viability of using IMUs to produce reliable prediction of runners’ performance levels and biomechanical parameters.

Asynchronous Averaging of Gait Cycles for Classification of Gait and Device Modes

The problem of joint classification of gait and device mode from inertial measurement units (IMU) measurements is considered. For this, an approach for computing unique gait signature using measurements collected from body-worn inertial measurement units (IMUs) is proposed.The gait signature represents one full cycle of the human gait, and is suitable for off-line or on-line classification of the gait mode. The signature can also be used to jointly classify the gait mode and the device mode. The device mode identifies how the IMU-equipped device is being carried by the user. The method is based on precise segmentation and resampling of the measured IMU signal, as an initial step, further tuned by minimizing the variability of the obtained signature within each gait cycle. Finally, a Fourier series expansion of the gait signature is introduced which provides a low-dimensional feature vector well suited for classification purposes. The proposed method is evaluated on a large dataset involving several subjects, each one containing two different gait modes and four different device modes.

Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning.

In this paper, we perform a systematic study about the on-body sensor positioning and data acquisition details for Human Activity Recognition (HAR) systems. We build a testbed that consists of eight body-worn Inertial Measurement Units (IMU) sensors and an Android mobile device for activity data collection. We develop a Long Short-Term Memory (LSTM) network framework to support training of a deep learning model on human activity data, which is acquired in both real-world and controlled environments. From the experiment results, we identify that activity data with sampling rate as low as 10 Hz from four sensors at both sides of wrists, right ankle, and waist is sufficient in recognizing Activities of Daily Living (ADLs) including eating and driving activity. We adopt a two-level ensemble model to combine class-probabilities of multiple sensor modalities, and demonstrate that a classifier-level sensor fusion technique can improve the classification performance. By analyzing the accuracy of each sensor on different types of activity, we elaborate custom weights for multimodal sensor fusion that reflect the characteristic of individual activities.

Human crawling performance and technique revealed by inertial measurement units

Human crawling performance and technique are of broad interest to roboticists, biomechanists, and military personnel. This study explores the variables that define crawling performance in the context of an outdoor obstacle course used by military organizations worldwide to evaluate the effects of load and personal equipment on warfighter performance. Crawling kinematics, measured from four body-worn inertial measurement units (IMUs) attached to the upper arms and thighs, are recorded for thirty-three participants. The IMU data is distilled to four metrics of crawling performance; namely, crawl speed, crawl stride time, ipsilateral limb coordination, and contralateral limb coordination. We hypothesize that higher performance (as identified by higher crawl speeds) is associated with more coordinated limbs and lower stride times. A cluster analysis groups participants into high and low performers exhibiting statistically significant differences across the four performance metrics. In particular, high performers exhibit superior limb coordination associated with a “diagonal gait” in which contralateral limbs move largely in-phase to produce faster crawl speeds and shorter crawl stride times. In contrast, low performers crawl at slower speeds with longer crawl stride times and less limb coordination. Beyond these conclusions, a major contribution of this study is a method for deploying wearable IMUs to study crawling in contextually relevant (i.e. non-laboratory) environments.

UWB/PDR Tightly Coupled Navigation with Robust Extended Kalman Filter for NLOS Environments

The fusion of ultra-wideband (UWB) and inertial measurement unit (IMU) is an effective solution to overcome the challenges of UWB in nonline-of-sight (NLOS) conditions and error accumulation of inertial positioning in indoor environments. However, existing systems are based on foot-mounted or body-worn IMUs, which limit the application of the system to specific practical scenarios. In this paper, we propose the fusion of UWB and pedestrian dead reckoning (PDR) using smartphone IMU, which has the potential to provide a universal solution to indoor positioning. The PDR algorithm is based on low-pass filtering of acceleration data and time thresholding to estimate the step length. According to different movement patterns of pedestrians, such as walking and running, several step models are comparatively analyzed to determine the appropriate model and related parameters of the step length. For the PDR direction calculation, the Madgwick algorithm is adopted to improve the calculation accuracy of the heading algorithm. The proposed UWB/PDR fusion algorithm is based on the extended Kalman filter (EKF), in which the Mahalanobis distance from the observation to the prior distribution is used to suppress the influence of abnormal UWB data on the positioning results. Experimental results show that the algorithm is robust to the intermittent noise, continuous noise, signal interruption, and other abnormalities of the UWB data.

Examination of the perceived agility and balance during a reactive agility task.

In vehicle dynamics, it is commonly understood that there is an inverse relationship between stability and maneuverability. However, animal studies have found that stability and maneuverability can coincide. In this study, we examine humans running a reactive agility obstacle and consider the relationship between observational perceived agility and balance, as well as the relationship between quantified surrogates of agility and balance. Recreational athletes (n = 18) completed the agility task while wearing inertial measurement units (IMUs) on their body. The task was also video-recorded. An observational study was completed by a separate group of adults (n = 33) that were asked to view the videos and score each athlete on a Likert scale for balance and for agility. The data from the body-worn IMUs were used to estimate quantified surrogate measures for agility and balance, and to assess if the relationship between the quantified agility and balance was in the same direction as the perceived relationship from the Likert scale responses. Results indicate that athletes that were given a higher Likert agility score were also given a higher balance score (rs = 0.75,p < 0.001). Quantitative surrogates of agility and balance also showed this same relationship. Additional insights on technique for this reactive agility task were informed by the quantitative surrogates. We observed the importance of stepping technique in achieving the faster completion times. The fast performing athletes spent a greater proportion of the task in double support and lower overall time in single support indicating increased periods of static stability. The fast performing athletes did not have a higher body speed, but performed the task with a more efficient technique, using foot placement to enable heading changes, and thus may have had a more efficient path. Similar to animal studies, people use technique to enable agile strategies while also enabling increased balance across the task.

Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation

Quality assessment in cricket is a complex task that is performed by understanding the combination of individual activities a player is able to perform and by assessing how well these activities are performed. We present a framework for inexpensive and accessible, automated recognition of cricketing shots. By means of body-worn inertial measurement units, movements of batsmen are recorded, which are then analysed using a parallelised, hierarchical recognition system that automatically classifies relevant categories of shots as required for assessing batting quality. Our system then generates meaningful visualisations of key performance parameters, including feet positions, attack/defence, and distribution of shots around the ground. These visualisations are the basis for objective skill assessment thereby focusing on specific personal improvement points as identified through our system. We evaluated our framework through a deployment study where 6 players engaged in batting exercises. Based on the recorded movement data we could automatically identify 20 classes of unique batting shot components with an average F1-score greater than 88%. This analysis is the basis for our detailed analysis of our study participants’ skills. Our system has the potential to rival expensive vision-based systems but at a fraction of the cost.

Automated Rehabilitation System: Movement Measurement and Feedback for Patients and Physiotherapists in the Rehabilitation Clinic

In current physical rehabilitation protocols, patients typically perform exercises with intermittent feedback or guidance following the initial demonstrations from the physiotherapist. Although many patient-centered systems have been developed for home rehabilitation, few systems have been developed to aid the physiotherapist as well as patients in the rehabilitation clinic. This article proposes the Automated Rehabilitation System (ARS), a system designed specifically for rehabilitation clinics using an iterative design process, developed with physiotherapists and patients in a knee and hip replacement clinic. ARS consists of body-worn inertial measurement units that continuously measure the patient’s pose. The measured pose is graphically represented as an animation and overlaid with the instructed motion on a visual display shown to the patient during exercise performance. ARS allows physiotherapists to quantitatively measure patient movement, assess recovery progress, and manage and schedule exercise regimens for patients. The system requirements and design requirements were derived through a focus group with 13 physiotherapists. For patients, ARS provides visual feedback and a novel exercise guidance feature to aid them while exercising. The patient interface was evaluated in a user study with 26 outpatients. The results show that performing the exercises with the visual guidance tool improves the quality of exercise performance.

Magnetic Field Swimmer Positioning

Several projects have tracked the movement of swimmers in pools using body worn inertial measurement units. In swimming, inertial sensing is subject to large amounts of drift and accumulated error which can only be corrected for after a complete length has been swum. In this article, we present a new method for tracking swimmers by detecting variations in the magnetic field caused by the structure of pools. This method is complementary to inertial positioning, as it allows the direct extraction of position without requiring post-processing, and unlike inertial sensing which loses accuracy over time, magnetic field tracking becomes increasingly accurate towards the end of a length.

Improving rehabilitation exercise performance through visual guidance.

In current physical rehabilitation protocols, patients typically perform exercises without feedback or guidance following the initial demonstrations from the physiotherapist. This paper proposes a system providing continuous visual feedback and guidance to patients to improve quality of motion performance and adherence to instructions. The system consists of body-worn inertial measurement units which continuously measure the patient's pose. The measured pose is overlaid with the instructed motion on a visual display shown to the user during exercise performance. Two user studies were conducted with healthy participants to evaluate the usability of the visual guidance tool. Motion data was collected by the inertial measurement sensors and used to evaluate quality of motion, comparing user performance with and without visual feedback and with or without exercise guidance. The quantitative and qualitative results of the studies confirm that performing the exercises with the visual guidance tool promotes more consistent exercise performance and proper technique.

Gaussian process framework for pervasive estimation of swimming velocity with body‐worn IMU

Presented is an accurate swimming velocity estimation method using an inertial measurement unit (IMU) by employing a simple biomechanical constraint of motion along with Gaussian process regression to deal with sensor inherent errors. Experimental validation shows a velocity RMS error of 9.0 cm/s and high linear correlation when compared with a commercial tethered reference system. The results confirm the practicality of the presented method to estimate swimming velocity using a single low-cost, body-worn IMU.

Multimodal recognition of reading activity in transit using body-worn sensors

Reading is one of the most well-studied visual activities. Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading. In this work we recognize reading activity by jointly analyzing eye and head movements of people in an everyday environment. Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units. We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data. We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading. Using person-independent training, we obtain an average precision for recognizing reading of 88.9% (recall 72.3%) using STR and of 87.7% (recall 87.9%) using SVM over all participants. We show that the proposed segmentation scheme improves the performance of recognizing reading events by more than 24%. Our work demonstrates that the joint analysis of eye and body movements is beneficial for reading recognition and opens up discussion on the wider applicability of a multimodal recognition approach to other visual and physical activities.