Motion Capture Sensors Research Articles

Three-Dimensional Aerial Image Interface, 3DAII

In this paper, we introduce the three-dimensional aerial image interface, 3DAII. This interface reconstructs and aerially projects a three-dimensional object image, which can be simultaneously observed from various viewpoints or by multiple users with the naked eye. A pyramid reflector is used to reconstruct the object image, and a pair of parabolic mirrors is used to aerially project the image. A user can directly manipulate the three-dimensional object image by superimposing a user’s hand-finger or a rod on the image. A motion capture sensor detects the user’s hand-finger that manipulates the projected image, and the system immediately exhibits some reaction such as deformation, displacement, and discoloration of the object image, including sound effects. A performance test is executed to confirm the functions of 3DAII. The execution time of the end-tip positioning of a robotic arm has been compared among four operating devices: touchscreen, gamepad, joystick, and 3DAII. The results exhibit the advantages of 3DAII; we can directly instruct the movement direction and movement speed of the end-tip of the robotic arm, using the three-dimensional Euclidean vector outputs of 3DAII in which we can intuitively make the end-tip of the robotic arm move in three-dimensional space. Therefore, 3DAII would be one important alternative to an intuitive spatial user interface, e.g., an operation device of aerial robots, a center console of automobiles, and a 3D modelling system. A survey has been conducted to evaluatecomfortandfatiguebased on ISO/TS 9241-411 andease of learningandsatisfactionbased on the USE questionnaire. We have identified several challenges related to visibility, workspace, and sensory feedback to users that we would like to address in the future.

YogaNet: 3-D Yoga Asana Recognition Using Joint Angular Displacement Maps With ConvNets

Representing 3-D motion-capture sensor data with 2-D color-coded joint distance maps (JDMs) as input to a deep neural network has been shown to be effective for 3-D skeletal-based human action recognition tasks. However, the joint distances are limited by their ability to represent rotational joint movements, which account for a considerable amount of information in human action classification tasks. Moreover, for the subject, view and time invariance in the recognition process, the deep classifier needs training on JDMs along different coordinate axes from multiple streams. To overcome the above shortcomings of JDMs, we propose integrating joint angular movements along with the joint distances in a spatiotemporal color-coded image called a joint angular displacement map (JADM). In the literature, multistream deep convolutional neural networks (CNNs) have been employed to achieve invariance across subjects and views for 3-D human action data, which is achieved by sacrificing training time for accuracy. To improve the recognition accuracy with reduced training times, we propose to test our JADMs with a single-stream deep CNN model. To test and analyze the proposed method, we chose video sequences of yoga. The 3-D motion-capture data represent a complex set of actions with lateral and rotational spatiotemporal variations. We validated the proposed method using 3-D traditional human action data from the publicly available datasets HDM05 and CMU. The proposed model can accurately recognize 3-D yoga actions, which may help in building a 3-D model-based yoga assistant tool.

The Synchronisation Of Motion Capture Results In The Animation Character Reinforcement Process

Animation is a collection of frames that express a motion activity. The animation consists of actors, characters and performance components that present a story — one of the technologies in making an animation with motion capture. The results of catching motion in the form of motion record data are then synchronised to animated characters. However, the problem is that there are difficulties in synchronising animated characters with the results of catching motion so that the motion can be more subtle. On capture, this motion uses Kinect as a motion capture sensor for actors. The results of this capture then become motion data. The synchronisation of motion data from the motion capture results with Kinect then adjusts the joint body character points, frames, time with three-dimensional (3D) animated characters according to the flow of character actors. This synchronisation is adjusted to the recording of character movements starting from the motion of humans, animals and objects that invite motion activity with motion capture methods both marked and without markers. In making 3D, there are models of giving a body or bone frame (rigging) after giving the body frames is done by giving visual effects, lighting, rendering, and compositing. Rigging is the installation of bones to characters used to place and manipulate animated controls on characters that will be animated to produce the desired gesture. Animation synchronisation is to create animated characters manually; the file is used for. The synchronisation is Biovision Hierarchy (BVH) with gesture giving movement without having to save the keyboard in every animation movement manually.

A Random Forest-based Approach for Hand Gesture Recognition with Wireless Wearable Motion Capture Sensors

Abstract Gesture Recognition has a prominent importance in smart environment and home automation. Thanks to the availability of Machine Learning approaches it is possible for users to define gestures that can be associated with commands for the smart environment. In this paper we propose a Random Forest-based approach for Gesture Recognition of hand movements starting from wireless wearable motion capture data. In the presented approach, we evaluate different feature extraction procedures to handle gestures and data with different duration. To enhance reproducibility of our results and to foster research in the Gesture Recognition area, we share the dataset that we have collected and exploited for the present work.

Physics-based keyframe selection for human motion summarization

Analysis of human motion is a field of research that attracts significant interest because of the wide range of associated application domains. Intangible Cultural Heritage (ICH), including the performing arts and in particular dance, is one of the domains where related research is especially useful and challenging. Effective keyframe selection from motion sequences can provide an abstract and compact representation of the semantic information encoded therein, contributing towards useful functionality, such as fast browsing, matching and indexing of ICH content. The availability of powerful 3D motion capture sensors along with the fact that video summarization techniques are not always applicable to the particular case of dance movement create the need for effective and efficient summarization techniques for keyframe selection from 3D human motion capture data sequences. In this paper, we introduce two techniques: a “time-independent” method based on k-means++ clustering algorithm for the extraction of prominent representative instances of a dance, and a physics-based technique that creates temporal summaries of the sequence at different levels of detail. The proposed methods are evaluated on two dance motion datasets and show promising results.

Frequency response of the leap motion controller and its suitability for measuring tremor

Although tremor is one of the most common movement disorders, it is evaluated using relatively coarse clinical scales. We propose to measure tremor in clinical settings using the Leap Motion Controller (LMC), which is a markerless motion capture sensor that has a low cost, zero set-up time, and dynamic accuracy of 1.2 mm. However, the frequency response of the LMC has not been characterized, so its ability to track oscillations such as tremor is unknown. To characterize the frequency response of the LMC, we measured the position of a mannequin hand simultaneously with the LMC and a high-resolution encoder while the mannequin hand oscillated at various combinations of frequency (1–15 Hz) and amplitudes (0.01–30 mm). We calculated the magnitude ratio and phase shift of the LMC and found the bandwidth of the LMC to range from 0–3 Hz to 0–5 Hz for tremor amplitudes greater than the dynamic accuracy. This bandwidth is too small to accurately measure most tremors. However, we developed an inverse filter to estimate the actual tremor amplitude and phase despite the limited bandwidth. Over the combinations of frequency and amplitude mentioned above, the inverse filter estimated the actual tremor amplitude and phase with errors of 3% and 2%, respectively.

Kinect-Based Exergames Tailored to Parkinson Patients

Parkinson’s disease (PD) is a progressive neurodegenerative movement disorder where motor dysfunction gradually increases. PD-specific dopaminergic drugs can ameliorate symptoms, but neurologists also strongly recommend physiotherapy combined with regular exercise. As there is no known cure for PD, traditional rehabilitation programs eventually tire and bore patients to the point of losing interest and dropping out of those programs just because of the predictability and repeatability of the exercises. This can be avoided with the help of current technology, character-based, interactive 3D games promote physical training in a nonlinear fashion, and can provide experiences that change each time the game is played. Such “exergames” (a combination of the words “exercise” and “game”) challenge patients into performing exercises of varying complexity in a playful and interactive environment. In this work we present a Unity3D-based platform hosting two exergames tailored to PD patients with mild to moderate symptoms. The platform employs Microsoft Kinect, an affordable off-the-shelf motion capture sensor that can be easily installed in both home and clinical settings. Platform navigation and gameplay rely on a collection of gestures specifically developed for this purpose and are based upon training programs tailored to PD. These gestures employ purposeful, large-amplitude movements intended to improve postural stability and reflexes and increase upper and lower limb mobility. When the patient’s movements, as detected by Kinect, “match” a preprogrammed gesture, an onscreen 3D cartoon avatar responds according to the game context at hand. In addition, ingame decision-making aims at improving cognitive reaction.

Personal Identification Through Pedestrians’ Behavior

This article focuses on a new approach for personal identification by exploring the features of pedestrian behavior. The recent progress of a motion capture sensor system enables personal identification using human behavioral data observed from the sensor. Kinect is a motion sensing input device developed by Microsoft for Xbox 360 and Xbox One. Personal identification using the Microsoft Kinect sensor (hereafter referred to as Kinect) is presented in this study. Kinect is used to estimate body sizes and the walking behaviors of pedestrians. Body sizes such as height and width, and walking behavior such as joint angles and stride lengths, for example, are used as explanatory variables for personal identification. An algorithm for the personal identification of pedestrians is defined by a traditional neural network and by a support vector machine. In the numerical experiments, pictures of body sizes and the walking behaviors are captured from fifteen examinees through Kinect. The walking direction of pedestrians was specified as 0°, 90°, 180°, and 225°, and then the accuracies were compared. The results indicate that identification accuracy was best when the walking direction was 180°. In addition, the accuracy of the vector machine was better than that of the neural network.

An adaptive framework for the creation of exergames for intangible cultural heritage (ICH) education

As of the early twentieth century, a significant body of research has been published that shows how effective game-based learning and gamification techniques can be, compared to other methods. These technologies are also very important for the learning and transmission of intangible cultural heritage (ICH) creations, which include, among others, dance, theater, and other skills where body motion has a primary role. However, creating games can be time consuming and usually demands a significant effort. Therefore, this paper focuses on the design and development of a novel framework for the rapid design of body-motion-based customizable game-like applications. This framework consists of two components: (i) an interface that allows the user to design the game and capture the motion data, and (ii) a customizable game for learning and training using off-the-shelf motion-capture sensors like the Microsoft Kinect. The game is automatically configured based on the output of the game design interface. In order to evaluate the proposed system, three pilot-use cases have been selected: (i) the Latin dance Salsa, (ii) the Greek traditional dance Tsamiko, and (iii) the Walloon (Belgian) traditional dance. Moreover, small-scaled experiments concerning the three different use cases were conducted, where both beginners and experts evaluated the game-like application for dance learning. Furthermore, a group of dance experts were asked to design and generate their own game and evaluate their experience. Results showed that the use of such a game-like application could be efficient, as positive feedback was obtained. In summary, participants found that the generated game-like application meets its objectives; it is generally efficient and satisfactory and offers a novel tool for ICH transmission and education. Last but not the least, the dance experts expressed their interest in developing more such games in the future, since they characterized the game design module as easy and intuitive to use.

Using the body kinematics to assess the utilization of transhumeral prostheses

There is a gap between the technical capability of upper limb prostheses, and the control strategy (commonly sequential myoelectric control) used to actuate the motorized joints. Surprisingly, the control method is the same for prosthetic hands and hand/wrist/elbow systems, which leads to a complex utilization. Research groups are investigating new control strategies to make the prosthetic control easier. Most studies are focusing on the task achievement, but it is nonetheless essential to assess the achievement method, especially since high-level amputees are prone to develop compensation-related musculoskeletal disorders. The study aims to show that body kinematics can be used to compare different control strategies, or prosthetic equipments. Transhumeral amputees performed a reaching task with different prosthetic parameters (different sockets, different control strategies). They performed the task with a prosthesis prototype that could be adapted to any socket, and implemented with any control strategy. They were equipped with motion capture sensors in order to record their body kinematics. The body motion analysis included the trunk and shoulder displacements, as well as the weight deviation. We could observe that each transhumeral amputee had a unique utilization of their prosthesis, and that they had their own residual capabilities. All the participants performed the task with the two tested control strategies successfully. However, the kinematic analysis showed differences between the control strategies in terms of coordination restauration, that the participants described but that the task assessment (success or not) could not provide. Differences could be highlighted between control strategies that the standard assessment method could not show. Hence the whole body kinematics can be a substantial source of information, especially given the potential severity of compensatory strategies in upper limb prosthesis utilization.

Low-dimensional dynamical characterization of human performance of cancer patients using motion data

BackgroundBiomechanical characterization of human performance with respect to fatigue and fitness is relevant in many settings, however is usually limited to either fully qualitative assessments or invasive methods which require a significant experimental setup consisting of numerous sensors, force plates, and motion detectors. Qualitative assessments are difficult to standardize due to their intrinsic subjective nature, on the other hand, invasive methods provide reliable metrics but are not feasible for large scale applications. MethodsPresented here is a dynamical toolset for detecting performance groups using a non-invasive system based on the Microsoft Kinect motion capture sensor, and a case study of 37 cancer patients performing two clinically monitored tasks before and after therapy regimens. Dynamical features are extracted from the motion time series data and evaluated based on their ability to i) cluster patients into coherent fitness groups using unsupervised learning algorithms and to ii) predict Eastern Cooperative Oncology Group performance status via supervised learning. FindingsThe unsupervised patient clustering is comparable to clustering based on physician assigned Eastern Cooperative Oncology Group status in that they both have similar concordance with change in weight before and after therapy as well as unexpected hospitalizations throughout the study. The extracted dynamical features can predict physician, coordinator, and patient Eastern Cooperative Oncology Group status with an accuracy of approximately 80%. InterpretationThe non-invasive Microsoft Kinect sensor and the proposed dynamical toolset comprised of data preprocessing, feature extraction, dimensionality reduction, and machine learning offers a low-cost and general method for performance segregation and can complement existing qualitative clinical assessments.

Elongation of the surface of the spine during lifting and lowering, and implications for design of an upper body industrial exoskeleton

The aim of this study was to assess the elongation of the skin surface of the spine for simulated industrial lifting and lowering tasks to aid the design of industrial exoskeletons worn on the back. Eighteen male participants lifted and lowered a box of varying loads (5 kg, 10 kg, 15 kg) using three techniques (squat, semi-squat, stooped) from the ground to a table. Motion capture sensors attached to the spine from C7 to S1 measured movement. Stoop lifting involved significantly more elongation (mean 71.1 mm; margin of error ±6.9) than squat lifting (mean 36.8 mm; margin of error ±6.9). Load and Task (lift vs. lower) did not have a significant effect on elongation. Elongation of the skin surface of the lumbar spine was greater than for the thoracic spine. These data detail example levels of elongation of the skin surface of the spine, which should be considered in upper body wearable industrial exoskeleton design. Further, exoskeleton design should take into account that the skin surface of the lumbar spine involves greater elongation than the skin surface of the thoracic spine during deep lifting.

Dance Exergame System for Health Using Wearable Devices

This paper proposed a dance training game applied using game theory by using wearable devices. The system extracts the teacher’s motions by using motion capture sensor technology. The pressure on the foot was sensed while dancing, and the motions of the ankle were extracted. The exergame enables students to watch the teacher’s dance in all directions by using an avatar and gaining the effects of both exercise and education by repeatedly practicing avatar’s dancing. To overcome the space constraints of the special equipment in the course of acquiring and measuring the students’ movements and to allow students to practice at any place, the movements of the user’s foot were measured by using a sock type wearable device that the user wears in their lives normally.

Marker-based motion tracking using Microsoft Kinect

Abstract Motion capture systems are used to quantify kinematics of a motion in various fields of research. Despite high accuracy, commercial systems are expensive and complicated to use. The Microsoft Kinect has been used as an inexpensive markerless motion capture sensor but it has shown imprecision in measuring delicate motions. To address these limitations, we developed a marker-based motion capture system using the Kinect version 2 sensor. Joint use of the Kinect IR camera as a pinhole camera model and its depth data was introduced in order to calculate the 3-dimensional coordinates of retroreflective markers. The proposed method enabled us to capture joint angles of a human-like ankle flexion/extension motion performed by an HRP-2 humanoid using Kinect version 2 which is known to be a challenging task for the Kinect. The humanoid motion data was used as ground-truth to validate the accuracy of the novel motion capture system. The results pave the way for a breakthrough affordable motion capture system used in both robotics and clinical applications.

Using an Optical Motion Sensor for Visualization and Analysis of Maintenance Work on Semiconductor Manufacturing Equipment

We developed a motion capture technique to record a worker’s movements during preventive maintenance, especially wiping actions during wet cleaning. Time-series 3-D coordinate data of the worker’s hand was successfully obtained with a motion capture sensor, and four different possible movements were distinguished. This data was also correlated with changes in the number of particles on the surface of a chamber part before and after wiping. The results show that key indicators of high-quality maintenance work can be extracted from workers’ motions using this technique.

Segmentation and detection of physical activities during a sitting task in Parkinson’s disease participants using multiple inertial sensors

IntroductionThe development of inertial sensors in motion capture systems enables precise measurement of motor symptoms in Parkinson’s disease (PD). The type of physical activities performed by the PD participants is an important factor to compute objective scores for specific motor symptoms of the disease. The goal of this study is to propose an approach to automatically detect the physical activities over a period time and segment the time stamps for such detected activities. MethodsA wearable motion capture sensor system using inertial measurement units (IMUs) was used for data collection. Data from the sensors attached to the shoulders, elbows, and wrists were utilized for detecting and segmenting the activities. An unsupervised machine learning algorithm was employed to extract suitable features from the appropriate sensors and classify the data points to the corresponding activity group. ResultsThe performance of the proposed technique was evaluated with respect to the manually labeled and segmented activities. The experimental results reveal that the proposed auto detection technique – by obtaining high average scores of accuracy (96%), precision (96%), and recall (98%) – is able to effectively detect the activities during the sitting task and segment them to the proper time stamps.

Task-space region-reaching control for medical robot manipulator

This paper addresses the remote control of robot skeleton based on gesture recognition of a human operator’s body. The control motion procedure is facilitated by the use of the Kinect motion capture sensor, which transmits appropriate signals to the robot’s joints to adequately replicate the operator’s movement. Since the movement of the human operator is allowed to occupy the whole Kinect’s space field of view, the arm robot however might have limited range of operation and therefore is subjected to operate in constrained region of the task space. The concept used for the control of the arm robot relies on the employment of the potential energy of a specified geometric desired region. The controller also incorporates high filtered version of joint errors to regulate the system without joint velocity measurement. The approach is experimentally demonstrated on five degrees of freedom articulated robot arm.

A New Multi-Sensor Fusion Scheme to Improve the Accuracy of Knee Flexion Kinematics for Functional Rehabilitation Movements.

Exergames have been proposed as a potential tool to improve the current practice of musculoskeletal rehabilitation. Inertial or optical motion capture sensors are commonly used to track the subject’s movements. However, the use of these motion capture tools suffers from the lack of accuracy in estimating joint angles, which could lead to wrong data interpretation. In this study, we proposed a real time quaternion-based fusion scheme, based on the extended Kalman filter, between inertial and visual motion capture sensors, to improve the estimation accuracy of joint angles. The fusion outcome was compared to angles measured using a goniometer. The fusion output shows a better estimation, when compared to inertial measurement units and Kinect outputs. We noted a smaller error (3.96°) compared to the one obtained using inertial sensors (5.04°). The proposed multi-sensor fusion system is therefore accurate enough to be applied, in future works, to our serious game for musculoskeletal rehabilitation.

Effects of Large Saddle Panels on the Biomechanics of the Equine Back During Rising Trot: Preliminary Results

The saddle panels, directly in contact with the horse's back, are likely an important element to optimize the fitting of the saddle, the comfort of the horse, and subsequently, the pain management in dorsalgic horses. The aim of this study was to better understand the effect of the saddle panels on the horse's back, by evaluating a prototype saddle (comfort panels: CP) compared to a standard saddle (STD). The horse's back movements were measured using inertial measurement units (IMUs) fixed at the levels of thoracic vertebrae T6, T12, T16 (under the saddle) and lumbar vertebrae L2 and L5. The centers of mass (COMs) of the horse and the rider and limb's protraction-retraction angles, pressure between saddle and horse's back, and force on the stirrups were measured using respectively 2D motion capture, pressure mat and force sensors in the stirrup leather. Three horses were trotted at the rising trot (sitting: left diagonal-rider seated; standing: right diagonal-rider standing) by the same rider. To compare saddles, linear mixed-effects regression models were used. The estimated means (SE) were calculated. During sitting phase, pressure in the cranial and middle areas of the saddle significantly increased for CP compared to STD (+0.9 (0.2) kPa and +1.0 (0.1) kPa, respectively) whereas caudal pressure decreased (−1.8 (0.4) kPa). Concurrently, the range of motion of angles T12-T16 and T16-L2 under the saddle significantly increased (+1.8 (0.2)° and +2.3 (0.3)°, respectively). The results showed that modifications of the panels' shape not only affect the pressure distribution but also the kinematics of the thoracic and lumbar regions of the equine back.

User acceptance of a touchless sterile system tocontrol virtual orthodontic study models.

In this article, we present an evaluation of user acceptance of our innovative hand-gesture-based touchless sterile system for interaction with and control of a set of 3-dimensional digitized orthodontic study models using the Kinect motion-capture sensor (Microsoft, Redmond, Wash). The system was tested on a cohort of 201 participants. Using our validated questionnaire, the participants evaluated 7hand-gesture-based commands that allowed the user to adjust the model in size, position, and aspect and to switch the image on the screen to view the maxillary arch, the mandibular arch, or models in occlusion. Participants' responses were assessed using Rasch analysis so that their perceptions of the usefulness of the hand gestures for the commands could be directly referenced against their acceptance of the gestures. Their perceptions of the potential value of this system for cross-infection control were also evaluated. Most participants endorsed these commands as accurate. Our designated hand gestures for these commands were generally accepted. We also found a positive and significant correlation between our participants' level of awareness of cross infection and their endorsement to use this system in clinical practice. This study supports the adoption of this promising development for a sterile touch-free patient record-management system.