Vive Tracker Research Articles

Methods for Implementing Tactile Sensations in Virtual Space by Integrating Physical Objects with Their Computer Models

The paper presents original methods and approaches for integrating, from the user's point of view, three-dimensional virtual models visible to them in a virtual reality headset, and their physical prototypes. This allows for the addition of some important elements of physical reality to the virtual environment, such as tactile and muscle-motor sensations. Developed solutions are based on the use of such wireless devices as HTC Vive Trackers, which determine their position and orientation in space, and can be placed on various objects of the real world. An original approach to making a tracking system by means of two HTC Base Stations and several Vive Trackers with selecting one anchor element is proposed, as well as new methods for real-time integration of physical reality objects with their virtual models by using functional control schemes with own blocks for trackers. Based on methods and approaches proposed in this work, software modules were implemented. Approbation of these modules in the VirSim virtual environment system, which was developed in Scientific Research Institute for System Analysis of the Russian Academy of Sciences, showed that the solutions developed are adequate and effective, and can have practical application in virtual environment systems and training complexes.

DTP: learning to estimate full-body pose in real-time from sparse VR sensor measurements

For virtual reality (VR) applications, estimating full-body pose in real-time is becoming increasingly popular. Previous works have reconstructed full-body motion in real time from an HTC VIVE headset and five VIVE Tracker measurements by solving the inverse kinematics (IK) problem. However, an IK solver may yield unnatural poses and shaky motion. This paper introduces Deep Tracker poser (DTP): a method for real-time full-body pose estimation in VR. This task is difficult due to the ambiguous mapping from the sparse measurements to full-body pose. The data obtained from VR sensors is calibrated, normalized and fed into the deep neural networks (DNN). To learn from sufficient data, we propose synthesizing a VR sensor dataset called AMASS-VR from the AMASS, a collection of various motion capture datasets. Furthermore, feet tracking loss is a common problem of VIVE Tracker. To improve the accuracy and robustness of DTP to the occlusion noise, we simulate the occlusion noise by Gaussian random noise. Then we synthesize an occlusion dataset AMASS-OCC and fine-tune DTP on that. We evaluate DTP by comparing with other popular methods in terms of the accuracy and computational cost. The results indicate that DTP outperforms others in terms of the positional error (1.04 cm) and rotational error (4.22 °). The quantitative and qualitative results show that DTP reconstructs accurate and natural full-body pose even under serious feet occlusion, which indicates the superiority of the DTP in modelling the mapping from sparse joint data to the full-body pose.

Integration of Physical Reality Objects with Their 3D Models Visualized in Virtual Environment Systems

The paper describes methods and approaches for integrating, from the user's point of view, three-dimensional virtual models visible to them in a virtual reality headset, and their physical prototypes. This allows for the addition of some important elements of physical reality to the virtual environment, such as tactile and muscle-motor sensations. Developed solutions are based on the use of wireless HTC Vive trackers, which determine their position and orientation in space and have the ability to be attached to various objects in the real world. An original approach to building a tracking system from several controllers of such type with one anchor element selection is proposed, as well as new methods for real-time integration of physical reality objects with their virtual models by means of functional control schemes with blocks for the Vive trackers. Software modules were created based on developed methods and approaches. Approbation of them was carried out in virtual environment system VirSim and showed adequacy and effectiveness of proposed solutions when using in virtual environment systems and training complexes.

Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario

Background: Stethoscopes and mannequins are basic tools for medical practitioners to diagnose various problems. Restricted by space, resources, and time, these tools and common video training are sometimes inaccessible. These obstacles may be solved by immersive virtual reality (VR) that improves interaction and blurs the line between virtual and reality. Therefore, this study combines virtual and real interaction technologies to investigate the impact of multi-sensory interaction on learning outcomes in chest auscultation.Methods: This study integrated VIVE Tracker technology to provide synchronized tactile stimulation in VR. University students in medical-related disciplines participated in the study. Interviews were conducted to understand how students perceived the effectiveness and usefulness of the thoracic auscultation VR.Conclusion: Most students expressed that tactile stimulation gave realistic sensations and effectively reinforced the immersive effect. Simultaneously, it increased their interest and motivation in learning. The integration of synchronized tactile stimulation is a novel approach that expands the realm of interactivity beyond handheld controller manipulation and creates a platform to simulate medical-related scenarios to learn complex knowledgeand skills.

Development and Validation of a Kinematically Accurate Upper-Limb Exoskeleton Digital Twin for Stroke Rehabilitation.

Rehabilitation robotics combined with virtual reality using head-mounted displays enable naturalistic, immersive, and motivating therapy for people after stroke. There is growing interest in employing digital twins in robotic neurore-habilitation, e.g., in telerehabilitation for virtual coaching and monitoring, as well as in immersive virtual reality applications. However, the kinematic matching of the robot's visualization with the real robot movements is hardly validated, potentially affecting the users' experience while immersed in the virtual environment due to a visual-proprioceptive mismatch. The kinematic mismatch may also limit the validity of assessment measures recorded with the digital twin. We present the development and low-cost kinematic validation of a digital twin of a seven active degrees-of-freedom exoskeleton for stroke rehabilitation. We validated the kinematic accuracy of the digital twin end-effector by performing two tasks-a planar reaching task and a 3D functional task-performed by a single healthy participant. We computed the end-effector position and rotation from the forward kinematics of the robot, the digital twin, and data recorded from the real robot using a low-cost tracking system based on HTC VIVE trackers and compared them pair-wise. We found that the digital twin closely matches the forward kinematics and tracked movement of the real robot and thus provides a reliable platform for future research on digital twins for stroke rehabilitation.

Dynamic Pose Tracking Accuracy Improvement via Fusing HTC Vive Trackers and Inertia Measurement Units

Dynamic Pose Tracking Accuracy Improvement via Fusing HTC Vive Trackers and Inertia Measurement Units

Static and Dynamic Accuracy and Occlusion Robustness of SteamVR Tracking 2.0 in Multi-Base Station Setups.

The tracking of objects and person position, orientation, and movement is relevant for various medical use cases, e.g., practical training of medical staff or patient rehabilitation. However, these demand high tracking accuracy and occlusion robustness. Expensive professional tracking systems fulfill these demands, however, cost-efficient and potentially adequate alternatives can be found in the gaming industry, e.g., SteamVR Tracking. This work presents an evaluation of SteamVR Tracking in its latest version 2.0 in two experimental setups, involving two and four base stations. Tracking accuracy, both static and dynamic, and occlusion robustness are investigated using a VIVE Tracker (3.0). A dynamic analysis further compares three different velocities. An error evaluation is performed using a Universal Robots UR10 robotic arm as ground-truth system under nonlaboratory conditions. Results are presented using the Root Mean Square Error. For static experiments, tracking errors in the submillimeter and subdegree range are achieved by both setups. Dynamic experiments achieved errors in the submillimeter range as well, yet tracking accuracy suffers from increasing velocity. Four base stations enable generally higher accuracy and robustness, especially in the dynamic experiments. Both setups enable adequate accuracy for diverse medical use cases. However, use cases demanding very high accuracy should primarily rely on SteamVR Tracking 2.0 with four base stations.

Motion Strategies for a Cobot in a Context of Intermittent Haptic Interface

Abstract From the list of interfaces used in virtual reality systems, haptic interfaces allow users to touch a virtual world with their hands. Traditionally, the user’s hand moves the end-effector of a robotic arm. When there is no contact in the virtual world, the robotic arm is passive; when there is contact, the arm suppresses mobility to the user’s hand in certain directions. Unfortunately, the passive mode is never completely seamless to the user. Haptic interfaces with intermittent contacts are interfaces using industrial robots that move towards the user when contact needs to be made. As the user is immersed in the virtual world via a virtual reality head mounted display (HMD), he cannot perceive the danger of a collision when he changes his area of interest in the virtual environment. The objective of this article is to describe four movement strategies for the robot to be as fast as possible on the contact zone while guaranteeing safety. This work uses the concept of predicting the user’s intention through his gaze direction and the position of his dominant hand (the one touching the object) and safe-points outside the human workspace. Experiments are done and analyzed with a Pareto front with a UR5 robot, an HTC vive tracker system for an industrial application involving the analysis of materials in the interior of a car.

PE-DLS: a novel method for performing real-time full-body motion reconstruction in VR based on Vive trackers

PE-DLS: a novel method for performing real-time full-body motion reconstruction in VR based on Vive trackers

Proyectos de digitalización y realidad virtual en el patrimonio arqueológico. El caso del yacimiento de la Motilla del Azuer en Daimiel (Ciudad Real)

This article presents the virtualization project of the Motilla del Azuer archaeological site in Daimiel (Ciudad Real, Spain). This project is using new and unique tools for the research, documentation, interpretation and dissemination of this important prehistoric site. The incorporation of advanced technological tools in the field of archaeological and cultural heritage, such as digital documentation and virtual reality (VR), are creating new avenues for managing cultural legacies. Modern advances in 3D digitization and VR allow the application of these techniques on a wide range of cultural assets with different characteristics and chronologies, opening a broad spectrum of new possibilities in research, education, and public outreach. This is an expanding area of heritage management as it creates a positive impact on the economic, cultural and social activities of local communities and regions. Because of these positive impacts, more and more stakeholders such as institutions, administrations, and heritage organisations are taking an active interest in this sector of technological development. In the case of the Motilla del Azuer archaeological site, the digitization and virtualization work undertaken, along with other dissemination activities, have provided critical products for the presentation, interpretation, and promotion of this monument.La Motilla del Azuer is one of the most unique archaeological sites in Spain. Dating from the Bronze Age, it is a fortification with a central plan, formed by three lines of concentric walls distributed around a central tower (Fig. 2). Inside, the labyrinthine shape of the access systems is remarkable, made up of narrow corridors, stairs, ramps... This defensive enclosure allowed the protection and control of essential products for the people of the Bronze Age. Its true uniqueness is that it encloses monumental water well more than 15 m deep.La Motilla del Azuer is open to the public and receives thousands of visitors each year. But for conservation reasons, the number of people who can access the site is highly controlled. Likewise, due to its defensive nature and its age, it is a space through which it is difficult to circulate, which makes it impossible for people with reduced mobility to visit. For these and other reasons, the Daimiel City Council in collaboration with the US non-profit entity Global Digital Heritage and the Spanish company Myou Software, has developed a project to digitize the archaeological site and make it accessible through an installed VR system, located permanently in the Daimiel Museum, the interpretation centre for the site (Fig. 10). This facility is designed to be used both by people in wheelchairs or with reduced mobility, as well as all visitors to the museum, and provides a realistic virtual tour of the site. To achieve this level of realism, a detailed 3D digitization of the site using photogrammetry, laser scanning, and 3D spherical photography was undertaken, and a VR installation based on the use of an HTC Vive Pro device that has motion controllers for hands (HTC Vive controllers) and feet (HTC Vive Trackers) was developed. The primary software is called Myou Engine, an open-source 3D engine compatible with XR, developed by the Myou Software company. In order to increase the immersiveness as much as possible, the Myou Software company has also developed a control system called Natural Locomotion that works by moving the arms and legs (Fig. 11).The final result of the project has allowed the implementation of an innovative VR space that increases accessibility to the archaeological site, represents a new tourist attraction for the town, and promotes new ways of managing and enjoying archaeological heritage.

Three-dimensional gait analysis using a virtual reality tracking system

The development of virtual reality (VR) technology has prompted low-budget hardware accessories for VR headsets that support six degrees of freedom (6DOF) tracking. In this study, two methods for describing lower-limb joint kinematics were proposed, implemented in custom software, and validated: Simplified 6DOF protocol and a previously proposed ISB 6DOF based on reporting standard for joint kinematics introduced by International Society of Biomechanics (ISB) adjusted to the HTC Vive Trackers. The low-cost system allowed for the collection of complete kinematic data of the lower limb, obtaining continuous tracker trajectories without time-consuming system calibration. The obtained data can serve as reference data for normal gait. The performed analysis confirmed the high repeatability, comparable with advanced optoelectronic motion capture systems. The proposed tools enable low-cost analysis of the kinematics without the use of additional equipment but in accordance with the protocols used in biomechanical research and consistent with clinical practice.

A Pilot Study Quantifying Center of Mass Trajectory during Dynamic Balance Tasks Using an HTC Vive Tracker Fixed to the Pelvis.

Fall rates are increasing among the aging population and even higher falls rates have been reported in populations with neurological impairments. The Berg Balance Scale is often used to assess balance in older adults and has been validated for use in people with stroke, traumatic brain injury, and Parkinson’s disease. While the Berg Balance Scale (BBS) has been found to be predictive of the length of rehabilitation stay following stroke, a recent review concluded the BBS lacked predictive validity for fall risk. Conversely, sophisticated measures assessing center of mass (COM) displacement have shown to be predictive of falls risk. However, calculating COM displacement is difficult to measure outside a laboratory. Accordingly, we sought to validate COM displacement measurements derived from an HTC Vive tracker secured to the pelvis by comparing it to COM derived from ‘gold’ standard laboratory-based full-body motion capture. Results showed that RMS between the COM calculated from HTC Vive tracker and full body motion capture agree with an average error rate of 2.1 ± 2.6 cm. Therefore, we conclude measurement of COM displacement using an HTC Vive tracker placed on the pelvis is reasonably representative of laboratory-based measurement of COM displacement.

An Evaluation of Motion Trackers with Virtual Reality Sensor Technology in Comparison to a Marker-Based Motion Capture System Based on Joint Angles for Ergonomic Risk Assessment.

The reproduction and simulation of workplaces, and the analysis of body postures during work processes, are parts of ergonomic risk assessments. A commercial virtual reality (VR) system offers the possibility to model complex work scenarios as virtual mock-ups and to evaluate their ergonomic designs by analyzing motion behavior while performing work processes. In this study a VR tracking sensor system (HTC Vive tracker) combined with an inverse kinematic model (Final IK) was compared with a marker-based optical motion capture system (Qualisys). Marker-based optical motion capture systems are considered the gold standard for motion analysis. Therefore, Qualisys was used as the ground truth in this study. The research question to be answered was how accurately the HTC Vive System combined with Final IK can measure joint angles used for ergonomic evaluation. Twenty-six subjects were observed simultaneously with both tracking systems while performing 20 defined movements. Sixteen joint angles were analyzed. Joint angle deviations between and were identified. These high deviations must be considered in ergonomic risk assessments when using a VR system. The results show that commercial low-budget tracking systems have the potential to map joint angles. Nevertheless, substantial weaknesses and inaccuracies in some body regions must be taken into account. Recommendations are provided to improve tracking accuracy and avoid systematic errors.

Abstract P202: Estimation of Arm Kinematics in Stroke Survivors With Wearable Sensors

Introduction: Stroke is the leading cause of long-term disability in the US, often resulting in upper extremity (UE) motor impairment. Numerous clinical metrics have been developed and used to evaluate UE motor function to assess rehabilitation outcomes, but most of these are innately subjective and/or have relatively low sensitivity to change due to the use of ordinal scales. As a result, there is a growing interest in the use of kinematics to evaluate UE motor function, as they provide completely objective, high resolution quantitative measurements. However, obtaining kinematic measurements in stroke survivors can be a challenging task, as non-ideal environments (e.g., hospital rooms), expense, and a limited ability to perform calibration poses can make traditional optical tracking systems impractical. To overcome these challenges, we developed and compared two methods using different wearable sensors to estimate elbow angle and wrist position during reaching movements in people with stroke. Methods: We developed frameworks specific for two different types of sensors, inertial measurement units (IMU) and virtual reality (Vive) trackers, to estimate elbow angle and wrist position. We assessed each sensor’s estimation accuracy during pure flexion-extension motion, performed by a manual goniometer, and pure pronation-supination motion, performed by a healthy participant. We also compared each sensor’s ability to longitudinally track a stroke survivor’s UE function by using the wrist position estimates to compute sweep areas during a sweeping task. Results: For the IMUs, our results demonstrated accuracy to within 4.8° and 1.2 cm for elbow angle and wrist position, respectively. For the Vive, our results demonstrated accuracy to within 2.2° and 0.9 cm. The change in sweep area estimated by each method agreed with each other, as the difference between estimates was approximately 4% of the change in sweep area. Conclusion: These methods can be used to develop and accurately assess kinematic metrics for use in evaluating stroke rehabilitation outcomes.

Feedback motion control of a spatial double pendulum manipulator relying on swept laser based pose estimation

Spatial pose estimation devices for mobile and cable-suspended robots have been rapidly developed. The HTC Vive sensor, which operates with swept laser beams, has aroused many researchers’ interest. We present experiments with a double pendulum robot equipped with the HTC Vive Tracker. A linear feedback controller ensured the tracking of pre-defined end-effector trajectories of various speeds. The pose feedback of the controller was provided by the HTC Vive. As a reference, the realized trajectory was measured by the OptiTrack motion capture system. We report that the motion control of a spatial double pendulum robot can be achieved by using a linear feedback controller together with cable winch and fan actuators providing six independent inputs. The accuracy of the HTC Vive was proved to be sufficient for the feedback position control of indoor mobile robots. We report that the error strongly correlates with the linear/angular velocity, the acceleration and the jerk.

HTC Vive Tracker: Accuracy for Indoor Localization

Researchers have extensively explored indoor localization in recent years. Robotic applications often require precise positioning, which is difficult when access to a GPS is limited or compromised. Many of the existing approaches require either prior knowledge of the local environment and fixed landmarks or complex and expensive hardware to achieve the necessary degree of accuracy. In this article, we examine the use of the HTC virtual reality hardware along with our novel approach, VIVEPOSE, to perform indoor robotic localization. We then compare the accuracy of our proposed indoor localization system to current approaches that use traditional odometry sensor-based localization. Finally, we present and demonstrate a leader-follower approach using VIVEPOSE and show how the HTC Vive tracker can be successful for indoor localization for a robotics application.

Cyber-physical postural training system for construction workers

Risks of work-related musculoskeletal injuries can be reduced by adequately training construction workers on performing work in safe postures. Traditional training approaches provide limited support for performing work tasks and receiving real-time feedback. This paper describes a cyber-physical postural training environment where workers can practice to perform work with reduced ergonomic risks. The proposed system uses wearable sensors, Vive trackers, machine learning and virtual reality to track body kinematics, and engagement with physical construction resources, and provides feedback via an interactive user interface. The postural training system was developed for training workers engaged in wood frame construction. User study of the effectiveness of the feedback and user interface was conducted. Findings showed that the user interface was perceived as convenient with limited interference with the workspace. The feedback was understandable in learning risks associated with participant's postures. Further research will involve conducting formative workload evaluation of the user interface and developing a reinforcement learning model for adapting the feedback based on the state of learning.

Calibration and Evaluation for Visuo-haptic Collocation in Haptic Augmented Virtuality Systems

Haptic augmented virtuality systems can provide users with a highly immersive haptic experience by visuo-haptic collocation of physical objects with their corresponding virtual objects. The collocation can be accomplished by an encounter-type haptic display system where a robotic device provides a physical prop that represents a whole or partial virtual object in real-time and in the space occupied by the virtual object in the virtual environment. For efficient and accurate visuo-haptic collocation, this paper proposes a novel yet simple, efficient and accurate calibration method. The method employs a high-accuracy position device such as an industrial robot that can place physical objects quickly and accurately. The transformation between the reference frames of the real (robot) and virtual worlds is obtained by the simple motion of the robot’s end-effector equipped with a real object that is in turn tracked by a VR tracker system such as the HTC VIVE tracker. An evaluation of the proposed method indicates that it is possible to achieve acceptable accuracy with a few data points as compared to existing calibration methods that require a large number of data points.

Agreement Analysis between Vive and Vicon Systems to Monitor Lumbar Postural Changes.

Immersive virtual reality has recently developed into a readily available system that allows for full-body tracking. Can this affordable system be used for component tracking to advance or replace expensive kinematic systems for motion analysis in the clinic? The aim of this study was to assess the accuracy of position and orientation measures from Vive wireless body trackers when compared to Vicon optoelectronic tracked markers attached to (1) a robot simulating trunk flexion and rotation by repeatedly moving to know locations, and (2) healthy adults playing virtual reality games necessitating significant trunk displacements. The comparison of both systems showed component tracking with Vive trackers is accurate within 0.68 ± 0.32 cm translationally and 1.64 ± 0.18° rotationally when compared with a three-dimensional motion capture system. No significant differences between Vive trackers and Vicon systems were found suggesting the Vive wireless sensors can be used to accurately track joint motion for clinical and research data.

Estimating Forearm Axial Rotation Using Vive Trackers for Interaction With Serious Games.

One of the most common upper limb dysfunction seen among cerebral palsy children is the limited supination or pronation of the forearm, thus training forearm axial rotation is a common demand for interventions. We propose a method to estimate forearm axial rotation using commercially available tracking device HTC Vive tracker to support interaction with serious games with forearm axial rotation movement for rehabilitation in clinic or home environment. The proposed method provides an easy-to-access and affordable way for forearm axial angle estimation when arm is at different positions relative to torso and when tracking devices are worn in different orientations. Effectiveness of the proposed method is examined, and a serious game system that uses the method for interaction is designed which was installed in The First Hospital of Jilin University as an option of forearm movement practices for children with cerebral palsy.