HTC Vive Research Articles

Development of virtual laboratory applications based on virtual reality using the requirements engineering method

The virtual laboratory application that has been designed in Industrial Engineering Department of Diponegoro University can simulate several work operations on turning machines as well as the introduction of tools to the work process bench. In addition to these aspects, applications that have been designed also do not provide a virtual environment that has complete equipment such as in a production system laboratory. Another drawback is that there is no usage guide in the form of a video tutorial for an initial understanding of users when operating a work operation on a turning machine. In practice, mistakes still occur frequently in working on turning machines. According to students, errors often occur due to a lack of understanding from students themselves even though virtual reality tools have been provided in the training of turning machine operations. Based on that problem this research tries to develop virtual laboratory using the Requirement Engineering method and HTC VIVE Pro Full Kit hardware that can correct the shortcomings of the previous development. The output of this research is a virtual reality-based virtual laboratory application that has been tested and evaluated by considering several aspects such as interaction, visualization, information, reusability, performance, and reliability.

P‐15.4: Virtual reality gesture recognition based on depth information

With the rapid development of computer technology, the information revolution is constantly affecting and changing our lives. Today's virtual reality technology is no longer just a virtual model for viewing. Experiencers are more willing to personally interact with the virtual environment in a virtual scene to increase their sense of gaming experience.Today's virtual reality interaction methods mainly interact through the handles that come with VR helmets. Previously, some scholars have researched gesture interaction in virtual environments. They use data gloves to obtain information about hand movements. This method makes users have to wear heavy data gloves, which limits the natural movement of their hands. To overcome this shortcoming, the need for low‐cost vision‐based virtual reality gesture interaction is becoming more and more urgent.Traditional gesture recognition technologies based on artificial modeling include template‐based matching, data‐based gloves, and machine learning. As machine learning gesture classifiers, Hidden Markov Model (HMM) and Support Vector Machine (SVM) [1] have been widely used. However, in the hand motion detection of ordinary cameras, the image is prone to speckles, lighting changes, and other factors, which seriously affects the performance of the classifier, making it difficult to accurately and efficiently detect the first image when the background is complex and the lighting is affected. In recent years, deep learning‐based object detection methods have achieved great breakthroughs [2]. In the past decade, many deep learning‐based dynamic gesture recognition algorithms have been introduced. In the case of changing lighting conditions, in order to improve the classification accuracy of gesture recognition [3]. Therefore, this article chooses a depth camera combined with HTC VIVE device for gesture recognition in virtual reality. First obtain the depth image of the hand through the depth camera, then label the depth image, and then learn through the deep learning network hand movement training [4], preprocess the detected human hand image, and recognize it by hand detection The position of the hand in the image is obtained, and the movement of the hand is recognized as a result of recognition.The algorithm in this paper uses the advanced deep learning network YOLOv3, and changes the network structure and multiscale structure based on the original network. This algorithm is compared with other deep learning algorithms for gesture recognition, and the recognition effect is better than other algorithms in the same category.

Robot apology as a post-accident trust-recovery control strategy in industrial human-robot interaction

Due to safety requirements for Human-Robot Interaction (HRI), industrial robots have to meet high standards of safety requirements (ISO 10218). However, even if robots are incapable of causing serious physical harm, they still may influence people's mental and emotional wellbeing, as well as their trust, behaviour and performance in close collaboration. This work uses an HTC Vive Virtual Reality headset to study the potential of using robot control strategies to positively influence human post-accident behaviour. In the designed scenario, a virtual industrial robot first makes sudden unexpected movements, after which it either does or does not attempt to apologise for them. The results show that after the robot tries to communicate with the participants, the robot is reported to be less scary, more predictable and easier to work with. Furthermore, postural analysis shows that the participants who were the most affected by the robot's sudden movement recover 74% of their postural displacement within 60 s after the event if the robot apologised, and only 34% if it did not apologise. It is concluded, that apologies, which are commonly used as a trust-recovery strategy in social robotics, can positively influence people engaged with industrial robotics as well. Relevance to industryFindings can be used as guidelines for designing robot behaviour and trust-recovery control strategies meant to speed up human recovery after a trust-violating event in industrial Human-Robot Interaction.

Concept evaluation of a new aircraft passenger privacy bubble using virtual prototyping: A Human-Centered Design framework.

The Human-Centered Design methodology advocates VR prototyping, as an effective tool to evaluate concepts in a cost-efficient, time-saving way. It is the question of whether it works in the development of a product intended to increase privacy while flying. The current study aims at the application of virtual reality on the evaluation of a new privacy bubble called PRIVA for the passenger cabin. An interactive VR was created and aligned with the HTC VIVE headset. 40 participants took part in the experiment as well as in the post-experiment survey. The concept was in overall, desirable as it was perceived to be more private, comfortable, satisfactory, effective, and appealing to participants compared to the current seat experiences. It was also perceived as more satisfactory with regards to the activities. The VR was effective, although there are limitations, the product seems promising and should be developed further.

Feedback motion control of a spatial double pendulum manipulator relying on swept laser based pose estimation

Spatial pose estimation devices for mobile and cable-suspended robots have been rapidly developed. The HTC Vive sensor, which operates with swept laser beams, has aroused many researchers’ interest. We present experiments with a double pendulum robot equipped with the HTC Vive Tracker. A linear feedback controller ensured the tracking of pre-defined end-effector trajectories of various speeds. The pose feedback of the controller was provided by the HTC Vive. As a reference, the realized trajectory was measured by the OptiTrack motion capture system. We report that the motion control of a spatial double pendulum robot can be achieved by using a linear feedback controller together with cable winch and fan actuators providing six independent inputs. The accuracy of the HTC Vive was proved to be sufficient for the feedback position control of indoor mobile robots. We report that the error strongly correlates with the linear/angular velocity, the acceleration and the jerk.

Pre-Trained-Based Individualization Model for Real-Time Spatial Audio Rendering System

Spatial audio has attracted more and more attention in the fields of virtual reality (VR), blind navigation and so on. The individualized head-related transfer functions (HRTFs) play an important role in generating spatial audio with accurate localization perception. Existing methods only focus on one database, and do not fully utilize the information from multiple databases. In light of this, a pre-trained-based individualization model is proposed to predict HRTFs for any target user in this paper, and a real-time spatial audio rendering system built on a wearable device is implemented to produce an immersive virtual auditory display. The proposed method first builds a pre-trained model based on multiple databases using a DNN-based model combined with an autoencoder-based dimensional reduction method. This model can capture the nonlinear relationship between user-independent HRTFs and position-dependent features. Then, fine tuning is done using a transfer learning technique at a limit number of layers based on the pre-trained model. The key idea behind fine tuning is to transfer the pre-trained user-independent model to the user-dependent one based on anthropometric features. Finally, real-time issues are discussed to guarantee a fluent auditory experience during dynamic scene update, including fine-grained head-related impulse response (HRIR) acquisition, efficient spatial audio reproduction, and parallel synthesis and playback. These techniques ensure that the system is implemented with little computational cost, thus minimizing processing delay. The experimental results show that the proposed model outperforms other methods in terms of subjective and objective metrics. Additionally, our rendering system runs on HTC Vive, with almost unnoticeable delay.

Reliability and Validity of a Virtual Reality-Based System for Evaluating Postural Stability.

Postural stability is an important indicator of balance and is commonly evaluated in neurorehabilitation. We proposed a system based on a virtual reality (HTC Vive) system with a tracker at the lumbar area. The position data of the tracker were obtained through detection of the sensors on the tracker by the VR system. The reliability and validity of these sway parameters to measure postural stability were evaluated. Twenty healthy adults had their postural sway measured with this system and a force platform system under four stance conditions, with wide- or narrow-stance and eyes open or closed. The path data from both systems were computed to obtain the following parameters: the mean distance and the mean velocity in the medial-lateral and anterior-posterior directions and the 95% confidence ellipse area. The reliability of the Vive-based sway measures was tested with intraclass correlation coefficients (ICCs). The convergent validity was tested against the center of pressure (COP) parameters from the force platform system. Finally, the discriminative validity was tested for the above four conditions. The results indicated that the Vive-based sway parameters had moderate to high reliability (ICCs: 0.56 ~ 0.90) across four conditions and correlated moderately to very highly with the COP parameters ( r = 0.420 ∼ 0.959 ). Bland-Altman plotting showed generally good agreement, with negative offset for the Vive-based sway parameters. The sway parameters obtained by the Vive-based system also discriminated well among the tasks. In conclusion, the results support this system as a simple and easy-to-use tool to evaluate postural stability with acceptable reliability and validity.

Quantitative analysis of temporal characteristics of opto-kinetic nystagmus for vection detection

When a person in a motionless position, observes moving visual stimulus that occupies a significant part of the visual field, the self-motion illusion can be induced. Such illusion is called vection. We conducted an experiment to test the hypothesis that vection appearance and elongation of optokinetic nystagmus slow phases occur at the same time. Vection was induced by rotating with different velocities the optokinetic drum in HTC Vive Pro Eye virtual reality headset. Also the eye movements were recorded during stimulus observation It was found that the duration of the slow phases of optokinetic nystagmus was longer in vection perception period in comparison to situations when vection was not perceived. Thus, we suggested that the elongation of nystagmus slow phases can be an indicator of the vection occurrence in virtual reality.

Design of a Research and Training Platform for Operating Portable Chainsaws Using Virtual Reality Technology

The logging industry belongs to one of the industrial processing branches with the greatest risk of accidents at work. Unfortunately, such situations often result in serious injuries and even the deaths of employees. In most cases, the casualties are people who are badly trained and inexperienced, with a short employment history. One solution to avoid these tragedies is to support workers employed in this industry with training applications. This paper presents a description of a research and training platform for operating portable chainsaws using virtual reality (VR) technology. The platform simulates phenomena occurring while working with a chainsaw, such as falling tree parts, the occurrence of chips and characteristic sound effects. The study used the HTC Vive Pro set and a dummy chainsaw with an electric module, thanks to which the application lets you control a virtual chainsaw using a real life device tracked by a system for mapping objects in space. For the purpose of this work, a series of 32 tests were conducted with 16 subjects and conclusions were drawn on the basis of test reports and surveys, in which the subjects expressed their opinions about the course of events in the platform application.

Hands-free Adjustment of the Microscope in Microneurosurgery

In microneurosurgery, the operating microscope plays a vital role. The classical neurosurgical operation is bimanual, that is, the microsurgical instruments are operated with both hands. Often, operations have to be carried out in narrow corridors at the depth of several centimeters. With current technology, the operator must manually adjust the field of view during surgery-which poses a disruption in the operating flow. Until now, technical adjuncts existed in the form of a mouthpiece to move the stereo camera unit or voice commands and foot pedals to control other interaction tasks like optical configuration. However, these have not been widely adopted due to usability issues. This study tests 2 novel hands-free interaction concepts based on head positioning and gaze tracking as an attempt to reduce the disruption during microneurosurgery and increase the efficiency of the user. Technical equipment included the Pentero 900 microscope (Carl Zeiss Microscopy GmbH, Jena, Germany), HTC Vive Pro (HTC, Taoyuan District (HQ), Taiwan), and an inbuilt 3D-printed target probe. Eleven neurosurgeons including 7 residents and 4 consultants participated in the study. The tasks created for this study were with the intention to mimic real microneurosurgical tasks to maintain applicative accuracy while testing the interaction concepts. The tasks involved visualization system adjustment to the specific target and touching the target. The first trial was conducted in a virtual reality setting applying the novel hands-free interaction concepts, and the second trial was conducted performing the same tasks on a 3D-printed target probe using manual field of view adjustment. The participants completed both trials with the same predetermined tasks, in order to validate the feasibility of the novel technology. The data collected for this study were obtained with the help of review protocols, detailed post-trial interviews, video and audio recordings, along with time measurements while performing the tasks. The user study conducted at the Charité Hospital in Berlin found that the gaze-tracking and head-positioning- based microscope adjustment were 18% and 29% faster, respectively, than the classical bimanual adjustment of the microscope. Focused user interviews showed the users' proclivity for the new interaction concepts, as they offered minimal disruption between the simultaneous target selection and camera position adjustment. The hands-free interaction concepts presented in this study demonstrated a more efficient execution of the microneurosurgical tasks than the classical manual microscope and were assessed to be more preferable by both residents and consultant neurosurgeons.

Dynamic Pose Tracking Performance Evaluation of HTC Vive Virtual Reality System

Virtual reality tracking devices are rapidly becoming the go-to system for cost-effective motion tracking solutions across different communities such as robotics, biomechanics, sports, rehabilitation, motion simulators, etc. This article focuses on the spatial tracking performance of HTC Vive's lighthouse tracking system (VLTS) devices (tracker, controller, and head mount display). A comprehensive literature survey on the performance analysis of VLTS on the various aspects is presented along with its shortcomings in terms of spatial tracking evaluation. The two key limitations have been identified: in static cases, there is a lack of standard procedures and criteria, and in dynamic cases, the entire study of spatial tracking. We address the first by assessing VLTS using the optical tracking system standard specified by ASTM International, and the latter by revising the standards to determine the upper-velocity limit for reliable tracking. The findings are substantiated with the trajectories of human wrist motion. Each evaluation's results are systematically analyzed with statistical hypothesis tests and criteria fulfillment. Comau NS16, an industrial serial robot, was used as the ground truth motion generator due to its repeatability and 6 degrees of workspace freedom. One of the major reasons for not having more generalized spatial tracking studies is that the tracking performance heavily depends on the configurations of the setup, work volume, environment, etc. Thus, the guidelines for configuring VLTS and the approach adapted from ASTM standards for evaluating VLTS for custom applications using our reported findings for both static and dynamic cases are included in the appendix.

BlinKey

Virtual Reality (VR) has shown promising potentials in many applications, such as e-business, healthcare, and social networking. Rich information regarding user's activities and their online accounts is stored in VR devices. If it is carelessly unattended, then attackers, including insiders, can make use of the stored information to, for example, perform in-app purchases at the legitimate owner's expenses. Current solutions, mostly following schemes designed for general personal devices, have been proved vulnerable to shoulder-surfing attacks due to the sight blocking caused by the headset. Although there have been efforts trying to fill this gap, they either rely on some highly advanced equipment, such as electrodes to read brainwaves, or introduce heavy cognitive load that has users perform a series of cumbersome authentication tasks. Therefore, an authentication method for VR devices that is robust and convenient is in dire need. In this paper, we present the design, implementation, and evaluation of a two-factor user authentication scheme, BlinKey, for VR devices that are equipped with an eye tracker. A user's secret passcode is a set of recorded rhythms when he/she blinks, together with the unique pupil size variation pattern. We call this passcode as a blinkey, which can be jointly characterized by knowledge-based and biometric features. To examine the performances, BlinKey is implemented on an HTC Vive Pro with a Pupil Labs eye tracker. Through extensive experimental evaluations with 52 participants, we show that our scheme can achieve the average EER as low as 4.0% with only 6 training samples. Besides, it is robust against various types of attacks. BlinKey also exhibits satisfactory usability in terms of login attempts, memorability, and impact of user motions. We also carry out questionnaire-based pre-/post-studies. The survey result indicates that BlinKey is well accepted as a user authentication scheme for VR devices.

Who is in the Room? Notification for Intrusions While in Virtual Reality

The development of head-mounted display virtual reality systems (e.g., Oculus Rift, HTC Vive) has resulted in an increasing need to represent the physical world while immersed in the virtual. Current research has focused on representing static objects in the physical room, but there has been little research into notifying VR users of changes in the environment. This study investigates how different sensory modalities affect noticeability and comprehension of notifications designed to alert head-mounted display users when a person enters his/her area of use. In addition, this study investigates how the use of an orientation type notification aids in perception of alerts that manifest outside a virtual reality users’ visual field. Results of a survey indicated that participants perceived the auditory modality as more effective regardless of notification type. An experiment corroborated these findings for the person notifications; however, the visual modality was in practice more effective for orientation notifications.

Commercial head-mounted display virtual reality for upper extremity rehabilitation in chronic stroke: a single-case design study

BackgroundRehabilitation is crucial for maximizing recovery after stroke. Rehabilitation activities that are fun and rewarding by themselves can be more effective than those who are not. Gamification with virtual reality (VR) exploits this principle. This single-case design study probes the potential for using commercial off-the-shelf, room-scale head-mounted virtual reality for upper extremity rehabilitation in individuals with chronic stroke, the insights of which can inform further research.MethodsA heterogeneous volunteer sample of seven participants living with stroke were recruited through advertisement. A single-case design was employed with a 5-week baseline (A), followed by a 10-week intervention (B) and a 6-month follow-up. Upper extremity motor function was assessed with validated kinematic analysis of drinking task. Activity capacity was assessed with Action Research Arm Test, Box and Block Test and ABILHAND questionnaire. Assessments were done weekly and at follow-up. Playing games on a VR-system with head-mounted display (HTC Vive) was used as rehabilitation intervention. Approximately 300 games were screened and 6 tested. Visual analysis and Tau-U statistics were used to interpret the results.ResultsVisual analysis of trend, level shift and overlap as well as Tau-U statistics indicated improvement of Action Research Arm Test in six participants. Four of these had at least a moderate Tau-U score (0.50–0.92), in at least half of the assessed outcomes. These four participants trained a total of 361 to 935 min. Two out of four participants who were able to perform the drinking task, had the highest training dose (> 900 min) and showed also improvements in kinematics. The predominant game played was Beat Saber. No serious adverse effects related to the study were observed, one participant interrupted the intervention phase due to a fall at home.ConclusionsThis first study of combining commercial games, a commercial head-mounted VR, and commercial haptic hand controls, showed promising results for upper extremity rehabilitation in individuals with chronic stroke. By being affordable yet having high production values, as well as being an easily accessible off-the-shelf product, this variant of VR technology might facilitate widespread adaption. Insights garnered in this study can facilitate the execution of future studies.Trial registration The study was registered at researchweb.org (project number 262331, registered 2019-01-30, https://www.researchweb.org/is/vgr/project/262331) prior to participant enrolment.

A Scoping Review on Virtual Reality-Based Industrial Training

The fourth industrial revolution has forced most companies to technologically evolve, applying new digital tools, so that their workers can have the necessary skills to face changing work environments. This article presents a scoping review of the literature on virtual reality-based training systems. The methodology consisted of four steps, which pose research questions, document search, paper selection, and data extraction. From a total of 350 peer-reviewed database articles, such as SpringerLink, IEEEXplore, MDPI, Scopus, and ACM, 44 were eventually chosen, mostly using the virtual reality haptic glasses and controls from Oculus Rift and HTC VIVE. It was concluded that, among the advantages of using this digital tool in the industry, is the commitment, speed, measurability, preservation of the integrity of the workers, customization, and cost reduction. Even though several research gaps were found, virtual reality is presented as a present and future alternative for the efficient training of human resources in the industrial field.

Evaluating an Intervention to Reduce Risky Driving Behaviors: Taking the Fear Out of Virtual Reality.

Educational programs are the most common type of intervention to reduce risky driving behavior. Their success, however, depends on the content of the material used and the mode of delivery. In the present study, we examined the impact of fear versus positively framed road safety films and traditional technologies (2D) versus emerging technologies (VR) on young drivers' self-reported risky driving behaviors. One hundred and forty-six university students completed a similar set of questionnaires pre-intervention and post-intervention, two weeks later. In addition, they were randomly assigned to one of the four experimental conditions (VR vs. 2D; positive vs. negative). In the VR conditions, the film was presented using an HTC VIVE Virtual Reality headset. In the 2D conditions, the film was presented on a computer screen. Measures evaluating attitudes toward risky driving behavior were completed at both time frames, questions regarding the participants' emotional arousal were asked at pre-intervention as a manipulation check, and questions regarding willingness to take risks in potentially dangerous driving situations were asked at follow-up. The findings indicate that the positively framed films significantly decreased self-reported risky driving behaviors in both modalities, but especially when viewed in VR format. In contrast, the fear appeal film, when shown in VR, failed to reduce risky driving behaviors, and in fact, increased young drivers' self-reported risky driving behaviors. Theoretical frameworks regarding the strengths and weaknesses of fear appeals and positively framed appeals are discussed to aid future research to reduce risky driving. Practical implications on the future usage of VR are also considered.

MarineVerse Enables More People to Sail!

MarineVerse wants to promote and share the sport of sailing with the world by providing sailing education,growing as a sailor, socializing, and having entertainment using virtual reality technology. Current VR sailingapplications are designed for Valve Index, HTC Vive, Oculus Rift, and Oculus Quest, and flagship products areMarineVerse Cup and VR Regatta.

Virtual Reality bridge between Chemistry and Cultural Heritage: the “Sala degli Stemmi” Case Study.

In this contribution, we present a multiscale and multidisciplinary VR architecture that aims at creating a common environment where cultural heritage and chemistry meet in order to strengthen the role already played by chemistry in the process of restoration of cultural goods. Our aim is to create a user friendly platform where experts of both fields can share data and ideas in a direct way, in order to achieve deeper insights into cultural goods combining the scientific and historical points of view. As a case study we present the 3D reconstruction of the “Sala degli Stemmi”, which is one of the two historical rooms at Palazzo della Carovana in Pisa, presenting a number of artworks that underwent a process of chemical analysis and restoration in 2012. The whole architecture has been developed using the Unity game engine, and it is usable with HTC Vive headsets. The implementation of the VR environment and the potential applications, from both the scientific and educational points of view, are discussed in some detail.

The Effect of Gender and Attractiveness of Motion on Proximity in Virtual Reality

In human interaction, people will keep different distances from each other depending on their gender. For example, males will stand further away from males and closer to females. Previous studies in virtual reality (VR), where people were interacting with virtual humans, showed a similar result. However, many other variables influence proximity, such as appearance characteristics of the virtual character (e.g., attractiveness). Our study focuses on proximity to virtual walkers, where gender could be recognised from motion only, since previous studies using point-light displays found walking motion is rich in gender cues. In our experiment, a walking wooden mannequin approached the participant embodied in a virtual avatar using the HTC Vive Pro HMD and controllers. The mannequin animation was motion captured from several male and female actors and each motion was displayed individually on the character. Participants used the controller to stop the approaching mannequin when they felt it was uncomfortably close to them. Based on previous work, we hypothesised that proximity will be affected by the gender of the character, but unlike previous research, the gender in our experiment could only be determined from character’s motion. We also expected differences in proximity according to the gender of the participant. We additionally expected some motions to be rated more attractive than others and that attractive motions would reduce the proximity measure. Our results show support for the last two assumptions, but no difference in proximity was found according to the gender of the character’s motion. Our findings have implications for the design of virtual characters in interactive virtual environments.

Virtual reality technologies for clinical education: evaluation metrics and comparative analysis

ABSTRACT With recent advances of Virtual Reality (VR) technology, the deployment of such will dramatically increase in non-entertainment environments, such as professional education and training, manufacturing, service, or low frequency/high-risk scenarios. Clinical education is an area that especially stands to benefit from VR technology due to the complexity, high cost, and difficult logistics. The effectiveness of the deployment of VR systems in healthcare environments is subject to factors that may not be necessarily considered for devices targeting the entertainment market. In this work, we systematically compare a wide range of VR head-mounted displays (HMDs) technologies and designs by defining a new set of metrics that are 1) relevant to medical VR solutions and 2) are of paramount importance for VR-based education and training. We evaluated 10 technologies based on various criteria, including neck strain, heat development, cleanability, and colour accuracy. Other metrics such as text readability, comfort, and contrast perception were evaluated in a multi-user study on three selected technologies, namely the ones offered by Oculus Rift S, HTC Vive Pro and Samsung Odyssey+. Results indicate that the HTC Vive Pro performs best with regards to comfort, display quality and compatibility with glasses.