Virtual Human Research Research Articles

Sign language recognition based on depth image processing

Deaf and mute people mainly rely on sign language to communicate, but they still face many problems in their daily life. The interaction between humans and computers has developed extensively in recent years, the flexible and easy-to-understand characteristics of sign language make it suitable for application in the technology of computer vision. This paper mainly introduces sign language recognition from the current research status, application of deep image processing technology, collection of sign language datasets and the final results. After analysis, this paper finally established a recognition model with the wrist joint as the center point to complete the extraction of hand features. This paper input a large number of pictures to train the model, and the final recognition rate reached 91.3%. This paper can widely apply this recognition technology in computer-aided teaching, TV program bilingual broadcasting, virtual human research, medical research, game entertainment and many other aspects.

Virtual discrete trial training for teacher trainees

It is crucial that teaching methods are conducted accurately when teaching children with autism spectrum disorder. This can be addressed by providing new teachers with ample opportunity to practice their teaching skills while receiving feedback from a supervisor. Unfortunately, due to lack of resources, this is not always possible. Advances in Virtual Reality and Virtual Human research are making technical solutions possible, where teachers can receive situational training in simulated environments with virtual students, virtual props and automated feedback. The aim of the present work was to apply this approach to the training of special education teachers that needed to master a teaching method called Discrete Trial Training (DTT), which is particularly well suited for teaching children with autism. The first phase of the project focused on supporting the method itself with constructive feedback and basic interaction with a virtual child. A study, based on in-depth single-subject design, with a group of real teacher trainees, indicates that the teachers were able to demonstrate basic DTT skills after experiencing teaching trials in a VR environment, indicating that this approach is viable.

Exploring the influence of a human-like dancing virtual character on the evocation of human emotion

ABSTRACTDance has universally been used as a form of human expression for thousands of years. This common human behaviour and communication method has not been explored much in the context of computer-based technology, even within the field of virtual human research. This paper presents an experimental study investigating the impact of watching dancing virtual characters on human emotions. The study analysed the responses of 55 participants, composed of a mix of dancers and non-dancers, who watched a dancing virtual character perform 3 different dances that represented anger, sadness and happiness in different display orders. The participants’ reported changes in their emotions and their feelings of anger, sadness and happiness were significantly dependent on which dancing character’s emotion they watched and the emotional change did not rely on correct recognition of the depicted emotion. For experimental control, our characters were faceless and danced without music. Our results suggest that just by watching a dancing virtual character some of the benefits associated with dancing could be accessed in circumstances where it is not desirable or feasible to dance, justifying further research to develop a personalised character with a face and music that adapts according to the humans’ emotions and preferences.

Guest Editors' Introduction: Special Section on Virtual Reality

IN this special section, we are pleased to present extended versions of four outstanding papers that were originally presented at the IEEE Virtual Reality 2006 Conference (VR 2006). IEEE Virtual Reality is the premier international conference on all aspects of virtual, augmented, and mixed reality. The conference program at VR 2006 consisted of nine sessions on the following topics: perception, simulation and visualization, applications of VR, distributed and collaborative systems, evaluation and user studies, augmented reality, tracking and projection displays, 3D interaction, and haptic and olfactory displays. For this special section, the international program committee selected four excellent papers from the 28 accepted research papers. As always, the choice was difficult since many of the other papers were also excellent candidates. The first paper, by Sean D. Young, Bernard D. Adelstein, and Stephen R. Ellis, received the best paper award at the VR 2006 for its high relevance to the field of virtual reality and simulation. The authors asked the question, “Does taking a motion sickness questionnaire make you motion sick?” Surprisingly, their research indicates that the answer is “yes!” The paper demonstrates that the administration of the questionnaire itself makes the participant aware that the virtual environment may produce motion sickness. The study shows that reports of motion sickness after immersion are much greater when both pre and posttest questionnaires are given than when only a posttest questionnaire is used. Since pretest questionnaires cannot simply be dropped in most cases, the authors suggest a number of ways to reduce this effect and discuss the implications of their observations. Augmented reality (AR) systems, which combine realworld and virtual imagery, present a unique set of perceptual issues for the user. The paper by J. Edward Swan II, Adam Jones, Eric Kolstad, Mark A. Livingston, and Harvey S. Smallman addresses such a problem: the accuracy of depth judgments made by users of optical see-through AR displays. These displays allow users to view the physical world directly, while overlaying virtual objects on the real scene. In many applications, it is critical that the user perceives the virtual objects to be in the correct position relative to the real world, but differences in depth perception between the virtual and real imagery may prevent this. Moreover, measuring the accuracy of users’ depth judgments is not trivial. The authors review previous work and methods used to address this problem, and then present two experiments of their own. The experiments use a perceptual matching technique and a blind walking technique to measure depth judgments, and reveal some interesting and surprising results. An emerging area of research in the VR community focuses on virtual humans. In the past, virtual human research has mainly addressed technical issues—making the virtual characters realistic in appearance, movements, emotions, behaviors, etc. With many of these problems at least partially solved, however, researchers can now begin to evaluate the social aspects of virtual humans; that is, how real users interact with virtual characters. Andrew B. Raij, Kyle Johnsen, Robert F. Dickerson, Benjamin C. Lok, Marc S. Cohen, Margaret Duerson, Rebecca Rainer Pauly, Amy O. Stevens, Peggy Wagner, and D. Scott Lint present a paper along these lines, describing two studies in which medical students interacted with a simulated patient. The simulated patient was either a real person acting the part of a patient, or a virtual human playing this role. The studies show that while the interpersonal interactions with the virtual human were similar to interactions with the real human in many ways, there were also subtle differences in the participants’ nonverbal behavior and attitude toward the virtual human. Such studies are critical for improving our understanding of how to use virtual characters in real-world VR applications. Believable haptic interaction with complex virtual objects is still a challenging research topic. Michael Ortega, Stephane Redon, and Sabine Coquillart have generalized the god-object method to enable high quality haptic interaction with rigid bodies consisting of tens of thousands of triangles. They suggest separating the computation of the motion of the six-degree-of-freedom god-object from the computation of the force applied to the user. The constraintbased force felt by the user can be computed within a few microseconds, which is necessary for the tactile simulation of fine surface details. The force is computed using a novel constraint-based quasistatic approach, which allows the suppression of force artifacts typically found in previous methods. The update of the pose of the rigid god-object is performed within a few milliseconds, which allows visual display at appropriate frame rates. 420 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 13, NO. 3, MAY/JUNE 2007

Creating interactive virtual humans: some assembly required

Discusses some of the key issues that must be addressed in creating virtual humans, or androids. As a first step, we overview the issues and available tools in three key areas of virtual human research: face-to-face conversation, emotions and personality, and human figure animation. Assembling a virtual human is still a daunting task, but the building blocks are getting bigger and better every day.