Full-body Interaction Research Articles

KAVE

While CAVE Automatic Virtual Environments (CAVE) have been around for over 2 decades they remain complex to setup, unaffordable to most, and generally limited to data and model visualization applications for academia and industry. In this paper, we present a solution to create a monocular CAVE using the Unity 3D game engine by adding motion parallax and full-body interaction support via the use of a Kinect V2 low-cost sensor. More importantly, we provide a functional and easy to use plugin for that effect, the KAVE, and its configuration tool. Here, we describe our own low-cost CAVE setup, a range of alternative configurations to CAVE systems using this technology and example applications. Finally, we discuss the potential of such an approach considering the current advancements in VR and gaming.

The value of learning talk: applying a novel dialogue scoring method to inform interaction design in an open-ended, embodied museum exhibit

Museum researchers have long acknowledged the importance of dialogue in informal learning, particularly for open-ended exploratory exhibits. Novel interaction techniques like full-body interaction are appealing for these exploratory exhibits, but designers have not had a metric for determining how their designs are supporting productive learning talk. Moreover, with the incorporation of digital technologies into museums, researchers and designers now have the opportunity for in situ A/B testing of multiple exhibit designs not previously possible with traditionally constructed exhibits, which once installed were difficult and expensive to iterate. Here we present a method called Scoring Qualitative Informal Learning Dialogue (SQuILD) for quantifying idiosyncratic social learning talk, in order to conduct in situ testing of group learning at interactive exhibits. We demonstrate how the method was applied to a 2 × 2 experiment varying the means of control (full-body vs. handheld tablet controller) and the distribution of control (single-user-input vs. multi-user-input) of an interactive data map exhibit. Though pilot testing in the lab predicted that full-body and multi-input designs would best support learning talk, analysis of dialogue from 119 groups’ interactions revealed surprising nuances in the affordances of each. Implications for embodied interaction design are discussed.

Impact of Interaction Paradigms on Full-Body Interaction Collocated Experiences for Promoting Social Initiation and Collaboration

This article addresses the design and study of novel interaction settings and systems supporting collocated interaction. More specifically, we aim to analyze the implications of two different interaction approaches, namely first-person and third-person interaction paradigms, and the corresponding theoretical approaches when designing and developing collocated experiences for children with Autism Spectrum Disorders (ASDs). This analysis is supported by the outcomes of several examples of full-body interaction collocated experiences aimed at promoting social initiation and collaboration. Moreover, we present a series of goal-driven guidelines to consider when designing with various interaction paradigms. Finally, we discuss future work to better understand the implications of constructing paradigms for the intuitive use of these full-body interaction systems.

タンジブルインターフェースとコンピュータインタラクションを連動させた生物教育支援システム

In the education of children, learning based on the direct experiences is effective. However, a lot of things have difficulty in direct experiences. Therefore, Authors have developed the learning support system that provide experiences in real and virtual environment. Leaners can get experience like touching and seeing in real and virtual environment by using this system. At the first stage of this study, we have developed larval mimesis experience system. This system consists of Kinect sensor, larval models, load sensors which was controlled in Arduino. By using this system, learners can do full body interaction on the virtual environment, larva models observation experience in the real environment. In experiment, we have a primary schoolchild use this system to evaluate usefulness of the system. This paper summarizes the current system and describes the evaluation results.

Quality of physical activity of children in exergames: Sequential body movement analysis and its implications for interaction design

Exergames are videogames based on full-body interaction that foster physical exercise during gameplay. Much research has focused on whether exergames foster sufficient physical activity and with which intensity they do so. This is usually measured through energy expenditure and compared to traditional physical activities such as sports or physical education exercises. However, little research has been undertaken on the quality of the physical activity, understood not as how well the movement is done, but as the richness and diversity of physical activity defined by experts, such as diverse types of motor skills (jumps, turns, pivots, manipulative actions, etc) and diverse types of conditioning capabilities (endurance, stretch, general strength, coordination, etc.). Hence, exergames typically address only aerobic (cardiovascular) activity and other types tend to be ignored. In this paper we propose a method for analyzing the types of physical activity (quality) that an exertion interface elicits. We propose this method as an analytical tool which can provide key information to categorize exergames, aid to their design and help designers make sure they achieve game mechanics that elicit the desired types of physical activity. We describe the method through the analysis of two exergames that we developed in the past for our exertion interface for children, called the Interactive Slide. We present the results of a systematic observational method, until now used mainly in sports analysis, which is based on applying sequential body movement analysis to obtain the play characteristics of 48 children. These results -action events and time-based patterns- typify the movement found in the two aforementioned exergames for this platform. We have found significant movement differences between the two, as well as some differences associated to age. Moreover, we show that sequential body movement analysis, which has already been successfully used in the past to analyze sports, can also be effective in analyzing exergames and hence allow grounded discussion on health issues related to their use. We present the results of a systematic observational method to obtain the play characteristics.We give an interaction design strategy to improve the quality of physical activity with exergames.We show that sequential body movement analysis can be effective in analyzing exergames.

Sparking social initiation behaviors in children with Autism through full-body Interaction

Abstract We present the full-body interaction system Lands of Fog, a multi-user experience designed with and for children with ASD to foster social initiation and collaborative behaviors. Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder which negatively impacts social communication abilities, resulting in significant problems forming and maintaining relationships with peers. Research suggests the use of Information and Communication Technologies to offer structured activities where individuals with ASD will be motivated to practice and develop these social behaviors, activities which deployed in inclusive schools could also help for social inclusion. We have completed two studies where children with ASD played with neurotypical children, one in a laboratory controlled setting and one in an inclusive elementary school. Results indicate that the system was successful in fostering motivation, socialization and collaboration. Moreover, the system was positively perceived by children and school professionals.

An evaluation-driven design approach to develop learning environments based on full-body interaction

The development of learning environments based on full-body interaction has become an increasingly important field of research in recent years. However, the design and evaluation strategies currently used present some significant limitations. Two major shortcomings are: the inadequate involvement of children in the design process and a lack of research into what meanings children construct within these learning environments. To tackle these shortcomings we present an evaluation-driven design approach, which aims at analyzing situated interpretations made by children. These interpretations are then used to guide and optimize design in an iterative process of design and assessment. This evaluation-driven design method was applied in the development of the EcoSystem Project, a full-body interaction learning environment for children aimed at supporting learning about environmental relationships. The application of this iterative approach proved to be highly effective both in facilitating continuous improvements in the proposed design and in reducing misconceptions by children using the environment. Moreover, experimental evaluation reported significant learning gains in children. This suggests both the potential of using full-body interaction to support learning and the effectiveness of our evaluation-driven approach in optimizing design solutions through the analysis of children’s interpretations.

Towards methods for evaluating and communicating participatory design: A multimodal approach

Participatory Design (PD) has been proposed as a useful strategy to address pitfalls in the design of serious games for children with special needs. Nonetheless, methodological weaknesses in the analysis of the results of PD workshops may hinder its effectiveness in providing useful and robust design contributions and facilitating communication in multidisciplinary teams. To address this issue we propose the use of multimodal analysis to evaluate participants' contributions during PD workshops. We present an analysis that was applied in an informant workshop with autistic children, aimed at refining the design of a serious game based on Full-Body Interaction. Results show that multimodal analysis constitutes an effective and coherent method to capture and analyze users' contributions across a wide range of semiotic resources, thus extending the richness of insights that can be derived from a PD workshop and communicated to the rest of the team. Furthermore, the analysis allowed the identification of fundamental design questions, thus offering a robust empirical ground for supporting dialogue and reflection between multiple stakeholders. Use of Multimodal Analysis to analyze users' contributions in Participatory Design.Case study with 4 children in Autistic Spectrum Disorder.Multimodal analysis effectively grasped contributions across different resources.Usefulness for guiding design of Serious Games in interdisciplinary teams.

フルボディインタラクションを用いた子供の協調性向上のための協同ゲーム

フルボディインタラクションを用いた子供の協調性向上のための協同ゲーム

A Framework for User-Defined Body Gestures to Control a Humanoid Robot

This paper presents a framework that allows users to interact with and navigate a humanoid robot using body gestures. The first part of the paper describes a study to define intuitive gestures for eleven navigational commands based on analyzing 385 gestures performed by 35 participants. From the study results, we present a taxonomy of the user-defined gesture sets, agreement scores for the gesture sets, and time performances of the gesture motions. The second part of the paper presents a full body interaction system for recognizing the user-defined gestures. We evaluate the system by recruiting 22 participants to test for the accuracy of the proposed system. The results show that most of the defined gestures can be successfully recognized with a precision between 86\(-\)100 % and an accuracy between 73\(-\)96 %. We discuss the limitations of the system and present future work improvements.

사용자 경험 증진을 위한 과학관 전시물의 사용성 평가

Many museums have recently employed digital technologies in exhibition installations to provide visitors with interactive experiences with the installations, not just audiences. However, most of them have focused on the adoption of new prototypes or technologies, not considering user experiences of those systems carefully. This study developed an evaluation tool for usability of the tangible user interfaces and conducted a usability study on museum installations emphasizing user interaction and experience. The evaluation tool is composed of 5 features of tangible user interfaces such as tangible, interaction, convenience, representation, spatial interaction and social interaction, and 24 items. The museum we investigated is the Gwacheon National Science Museum, where 8 installations, classified 4 categories, were selected for the usability study. We recruited 6 undergraduate students, who were divided into 2 teams, each team having 3 students. Three students in a team manipulated and experience each installation together and reported their evaluation score through the questionnaire and interviews. The results showed that the score of the usability for the category 3, which requires students to move their bodies for the interaction, is the highest one because it features with spatial interaction. Students expressed much interest in the category 4, which utilizes users' other senses, however, the score of the usability is the lowest because the interaction is temporary and repetitive. Most installations are well designed in terms of control constraints, legibility, lower thresholds, participation encouragement, and open to the public, but pooly designed in terms of multiple access points, configurability, accurate movement, ambient media, and full-body interaction.

Three‐Dimensional, Virtual Reality Vestibular Rehabilitation for Ménière's Disease‐Induced Chronic Imbalance Problem

Objectives: Patients with Ménière’s disease (MD) may suffer from chronic imbalance problem that cannot be cured by surgery or medication. For these patients, vestibular rehabilitation is another treatment option. Three-dimensional (3D), virtual reality (VR) technology may be applied to facilitate rehabilitation exercise. In this study we report outcomes of 3D, VR rehabilitation for patients with MD-related chronic vestibular dysfunction. Methods: A VR-based rehabilitation system was developed. The system consists of the Kinect Xbox for full-body interaction, 3D-ready projector with shutter-glasses for stereo imaging, Wii Fit pressure pad (for tracing the center of gravity sway and statokinesigrams [SKG], in cm2), and Unity game engine for leveled-difficulty design. The modified Cawthorne-Cooksey exercises (MCCE) was adapted into this system; the MCCE scores were compared following intervention. 50 MD patients with chronic balance were recruited for this prospective, longitudinal, uncontrolled clinical study. Patients followed the instructions of MCCE to complete at least 6 sessions of vestibular rehabilitation within a 4-week period. Results: All patients experienced improvement of balance function performing daily activities and were overall positive about the training protocol. After intervention, the fast components of postural sway and SKG ( P < 0.05) were reduced. The MCCE scores significantly improved (66.6 to 74.6, P < 0.05). We found age (Î2 = -0.52, P < 0.05) and number of training sessions (Î2 = 2.53X, P < 0.05) were significant predictors for clinical improvement. Conclusions: Vestibular rehabilitation can assist MD patients who cannot benefit from medication and surgery. The data support the applicability of 3D, VR technology in the field of vestibular rehabilitation.

Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit

We present the Flexible Action and Articulated Skeleton Toolkit (FAAST), a middleware software framework for integrating full-body interaction with virtual environments, video games, and other user interfaces. This toolkit provides a complete end-to-end solution that includes a graphical user interface for custom gesture creation, sensor configuration, skeletal tracking, action recognition, and a variety of output mechanisms to control third party applications, allowing virtually any PC application to be repurposed for gestural control even if it does not explicit support input from motion sensors. To facilitate intuitive and transparent gesture design, we define a syntax for representing human gestures using rule sets that correspond to the basic spatial and temporal components of an action. These individual rules form primitives that, although conceptually simple on their own, can be combined both simultaneously and in sequence to form sophisticated gestural interactions. In addition to presenting the system architecture and our approach for representing and designing gestural interactions, we also describe two case studies that evaluated the use of FAAST for controlling first-person video games and improving the accessibility of computing interfaces for individuals with motor impairments. Thus, this work represents an important step toward making gestural interaction more accessible for practitioners, researchers, and hobbyists alike.

Using Health Information Technology to Prevent and Treat Diabetes

Using Health Information Technology to Prevent and Treat Diabetes

We hunters: interactive communication for young cavemen

This paper presents a multi-user virtual heritage application that allows children to experience and learn the hunting strategies depicted in a cave painting from 6000 BC. The application is based on a multi-touch vertical screen that allows for full-body interaction through carefully defined gestures that search to provide sufficient ecological naturalness and experimentation freedom to the users. This paper gives a detailed account on the experience and interaction design, agent programming and multi-touch technology. It also presents an experimental set-up, especially designed to assess the impact of the installation in the educational goal, the usability of the experience and the results obtained.

Virtual Reality Rehabilitation for Vestibular Dysfunction

Objective: People with chronic vestibular dysfunction can be treated with balance rehabilitation. Virtual reality (VR) technology may serve as a useful therapeutic tool to facilitate rehabilitation. The aim of this pilot study was to investigate the clinical applicability of VR technology in treating patients with vestibular dysfunction. Method: Five patients with chronic vestibular dysfunction were recruited for this uncontrolled clinical study. The rehabilitation protocol (modified Cawthorne-Cooksey exercises, MCCE) was adapted into a VR-based training system. Patients followed the instructions of MCCE to complete 20 sessions of vestibular rehabilitation within a 4-week period. Results: A VR-based rehabilitation system was developed with various latest technologies, including Kinect Xbox for full-body interaction, 3D-ready projector with shutter-glasses for stereo imaging, Wii Fit pressure pad for tracing the center of gravity, and Unity game engine for leveled-difficulty design. Five VR tasks were built, and more than 100 training trials were designed. Subjectively, all patients experienced the improvement of balance function performing daily activities and were overall positive to the training protocol. After intervention the fast component of postural sway ( P < .05) was reduced. The follow-up interview showed decreased disability and increased general health. Conclusion: VR has many advantages over traditional rehabilitation techniques in improving the testing or training environment of human performance. This study demonstrates that VR technology offers great promise in the field of vestibular rehabilitation.

Virtual Reality and Interactive Digital Game Technology: New Tools to Address Obesity and Diabetes

The convergence of the exponential advances in virtual reality (VR)-enabling technologies with a growing body of clinical research and experience has fueled the evolution of the discipline of clinical VR. This article begins with a brief overview of methods for producing and delivering VR environments that can be accessed by users for a range of clinical health conditions. Interactive digital games and new forms of natural movement-based interface devices are also discussed in the context of the emerging area of exergaming, along with some of the early results from studies of energy expenditure during the use of these systems. While these results suggest that playing currently available active exergames uses significantly more energy than sedentary activities and is equivalent to a brisk walk, these activities do not reach the level of intensity that would match playing the actual sport, nor do they deliver the recommended daily amount of exercise for children. However, these results provide some support for the use of digital exergames using the current state of technology as a complement to, rather than a replacement, for regular exercise. This may change in the future as new advances in novel full-body interaction systems for providing vigorous interaction with digital games are expected to drive the creation of engaging, low-cost interactive game-based applications designed to increase exercise participation in persons at risk for obesity.

Immersive singularity-free full-body interactions with reduced marker set

Despite the large success of games grounded on movement-based interactions the current state of full-body motion capture technologies still prevents the exploitation of precise interactions with complex environments. The first key requirement in the line of work we present here is to ensure a precise spatial correspondence between the user and the Avatar. For that purpose, we build upon our past effort in human postural control with a prioritized inverse kinematics (PIK) framework. One of its key advantages is to ease the dynamic- and priority-based combination of multiple conflicting constraints such as ensuring the balance and reaching a goal. However, its reliance on a linearized approximation of the problem makes it vulnerable to the well-known full extension singularity of the limbs. We address this issue by presenting a new type of 1D analytic constraint that smoothly integrates within the PIK framework under the name of FLEXT constraint (for FLexion-EXTension constraint). We further ease the full-body interaction by combining this new constraint with a recently introduced motion constraint to exploit the data-based synergy of full-body reach motions. The combination of both techniques allows immersive full-body interactions with a small set of active optical marker. Copyright © 2010 John Wiley & Sons, Ltd.

Design and evaluation of user’s physical experience in an Ambient Interactive Storybook and full body interaction games

The paper presents the design and evaluation of an original Ambient Interactive Storybook (AIS) for children, including its platform, the background story, and 10 full body interactive games. The evaluation, which focused on the user's physical experience and elements important to the designer, has been methodologically derived from the Krofli?'s and Laban's framework Body, Space, Time and Relationship, and additionally supported by sport science measurements. An experiment with 8 participants playing 10 games for 20 min was conducted and recorded to a digital video. Participants' physical experience was evaluated through the analysis of postures, the quality of the movement, the body parts used in the interaction, the playing area, the direction of movement, direction of gaze, tempo, dynamics and the quantity of motion (QoM). Results of the experiment are discussed in relation and with implications for game design. Conclusions are drawn with the summary of main findings, to better understand the mechanisms and principles involved the design of user's physical activity in full body interactive games for children. The theoretical work of Laban and Krofli? proved to be useful for interaction and games design in the transition from desktop to full body interactive games.

Social cooperation and competition in the mixed reality space eXperience Induction Machine XIM

Although the architecture of mixed reality spaces is becoming increasingly more complex, our understanding of human behavior in such spaces is still limited. Despite the sophisticated methods deployed in ethology and behavioral biology to track and analyze the actions and movements of animals, we rarely find studies that focus on the understanding of human behavior using such instruments. Here, we address this issue by analyzing social behavior and physical actions of multiple humans who are engaging in a game. As a paradigm of social interaction, we constructed a mixed reality football game in which two teams of two players have to cooperate and compete in order to win. This paradigm was deployed in the, so-called, eXperience Induction Machine (XIM), a human accessible, fully instrumented space that supports full body interaction in mixed reality without the need for body-mounted sensors. Our results show that winning and losing strategies can be discerned by specific behavioral patterns and proxemics. This demonstrates that mixed reality systems such as XIM provide new paradigms for the investigation of human social behavior.