Movement-based Interaction Research Articles

Adaptive shift of active electroreception in weakly electric fish for troglobitic life

The adaptive-shift hypothesis for the evolution of cave-dwelling species posits that ancestor species in surface habitats had exaptations for subterranean life that were exploited when individuals invaded caves. Weakly electric Gymnotiform fishes, nocturnal South American teleost fishes, have features that appear to be likely exaptations for troglobitic life. These fishes have active electrosensory systems in which fish generate weak electric fields that are detected by specialized electroreceptors. Gymnotiform fishes use their electric fields for navigation, prey capture (scene analysis), and social communication. Although active electrosensory systems appear to be exaptations for troglobitic life, as fish use these systems to “see in the dark”, producing electric fields is energetically costly. Cave habitats, which often are low in resources, may not be able to support such high energetic demands. Eigenmannia vicentespelaea, a species of weakly electric fish that is endemic to the São Vicente II cave in central Brazil, surprisingly generates stronger electric fields than their surface relatives. The increase in strength of electric fields may result simply from differences in size between cave and surface populations, but may also be due to lack of predation pressure in the cave or increases in “sensory volumes” and acuity that improve prey localization and capture. Eigenmannia vicentespelaea exhibits the classical phenotypes of any troglobitic fish: these fish have small to nonexistent eyes and loss of pigmentation. The closest living surface relative, Eigenmannia trilineata, inhabits streams nearby and has eyes and pigmentation. The electrosensory and locomotor behavior of both species of fish were measured in their natural habitats using a grid recording system. Surface Eigenmannia exhibited dramatic circadian changes in social behavior, such as hiding under rocks during the day and foraging in groups at night, while cave Eigenmannia displayed territorial behavior with no apparent circadian modulations. The territorial behavior involved electrical and movement-based interactions that may be a form of boundary patrolling. Electrosocial behavior and scene analysis are mechanistically interlinked because both stem from active sensing tactics.

Investigating the effect of synchronized movement on toddlers' word learning.

The effect of interpersonal behavioral synchrony on children's behavior is an emerging field rich with research potential. While studies demonstrate its effect on affiliative and prosocial outcomes, the role of synchronized movement on children's specific learning outcomes has not yet been investigated experimentally. One possibility is that synchrony, as a coordinated social activity, encourages perceived social bonds, leading to heightened attention, and better information retention. Equally likely is that physiological, rather than social learning, mechanisms mediate the effect, given the previously demonstrated role of autonomic arousal in attentional fluctuations, cognitive engagement, problem solving, exploration, and curiosity. The present study investigated the behavioral and physiological effects of synchrony conceptualized as induced, interpersonal, behavioral, movement-based interaction, on word learning in 2.5-year-old children. In a laboratory experiment, toddlers engaged in either a synchronous or an asynchronous movement-based interaction with an adult experimenter while listening to an upbeat children's song. After the (a)synchronous movement episode, the same experimenter engaged children in a word learning task. During the (a)synchrony and learning phases, children's physiological arousal was continuously recorded, resulting in heart rate and skin conductance response measures. Following a caregiver-child free play break, children were tested on their novel word retention. The results indicated that children learned novel labels at equal rates during the learning phase in both conditions, and their retention at test did not differ between conditions: although above chance retention of novel labels was found only following the synchronous, but not the asynchronous episode, the cross-episode comparisons did not reach statistical significance. Physiological arousal indices following the (a)synchrony episode did not differ between conditions and did not predict better word learning, although skin conductance response was higher during the learning than the movement episode. This study contributes to our understanding of the underlying cognitive and physiological mechanisms of interpersonal behavioral synchrony in the knowledge acquisition domain and paves the way to future investigations.

Designing postures for rehabilitation therapies in a multimodal system based on a 3D virtual environment and movement-based interaction

Technological advances have facilitated new approaches to support different needs in healthcare environments. In particular, in the field of rehabilitation therapies, we can find software applications that have been developed to support the performance of specific exercises, often with different customization options. However, there are still many gaps that need to be addressed to provide more and better global solutions. In this article, we present a novel system aimed at physiotherapists, which allows them to create new rehabilitation exercises designed to meet the specific needs of their patients. This implies the creation of individualized therapies that contribute to a better and faster recovery of patients. The system consists of a virtual 3D environment with a 3D skeleton representing the patient. The physiotherapist can interact with the skeleton to design the desired postures that the patient should practice according to their personal limitations. The system allows physiotherapists to compose a complete and personalized set of exercises. Alternatively, the physiotherapist may also create the postures by means of a motion sensing device using motion- and voice-based interaction. Both options pose research challenges that the authors have addressed to provide the solution presented in this paper.

Designing felt experiences with movement-based, wearable musical instruments: From inclusive practices toward participatory design.

Inclusive musical instruments benefit from incorporating wearable interfaces into digital musical instrument design, creating opportunities for bodily felt experiences and movement-based interactions. In this article, we discuss the evolution of our inclusive design approach behind the design and performance practices of three wearable musical instruments. By focusing on the embodied, somatic, and tacit dimensions of movement-based musical interaction, we evaluate these case studies, combining the third and first-person perspectives. The design and implementation of the wearable sensing, utilizing the additive manufacturing techniques, are discussed for each instrument and its performer in specific cases of musical expression. This article further discusses how our approach integrates music performance as a crucial step into design and evaluation, utilizing these performance practices and such collaborative settings for improved diversity and inclusion. Finally, we examine how our design approach evolves from user-centered design to more participatory practices, offering people with diverse abilities a shared music performance space.

Viewpoints AI: Demonstration

This article describes a technical approach to movement-based interactions between a human interactor and an intelligent agent based on the theatrical Viewpoints movement framework. The Viewpoints AI system features procedural gesture interpretation using shallow semantics and deep aesthetics analysis from the Viewpoints framework. The installation creates a liminal virtual / real space for the human and AI to interact by the use of digital projection for the AI visualization and shadow play to represent the human. Observations from a recent public demonstration of the system and future directions of work are also discussed.

Viewpoints AI

This article describes a technical approach to movement-based interactions between a human interactor and an intelligent agent based on the theatrical Viewpoints movement framework. The Viewpoints AI system features procedural gesture interpretation using shallow semantics and deep aesthetics analysis from the Viewpoints framework. The installation creates a liminal virtual / real space for the human and AI to interact by the use of digital projection for the AI visualization and shadow play to represent the human. Observations from a recent public demonstration of the system and future directions of work are also discussed.

Movement-Based Interactive Textiles Design for Emotion Regulation

OVERVIEW Textiles have always been an important carrier for people to express their emotions. Wearable technology brings the power of computing closer to body, which has led to the rising focus of the design of bodily emotion-related interactive textiles. Body engaged affective design for emotion regulation has become a trend in the field of games and interactive art, but the advantages of interactive textiles in this field have not been systematically studied and fully developed. This research aims to explore ways to enhance the potential of smart textiles in assisting emotion regulation of users with movement-based interaction. The research follows the Research through Design approach. Preliminary results show that several different emotion regulation strategies can be applied to the design of movement-based interactive textiles. This research offers a new perspective on the potential of emotion regulation with interactive textiles and will contribute to a deeper understanding of movement engaged affective design.

On the Use of Movement-Based Interaction with Smart Textiles for Emotion Regulation.

Research from psychology has suggested that body movement may directly activate emotional experiences. Movement-based emotion regulation is the most readily available but often underutilized strategy for emotion regulation. This research aims to investigate the emotional effects of movement-based interaction and its sensory feedback mechanisms. To this end, we developed a smart clothing prototype, E-motionWear, which reacts to four movements (elbow flexion/extension, shoulder flexion/extension, open and closed arms, neck flexion/extension), fabric-based detection sensors, and three-movement feedback mechanisms (audio, visual and vibrotactile). An experiment was conducted using a combined qualitative and quantitative approach to collect participants’ objective and subjective emotional feelings. Results indicate that there was no interaction effect between movement and feedback mechanism on the final emotional results. Participants preferred vibrotactile and audio feedback rather than visual feedback when performing these four kinds of upper body movements. Shoulder flexion/extension and open-closed arm movements were more effective for improving positive emotion than elbow flexion/extension movements. Participants thought that the E-motionWear prototype were comfortable to wear and brought them new emotional experiences. From these results, a set of guidelines were derived that can help frame the design and use of smart clothing to support users’ emotional regulation.

Recognizing Running Movement Changes with Quaternions on a Sports Watch

Sports watches are popular amongst runners but limited in terms of sensor locations (i.e., one location at the wrist). Thus, apps on the watch cannot directly sense movement changes in arbitrary body locations. This, in turn, severely limits training support services, contextual awareness and seamless interaction mechanisms. Our approach addresses this gap and connects the watch with two strategically placed inertial measurement units (IMUs) via bluetooth low energy (BLE). Our prototypical app for Wear OS receives orientation (quaternion) data and matches the sensors to the arm or leg segments using a flexible and simple procedure. We collected data from eight runners and used support vector machines (SVMs) to recognize different movements. Findings from our evaluation with different parameters indicate feasible recognition and low false-positive rates for two to three different movements, for both placements. Our approach can thus help to improve applications that support training and thus contributes to developing motion capture for personal use; it also enables movement-based interaction while running.

Target selection by gaze pointing and manual confirmation: performance improved by locking the gaze cursor

Eye movement-based human-computer interactions are emerging in diverse scenarios. When selecting targets on a user interface, the method of combining fast gaze pointing with reliable manual action is becoming increasingly popular. However, this method suffers from noise in gaze pointing caused by eye jitters and users’ habitual early move-away of gaze before manual actions. Here we propose a novel solution to mitigate these problems by locking the gaze cursor at the target for imminent manual selection. We compared this gaze-lock cursor with a conventional gaze cursor in a typing task with varying key sizes and key gaps. Results show that typing performance was significantly better with larger key size and gap. More importantly, the gaze-lock cursor significantly increased speed and decreased errors when compared to a conventional gaze cursor. Our findings demonstrate that the gaze-lock cursor is a promising tool for gaze interactions involving frequent target selections. Practitioner summary: Target selection by gaze pointing and manual confirmation suffers from eye jitters and users’ habitual early move-away of gaze before manual actions. The performance of this method can be improved by applying the gaze-lock cursor we proposed, increasing target size or increasing the target gap. Abbreviations: WTC: warping to target center; ALCM: automatic lock of cursor movement; LCD: liquid crystal display; EWMA: exponential weighted moving average; ER: error rate; ET: execution time; ED: edit distance; CV: coefficients of variation; ANOVA: analysis of variance; GUIs: general user interfaces

Walkable Mixed Reality Map as interaction interface for Virtual Heritage

Studies in the Virtual Heritage (VH) domain have led to underlining the significance of a contextual relationship between users, immersive reality technologies and interactive and engaging cultural context as essential aspects towards enabling cultural learning in VH applications. Interaction methods, therefore, play a great role in terms of enabling interactive and engaging experience under various VH settings. This paper proposes a novel approach to use maps as interaction interfaces in a Mixed Reality (MxR) environment that could be applied to specific VH settings with a predefined cultural and historical context. The primary focus of the proposed interaction method named ‘Walkable MxR Map’ is to use interactive, immersive and walkable maps to allow users interact with cultural content, 3D models, and different multimedia content at museums and heritage sites. Hence, the applicability of the proposed systems will not be limited to museums' indoor settings; its applicability extends outdoors at the natural location of cultural heritage assets. To this end, immersive reality technologies, interaction methods, development platforms and mapping and cloud storage services have been combined to realise the interaction method. The Walkable MxR Map allows users to interact with virtual objects via maps that are virtually projected on the floor and viewable through MxR devices, specifically the Microsoft HoloLens. The projected maps are room-scale and walkable with a potential global scalability. Besides movement-based interaction, users can interact with virtual objects, multimedia content and 3D models using HoloLens's standard gesture, gaze and voice interaction methods.

Investigating Immersion and Learning in a Low-Embodied versus High-Embodied Digital Educational Game: Lessons Learned from an Implementation in an Authentic School Classroom

Immersion is often argued to be one of the main driving forces behind children’s learning in digital educational games. Researchers have supported that movement-based interaction afforded by emerging embodied digital educational games may heighten even more immersion and learning. However, there is lack of empirical research warranting these claims. This case study has investigated the impact of high-embodied digital educational game, integrated in a primary school classroom, on children’s immersion and content knowledge about nutrition (condition1 = 24 children), in comparison to the impact of a low-embodied version of the game (condition2 = 20 children). Post-interventional surveys investigating immersion indicated that there was difference only on the level of engagement, in terms of perceived usability, while children’s learning gains in terms of content knowledge did not differ among the two conditions. Interviews with a subset of the children (n = 8 per condition) resulted in the identification of (a) media form, (b) media content and (c) context-related factors, which provided plausible explanations about children’s experienced immersion. Implications are discussed for supporting immersion in high-embodied educational digital games.

Documenting the Elusive and Ephemeral in Embodied Design Ideation Activities

Documenting embodied ideation activities is challenging, as they often result in ephemeral design constructs and elusive design knowledge difficult to document and represent. Here, we explore documentation forms designers can use internally during the design process in the domain of movement-based interaction in collocated, social settings. Using previous work and our experience from embodied ideation workshops, we propose three documentation forms with complementing perspectives of embodied action from a first and a third person view. We discuss how they capture ephemeral embodied action and elusive design and experiential knowledge, in relation to two interdependent aspects of documentation forms: their performativity and the medium they use. The novelty of these forms lies in what is being captured: ephemeral design constructs that emerge as designers engage with the embodied ideation activity; how it is portrayed: in aggregation forms that highlight elusive design knowledge; and their purpose: to clarify and augment analytical results improving the designer-researchers’ understanding of key aspects of the embodied ideation process and its outcomes, useful to advance the design process and for research dissemination.

The use of joint coordinates to monitor patients in a movement-based interaction system

Movement-based interaction has brought about new possibilities in healthcare environments. Concretely, rehabilitation therapies represent an adequate domain in which this kind of systems can produce significant benefits. In this context, the authors have developed SIVIRE, a movement-based interaction system whose main contributions are a virtual online editor of exercises with the capacity to adapt the rehabilitation to the patients’ conditions and evolution. During the development process, we have considered some aspects regarding the monitoring process to improve the quality and accuracy of the system, such as monitoring the patient’s postures, which is a critical procedure in order to properly correct and guide them. Specifically, this paper presents a descriptive study of three methods that address the monitoring process in a different way: the first using joint coordinates, the second based on bones coordinates and the last using the angles generated by the bones. Following the analysis of these three different mechanisms, the authors discuss the reason to identify the use of joint coordinates as the best technique to monitor patients in this kind of movement-based interaction systems. Such technique has been applied in the development of SIVIRE. Finally, we present the results of the evaluation of the system implemented with this monitoring mechanism. The outcomes prove that SIVIRE offers a reliable and useful tool to support rehabilitation therapies, both for patients and physiotherapists.

Bodystorming for movement-based interaction design

After a decade of movement-based interaction in human–computer interaction, designing for the moving body still remains a challenge. Research in this field requires methods to help access, articula ...

Interactive Visuals as Metaphors for Dance Movement Qualities

The notion of “movement qualities” is central in contemporary dance; it describes the manner in which a movement is executed. Movement qualities convey information revealing movement expressiveness; their use has strong potential for movement-based interaction with applications in arts, entertainment, education, or rehabilitation. The purpose of our research is to design and evaluate interactive reflexive visuals for movement qualities. The theoretical basis for this research is drawn from a collaboration with the members of the international dance company Emio Greco|PC to study their formalization of movement qualities. We designed a pedagogical interactive installation called Double Skin/Double Mind (DS/DM) for the analysis and visualization of movement qualities through physical model-based interactive renderings. In this article, we first evaluate dancers’ perception of the visuals as metaphors for movement qualities. This evaluation shows that, depending on the physical model parameterization, the visuals are capable of generating dynamic behaviors that the dancers associate with DS/DM movement qualities. Moreover, we evaluate dance students’ and professionals’ experience of the interactive visuals in the context of a dance pedagogical workshop and a professional dance training. The results of these evaluations show that the dancers consider the interactive visuals to be a reflexive system that encourages them to perform, improves their experience, and contributes to a better understanding of movement qualities. Our findings support research on interactive systems for real-time analysis and visualization of movement qualities, which open new perspectives in movement-based interaction design.

TOWARDS MOVEMENT-BASED INTERACTION FOR TAWAF TRAINING INVESTIGATING ITS SUITABILITY FOR OLDER ADULTS

Training of Tawaf (a rite in Hajj- the Muslim pilgrimage) has been limited to traditional method of text-reading and talk-listening. An edutainment approach of virtual environment simulation training could offer a more immersive experience through realistic Tawaf settings. Constructing a more natural interaction style for the virtual training, besides enhancing its realism value, could be the missing link in providing the engagement factor that appeal to older users. This paper describe a preliminary study in finding out a first indication on whether a natural interaction style through body movements could also be suitable for older users. Encouraging results showed that there is little difference in gesture performance of Istilam (a Tawaf gesture) between group of adult participants and group of older (age above 60) participants, suggesting the applicability of movement-based Tawaf training interaction to older adults as well.

A classification of user experience frameworks for movement-based interaction design

Recent technological developments have made it possible for digital technology to offer interaction modes that can engage the whole body. These systems can track the users’ movements to support rehabilitation, to play games or to create music, among other activities. One important implication of interacting with the whole body is the increased relevance of the user’s experience. A number of approaches and frameworks have been proposed to design these movement-based systems and some of them pay special attention to the user’s experience. This paper maps the space of design frameworks for movement-based interaction that focus on user experience. It classifies existing frameworks by their type and by characteristics that could help practitioners intending to design applications of that sort to make informed decisions about what framework to employ. Additionally, and to illustrate the use of the classification, we include a case study related with selecting and deploying some of these frameworks for a design exercise.

Movement-based interaction applied to physical rehabilitation therapies.

BackgroundHealth care environments are continuously improving conditions, especially regarding the use of current technology. In the field of rehabilitation, the use of video games and related technology has helped to develop new rehabilitation procedures. Patients are able to work on their disabilities through new processes that are more motivating and entertaining. However, these patients are required to leave their home environment to complete their rehabilitation programs.ObjectiveThe focus of our research interests is on finding a solution to eliminate the need for patients to interrupt their daily routines to attend rehabilitation therapy. We have developed an innovative system that allows patients with a balance disorder to perform a specific rehabilitation exercise at home. Additionally, the system features an assistive tool to complement the work of physiotherapists. Medical staff are thus provided with a system that avoids the need for them to be present during the exercise in specific cases in which patients are under suitable supervision.MethodsA movement-based interaction device was used to achieve a reliable system for monitoring rehabilitation exercises performed at home. The system accurately utilizes parameters previously defined by the specialist for correct performance of the exercise. Accordingly, the system gives instructions and corrects the patient’s actions. The data generated during the session are collected for assessment by the specialist to adapt the difficulty of the exercise to the patient’s progress.ResultsThe evaluation of the system was conducted by two experts in balance disorder rehabilitation. They were required to verify the effectiveness of the system, and they also facilitated the simulation of real patient behavior. They used the system freely for a period of time and provided interesting and optimistic feedback. First, they evaluated the system as a tool for real-life rehabilitation therapy. Second, their interaction with the system allowed us to obtain important feedback needed to improve the system.ConclusionsThe system improves the rehabilitation conditions of people with balance disorder. The main contribution comes from the fact that it allows patients to carry out the rehabilitation process at home under the supervision of physiotherapists. As a result, patients avoid having to attend medical centers. Additionally, medical staff have access to an assistant, which means their presence is not required in many exercises that involve constant repetition.

A proposal of body movement-based interaction towards remote collaboration for concurrent engineering

The goal of concurrent engineering is to achieve the reduction of cost and time-to-market as well as to improve the quality of products, where collaboration of participants among different locations is critical. Various types of communication tools are available. However, the participants do not have the same feeling as a face-to-face meeting in remote communication. This study focuses on the issues of tele-presence and virtual immersion in remote communication, and proposes an idea of body movement-based interaction. The first approach is a hand gesture-based interaction called ATPI, which covers the natural control and mimicry of movement towards a distant object. The second approach is a head movement-based interaction called ARM-COMS, which covers the physical challenge as well as the mimicry of head movement using a physical object. Based on the preliminary experiments for these two approaches, feasibility of the idea of body movement-based interaction will be discussed in this paper.