Non-verbal Interaction Research Articles

Better alone than in bad company

AbstractUsing artificial emotions helps in making human-robot interaction more personalised, natural, and so more likeable. In the case of humanoid robots with constrained facial expression, the literature concentrates on the expression of emotions by using other nonverbal interaction channels. When using multi-modal communication, indeed, it is important to understand the effect of the combination of such non-verbal cues, while the majority of the works addressed only the role of single channels in the human recognition performance. Here, we present an attempt to analyse the effect of the combination of different animations expressing the same emotion or different ones. Results show that when an emotion is successfully expressed using a single channel, the combination of this channel with other animations, that may have lower recognition rates, appears to be less communicative.

Assessment of Apoptosis Pathway in Peripheral Blood of Autistic Patients.

Autism spectrum disorder (ASD) includes a number of severe neurodevelopmental disorders known by defects in social interaction, impaired verbal and non-verbal interactions, and stereotypic activities and limited interests. Dysregulation of apoptotic pathways have been demonstrated in brain tissues of affected individuals. In the present study, we evaluated expression levels of apoptosis-related genes and miRNAs in peripheral blood of ASD patients compared with healthy subjects. Transcript levels of BCL2, CASP8, and hsa-29c-3p were significantly lower in total ASD patients compared with total normal children (P values = 0.003, 0.002, and 0.01 respectively). When sex of study participants was considered in the analysis, the difference in transcript levels of these genes was significant only in male subjects. Peripheral expression of BCL2 and hsa-29c-3p had 100% sensitivity 92% specificity in ASD diagnosis. The diagnostic power of combination of transcript levels of these genes was estimated to be 78% based on the calculated AUC value. The present study provides evidences for dysregulation of apoptotic pathways in peripheral blood of ASD patients and suggests certain apoptosis-related genes as biomarkers in this regard.

Coordinated Collaboration and Nonverbal Social Interactions: A Formal and Functional Analysis of Gaze, Gestures, and Other Body Movements in a Contemporary Dance Improvisation Performance

This study presents a microanalysis of what information performers “give” and “give off” to each other via their bodies during a contemporary dance improvisation. We compare what expert performers and non-performers (sufficiently trained to successfully perform) do with their bodies during a silent, multiparty improvisation exercise, in order to identify any differences and to provide insight into nonverbal communication in a less conventional setting. The coordinated collaboration of the participants (two groups of six) was examined in a frame-by-frame analysis focusing on all body movements, including gaze shifts as well as the formal and functional movement units produced in the head–face, upper-, and lower-body regions. The Methods section describes in detail the annotation process and inter-rater agreement. The results of this study indicate that expert performers during the improvisation are in “performance mode” and have embodied other social cognitive strategies and skills (e.g., endogenous orienting, gaze avoidance, greater motor control) that the non-performers do not have available. Expert performers avoid using intentional communication, relying on information to be inferentially communicated in order to coordinate collaboratively, with silence and stillness being construed as meaningful in that social practice and context. The information that expert performers produce is quantitatively less (i.e., producing fewer body movements) and qualitatively more inferential than intentional compared to a control group of non-performers, which affects the quality of the performance.

A Hybrid Approach for Facial Expression Recognition Using Extended Local Binary Patterns and Principal Component Analysis

Facial feature extraction and recognition plays a prominent role in human non-verbal interaction and it is one of the crucial factors among pose, speech, facial expression, behaviour and actions which are used in conveying information about the intentions and emotions of a human being. In this article an extended local binary pattern is used for the feature extraction process and a principal component analysis (PCA) is used for dimensionality reduction. The projections of the sample and model images are calculated and compared by Euclidean distance method. The combination of extended local binary pattern and PCA (ELBP+PCA) improves the accuracy of the recognition rate and also diminishes the evaluation complexity. The evaluation of proposed facial expression recognition approach will focus on the performance of the recognition rate. A series of tests are performed for the validation of algorithms and to compare the accuracy of the methods on the JAFFE, Extended Cohn-Kanade images database.

Learner-teacher non-verbal interaction effect on academic achievement of learners in chemistry

Learner-teacher non-verbal interaction effect on academic achievement of learners in chemistry

“Say ‘What happened?’ in Hebrew. He does not speak Arabic!” Early language awareness as expressed in verbal and nonverbal interactions in the preschool bilingual classroom

The aim of this longitudinal ethnographic study was to explore how children's verbal and nonverbal behavior reflects their language awareness at a bilingual Arabic–Hebrew-speaking preschool in Israel. We adopted the perspective that children's L2 acquisition is situated within the social events and interactional practices of the classroom community. We took a close look at six bilingual 3-year-old children—three L1 Arabic-speaking children and three L1 Hebrew-speaking children. To enhance the credibility of our study, we triangulated the collected observations of the children's talk via teachers and parents’ testimonies. The analysis revealed that children applied diverse verbal and nonverbal mediating cues to solve their interlocutors’ communicative troubles. Their pragmatic awareness of the interlocutors’ communicative troubles was encouraged by a unique classroom context where the teachers stimulated children's language mediation and peers’ backup. The study is beneficial for language teachers’ understanding of how they can support children's language awareness.

Emotion regulation during conflict interaction after a systemic music intervention: Understanding changes for parents with a trauma history and their adolescent

ABSTRACTIntroduction: For parents who have experienced childhood maltreatment, parenting an adolescent may trigger memories of abuse or neglect, intensifying parent-adolescent conflict. This paper extends an earlier report of outcomes from a pilot randomised controlled trial (RCT) of Tuning Relationships with Music™, an intervention for parent-adolescent dyads where parents have a trauma history and dyads are experiencing conflict, in order to look systemically at dyads’ emotion regulation during their musical representation of nonverbal conflict interaction (NCI).Method: The RCT randomly allocated 26 parent-adolescent dyads into intervention or control conditions, and analysed dyads’ self-report and observation measures completed at baseline and 4-month post-baseline follow-up. Aims of the current paper were (1) to examine relationships between parents’ trauma history, parent-adolescent conflict, parents’ reactivity and responsiveness, and dyads’ emotion regulation, consistency and predictability during NCI; and (2) whether a systemic music intervention focused on nonverbal communication (NVC) and emotion regulation would change the dynamics of dyads’ relational conflict.Results: Higher parents’ childhood betrayal trauma and adolescent-reported conflict scores were correlated with predictable NVC sequences whilst dyads were emotionally dysregulated, and parents’ reactivity was correlated with dyads’ inconsistent NVC during NCI. Post-intervention dyads were more emotionally regulated, consistent and predictable during NCI, reported lower levels of conflict and showed increased parent emotional responsiveness and reduced reactivity.Discussion: Findings indicate that where parents have experienced childhood maltreatment, conflictual parent-adolescent dyads may benefit from intervention focusing on skills that promote emotionally regulated, predictable and consistent NVC during conflict interactions. Trials with a larger sample are warranted.

Conversational engagement and mobile technology use

The present work investigated effects of mobility constraints on gesturing and other aspects of conversational engagement when using mobile technology. Based on studies of non-verbal communication and interaction quality, we compared mobile and static forms of video call in structured interviews. Using real-time motion capture and content coding of recordings as well as self-reports, several data sources were considered: hand velocity as an indicator of gesturing, observed levels of engagement and remembered conversational content. Gesturing, in contrast to other indicators of engagement, was reduced under high mobility constraints and was more pronounced under low constraints. Gesturing was further positively related to observed engagement. Additionally, lower constraints resulted in more memories of conversational content. These findings emphasise the relevance of mobility constraints and bodily movement during mediated interaction and call for further model development in computer-mediated communication. Potential for application lies in interactive scenarios such as remote interviewing, testimonials, and relationship maintenance.

A Socially Assistive Mobile Platform for Weight-Support in Gait Training

Although the aspects of physical support for patients with lower-limb sensory motor paralysis during walk rehabilitation are well studied, it is also important to consider the social factors which may affect them during gait training. In this study, we propose a robotic platform able to autonomously accompany users while providing social assistance by using an anthropomorphic interface on the front side. The developed mobile platform is based on a weight-support device and is capable to move in accordance with the users’ gait speed and to acquire information such as step-length, step-width and cadence, which can be used further to monitor their progress during the task. The non-verbal interaction provided by the anthropomorphic robot is done by head and hand movement which is remotely controlled during the exercise. Experiments indicate that the platform can adapt to the walking pattern of participants in a similar manner as a therapist does. Furthermore, assessment from questionnaires shows improvements in the participants’ feeling of comfort, independence and motivation in the scenario with the social robot.

Collective benefit in joint perceptual judgments: Partial roles of shared environments, meta-cognition, and feedback

Recent research has demonstrated that two persons can optimally combine their observations to make better decisions when they can verbally communicate their confidence to each other. The present study investigated whether non-verbal interaction in a shared environment can be sufficient to achieve similar collective benefit. Pairs of individuals performed a localization task in a shared virtual 3D environment. In Experiments 1 and 2, partners had access to orthogonal viewpoints providing complementary information. The results showed robust collective benefit compared to individual performance from a single viewpoint, which could be obtained without any opportunity for verbal communication and even when no feedback about joint accuracy was provided (Experiment 2). When partners shared the same viewpoint (Experiment 3), collective benefit was achieved only when feedback on accuracy was provided (Experiment 3a). The findings indicate that sharing an environment can be sufficient for achieving integration of complementary perceptual information. Communicating confidence might not be necessary when an environment is shared. Another possibility is that processes for integrating interpersonally continuous information generally differ from the processes invoked when making a joint dichotomous choice.

Musical communication in scaffolding young learners’ expressive agency

In a qualitative study of the nature of musical communication during scaffolding music learning, the most important themes to emerge reflected the role of musical communication in blossoming young learners’ expressive agency. The study focused on two different groups of young piano learners (aged 4–9) during collaborative (listening, creating, and performing) problem-solving experiences. Working as a teacher-researcher in the context of my own studio piano classes, I documented verbal and nonverbal interactions that occurred during the lessons. Data were collected primarily through video observation, field notes and a reflective journal. Analysis included the construction of narrative vignettes from these data. Analysis revealed that the children’s music learning was a creative process of transformation, as they negotiated and renegotiated their own meaning and that of others through musical communication. Possessing their own communicative musicality, learners exhibited ability to share a range of musical understanding and sensitivities through both sound and physical motion. Findings suggest that children’s music learning is not only located in individual minds but is anchored in a communicative landscape and when learners are engaged in musical scaffolding their expressive agency is enabled.

Adults imitate to send a social signal

Humans frequently imitate each other’s actions with high fidelity, and different reasons have been proposed for why they do so. Here we test the hypothesis that imitation can act as a social signal, with imitation occurring with greater fidelity when a participant is being watched. In a preregistered study, 30 pairs of naïve participants played an augmented-reality game involving moving blocks in space. We compared imitation fidelity between trials where the leader watched the followers’ action, and trials where the leader did not watch. Followers imitated the trajectory height demonstrated by the leader, and critically, the strength of this correlation was greater in trials where the follower knew the leader was watching them. This suggests that followers spontaneously used imitation as a social signal in a nonverbal interaction task, supporting socio-communicative hypotheses of imitation.

The influence of prosocial priming on visual perspective taking and automatic imitation.

Imitation and perspective taking are core features of non-verbal social interactions. We imitate one another to signal a desire to affiliate and consider others’ points of view to better understand their perspective. Prior research suggests that a relationship exists between prosocial behaviour and imitation. For example, priming prosocial behaviours has been shown to increase imitative tendencies in automatic imitation tasks. Despite its importance during social interactions, far less is known about how perspective taking might relate to either prosociality or imitation. The current study investigates the relationship between automatic imitation and perspective taking by testing the extent to which these skills are similarly modulated by prosocial priming. Across all experimental groups, a surprising ceiling effect emerged in the perspective taking task (the Director’s Task), which prevented the investigation of prosocial priming on perspective taking. A comparison of other studies using the Director’s Task shows wide variability in accuracy scores across studies and is suggestive of low task reliability. In addition, despite using a high-power design, and contrary to three previous studies, no effect of prosocial prime on imitation was observed. Meta-analysing all studies to date suggests that the effects of prosocial primes on imitation are variable and could be small. The current study, therefore, offers caution when using the computerised Director’s Task as a measure of perspective taking with adult populations, as it shows high variability across studies and may suffer from a ceiling effect. In addition, the results question the size and robustness of prosocial priming effects on automatic imitation. More generally, by reporting null results we hope to minimise publication bias and by meta-analysing results as studies emerge and making data freely available, we hope to move towards a more cumulative science of social cognition.

Communicative misalignment in Autism Spectrum Disorder

Communication deficits are a defining feature of Autism Spectrum Disorder (ASD), manifest during social interactions. Previous studies investigating communicative deficits have largely focused on the perceptual biases, social motivation, cognitive flexibility, or mentalizing abilities of isolated individuals. By embedding autistic individuals in live non-verbal interactions, we characterized a novel cause for their communication deficits. Adults with ASD matched neurotypical individuals in their ability and propensity to generate and modify intelligible behaviors for a communicative partner. However, they struggled to align the meaning of those behaviors with their partner when meaning required referencing their recent communicative history. This communicative misalignment explains why autistic individuals are vulnerable in everyday interactions, which entail fleeting ambiguities, but succeed in social cognition tests involving stereotyped contextual cues. These findings illustrate the cognitive and clinical importance of considering social interaction as a communicative alignment challenge, and how ineffective human communication is without this key interactional ingredient.

Inferring power and dominance from dyadic nonverbal interactions in autism spectrum disorder.

Research studies to date have revealed conflicting results with respect to the processing of nonverbal cues from social interactions in autism spectrum disorder (ASD). Therefore, the aim of the present study was to investigate the contribution of two important factors for the perception of dyadic social interactions, namely (a) the movement contingency and (b) the spatial context. To this end, 26 adult participants with ASD and 26 age-, sex-, and IQ-matched typically developed control participants observed animations presenting nonverbal interactions between two human virtual characters enacting power relationships. We manipulated (a) movement contingency by exchanging one of the two original agents with an agent from another dyad and (b) spatial context by changing agents' spatial orientation to a back-to-back position. Participants were asked to rate dominance and submissiveness of these agents. Results showed that the movement contingency manipulation affected accuracy and consistency of power perception and that the spatial context manipulation slowed down reaction times comparably in both groups. With regard to group differences, individuals with ASD were found to judge power relationships slower compared to control participants, potentially suggesting a more explicit processing style in ASD. Furthermore, the spatial context manipulation slowed down the reaction times more in the contingent compared to the non-contingent conditions only in the ASD group. These findings contribute to the ongoing debate whether individuals with ASD have difficulties in understanding nonverbal cues in a dyadic context by suggesting that they do so in more subtle ways than previously investigated. Autism Res 2019, 12: 505-516 © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: This study shows that the ability and speed of judging who is dominant in a social interaction depends on two factors: (a) whether their movements are matched and (b) whether they are facing each other or not. This is similarly the case for participants with and without autism. Interestingly, however, individuals with autism seem to judge generally slower, suggesting a more explicit processing style. The two factors seem to interact, suggesting that nonverbal processing difficulties are subtler than previously thought.

A Systems Model of Dyadic Nonverbal Interaction

This article discusses a new systems model of dyadic nonverbal interaction. The model builds on earlier theories by integrating partners’ parallel sending and receiving nonverbal processes into a broader, dynamic ecological system. It does so in two ways. First, it moves the level of description beyond the individual level to the coordination of both partners’ contributions to the interaction. Second, it recognizes that the relationships between (a) individuals’ characteristics and processes and (b) the social ecology of the interaction setting are reciprocal and best analyzed at the systems level. Thus, the systems model attempts to describe and explain the dynamic interplay among individual, dyadic, and environmental processes in nonverbal interactions. The potential utility and the limitations of the systems model are discussed and the implications for future research considered. Although the systems model is focused explicitly on face-to-face nonverbal communication, it has considerable relevance for digital communication. Specifically, this model provides a useful framework for examining the social effects of mobile device use and as a template for studying human–robot interactions.

Representational challenges in animated chemistry: self-generated animations as a means to encourage students’ reflections on sub-micro processes in laboratory exercises

A central aspect of learning chemistry is learning to relate observations of phenomena to models of the sub-microscopic level of matter, and hence being able to explain the observable phenomena. However, research shows that students have difficulties discerning and comprehending the meaning of the sub-micro level and its models, and that practical work in its traditional form fails to help students to discern the relation between observations and models. Consequently, there is a strong call for new teaching activities to address these issues. This paper emerges from a growing number of studies showing that learning is supported when students are set to cooperatively create their own multimodal representations of science phenomena. In this paper, we explore the approach of letting students create their own stop-motion animation as a means to explain observations during practical work. The students’ work of producing a phenomenon in the laboratory and creating an animation was recorded (audio–video) to capture students’ verbal and non-verbal interactions and use of resources. Data was analysed using a thematic content analysis with a deductive approach aimed at identifying the aspects of chemistry content that are being reasoned. The analysis showed that the task enabled students to engage in reasoning concerning both the observations and the sub-micro-level models, and how they relate to each other. The task also enabled students to reason about features of the representation that are needed to make sense of both the observational and sub-microscopic aspects of a phenomenon, as well as reflecting upon the meaning of a model.

Research of psychological support of the personality of the senior preschool child and his readiness for school in the conditions of transitive society

This article describes the psychological support as a scientific category; analyzes different approaches to the study of psychological support patterns supporting verbal and nonverbal interaction plan affecting the efficiency of psychological support of the individual, highlighted the constructive changes facility wide the influence of psychological support on developing personality. It is proved that in a transitive society psychological support will help the child to adapt to rapidly changing factors of environmental development, to a new type of activity, to new social roles. In practical terms, psychological support can be applied using specific, both individual and group psychological methods with components of modern exercises and psychotechnics. Here the developing subject needs the support of a competent partner in the event community. This article describes the supporting forms, means, methods, techniques that can be used by teachers and educators at the stage of adaptation of first-graders to school.

Emotion-Specific Facial Activation Maps Based on Infrared Thermal Image Sequences

Research on emotion recognition has recently started to gain increased attention. Inner emotions or thought activity can be determined by analyzing facial expression, behavioral responses, audio, and physiological signals. Facial expression is now recognized as an important form of non-verbal interaction. In this paper, emotion-specific activation maps were constructed to establish infrared thermal facial image sequences as an alternative approach to the determination of the correlation between emotional triggers and changes in facial temperature. During the testing process, data stored in the International Affective Picture System were used to create emotional clips that triggered three different types of emotion in the subjects, and their infrared thermal facial image sequences were simultaneously recorded. For processing, an image calibration protocol was first employed to reduce the variance produced by irregular micro-shifts in the faces of the subjects, followed by independent component analysis and statistical analysis protocols to create the facial emotional activation maps. The test results showed that the problem of selecting local regions when analyzing frame temperature had been resolved. The emotion-specific facial activation maps provide visualized results that facilitate the observation and understanding of information.

Video analysis in Design-Based Research – Findings of a project on self-organised learning at a vocational school

The use of video analysis in Design-Based Research (DBR) seems to be promising, because the quality of video data matches the reality of educational fields. Educational fields are multidimensional and complex. And more than other types of data, video may capture, for example, the simultaneity of verbal and non-verbal interactions. This seems to be valuable in the quest for new insights and better designs of educational interventions. However, to date there has been limited use of video data in researching their design. This paper aims at reflecting how the benefits of video-based analysis may be utilised in DBR. Experiences with the collection and analysis of video data in a project to design self-organised learning (SOL) at a vocational school in Germany will be used as a case study to illustrate the type of findings that may feed into the DBR process. In this case, the project school had already introduced a sophisticated SOL model but was experiencing various implementation difficulties. Resolving issues like this requires insights into how exactly a concept is realised and what happens in the field. Therefore, video data on classroom interactions was gathered and sub-sequently analysed using the documentary method. This led to the reconstruction of two different types of orientation that were guiding the students when they dealt with their self-organised learning environment. In a subversive orientation, students playfully infiltrate the formal learning space with peer activities. In a confirming orientation, students stick to both, the (informal) rules of the (formal) learning arrangement and of the peer environment, thus expressing respect for the boundary between these two worlds. These findings have been used to redesign the SOL intervention.