Nonverbal Behavior Research Articles

The body says it all: Non-verbal indicators of choice awareness

Abstract Recent cognitive science research suggests that occasional “blindness” to choice manipulations indicates a lack of awareness in choice making. This claim is based on participants’ tendency not to detect choice manipulations and the similarity between their justifications for choices they made and those they were tricked into believing they made. Using a cognitive-semiotic framework, we argue that such conclusions underestimate the embodied, intersubjective nature of human meaning-making. We support this by investigating choice awareness beyond language to include non-verbal behavior. Forty-one participants were asked to choose from pairs of photographs of human faces the one they found most attractive and then to justify their choices, without knowing that for some of the trials they were asked to justify a choice that they had not made. Verbal responses were categorized as (i) non-manipulated, (ii) detected manipulated, and (iii) undetected manipulated trials. Bodily expressions, assessed using five different Categories of Bodily Expression (CBE): Adaptors, Torso, Head, Face and Hand expressions, revealed differences in: (a) duration, (b) rates of occurrence and (c) variety of the CBEs across trials. Thus, even when manipulations were not verbally detected, participants took longer to assess choices, showed increased bodily expressions, and engaged more body parts in undetected manipulations compared to non-manipulated choice trials. This suggests a degree of awareness to the choice manipulation, even if pre-reflective, manifested in participants’ bodily expressions.

Correlation Among Teacher ICT Teaching, Teacher Immediacy Behaviors, Teacher–Student Rapport, and Student Engagement in Smart Classroom Teaching

With the constant evolution of information and communication technology (ICT), smart classrooms have profoundly influenced Education for Sustainable Development (ESD) by presenting advanced ICT that markedly improved the effectiveness and quality of teaching and learning. Teacher communication technology teaching and immediacy behaviors are crucial for leading and reforming the teaching and learning process in smart classrooms. Meanwhile, teacher–student rapport and student engagement are also key factors that influence ESD. This study aims to investigate the correlation among teacher ICT teaching, teacher immediacy behaviors, teacher–student rapport and student engagement in smart classroom teaching. We surveyed 1032 Chinese university students using the questionnaire method and analyzed the data using structural equation modeling (SEM). The findings revealed that teacher ICT teaching and immediacy behaviors exerted a noteworthy positive impact on student engagement and teacher–student rapport. Meanwhile, teacher ICT teaching markedly positively correlated with verbal immediacy but markedly negatively correlated with nonverbal immediacy. These findings have practical implications for ESD: in smart classrooms, teacher ICT teaching and their immediacy behaviors serve as vital factors in augmenting teaching quality, encouraging student engagement and fostering harmonious teacher–student rapport. Thus, teachers must harness their ICT teaching skills, flexibly assimilate instant behaviors into the teaching process and interact with students in a richer and more diversified manner to effectively augment teaching quality and promote the overall and sustainable growth of students. Furthermore, this study can inform the expansion of smart classrooms, which in the future should not only offer teachers a convenient teaching and learning environment but also evade ICT that limits teacher nonverbal immediacy behaviors.

Does (Biasing) Nonverbal Information Deteriorate the Accuracy of the Take‐the‐Best Heuristic for Deception Detection?

ABSTRACTPeople are poor lie detectors, partly because they hold false beliefs about nonverbal cues to deception. Here, we investigated if guiding people to rely only on a message's detailedness (“take‐the‐best”) boosts their lie detection and to what extent such heuristic judgments are immune to nonverbal information. In three studies (Ns = 109, 88 and 144), participants made detailedness‐based veracity judgements, of text versus video statements (Study 1), or of statements without or with biasing nonverbal behavior (truth tellers diverting, liars maintaining gaze; Studies 2 and 3). Compared to unguided judgements, participants using the heuristic method achieved higher deception detection accuracy throughout. Mere access to nonverbal behavior did not deteriorate performance (Study 1), but the heuristic was not fully immune to biasing nonverbal behavior (Studies 2, 3). Our findings challenge the lay notion that access to nonverbal behavior benefits deception detection and suggest that only focusing on diagnostic cues improves lie detection.

Curiosity and Digital Stories: Exploring Preschoolers’ Behaviors

<p style="text-align:justify">Given curiosity’s fundamental role in motivation and learning and considering the widespread use of digital stories as educational tools from the preschool age, we pursued measuring preschoolers’ curiosity when interacting with digital stories. Using 129 toddlers and preschoolers as a sample, three groups (one for each class) were given different versions of the same digital story to listen to: interactive, non-interactive, and animated. Toddlers' verbal and nonverbal behaviors were utilized to quantify curiosity as a condition brought on by the app. The participants' verbal and nonverbal behaviors were recorded during the digital reading aloud. Every child's data was encoded at one-minute intervals to examine concurrent behavior, and the results were then compiled. The findings show that interactive presentation formats encourage more touching and language use but less noise production and that interaction and the creative use of hot spots in digital illustrations are key elements in piquing viewers' curiosity while contributing to the strengthening of the engagement to the activity and the cultivation of critical thinking, creativity, and imagination.</p>

A lightweight network-based sign language robot with facial mirroring and speech system

Human–robot interaction is an essential capability for humanoid robots to enter the physical world and become companions in people’s lives, learning, and work. While the majority of current research focuses on the voice-based interactions of robots, yet over 60% of communication occurs through nonverbal behaviors, such as facial expressions and hand gestures. Endowing robots with the ability to communicate through nonverbal behavior not only enhances the interactive experience with robots but also provides a potential communication tool for individuals with hearing or speech impairments. Here, we develop a humanoid robot capable of adjusting facial movements by driving servos, and design a novel framework for the robot to integrate sign language recognition and facial landmark detection algorithms. This framework facilitates the robot recognize sign language and translate it into spoken language, while also imitating the facial expressions of the signers. To achieve this, we also propose a lightweight deep learning network called RealTimeSignNet for real-time sign language recognition. Leveraging lightweight 3D convolution modules and time-dependent constraints, this model adapts to various time scales, ensuring efficient processing of sign language recognition tasks. Experimental results demonstrate the outstanding performance of the RealTimeSignNet model on mainstream sign language datasets, achieving an accuracy of 88.1% on the large continuous sign language dataset (continuous SLR), 98.2% on the isolated sign language dataset (SLR 500), and 91.50% on the English sign language dataset (WLAS). The overall assessment demonstrates that our humanoid robot is capable of recognizing sign language and translating it into spoken language, while imitating the facial emotions, providing a comprehensive solution to the communication challenges faced by individuals with hearing and speech impairments.

My Child Does More Than Sit for Virtual Read‐Aloud: An Exploratory Human Pose Estimation Study

ABSTRACTVirtual storytime programs supported young children and families when in‐person library services were discontinued in response to the COVID‐19 pandemic in the United States. These virtual storytimes offered via videoconferencing systems allowed for the inclusion of some in‐person storytime elements such as the librarian read‐aloud, and children are the target audience of these picturebook read‐alouds. Children's nonverbal behavior symbolizes a powerful clue as to when they are engaged with the read‐aloud story. What is unclear is how a child nonverbally engages with the storytime readings. This study shows how human pose detection software, OpenPose, was instrumented with custom Python scripts to track and code a preschooler's nonverbal behavior during a virtual storytime read‐aloud. Results demonstrate considerable nonverbal attentiveness and engagement by this child during the read‐aloud.

Unveiling the Emotional Dynamics of Sports

Abstract: The variety of visible behaviors after winning or losing in sports reveals a lot about the psychological challenges players face during a sport competition. In this article, grounded in appraisal emotion approaches, self-regulation theory, as well as the biocultural framework of nonverbal behavior, I present findings on (a) how these affective expressive behaviors can partially be explained (i. e., their antecedents) and (b) how they might influence sport performance (i. e., their consequences). Regarding their antecedents, situational factors related to the importance of the situation, the outcome of the point, as well as players’ self-talk have been shown to be associated with these behaviors. While players report various performance-related inter- and intrapersonal consequences, the findings on associations with objective performance indicators show some inconsistency. Several directions for future research are subsequently discussed that may help improve the understanding of affective expressive behaviors in sport.

Computational elucidation of nonverbal behavior and body language in music therapy

Abstract Music therapy has shown efficacy in serious and chronic conditions, mental disorders, and disabilities. However, there is still much to explore regarding the mechanisms through which music interventions exert their effects. A typical session involves interactions between the therapist, the client, and the musical work itself, and to help address the challenges of capturing and comprehending its dynamics, we extend our general computational paradigm (CP) for analyzing the expressive and social behavioral processes in arts therapies. The extension includes bodily and nonverbal aspects of the behavior, offering additional insights into the client's emotional states and engagement. We have used this version of the CP, which employs AI pose estimation technology, image processing, and audio analysis, to capture therapy-related psychometrics and their intra- and inter-session analysis. The CP is applied in a real-world proof-of-concept study, and the results enable us to pinpoint meaningful events and emergent properties not captured by the human eye, complementing the therapist's interpretations. The resulting data may also be useful in other scientific and clinical areas.

Is Full Bright Smile a Positive Factor for Higher WTP?: Evidence from a Cross-Cultural Study on the Integrated Effects of Salesperson’s Non-verbal Behavior

This study examined the extent to which culture and familiarity with a shopping situation moderate the impact of a salesperson’s smile intensity on consumer decisions. Koreans, Indonesians, and Filipinos and Malaysians were recruited to represent low, mid, and high power distances, respectively. Participants were randomly assigned to one of two scenarios which asked to imagine shopping for a cosmetic product and being approached by a salesperson who either had a full smile and proactive service or had a limited smile and passively promoted sales. As predicted, the intentions to stay at the shop, purchase recommended items, and pay extra money for shopping were higher among the Korean participants when imagining a salesperson with a full smile. The Indonesians, in contrast, were more willing to pay extra money for shopping when approached by a salesperson with a limited smile. Implications for cross-cultural scholars and marketing practitioners are also discussed.

Understanding Therapeutic Lying In The Care Of A Person with Dementia: An Evolutionary Concept Analysis

Abstract Background Therapeutic lying, described as any verbal or non-verbal behaviour or communication that involves the manipulation of information has been shown to be pervasive in dementia care. However, its theoretical foundation is difficult to underpin and appears to have evolved through practice without a clear definition. The aim of this study is to establish conceptual clarity around the concept of therapeutic lying in the care of a person with dementia. Methods Rodgers’ evolutionary concept analysis was used to focus on the contextual qualities of therapeutic lying. This dispositional view recognises that concepts are fluid in nature. The analysis involves deconstructing the concept into surrogate terms, attributes, antecedents, consequences and related terms to generate a greater understanding of its innate features. A systematic search relating to “therapeutic lying” and “dementia” was undertaken. The final study sample consisted of 37 papers. Results This analysis revealed therapeutic lying’s surrogate terms, attributes, antecedents, consequences and related terms in the care of a person with dementia. This study highlighted this concept is intended to be used in the person's best interests, for the purpose of doing good. However, its subjective nature makes it an extremely difficult concept to grasp collectively. Conclusion Therapeutic lying is highly dependent on individual interpretation and the caregiver’s moral compass. It can potentially damage trust in a caring relationship and may contribute to the “othering” of a person with dementia. The negative consequences surrounding this concept must be studied comprehensively before it is considered in practice. Therapeutic lying should be used only as a last resort when a person with dementia is exhibiting truth related distress. As the therapeutic effect of a therapeutic lie cannot be guaranteed, the term “well intentioned lying” is proposed as a potential alternative for the concept as this term highlights the caregiver’s good intention behind incorporating a lie during care.

The Influence of “Body Language” on the Assessment of Witness Credibility in Australian Written Court Judgments

ABSTRACT This article examines how “body language” has been used in Australian court judgments when assessing witness credibility, and how it can influence legal proceedings. We first address the use (and misuse) of nonverbal communication. Then, we address the guidelines of the High Court of Australia regarding nonverbal behavior, also known as demeanor within the legal community. The methodology of the doctrinal analysis we conducted and the results are presented. Their implications considering research on nonverbal communication are discussed, and we conclude with a call for legal scholars and practitioners to further address the influence of “body language” in legal proceedings.

Acoso sexual experimentado por estudiantes universitarios ecuatorianos: prevalencia, impacto y acciones institucionales

This study addresses sexual harassment at the Technical University of Machala, aiming to examine its prevalence, characteristics, and emotional effects on students. Quantitative research with descriptive, exploratory, and explanatory scopes, conducted on a simple random sample of 776 university students, who were administered the ASIES instrument to collect data, which has been previously validated in the Ecuadorian context. The results reveal that a significant fraction of the students have experienced harassment, with verbal and non-verbal behaviors predominating, and women reporting a higher incidence. It was observed that,despite the implemented policies, there is still a lack of knowledge about the resources and reporting procedures. We conclude that it is essential to expand the research and improve information dissemination to address this issue, in addition to emphasizingthe need for educational and prevention strategies that promote a safe and inclusive university environment for all genders.

Dynamics of following and leading: association of movement synchrony and depression severity.

Depression negatively affects interpersonal functioning and influences nonverbal behavior. Interpersonal theories of depression suggest that depressed individuals engage in behaviors that initially provoke others' support and reassurance, but eventually lead to rejection that may also be expressed nonverbally. This study investigated movement synchrony as a nonverbal indicator of support and rejection and its association with depression severity in a sample of depressed and healthy individuals. Semi-standardized diagnostic interview segments with N = 114 dyads were video recorded. Body movement was analyzed using Motion Energy Analysis, synchrony intervals were identified by computing windowed cross-lagged correlation and a peak-picking-algorithm. Depression severity was assessed via both self-rating (BDI-II) and clinician rating (HAMD). Both self-rated and clinician-rated depression severity were negatively correlated with patient-led, but not clinician-led movement synchrony measures. The more depressed patients were, the less they initiated movement synchrony with their clinicians. These correlations remained significant after controlling for gender, age, gross body movement, and psychopharmacological medication. Findings suggest that depression may negatively affect patients' active initiative in interaction situations. Automatized methods as used in this study can add valuable information in the diagnosis of depression and the assessment of associated social impairments.

Investigating spoken classroom interactions in linguistically heterogeneous learning groups – An interdisciplinary approach to process video-based data in second language acquisition classrooms

Speaking the local language is central for successful integration into society. The teacher's language in second language (L2) classrooms serves as a crucial tool in language learning. Heterogeneity of learners’ language proficiency levels challenges teachers to adapt their language and accompanied instructional behavior. We offer an approach to study language acquisition processes and how teachers adapt their instructional language. This article presents our language-independent guidelines for processing video-based data of classroom interactions and demonstrate their reliability in a German as Second Language (GSL) classroom. These guidelines enable transcriptions of spoken language in noisy environments and detailed annotations of non-verbal classroom behavior. We outline research avenues at the intersection of empirical education research and linguistics that become feasible through these resources focusing on studying (non-)verbal adaptation strategies of teachers for learners at different proficiency levels. Our work directly fosters the interdisciplinary study of teacher-learner interactions, teacher competencies, and language acquisition.

Differences between Chinese and Western English-speaking teachers in providing disagreement in EFL teaching in China

ABSTRACT Disagreement is face-threatening but necessary for teachers to correct errors and prompt learning. However, little research has examined how teachers employ disagreement as feedback, especially from a multimodal perspective. This study investigated the differences in linguistic markers (LMs) and accompanying nonverbal behaviours (NVBs) associated with disagreements between Western English-speaking teachers (L1 teachers) and Chinese English teachers (L2 teachers). The classes of four L1 teachers and four L2 teachers were video-recorded and analysed. Follow-up interviews were analysed to examine the possible factors pertaining to the differences between the two groups. The results showed that both groups preferred to employ questions and counterclaims but differed in terms of the forms or contents they used. L1 teachers tended to use downtoners and humour as mitigators, while L2 teachers were inclined to employ token agreements and partial agreements. Smiling and gazing were the primary accompanying NVBs employed by the L1 teachers, whereas gazing and postures dominated the L2 teachers’ data. Possible factors for such differences were identified, such as perceived students’ needs and views on the functions of disagreements. The pedagogical implications for L1 and L2 teachers are also discussed.

Non-verbal effecting - animal research sheds light on human emotion communication.

Cracking the non-verbal "code" of human emotions has been a chief interest of generations of scientists. Yet, despite much effort, a dictionary that clearly maps non-verbal behaviours onto meaning remains elusive. We suggest this is due to an over-reliance on language-related concepts and an under-appreciation of the evolutionary context in which a given non-verbal behaviour emerged. Indeed, work in other species emphasizes non-verbal effects (e.g. affiliation) rather than meaning (e.g. happiness) and differentiates between signals, for which communication benefits both sender and receiver, and cues, for which communication does not benefit senders. Against this backdrop, we develop a "non-verbal effecting" perspective for human research. This perspective extends the typical focus on facial expressions to a broadcasting of multisensory signals and cues that emerge from both social and non-social emotions. Moreover, it emphasizes the consequences or effects that signals and cues have for individuals and their social interactions. We believe that re-directing our attention from verbal emotion labels to non-verbal effects is a necessary step to comprehend scientifically how humans share what they feel.

Communication Clashes Between Autistic Children and Parents in Non-Verbal Behavior

Communication within a family will indirectly create harmony and a close bond between parents and children. However, not all families can realize harmony. This is because there is a family given a particular child or a child with special needs, one of which is a child with autism. The purpose of this study is to understand the interpersonal relationship between parents and children with autism who have limitations in communication using the Relational Dialectics Theory (RDT) approach. This research applied a qualitative research method with a descriptive approach. The research results indicated that the relational dialectics of interpersonal relationships between parents and children with autism raise tension or tension when building communication. Of course, this will challenge parents to address tension when building communication with children with autism to create communication that can be understood by each other.

Alone Together, Together Alone: The Effects of Social Context on Nonverbal Behavior in Virtual Reality

Abstract Social virtual reality (VR), by definition, focuses on people using networked VR systems to bring avatars together. Previous studies have examined how different factors affect social interaction, in small groups such as dyads or triads. However, in a typical social VR scene there tends to be dozens of avatars, even those not directly interacting with a given user. Furthermore, beyond the virtual environment, VR users are also situated in various immediate physical social contexts. In two field experiments, we investigate how the presence of virtual and physical people contextualize and influence nonverbal behaviors. Study 1 examines virtual context and asks how interacting with others in a private or public virtual environment influences nonverbal outcomes during interactions in a social VR platform. Across two sessions, participants (n = 104) met either in a private virtual environment with their group members alone or in a public environment surrounded by four other groups. Results showed that participants moved their avatars slower and stood closer to group members in public versus private environments. Study 2 examines physical context and asks how interacting with virtual others while physically together or alone influences nonverbal behaviors. Participants (n = 61) met in virtual environments while they were in either a shared physical environment or separated physical environments. Results showed that, compared to remote participants, participants who were physically together moved their bodies more slowly, but their avatars faster. Moreover, there was more mutual gaze among remote participants. We discuss implications to theories of social influence in VR.

Personality Expression using Co-speech Gesture

We express our personality through verbal and nonverbal behavior. While verbal cues are mostly related to the semantics of what we say, nonverbal cues include our posture, gestures, and facial expressions. Appropriate expression of these behavioral elements improves conversational virtual agents’ communication capabilities and realism. Although previous studies focus on co-speech gesture generation, they do not consider the personality aspect of the synthesized animations. We show that automatically generated co-speech gestures naturally express personality traits, and heuristics-based adjustments for such animations can further improve personality expression. To this end, we present a framework for enhancing co-speech gestures with the different personalities of the Five-Factor model. Our experiments suggest that users perceive increased realism and improved personality expression when combining heuristics-based motion adjustments with co-speech gestures.

Indicating engagement in online workplace meetings

Abstract Amid COVID-19 and the so-called “digital pivot”, online virtual communication is at the heart of our professional and private lives. As we move into a post-COVID context, the affordances of the digital turn have shown that we can operate professionally online but there is a need for better understanding of communication in the online workplace. This paper contributes to our understanding of the dynamics of indicators of engagement in multi-party communication online, as evidenced by a corpus-based multi-modal study. It showcases the importance of building naturally-occurring spoken corpora that go beyond written transcription and include annotation of non-verbal behaviour. The work focuses on the incidence, frequency, position, and function of spoken and head nod backchannels, exploring coordination and co-occurrence of these features in online talk. Findings point to a changing profile of how engagement is displayed in online workplace meetings, which appears to be linked to the functionality of platforms.