Nonverbal Behavior Research Articles

Affective Computing Model With Impulse Control in Internet of Things Based on Affective Robotics

The combination of Internet of Things (IoT) and artificial intelligence (AI) technology plays an important role in many fields, especially in the field of psychology and medical treatment. This work is mainly to study an affective robotics that can serve humans emotionally based on the IoT and AI technology. The design of affective robotics is important to understand the underlying mechanisms of human behaviors in real life. These mechanisms mainly include human nonverbal behaviors and affective states, which are important but difficult to be precisely modeled. To address this challenge, we introduce a human–robot interaction (HRI) architecture, including emotion recognition, affective computing, emotion diagnosis, and emotion control. First, we propose a system model based on HRI between affective robotics and human, in order to enhance the emotional service. Then, we develop a dynamical model with affective computing and control, where we provide a mathematical formulation method based on stochastic differential equations to quantify the emotional state. Furthermore, we perform the dynamic behavior analysis of the existence, boundedness, and stability of the model solution comprehensively. Numerical results are provided to demonstrate the validity and feasibility of the proposed design techniques.

초등교사의 비언어적 의사소통 행동 유형 연구

Objectives This study is to explore the types of nonverbal communication behaviors of elementary school teachers based on communication theory. Methods This study was conducted as a philosophical and psychological integrated approach to education that can scientifically support various perspectives on human behavior and human behavior and psychological processes for the study of teachers' nonverbal communication behavioral types. Based on the communication theoretical exploration of Jurgen Habermas and Satir, a literature study was conducted on the types of nonverbal communication behaviors of teachers. Results Looking at many previous studies, the nonverbal communication behaviors of teachers focused on the personal aspect with emphasis on the teacher's personality traits. In this study, the teacher's nonverbal communication behavior was considered not only in the personal aspect of the teacher but also in the social and environmental context through Habermas and Satir's communication theoretical exploration and classification of nonverbal behavioral factors according to previous studies on nonverbal communication behavior. The behavioral patterns that appear according to the necessity of children were divided into two types: body language type and environment language type. As sub-elements, body language type communication behavior types using body organs such as body movements, body sensations, and voices, and environmental language type communication behavior types using physical time and space, artificial symbols, and social systems as sub-elements. classified. Conclusions Conclusion When looking at the types of communication behaviors of teachers in the existing studies in the context of elementary school classrooms, our gaze is focused on the teacher's individual personality traits and the differences between verbal communication behavior and nonverbal communication behavior. There have been many studies that have measured the frequency. In this study, which was conducted to overcome the limitations that were difficult to categorize because the complexity of the nonverbal communication system and the range of its elements are very wide, based on the communication theory of Habermas and Satir, various nonverbal elements appearing in the nonverbal communication behavior of teachers were identified as common. In addition to understanding the characteristics of an individual teacher with a body, the teacher's communication behavior type is an integrated perspective of the temporal and spatial environment of school, classroom, class, and situation, which are the real life contexts in which the individual is involved. will have to be understood as.

The recognition of affective states associated with players’ non-verbal behavior in volleyball

The visible non-verbal behavior of volleyball players after a point might indicate their affective state. The purpose of the present study was to investigate whether the point outcome (i.e., win vs. loss) and the importance of the situation (i.e., first vs. fifth set) influence how well the affective states (indexed through non-verbal behavior) can be recognized. For this purpose, 111 participants were presented 60 video excerpts showing the non-verbal behavior of volleyball teams immediately after a rally and asked whether the shown team had just won or lost a point. The results indicated that the recognition rate was significantly higher for won (80.09%) than for lost points (65.56%) as well as for points of the fifth set (77.84%) than for points of the first set (68.51%). Furthermore, an explorative analysis showed that the recognition rate was significantly higher for won points of the fifth set (89.19%) than for those of the first set (72.13%).

Nonverbal Behavior as an Educational Symbolic Mediator in Class

Nonverbal Behavior as an Educational Symbolic Mediator in Class

Validation and application of the Non-Verbal Behavior Analyzer: An automated tool to assess non-verbal emotional expressions in psychotherapy

Emotions play a key role in psychotherapy. However, a problem with examining emotional states via self-report questionnaires is that the assessment usually takes place after the actual emotion has been experienced which might lead to biases and continuous human ratings are time and cost intensive. Using the AI-based software package Non-Verbal Behavior Analyzer (NOVA), video-based emotion recognition of arousal and valence can be applied in naturalistic psychotherapeutic settings. In this study, four emotion recognition models (ERM) each based on specific feature sets (facial: OpenFace, OpenFace-Aureg; body: OpenPose-Activation, OpenPose-Energy) were developed and compared in their ability to predict arousal and valence scores correlated to PANAS emotion scores and processes of change (interpersonal experience, coping experience, affective experience) as well as symptoms (depression and anxiety in HSCL-11). A total of 183 patient therapy videos were divided into a training sample (55 patients), a test sample (50 patients), and a holdout sample (78 patients). The best ERM was selected for further analyses. Then, ERM based arousal and valence scores were correlated with patient and therapist estimates of emotions and processes of change. Furthermore, using regression models arousal and valence were examined as predictors of symptom severity in depression and anxiety. The ERM based on OpenFace produced the best agreement to the human coder rating. Arousal and valence correlated significantly with therapists' ratings of sadness, shame, anxiety, and relaxation, but not with the patient ratings of their own emotions. Furthermore, a significant negative correlation indicates that negative valence was associated with higher affective experience. Negative valence was found to significantly predict higher anxiety but not depression scores. This study shows that emotion recognition with NOVA can be used to generate ERMs associated with patient emotions, affective experiences and symptoms. Nevertheless, limitations were obvious. It seems necessary to improve the ERMs using larger databases of sessions and the validity of ERMs needs to be further investigated in different samples and different applications. Furthermore, future research should take ERMs to identify emotional synchrony between patient and therapists into account.

Adapting conversational strategies in information-giving human-agent interaction.

In this work, we focus on human-agent interaction where the role of the socially interactive agent is to optimize the amount of information to give to a user. In particular, we developed a dialog manager able to adapt the agent's conversational strategies to the preferences of the user it is interacting with to maximize the user's engagement during the interaction. For this purpose, we train an agent in interaction with a user using the reinforcement learning approach. The engagement of the user is measured using their non-verbal behaviors and turn-taking status. This measured engagement is used in the reward function, which balances the task of the agent (giving information) and its social goal (maintaining the user highly engaged). Agent's dialog acts may have different impact on the user's engagement depending on several factors, such as their personality, interest in the discussion topic, and attitude toward the agent. A subjective study was conducted with 120 participants to measure how third-party observers can perceive the adaptation of our dialog model. The results show that adapting the agent's conversational strategies has an influence on the participants' perception.

Influence of lecturers’ verbal and non-verbal immediacy behaviour on perceived affective and cognitive learning in a multicultural context.

The intention of the present study was to determine whether (1) the immediacy behaviourof lecturers whose home language is Afrikaans or English contributes positively to theaffective and cognitive learning of learners whose home language is Afrikaans, Englishor one of the African languages; and (2) whether the form of immediacy behaviourdisplayed by English or Afrikaans lecturers functions differently for learners whose homelanguage is one of the African languages in relation to those whose home language isAfrikaans or English. Data was collected by means of a questionnaire that had separatesections on immediacy behaviour and learning. Positive correlations between theimmediacy behaviour of the lecturers and the affective and cognitive learning of thelearners were observed for the whole test group. The immediacy behaviour displayed bythe lecturers functions differently for learners whose home language is one of the Africanlanguages than for those whose home language is Afrikaans or English. Considering thesefindings, it is imperative that instructional communication in today’s South Africa beincreasingly characterized by a culture-centred approach.

A Clinician's Perspective on the Communicative Intention in Autism: Call to Reconsider our Understanding of Non-Verbal Behaviours, Private and Inner Speech and Echolalia in Children with Autism Spectrum Disorder

A Clinician's Perspective on the Communicative Intention in Autism: Call to Reconsider our Understanding of Non-Verbal Behaviours, Private and Inner Speech and Echolalia in Children with Autism Spectrum Disorder

Misinterpretation of speaker intent in a multilingual workforce

This article deals with communication problems in intercultural communication in the workplace.Findings from this study have practical implications for developers of both intercultural and languagecourses. The research established that a beckoning gesture commonly used by white people isregarded as offensive by most blacks. At the same time many whites are unaware of non-verbalpoliteness markers signalled by means of kinetics. What whites interpret as "blunt" requests (oftenregarded as demands) can be ascribed to different norms of communication in African languages.These findings have implications for successful intercultural communication in an industrial society.Awareness of differences in verbal and non-verbal behaviour (termed mutual ignorance by Reagan)needs to be included in training for all members of the multicultural workforce.

Trimodal prediction of speaking and listening willingness to help improve turn-changing modeling.

Participants in a conversation must carefully monitor the turn-management (speaking and listening) willingness of other conversational partners and adjust their turn-changing behaviors accordingly to have smooth conversation. Many studies have focused on developing actual turn-changing (i.e., next speaker or end-of-turn) models that can predict whether turn-keeping or turn-changing will occur. Participants' verbal and non-verbal behaviors have been used as input features for predictive models. To the best of our knowledge, these studies only model the relationship between participant behavior and turn-changing. Thus, there is no model that takes into account participants' willingness to acquire a turn (turn-management willingness). In this paper, we address the challenge of building such models to predict the willingness of both speakers and listeners. Firstly, we find that dissonance exists between willingness and actual turn-changing. Secondly, we propose predictive models that are based on trimodal inputs, including acoustic, linguistic, and visual cues distilled from conversations. Additionally, we study the impact of modeling willingness to help improve the task of turn-changing prediction. To do so, we introduce a dyadic conversation corpus with annotated scores of speaker/listener turn-management willingness. Our results show that using all three modalities (i.e., acoustic, linguistic, and visual cues) of the speaker and listener is critically important for predicting turn-management willingness. Furthermore, explicitly adding willingness as a prediction task improves the performance of turn-changing prediction. Moreover, turn-management willingness prediction becomes more accurate when this joint prediction of turn-management willingness and turn-changing is performed by using multi-task learning techniques.

Couples’ Sexual Communication: Results from a Questionnaire and Conclusions For School-Based Sex Education

Introduction: Previous research demonstrated that sexual communication is positively intertwined with sexual satisfaction. The aim of this study was to identify the specific elements (‘sexual languages’, nonverbal behavior, post-sex communication, self-disclosure of sexual fantasies, negotiation of safe sex practices) that contribute to sexual satisfaction, and to derive implications for school-based sex education.
 Methodology: A mixed-methods approach was applied based on a questionnaire with open-ended items and two scales measuring sexual communication and sexual satisfaction. A convenience sample of 52 students (16 men, 36 women) recruited from Ludwigsburg University of Education was used. Qualitative content analysis was utilized to analyze qualitative data. Group comparisons and Spearman’s rank-order correlation test were performed to analyze quantitative data.
 Results: Nearly all participants perceived their sexual satisfaction as high, while they rated their sexual communication as mediocre to high. Sexual communication correlated strongly (ρ = 0.51, P < .001) with overall sexual satisfaction. Respondents’ sexual satisfaction did not depend on whether their sexual communication was verbal or non-verbal. Participants with undisclosed sexual fantasies (which they would love to turn into reality) had lower sexual satisfaction rates. Around a quarter of participants described concrete teaching methods to facilitate communication skills in school-based sex education.
 Discussion: The present results confirm, in line with previous research, that sexual satisfaction is highly predictable with sexual communication.
 Conclusion: One of the challenges of sex education is the cultural lack of modeling on how to communicate effectively about sex. A central goal must be that girls and boys can perceive their sexual turn-ons and -offs and disclose them to a partner. This self-disclosure is a crucial factor in developing a positive intimate relationship. As suggested by one of six of the participants, role-play scenarios (behavioral skill training) seem to be well suited to learn how to communicate about safe sex practices, sexual wants, and turn-offs.

Power of nonverbal behavior in online business negotiations: understanding trust, honesty, satisfaction, and beyond

ABSTRACT Digital teamwork has become prevalent and is ever since becoming part of the human work- and life-style, globally. But in comparison with face-to-face setting, virtual teams face multifold challenges. To date, scarce empirical research has examined whether team-breaking challenges are associated with limited access to peer nonverbal signals. This study examines whether access to body signals is associated with effective teamwork, and whether limited access provokes key team challenges. We also examine what social-psychological team concepts can be detected from peers’ consciously or unconsciously displayed visual cues that cannot be as effectively gained without visual access. 14 dyadic teams of MBA students were examined in an online business negotiation task to reach an authentic commercial deal. Half of the teams negotiated only through voice and text, while the other half had camera access as well. Using an exploratory mixed methods analysis, we identified 12 unique team factors based on nonverbal data. We also found that teams with camera access could build mutual trust more rapidly, detect peer honesty better, and realize agreements on suggestions more accurately. Surprisingly, we also found instances where camera access became stressful and participants reported it as an additional burden. Conclusions and implications are reported at the end.

PARALINGUISTIC MEANS IN INTERCULTURAL COMMUNICATION

The article is devoted to the problem of nonverbal components of communication and their role in the process of intercultural communication, in connection with which knowledge and consideration of national characteristics of nonverbal behavior of communicants as representatives of different cultures which is necessary for the success and effectiveness of intercultural communications.

Language Style Matching in Older Couples and Marital Satisfaction

Research on marital communication has focused on nonverbal behaviors (e.g., affect), leaving specific word use relatively unexplored. Recent work, however, suggests that similarity in function words (e.g., articles) may be associated with relational functioning in younger couples. We explored if this language style matching (LSM) also occurred in more established relationships and whether it was related to marital satisfaction within a sample of 64 older adult married couples. Using a dyadic, two-wave, observational research design, our results suggest that LSM is highly prevalent in older couples’ conversations across three different tasks (reminiscence, problem solving, and health support). Only LSM during reminiscence, however, was related to wives’ concurrent marital satisfaction. We considered implications for future research and for potential clinical interventions targeting language and communication in older adults.

CONFLICT BEHAVIOR STRATEGIES IN RELATION TO COGNITIVE ATTITUDES AND DECISION-MAKING STYLE

Modern understanding of conflict inclines us to the idea that conflict is closely related to cognitive phenomena, which can radically change the course of the conflict by influencing them. In order to use constructive approaches to conflict resolution, it is necessary to master effective techniques that affect personal attitudes and the individual's world view. The article presents the results of research work aimed at studying the relationship of personality behavior strategies in conflict, cognitive attitudes and decision-making styles. Based on theoretical definitions and analysis of practical research, it has been suggested that strategies of behavior in conflict are interrelated with irrational cognitive attitudes and decision-making styles. According to the results of an empirical study obtained on the material of 88 male students studying in technical areas of training at various universities in Perm, in the age range from 20 to 24 years (Me=22), it is shown that behavioral strategies in conflict are interrelated with a number of cognitive attitudes and a decision-making style. Links have been identified at the level of significance (p ≤ 0.05) between: the level of rationality and the manifestation of rivalry, evaluative cognitive attitude and a protective behavior strategy together with a neutral-independent non-verbal behavior strategy, catastrophizing attitude and a compromise strategy, rational-analytical style of thinking and a strategy of adjustment. Also three factors of conflict behavior of personality depending on the expression of irrational cognitive attitudes were identified: "Rational-partner", "Irrational-avoidant", "Intuitive-manipulative". The results obtained are proposed to be used in conflict counseling in order to more effectively regulate the course of conflicts.

Understanding production variability in multimodal communication

A comprehensive view of speech communication requires a consideration of dynamic characteristics in language users’ multimodal behaviors. This talk discusses dynamic properties of both verbal and non-verbal communication behaviors, focusing on variability in voice, speech, and co-speech gesture production. I present a series of experiments that leverage quantitative approaches to examine how the multimodal production units of the cognitive system vary in their dynamic control and realization in executing communication goals. The overarching hypothesis is that surface variations in multimodal behaviors are structured to reflect communicative intentions, exhibiting a tight relation between the linguistic system and recruited modalities. Supporting evidence emerges from two types of data: dynamic voice signals drawn from speech corpora of various speakers, speaking styles, and languages, and time-aligned multidimensional signals of speech and co-speech gestures (simultaneously recorded audio, kinematic, and visual signals). Voice variations are acoustically structured by both biologically relevant factors and factors that vary with language-specific phonology. Furthermore, the spatiotemporal patterning of vocal tract actions and cooccurring body movements systematically varies, reflecting the prosodic structure of language. The findings and the utility of the analytical tools employed in these studies have implications for comparative investigations of communicative behaviors in animals. [Work supported by the NSF.]

Children with profound intellectual disabilities metod of communication in family environments in familiar situational contexts part I

The first part of this article presents the characteristics of communication in children with profound intellectual disabilities with an adult in a family environment in a familiar situational context during dressing and undressing activities. The second part of the article focuses on dressing and undressing activities in a new situational context. In the conducted research, a qualitative research orientation was applied using the method of observation and the technique of observing samples of events. Recordings of routine daily activities dressing and undressing, eating, playing were analyzed. The observed data were recorded in the authors Sheet for Observation, Analysis and Interpretation of Communicative Behavior in a Communicative Situation. The analysis of the research material was carried out according to the planned stages: (1) analysis of the collected recordings description of the situational context, recording of the childs and adults communicative behavior; (2) analysis of the childs communicative behavior identification of non-verbal and verbal communicative behavior according to the communicative channels of children with profound intellectual disabilities (breathing, body signals, facial expression, eye expression, gesture, body position/posture, speech); (3) interpretation of observed communicative behaviors assigning characteristics to communicativebehaviors (effectiveness and appropriateness) and attempting to assign meaning to them (identifying childrens behaviors that may indicate attempts to initiate, sustain or interrupt interactions). Selected fragments of communicative situations were also analyzed. Based on these, the factors determining the effectiveness and appropriateness of the method of communication used by children with profound intellectual disabilities were inferred. This study is of cognitive and practical value. It complements the existing knowledge in the literature on the topic and can also contribute to introducing practical solutions to support the development of the communication process of children with profound intellectual disabilities.

The embodied enactment of politeness metapragmatics

Abstract Whereas previous research on metapragmatic talk has tended to focus on what people say about politeness, the current paper additionally explores how people talk about politeness. More specifically, we extend our analysis to the embodied resources, including gestures and nonverbal behaviour, that people use when enacting politeness talk. The data comes from two semi-structured interviews that we conducted with South Korean participants, both of whom were middle-aged university professors. We subjected the data to three rounds of analysis. First, we thematically coded the content of the narratives, revealing important differences in how these two participants from the same language background conceived of politeness. Second, we analyzed the appearance of verbal and non-verbal markers of deferential politeness. This analysis revealed that the overall bodily comportment of the two participants closely matched the different politeness narratives that they inhabited. Third and finally, we examined how the participants used bodily movements when evoking specific embodied practices related to politeness, and used embodied behaviours to represent abstract politeness-related concepts and map them onto spatial locations. Overall, the analysis shows that metapragmatic talk about politeness is an embodied achievement and thus needs to be treated within the remit of the multimodal turn in politeness research.

Collaborating eye to eye: Effects of workplace design on the perception of dominance of collaboration robots.

The concept of Human-Robot Collaboration (HRC) describes innovative industrial work procedures, in which human staff works in close vicinity with robots on a shared task. Current HRC scenarios often deploy hand-guided robots or remote controls operated by the human collaboration partner. As HRC envisions active collaboration between both parties, ongoing research efforts aim to enhance the capabilities of industrial robots not only in the technical dimension but also in the robot’s socio-interactive features. Apart from enabling the robot to autonomously complete the respective shared task in conjunction with a human partner, one essential aspect lifted from the group collaboration among humans is the communication between both entities. State-of-the-art research has identified communication as a significant contributor to successful collaboration between humans and industrial robots. Non-verbal gestures have been shown to be contributing aspect in conveying the respective state of the robot during the collaboration procedure. Research indicates that, depending on the viewing perspective, the usage of non-verbal gestures in humans can impact the interpersonal attribution of certain characteristics. Applied to collaborative robots such as the Yumi IRB 14000, which is equipped with two arms, specifically to mimic human actions, the perception of the robots’ non-verbal behavior can affect the collaboration. Most important in this context are dominance emitting gestures by the robot that can reinforce negative attitudes towards robots, thus hampering the users’ willingness and effectiveness to collaborate with the robot. By using a 3 × 3 within-subjects design online study, we investigated the effect of dominance gestures (Akimbo, crossing arms, and large arm spread) working in a standing position with an average male height, working in a standing position with an average female height, and working in a seated position on the perception of dominance of the robot. Overall 115 participants (58 female and 57 male) with an average age of 23 years evaluated nine videos of the robot. Results indicated that all presented gestures affect a person’s perception of the robot in regards to its perceived characteristics and willingness to cooperate with the robot. The data also showed participants’ increased attribution of dominance based on the presented viewing perspective.

Automated patterns of head dynamics are associated with psychopathic traits in incarcerated women

Men with elevated psychopathic traits have been characterized by unique patterns of nonverbal communication, including more fixed and focused head positions during clinical interviews, compared to men scoring low on measures of psychopathy. However, it is unclear whether similar patterns of head dynamics help characterize women scoring high on psychopathic traits. Here, we utilized an automated detection algorithm to assess head position and dynamics during a videotaped clinical interview (i.e., the Psychopathy Checklist – Revised [PCL-R]) in a sample of n = 213 incarcerated women. PCL-R Total, Factor 1 (i.e., interpersonal and affective psychopathic traits), and Factor 2 (i.e., lifestyle/behavioral and antisocial/developmental psychopathic traits) scores were associated with a pattern of head dynamics indicative of a rigid head position. The current study extends analyses of nonverbal behavior studies in men to women and highlights how individuals with elevated psychopathic traits demonstrate unique nonverbal behaviors relative to individuals who score low on psychopathic traits. The implications and clinical value of these findings are discussed.