Interpretation Of Social Signals Research Articles

The significance of parental mentalizing for four-year-old children's solitary pretend play.

Pretend play is a signature behavior of early childhood and is considered to reflect the child's emerging symbolic function, enabling the interpretation of social signals, language development, and emotion understanding. While theory links parental mentalizing with children's pretend play, only a few studies have investigated this association. These studies are limited to infancy and early toddlerhood, and child pretend play is assessed during play with an adult (social play). Based on the assumption that child solitary pretend play reflects the child's 'baseline' pretend play ability, in this study, we investigated children's pretend play at its peak, i.e., during the preschool age, without the facilitation of another player. The overall objective was to investigate if parental mentalizing increases pretend play complexity in children. The sample consisted 99 Danish mothers and their 4-year-old children. Employing a cross-sectional design, we hypothesized that parental mental state language, as an indicator of 'online' mentalizing during interaction with the child, is a mechanism through which 'offline' mentalizing, measured as parental reflective functioning, is associated with child solitary pretend play. Child pretend play complexity was observed and coded with an adapted version of the 12-Step Play Scale. Maternal offline mentalizing was assessed with the Parental Reflective Functioning Questionnaire, and maternal online mentalizing was assessed by coding the mothers' mental state language during interaction with the child using a modified version of the mind-mindedness coding scheme. While there was no direct effect of maternal offline reflective functioning on child pretend play, online mental state language mediated the link between offline maternal reflective functioning and child pretend play. These results provide support for the theoretically assumed link between parental mentalizing and children's capacity for pretend play. Furthermore, our study contributes to the literature on parental mentalization, suggesting that parental mentalizing facilitates child development only if the parent can translate this ability into 'mentalizing in action'.

Эмпатия в структуре социально-коммуникативной компетентности личности

One of the urgent problems of our time has become the problem of empathy in the context of socio–psychological problems. Being a socio–psychological mechanism of mutual understanding in the process of communication, empathy determines mobility, pace and structuring of social interaction. In the structure of social competencies, empathic response becomes the basis for solving interpersonal problems, correct interpretation of social signals.

A sparkle in the eye: Illumination cues and lightness constancy in the perception of eye contact

In social interactions, our sense of when we have eye contact with another person relies on the distribution of luminance across their eye region, reflecting the position of the darker iris within the lighter sclera of the human eye. This distribution of luminance can be distorted by the lighting conditions, consistent with the fundamental challenge that the visual system faces in distinguishing the nature of a surface from the pattern of light falling upon it. Here we perform a set of psychophysics experiments in human observers to investigate how illumination impacts on the perception of eye contact. First, we find that simple changes in the direction of illumination can produce systematic biases in our sense of when we have eye contact with another person. Second, we find that the visual system uses information about the lighting conditions to partially discount or ‘explain away’ the effects of illumination in this context, leading to a significantly more robust sense of when we have eye contact with another person. Third, we find that perceived eye contact is affected by specular reflections from the eye surface in addition to shading patterns, implicating eye glint as a potential cue to gaze direction. Overall, this illustrates how our interpretation of social signals relies on visual mechanisms that both compensate for the effects of illumination on retinal input and potentially exploit novel cues that illumination can produce.

Brain circuits signaling the absence of emotion in body language

Adaptive social behavior and mental well-being depend on not only recognizing emotional expressions but also, inferring the absence of emotion. While the neurobiology underwriting the perception of emotions is well studied, the mechanisms for detecting a lack of emotional content in social signals remain largely unknown. Here, using cutting-edge analyses of effective brain connectivity, we uncover the brain networks differentiating neutral and emotional body language. The data indicate greater activation of the right amygdala and midline cerebellar vermis to nonemotional as opposed to emotional body language. Most important, the effective connectivity between the amygdala and insula predicts people's ability to recognize the absence of emotion. These conclusions extend substantially current concepts of emotion perception by suggesting engagement of limbic effective connectivity in recognizing the lack of emotion in body language reading. Furthermore, the outcome may advance the understanding of overly emotional interpretation of social signals in depression or schizophrenia by providing the missing link between body language reading and limbic pathways. The study thus opens an avenue for multidisciplinary research on social cognition and the underlying cerebrocerebellar networks, ranging from animal models to patients with neuropsychiatric conditions.

Requirements for Robotic Interpretation of Social Signals “in the Wild”: Insights from Diagnostic Criteria of Autism Spectrum Disorder

The last few decades have seen widespread advances in technological means to characterise observable aspects of human behaviour such as gaze or posture. Among others, these developments have also led to significant advances in social robotics. At the same time, however, social robots are still largely evaluated in idealised or laboratory conditions, and it remains unclear whether the technological progress is sufficient to let such robots move “into the wild”. In this paper, we characterise the problems that a social robot in the real world may face, and review the technological state of the art in terms of addressing these. We do this by considering what it would entail to automate the diagnosis of Autism Spectrum Disorder (ASD). Just as for social robotics, ASD diagnosis fundamentally requires the ability to characterise human behaviour from observable aspects. However, therapists provide clear criteria regarding what to look for. As such, ASD diagnosis is a situation that is both relevant to real-world social robotics and comes with clear metrics. Overall, we demonstrate that even with relatively clear therapist-provided criteria and current technological progress, the need to interpret covert behaviour cannot yet be fully addressed. Our discussions have clear implications for ASD diagnosis, but also for social robotics more generally. For ASD diagnosis, we provide a classification of criteria based on whether or not they depend on covert information and highlight present-day possibilities for supporting therapists in diagnosis through technological means. For social robotics, we highlight the fundamental role of covert behaviour, show that the current state-of-the-art is unable to characterise this, and emphasise that future research should tackle this explicitly in realistic settings.

Serious Games for Training Social Skills in Job Interviews

In this paper, we focus on experience-based role play with virtual agents to provide young adults at the risk of exclusion with social skill training. We present a scenario-based serious game simulation platform. It comes with a social signal interpretation component, a scripted and autonomous agent dialog and social interaction behavior model, and an engine for 3-D rendering of lifelike virtual social agents in a virtual environment. We show how two training systems developed on the basis of this simulation platform can be used to educate people in showing appropriate socioemotive reactions in job interviews. Furthermore, we give an overview of four conducted studies investigating the effect of the agents’ portrayed personality and the appearance of the environment on the players’ perception of the characters and the learning experience.

Context-Aware Automated Analysis and Annotation of Social Human--Agent Interactions

The outcome of interpersonal interactions depends not only on the contents that we communicate verbally, but also on nonverbal social signals. Because a lack of social skills is a common problem for a significant number of people, serious games and other training environments have recently become the focus of research. In this work, we present NovA ( No n v erbal behavior A nalyzer), a system that analyzes and facilitates the interpretation of social signals automatically in a bidirectional interaction with a conversational agent. It records data of interactions, detects relevant social cues, and creates descriptive statistics for the recorded data with respect to the agent's behavior and the context of the situation. This enhances the possibilities for researchers to automatically label corpora of human--agent interactions and to give users feedback on strengths and weaknesses of their social behavior.

SSI

Automatic detection and interpretation of social signals carried by voice, gestures, mimics, etc. will play a key-role for next-generation interfaces as it paves the way towards more intuitive and natural humancomputer interaction. In this article we introduce Social Signal Interpretation (SSI), a framework for real-time recognition of social signals. SSI supports a large range of sensor devices, filter and feature algorithms, as well as, machine learning and pattern recognition tools. It encourages developers to add new components using SSI's C++ API, but also addresses front end users by offering an XML interface to build pipelines with a text editor. SSI is freely available under GPL at openssi.net.

Mental representations of emotional facial expressions are more complex rather than less accurate in older observers

The brain ages as dramatically as the body, manifesting itself in the reorganization of brain networks mediating interpretation of social signals. In particular, previous research has indicated that older adults are less accurate in categorizing negative facial emotional expressions, but the specific nature of these changes is unclear. Here, using the 4D Generative Face Grammar (GFG) platform (Yu et al., 2012) we generated a set of random combinations of facial movements, which coupled with reverse correlation informed the subsets constituting socially meaningful facial expressions. Eighteen young (10 female, median age=21, 18-25) and eighteen older (10 female, median age=65, 56-88) observers each categorized 4,992 animations of young and old faces into six basic emotions (i.e. 'happy', 'surprise', 'fear', 'disgust', 'anger', and 'sad') plus 'other/don't know', and rated the intensity of each expression (Jack et al., 2012). We then reverse correlated the random facial movements and their temporal parameters with the observers' responses. This resulted in dynamic models of the 6 basic facial expressions at five levels of intensity for each observer. We then performed a group-level k-means clustering analysis on the individual models. Young observers showed high homogeneity within expression categories, with a characteristic confusion between 'anger' and 'disgust'. Older observers showed well-preserved homogeneity for the expression models of 'disgust' and 'happiness'. By contrast, models of negative emotions ('fear', 'anger', 'sadness', and 'surprise') were less homogeneous across older observers. To examine whether this arose from a higher diversity of models of negative emotions in older observers, we identified the recognition strategies of each individual observer (Delis et al., submitted) and found that older observers deploy more strategies, in particular for models of 'sadness'.

A psycho-ethological approach to social signal processing

The emerging field of social signal processing can benefit from a theoretical framework to guide future research activities. The present article aims at drawing attention to two areas of research that devoted considerable efforts to the understanding of social behaviour: ethology and social psychology. With a long tradition in the study of animal signals, ethology and evolutionary biology have developed theoretical concepts to account for the functional significance of signalling. For example, the consideration of divergent selective pressures responsible for the evolution of signalling and social cognition emphasized the importance of two classes of indicators: informative cues and communicative signals. Social psychology, on the other hand, investigates emotional expression and interpersonal relationships, with a focus on the mechanisms underlying the production and interpretation of social signals and cues. Based on the theoretical considerations developed in these two fields, we propose a model that integrates the processing of perceivable individual features (social signals and cues) with contextual information, and we suggest that output of computer-based processing systems should be derived in terms of functional significance rather than in terms of absolute conceptual meaning.

Social Signal Interpretation (SSI)

The development of anticipatory user interfaces is a key issue in human-centred computing. Building systems that allow humans to communicate with a machine in the same natural and intuitive way as they would with each other requires detection and interpretation of the user’s affective and social signals. These are expressed in various and often complementary ways, including gestures, speech, mimics etc. Implementing fast and robust recognition engines is not only a necessary, but also challenging task. In this article, we introduce our Social Signal Interpretation (SSI) tool, a framework dedicated to support the development of such online recognition systems. The paper at hand discusses the processing of four modalities, namely audio, video, gesture and biosignals, with focus on affect recognition, and explains various approaches to fuse the extracted information to a final decision.

Converging Evidence for the Advantage of Dynamic Facial Expressions

Neuroimaging evidence suggests that dynamic facial expressions elicit greater activity than static face stimuli in brain structures associated with social cognition, interpreted as greater ecological validity. However, a quantitative meta-analysis of brain activity associated with dynamic facial expressions is lacking. The current study investigated, using three fMRI experiments, activity elicited by (a) dynamic and static happy faces, (b) dynamic and static happy and angry faces, and (c) dynamic faces and dynamic flowers. In addition, using activation likelihood estimate (ALE) meta-analysis, we determined areas concordant across published studies that (a) used dynamic faces and (b) specifically compared dynamic and static emotional faces. The middle temporal gyri (Experiment 1) and superior temporal sulci (STS; Experiment 1 and 2) were more active for dynamic than static faces. In contrasts with the baseline the amygdalae were more active for dynamic faces (Experiment 1 and 2) and the fusiform gyri were active for all conditions (all Experiments). The ALE meta-analyses revealed concordant activation in all of these regions as well as in areas associated with cognitive manipulations (inferior frontal gyri). Converging data from the experiments and the meta-analyses suggest that dynamic facial stimuli elicit increased activity in regions associated with interpretation of social signals and emotional processing.

A biological perspective on empathy.

During the evolution of the primate CNS, organization of neural activity has been shaped by the need for rapid and accurate evaluation of the motivations of others. Using a broad biological approach, the author considers empathy from evolutionary, ontogenetic, and neurophysiological viewpoints. Emotional communication follows a developmental course in primate evolution and in individuals: specialized neural activity and CNS organization subserve the interpretation of social signals. Neurophysiological studies now in progress may shed light on fundamental questions about the nature of empathy.