Real-time Social Interaction Research Articles

Synchrony Across Brains.

Second-person neuroscience focuses on studying the behavioral and neuronal mechanisms of real-time social interactions within single and across interacting brains. In this review article, we describe the developments that have been undertaken to study socially interactive phenomena and have helped to focus on behavioral and neurobiological processes that extend across interaction partners. More specifically, we focus on the role that synchrony across brains plays in enabling and facilitating social interaction and communication and in shaping social coordination and learning, and we consider how reduced synchrony across brains may constitute a core feature of psychopathology.

The promoting effect of the absence of second-party's punishment power on third-party punishment in maintaining social fairness norms: An EEG hyper-scanning study

Third-party punishment (TPP) plays an irreplaceable role in maintaining social fairness. Punishment power is a significant area of study within economic games. However, the impact of whether or not the second-party possesses punishment power on TPP remains unexplored. The present study utilizes the high temporal resolution of EEG and time-frequency analysis, intra-barin functional connectivity analysis, inter-brain synchronization (IBS) analysis, and granger causality analysis(GCA) to comprehensively explore the neural mechanism of TPP from the perspective of third-party individual's decision-making and IBS in the real-time social interaction. Time-frequency results found that, the absence of the punishment power activated more theta-band and alpha-band power compare to when second-party has punishment power. When second-party has no punishment power, functional connection results observed stronger functional connectivity in theta band for medium unfair offers between rTPJ and PFC. Dual-brain analysis revealed that when the second-party has no punishment power, there is a significantly higher IBS in the alpha band between the frontal and frontal-central lobes of the second-party and the parietal and parietal occipital lobes of the third-party. GCA results further showed that the direction of IBS from third-party to second-party was significantly stronger than from second-party to third-party. This study demonstrates that the absence of the second-party's punishment power promote TPP, and similar cognitive process of thinking on how to maintain social fairness enhances IBS. The current study emphasizes the influence of punishment power on TPP, broadens the research perspective and contributes crucial insights into maintain social fairness.

Observers predict actions from facial emotional expressions during real-time social interactions

In face-to-face social interactions, emotional expressions provide insights into the mental state of an interactive partner. This information can be crucial to infer action intentions and react towards another person’s actions. Here we investigate how facial emotional expressions impact subjective experience and physiological and behavioral responses to social actions during real-time interactions. Thirty-two participants interacted with virtual agents while fully immersed in Virtual Reality. Agents displayed an angry or happy facial expression before they directed an appetitive (fist bump) or aversive (punch) social action towards the participant. Participants responded to these actions, either by reciprocating the fist bump or by defending the punch. For all interactions, subjective experience was measured using ratings. In addition, physiological responses (electrodermal activity, electrocardiogram) and participants’ response times were recorded. Aversive actions were judged to be more arousing and less pleasant relative to appetitive actions. In addition, angry expressions increased heart rate relative to happy expressions. Crucially, interaction effects between facial emotional expression and action were observed. Angry expressions reduced pleasantness stronger for appetitive compared to aversive actions. Furthermore, skin conductance responses to aversive actions were increased for happy compared to angry expressions and reaction times were faster to aversive compared to appetitive actions when agents showed an angry expression. These results indicate that observers used facial emotional expression to generate expectations for particular actions. Consequently, the present study demonstrates that observers integrate information from facial emotional expressions with actions during social interactions.

Abstract TP14: SocialBit: A Novel Smartwatch Sensor to Detect Social Interaction Frequency in Stroke Survivors

Introduction: Stroke survivors experience high levels of social isolation which hinder their rehabilitation and well-being. A gap in the field that impedes intervention development is a lack of real-time social interaction measurement tailored for this population. In response, we developed SocialBit, a smartwatch-based machine learning algorithm to discreetly and passively detect social interactions through privacy-preserving acoustic analysis. In this observational study, we estimated the accuracy of SocialBit to detect social interactions compared to human observers in stroke survivors. Methods: SocialBit was built as a neural network machine learning algorithm using features from YAMNet and a Transformer classifier. It was then trained and tested in stroke survivors for up to 8 days in hospital. Independently, human observers tallied the presence or absence of social interactions every minute to establish the ground truth. SocialBit performance of detecting social interactions was compared against the ground truth. Results: We analyzed 1137.7 hours of ambient audio data in 90 patients within 14 days of stroke onset. 16% of patients had aphasia, 25% reported symptoms of clinical depression, and 28% reported moderate to severe loneliness. SocialBit had an accuracy of 0.83 (SD = 0.2), sensitivity of 0.86 (SD = 0.2), specificity of 0.81 (SD = 0.02), and an area under the curve (AUC) of 0.90 (SD = 0.2) (Figure 1). Conclusion: SocialBit accurately detected the frequency of social interactions in stroke survivors in a hospital setting. These results will drive future planned studies to examine SocialBit's performance in patients’ home communities. SocialBit shows the potential to be the first real-time social sensor to quantify social interactions and social isolation in stroke survivors.

An Active-Inference Approach to Second-Person Neuroscience

Social neuroscience has often been criticized for approaching the investigation of the neural processes that enable social interaction and cognition from a passive, detached, third-person perspective, without involving any real-time social interaction. With the emergence of second-person neuroscience, investigators have uncovered the unique complexity of neural-activation patterns in actual, real-time interaction. Social cognition that occurs during social interaction is fundamentally different from that unfolding during social observation. However, it remains unclear how the neural correlates of social interaction are to be interpreted. Here, we leverage the active-inference framework to shed light on the mechanisms at play during social interaction in second-person neuroscience studies. Specifically, we show how counterfactually rich mutual predictions, real-time bodily adaptation, and policy selection explain activation in components of the default mode, salience, and frontoparietal networks of the brain, as well as in the basal ganglia. We further argue that these processes constitute the crucial neural processes that underwrite bona fide social interaction. By placing the experimental approach of second-person neuroscience on the theoretical foundation of the active-inference framework, we inform the field of social neuroscience about the mechanisms of real-life interactions. We thereby contribute to the theoretical foundations of empirical second-person neuroscience.

Tuning in to real-time social interactions in macaques

Movement synchronization between individuals has been implicated in reinforcing their cohesion. How might such interindividual motor entrainment be controlled by the social brain? The answer remains elusive owing largely to the lack of suitable animal models in which direct neural recordings are available. Here, we show that macaque monkeys exhibit social motor entrainment without human prompting. We found that repetitive arm movements for horizontal bar sliding were phase coherent between two monkeys. The nature of motor entrainment was specific to animal pairs, consistent across days, dependent on visual inputs, and affected by social hierarchy. Notably, the entrainment was diminished when paired with prerecorded movies of a monkey making the same movements or bar motion alone. These findings demonstrate that motor entrainment is facilitated by real-time social exchanges, providing a behavioral platform to study the neural basis of potentially evolutionarily conserved mechanisms that support group cohesion.

Feeling Small or Standing Tall? Height Manipulation Affects Speech Anxiety and Arousal in Virtual Reality.

Social performance situations often constitute one of the most challenging communication tasks across different cultures. In today's work environments, giving presentations and performing in front of others are often essential and expected. Therefore, public speaking anxiety can have a serious impact on an individual's job performance, career choice, and prospects. Contemporary consumer virtual reality hardware has made it possible to practice public speaking anywhere in a safe and private virtual reality environment (VRE). As VREs offer the means to practice real-life scenarios, they also make it possible to go beyond what is "real"; to replace simulations with more dynamic and innovative training environments. Furthermore, with occupational life undergoing a significant shift toward technology-mediated working conditions, innovative tools and methods could also be used during virtually implemented real-time social interactions. This research aimed to study the ways in which an illusion of height, that is, perceived tallness versus perceived shortness, without any visible virtual body or representation, influences state speech anxiety and emotional responses of participants during simulation of a stressful speech task. The experiment followed a strictly controlled between-subject procedure, and both self-reported and psychophysiological data were collected. Results indicate that participants perceiving the illusion of tallness felt less anxious and had lower self-reported arousal compared with participants with the illusion of shortness. This implies that even simple, visual, first-person perspective manipulation of the VRE could help individuals to reduce their stress responses during a task-oriented situation.

Cortico-Cerebellar neurodynamics during social interaction in Autism Spectrum Disorders

BackgroundExploring neural network dynamics during social interaction could help to identify biomarkers of Autism Spectrum Disorders (ASD). A cerebellar involvement in autism has long been suspected and recent methodological advances now enable studying cerebellar functioning in a naturalistic setting. Here, we investigated the electrophysiological activity of the cerebro-cerebellar network during real-time social interaction in ASD. We focused our analysis on theta oscillations (3–8 Hz), which have been associated with large-scale coordination of distant brain areas and might contribute to interoception, motor control, and social event anticipation, all skills known to be altered in ASD. MethodsWe combined the Human Dynamic Clamp, a paradigm for studying realistic social interactions using a virtual avatar, with high-density electroencephalography (HD-EEG). Using source reconstruction, we investigated power in the cortex and the cerebellum, along with coherence between the cerebellum and three cerebral-cortical areas, and compared our findings in a sample of participants with ASD (n = 107) and with typical development (TD) (n = 33). We developed an open-source pipeline to analyse neural dynamics at the source level from HD-EEG data. ResultsIndividuals with ASD showed a significant increase in theta band power over the cerebellum and the frontal and temporal cortices during social interaction compared to resting state, along with significant coherence increases between the cerebellum and the sensorimotor, frontal and parietal cortices. However, a phase-based connectivity measure did not support a strict activity increase in the cortico-cerebellar functional network. We did not find any significant differences between the ASD and the TD group. ConclusionsThis exploratory study uncovered increases in the theta band activity of participants with ASD during social interaction, pointing at the presence of neural interactions between the cerebellum and cerebral networks associated with social cognition. It also emphasizes the need for complementary functional connectivity measures to capture network-level alterations. Future work will focus on optimizing artifact correction to include more participants with TD and increase the statistical power of group-level contrasts.

Ecological and Convergent Validity of Experimentally and Dynamically Assessed Capacity for Social Contingency Detection Using the Perceptual Crossing Experiment in Adolescence.

The Perceptual Crossing Experiment (PCE) captures the capacity for social contingency detection using real-time social interaction dynamics but has not been externally validated. We tested ecological and convergent validity of the PCE in a sample of 208 adolescents from the general population, aged 11 to 19 years. We expected associations between PCE performance and (a) quantity and quality of social interaction in daily life, using Experience Sampling Methodology (ESM; ecological validity) and (b) self-reported social skills using a questionnaire (convergent validity). We also expected PCE performance to better explain variance in ESM social measures than self-reported social skills. Multilevel analyses showed that only self-reported social skills were positively associated with social experience of company in daily life. These initial results do not support ecological and convergent validity of the PCE. However, fueled by novel insights regarding the complexity of capturing social dynamics, we identified promising methodological advances for future validation efforts.

Examining the effect of seat location on students’ real-time social interactions in a smart classroom using experience sampling method

Examining the effect of seat location on students’ real-time social interactions in a smart classroom using experience sampling method

Greater Social Competence Is Associated With Higher Interpersonal Neural Synchrony in Adolescents With Autism.

Difficulty engaging in reciprocal social interactions is a core characteristic of autism spectrum disorder. The mechanisms supporting effective dynamic real-time social exchanges are not yet well understood. This proof-of-concept hyperscanning electroencephalography study examined neural synchrony as the mechanism supporting interpersonal social interaction in 34 adolescents with autism spectrum disorder (50% female), age 10–16 years, paired with neurotypical confederates of similar age. The degree of brain-to-brain neural synchrony was quantified at temporo-parietal scalp locations as the circular correlation of oscillatory amplitudes in theta, alpha, and beta frequency bands while the participants engaged in a friendly conversation. In line with the hypotheses, interpersonal neural synchrony was significantly greater during the social interaction compared to the baseline. Lower levels of synchrony were associated with increased behavioral symptoms of social difficulties. With regard to sex differences, we found evidence for stronger interpersonal neural synchrony during conversation than baseline in females with autism, but not in male participants, for whom such condition differences did not reach statistical significance. This study established the feasibility of hyperscanning during real-time social interactions as an informative approach to examine social competence in autism, demonstrated that neural coordination of activity between the interacting brains may contribute to social behavior, and offered new insights into sex-related variability in social functioning in individuals with autism spectrum disorders.

Three-year-olds’ Perspective-taking in Social Interactions: Relations with Socio-cognitive Skills

ABSTRACT Understanding others’ perspectives and integrating this knowledge in social interactions is challenging for young children; even adults struggle with this skill. While young children show the capacity to understand what others can and cannot see under supportive laboratory conditions, more research is necessary to understand how children implement their perspective-taking (PT) skill during interactions and which socio-cognitive skills support their ability to do so. This preregistered study examined children’s Level 1 visual PT in a real-time social interaction and tested whether social-cognitive skills (focusing on inhibition of imitation) predicted PT. Thirty-six 3-year-old children (mean age: 37.3 months) participated in a PT task and responded implicitly (via eye gaze) and explicitly (via toy choice) to situations where their communicative partner could see some objects but not others. Three-year-olds demonstrated sensitivity to another’s perspective via implicit responses, but did not consistently take their partner’s perspective into account in their actions when considering objects their partner could not see. Contrary to adult findings, children who struggled to inhibit imitating (those more affected by another’s actions) demonstrated better PT, again when considering objects outside their partner’s sight. Thus, 3-year-olds’ sensitivity to others’ perspectives was robust, while acting on PT knowledge may still be developing; further, children more affected by another’s actions demonstrated improved PT skills.

Neural responses to social acceptance predict behavioral adjustments following peer feedback in the context of a real-time social interaction task.

Strong social connections are important predictors of both mental and physical health. The ability to effectively process social feedback from other people and adjust behavior accordingly is a critical part of skillfully navigating the social landscape. However, relatively few studies have considered neural systems driving these behavioral adjustments. In this study, 254 participants engaged in a peer interaction game while electroencephalogram (EEG) was recorded. In this game, participants repeatedly "interact" with a small set of virtual peers over a series of rounds, in which they provide feedback to their peers, and receive feedback from them in turn. A reward-sensitive event-related potential called the Reward Positivity (RewP) was extracted from the EEG following positive feedback from peers, and multilevel modeling was used to examine whether the RewP moderated associations between the feedback participants received during the task and their subsequent behavior. Participants were more likely to accept coplayers who had previously voted to accept them, and to like coplayers who had voted to accept them on the same round. A larger RewP was associated with a stronger tendency to modify behavior following feedback from peers, both in terms of voting behavior and expressions of liking. These data suggest that initial neural responsiveness to reward within 300ms of positive social feedback may guide social behaviors. Thus, this line of research represents an important step toward a more complete understanding of the ways in which neural responses to feedback are involved in human social behaviors.

Interactive Psychometrics for Autism With the Human Dynamic Clamp: Interpersonal Synchrony From Sensorimotor to Sociocognitive Domains.

The human dynamic clamp (HDC) is a human–machine interface designed on the basis of coordination dynamics for studying realistic social interaction under controlled and reproducible conditions. Here, we propose to probe the validity of the HDC as a psychometric instrument for quantifying social abilities in children with autism spectrum disorder (ASD) and neurotypical development. To study interpersonal synchrony with the HDC, we derived five standardized scores following a gradient from sensorimotor and motor to higher sociocognitive skills in a sample of 155 individuals (113 participants with ASD, 42 typically developing participants; aged 5 to 25 years; IQ > 70). Regression analyses were performed using normative modeling on global scores according to four subconditions (HDC behavior “cooperative/competitive,” human task “in-phase/anti-phase,” diagnosis, and age at inclusion). Children with ASD had lower scores than controls for motor skills. HDC motor coordination scores were the best candidates for stratification and diagnostic biomarkers according to exploratory analyses of hierarchical clustering and multivariate classification. Independently of phenotype, sociocognitive skills increased with developmental age while being affected by the ongoing task and HDC behavior. Weaker performance in ASD for motor skills suggests the convergent validity of the HDC for evaluating social interaction. Results provided additional evidence of a relationship between sensorimotor and sociocognitive skills. HDC may also be used as a marker of maturation of sociocognitive skills during real-time social interaction. Through its standardized and objective evaluation, the HDC not only represents a valid paradigm for the study of interpersonal synchrony but also offers a promising, clinically relevant psychometric instrument for the evaluation and stratification of sociomotor dysfunctions.

Social Interaction and Pain Threshold in Virtual Reality.

This experiment examined the effects of social presence and perceived location of a virtual environment on participants' pain thresholds in a preregistered, within-subjects experiment. First, we examined the effects of social interaction versus being alone in a virtual environment. Second, we compared a virtual environment representing a remote location to a replication of the laboratory environment. Social interaction predicted increased pain tolerance, but there was no effect of the "location" of the virtual environment. To our knowledge, this research project is the first to use real-time social interaction in virtual reality as a distractor for experimental thermal pain, and the first to examine the potential interaction between social interaction and transportation to different virtual locations. While this task is not directly analogous to the experience of pain in a medical setting, this preliminary study indicates future avenues for patient treatment. Clinical Trial Registration number: 1701006910.

Neural Correlates of Group Versus Individual Problem Solving Revealed by fMRI.

Group problem solving is a prototypical complex collective intellectual activity. Psychological research provides compelling evidence that problem solving in groups is both qualitatively and quantitatively different from doing so alone. However, the question of whether individual and collective problem solving involve the same neural substrate has not yet been addressed, mainly due to methodological limitations. In the current study, functional magnetic resonance imaging was performed to compare brain activation when participants solved Raven-like matrix problems in a small group and individually. In the group condition, the participant in the scanner was able to discuss the problem with other team members using a special communication device. In the individual condition, the participant was required to think aloud while solving the problem in the silent presence of the other team members. Greater activation was found in several brain regions during group problem solving, including the medial prefrontal cortex; lateral parietal, cingulate, precuneus and retrosplenial cortices; frontal and temporal poles. These areas have been identified as potential components of the so-called “social brain” on the basis of research using offline judgments of material related to socializing. Therefore, this study demonstrated the actual involvement of these regions in real-time social interactions, such as group problem solving. However, further connectivity analysis revealed that the social brain components are co-activated, but do not increase their coupling during cooperation as would be suggested for a holistic network. We suggest that the social mode of the brain may be described instead as a re-configuration of connectivity between basic networks, and we found decreased connectivity between the language and salience networks in the group compared to the individual condition. A control experiment showed that the findings from the main experiment cannot be entirely accounted for by discourse comprehension. Thus, the study demonstrates affordances provided by the presented new technique for neuroimaging the “group mind,” implementing the single-brain version of the second-person neuroscience approach.

Interpersonal Distance During Real-Time Social Interaction: Insights From Subjective Experience, Behavior, and Physiology.

Physical distance is a prominent feature in face-to-face social interactions and allows regulating social encounters. Close interpersonal distance (IPD) increases emotional responses during interaction and has been related to avoidance behavior in social anxiety. However, a systematic investigation of the effects of IPD on subjective experience combined with measures of physiological arousal and behavioral responses during real-time social interaction has been missing. Virtual Reality allows for a controlled manipulation of IPD while maintaining naturalistic social encounters. The present study investigates IPD in social interaction using a novel paradigm in Virtual Reality. Thirty-six participants approached virtual agents and engaged in short interactions. IPD was varied between 3.5 and 1 m by manipulating the distance at which agents reacted to the participant's approach. Closer distances were rated as more arousing, less pleasant, and less natural than longer distances and this effect was significantly modulated by social anxiety scores. Skin conductance responses were also increased at short distances compared to longer distances. Finally, an interaction of IPD and social anxiety was observed for avoidance behavior, measured as participants' backward motion during interaction, with stronger avoidance related to close distances and high values of social anxiety. These results highlight the influence of IPD on experience, physiological response, and behavior during social interaction. The interaction of social anxiety and IPD suggests including the manipulation of IPD in behavioral tests in Virtual Reality as a promising tool for the treatment of social anxiety disorder.

Two-Person Approaches to Studying Social Interaction in Psychiatry: Uses and Clinical Relevance.

Social interaction is ubiquitous in human society. The two-person approach—a new, powerful tool to study information exchange and social behaviors—aims to characterize the behavioral dynamics and neural mechanisms of real-time social interactions. In this review, we discuss the benefits of two-person approaches compared to those for conventional, single-person approaches. We describe measures and paradigms that model social interaction in three dimensions (3-D), including eye-to-eye, body-to-body, and brain-to-brain relationships. We then discuss how these two-person measures and paradigms are used in psychiatric conditions (e.g., autism, mood disorders, schizophrenia, borderline personality disorder, and psychotherapy). Furthermore, the advantages of a two-person approach (e.g., dual brain stimulation, multi-person neurofeedback) in clinical interventions are described. Finally, we discuss the methodological and translational challenges surrounding the application of two-person approaches in psychiatry, as well as prospects for future two-/multi-person studies. We conclude that two-person approaches serve as useful additions to the range of behavioral and neuroscientific methods available to assess social interaction in psychiatric settings, for both diagnostic techniques and complementary interventions.

Investigating real-time social interaction in pairs of adolescents with the Perceptual Crossing Experiment.

The study of real-time social interaction provides ecologically valid insight into social behavior. The objective of the current research is to experimentally assess real-time social contingency detection in an adolescent population, using a shortened version of the Perceptual Crossing Experiment (PCE). Pairs of 148 adolescents aged between 12 and 19 were instructed to find each other in a virtual environment interspersed with other objects by interacting with each other using tactile feedback only. Across six rounds, participants demonstrated increasing accuracy in social contingency detection, which was associated with increasing subjective experience of the mutual interaction. Subjective experience was highest in rounds when both participants were simultaneously accurate in detecting each other's presence. The six-round version yielded comparable social contingency detection outcome measures to a ten-round version of the task. The shortened six-round version of the PCE has therefore enabled us to extend the previous findings on social contingency detection in adults to an adolescent population, enabling implementation in prospective research designs to assess the development of social contingency detection over time.

Using second-person neuroscience to elucidate the mechanisms of social interaction.

Although a large proportion of our lives are spent participating in social interactions, the investigation of the neural mechanisms supporting these interactions has largely been restricted to situations of social observation - that is, situations in which an individual observes a social stimulus without opportunity for interaction. In recent years, efforts have been made to develop a truly social, or 'second-person', neuroscientific approach to these investigations in which neural processes are examined within the context of a real-time reciprocal social interaction. These developments have helped to elucidate the behavioural and neural mechanisms of social interactions; however, further theoretical and methodological innovations are still needed. Findings to date suggest that the neural mechanisms supporting social interaction differ from those involved in social observation and highlight a role of the so-called 'mentalizing network' as important in this distinction. Taking social interaction seriously may also be particularly important for the advancement of the neuroscientific study of different psychiatric conditions.