Hyperscanning Technique Research Articles

Dynamic brain networks in spontaneous gestural communication

Gestures accent and illustrate our communication. Although previous studies have uncovered the positive effects of gestures on communication, little is known about the specific cognitive functions of different types of gestures, or the instantaneous multi-brain dynamics. Here we used the fNIRS-based hyperscanning technique to track the brain activity of two communicators, examining regions such as the PFC and rTPJ, which are part of the mirroring and mentalizing systems. When participants collaboratively solved open-ended realistic problems, we characterised the dynamic multi-brain states linked with specific social behaviours. Results demonstrated that gestures are associated with enhanced team performance, and different gestures serve distinct cognitive functions: interactive gestures are accompanied by better team originality and a more efficient inter-brain network, while fluid gestures correlate with individual cognitive fluency and efficient intra-brain states. These findings reveal a close association between social behaviours and multi-brain networks, providing a new way to explore the brain-behaviour relationship.

Relationship evolution shapes inter-brain synchrony in affective sharing: The role of self-expansion.

The development of social relationships influences a person's self-concept, which in turn affects their perceptions and neural correlates in social interactions. This study employed an EEG-based hyperscanning technique and a longitudinal design to investigate how the evolution of interpersonal relationships impacts inter-brain synchrony during nonverbal social-emotional interactions. The framework for this study is based on the self-expansion model. We found that dyads exhibited enhanced affective sharing abilities and increased brain-to-brain synchrony, particularly in the gamma rhythm across the frontal, parietal, and left temporoparietal regions, after seven months together compared to when they first met. Additionally, the results indicate that inter-brain coupling evolves as relationships develop, with synchrony in nonverbal social-emotional interactions increasing as self-expansion progresses. Crucially, in the deep learning model, interpersonal closeness can be successfully classified by inter-brain synchrony during emotional-social interactions. The longitudinal EEG-hyperscanning design of our study allows for capturing dynamic changes over time, offering new insights into the neurobiological foundations of social interaction and the potential of neural synchrony as a biomarker for relationship dynamics.

Neural mechanisms distinguishing two types of cooperative problem-solving approaches: An fNIRS hyperscanning study

Collaborative cooperation (CC) and division of labor cooperation (DLC) are two prevalent forms of cooperative problem-solving approaches in daily life. Despite extensive research on the neural mechanisms underlying cooperative problem-solving approaches, a notable gap exists between the neural processes that support CC and DLC. The present study utilized a functional near-infrared spectroscopy (fNIRS) hyperscanning technique along with a classic cooperative tangram puzzle task to investigate the neural mechanisms engaged by both friends and stranger dyads during CC versus DLC. The key findings of this study were as follows: (1) Dyads exhibited superior behavioral performance in the DLC task than in the CC task. The CC task bolstered intra-brain functional connectivity and inter-brain synchrony (IBS) in regions linked to the mirror neuron system (MNS), spatial perception (SP) and cognitive control. (2) Friend dyads showed stronger IBS in brain regions associated with the MNS than stranger dyads. (3) Perspective-taking predicted not only dyads’ behavioral performance in the CC task but also their IBS in brain regions associated with SP during the DLC task. Taken together, these findings elucidate the divergent behavioral performance and neural connection patterns between the two cooperative problem-solving approaches. This study provides novel insights into the various neurocognitive processes underlying flexible coordination strategies in real-world cooperative contexts.

Event-Related Potential-Based Collaborative Brain-Computer Interface for Augmenting Human Performance Using a Low-Cost, Custom Electroencephalogram Hyperscanning Infrastructure

The electroencephalogram (EEG) hyperscanning technique has been demonstrated to facilitate the applicability of a collaborative brain-computer interface (cBCI) to augment human performance with respect to the collective intelligence of multiple brains. However, assembling a hyperscanning platform with commercial products inevitably introduces practical and cost burdens regarding labor and hardware setup, hindering group scalability. This work thus explores how effectively a low-cost, custom-made EEG hyperscanning platform can be achieved by demonstrating a cBCI framework. This work quantifies BCI performance in collaborative and single-brain scenarios applied to the EEG dataset collected from three subjects simultaneously participating in a target-distractor differentiation task over 10 days. This work also compares various brain-fusion scenarios with feature-extraction methods for multiple brains. Given the 30 pseudo brains (i.e., 3 subjects × 10 day sessions), the decision-level committee voting outperformed the single-brain BCI performance and considerably improved by leveraging more pseudo brains. The 27-brain setting achieved the best information transfer rate (ITR) of 116.6±5.6 bits/min, which was a nearly 817% enhancement over the single-brain ITR (12.7±9.2 bits/min). In addition, the cBCI decision augmented the actual button-pressing time by 25 ms. Such a low-cost, custom-made hyperscanning infrastructure economically and practically favors multiple-brain applications in a larger group.

Influence of interpersonal distance on collaborative performance in the joint Simon task—An fNIRS-based hyperscanning study

Collaboration is a critical skill in everyday life. It has been suggested that collaborative performance may be influenced by social factors such as interpersonal distance, which is defined as the perceived psychological distance between individuals. Previous literature has reported that close interpersonal distance may promote the level of self-other integration between interacting members, and in turn, enhance collaborative performance. These studies mainly focused on interdependent collaboration, which requires high levels of shared representations and self-other integration. However, little is known about the effect of interpersonal distance on independent collaboration (e.g., the joint Simon task), in which individuals perform the task independently while the final outcome is determined by the parties. To address this issue, we simultaneously measured the frontal activations of ninety-four pairs of participants using a functional near-infrared spectroscopy (fNIRS)-based hyperscanning technique while they performed a joint Simon task. Behavioral results showed that the Joint Simon Effect (JSE), defined as the RT difference between incongruent and congruent conditions indicating the level of self-other integration between collaborators, was larger in the friend group than in the stranger group. Consistently, the inter-brain neural synchronization (INS) across the dorsolateral and medial parts of the prefrontal cortex was also stronger in the friend group. In addition, INS in the left dorsolateral prefrontal cortex negatively predicted JSE only in the friend group. These results suggest that close interpersonal distance may enhance the shared mental representation among collaborators, which in turn influences their collaborative performance.

The role of mindfulness on theta inter-brain synchrony during cooperation feedback processing: An EEG-based hyperscanning study

Mindfulness appears to improve empathy and understanding in relationships, which are necessary for successful cooperation. However, the impact of mindfulness on cooperation has not been fully studied. This study used hyperscanning technique to examine the effect of mindfulness on the inter-brain synchrony of interacting individuals during the cooperative tasks. Forty-one dyads were randomly assigned to a mindfulness group or a non-mindfulness group. Dyads of the mindfulness group performed a short mindfulness exercise following a 15-minute mindfulness audio guidance. Dyads of the non-mindfulness group were instructed to rest quietly with their eyes closed. Then, simultaneously and continuously EEG was recorded from all dyads when they completed a computer-based cooperative game task. Reaction times (RTs) and success rates were used to indicate the behavioral performance, and phase locking value (PLV) was used to indicate the inter-brain synchrony. The results showed that (1) Greater theta inter-brain synchrony during the cooperative computer game tasks was observed in the mindfulness group than in the non-mindfulness group; (2) Greater theta inter-brain synchrony was observed in the successful cooperation conditions as compared to those in the failure cooperation conditions; (3) Greater theta inter-brain synchrony was observed at the frontal region as compared to those at the parietal-occipital region in the successful cooperation condition. The results expand the neural basis of the effects of mindfulness on cooperation feedback processing.

The effect of speech-gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication.

In everyday face-to-face communication, speakers use speech to transfer information and rely on co-occurring nonverbal cues, such as hand and facial gestures. The integration of speech and gestures facilitates both language comprehension and the skill of the theory of mind. Consecutive dialogue interpreting (DI) allows dyads of different linguistic backgrounds to communicate with each other. The interpreter interprets after the interlocutor has finished a turn, so the interlocutor watches the gesture first and hears the target language a few seconds later, resulting in speech-gesture asynchrony. In this study, we used the functional near-infrared spectroscopy hyperscanning technique to investigate the influence of speech-gesture asynchrony on different levels of communication. Twenty groups were recruited for the DI experiments. The results showed that when the interpreter performed consecutive interpreting, the time-lagged neural coupling at the temporoparietal junction decreased compared to simultaneous interpreting. It suggests that speech-gesture asynchrony significantly weakened the ability of interlocutors to understand each other's mental state, and the decreased neural coupling was significantly correlated with the interpreter's interpretation skill. In addition, the time-aligned neural coupling at the left inferior frontal gyrus increased, which suggests that, as compensation, the interlocutor's verbal working memory increases in line with the communication process.

FNIRS Hyperscanning of Right-brain Synchronization between Therapist-client in Sandplay Therapy

This study investigated inter-right brain synchrony between therapist and client in Sandplay therapy, using hyperscanning technique based on functional near-infrared spectroscopy (fNIRS). fNIRS is a non-intrusive method that measures changes in oxyhemoglobin the cerebral blood. A total of seven therapist-client pairs-i.e., 14 participants-wore fNIRS devices on their heads and engaged in two sessions of Sandplay therapy, with each session lasting for 30 minutes. The study observed synchronization in the right and left prefrontal cortices of both therapists and clients in all seven pairs, during every session. Interestingly enough, synchronization occurred not only while the pairs engaged in verbal communication about the completed sandpicture but also during the non-verbal process of clients' creating sandpictures. The outcome of the study hence suggests neurobiological fundamentals for the therapeutic relationship between therapist and client, which is also called the therapeutic resonance, relational mutual regression, therapeutic alliance, and mother-child unity.

Close spatial distance and direct gaze bring better communication outcomes and more intertwined neural networks

Non-verbal cues tone our communication. Previous studies found that non-verbal factors, such as spatial distance and gaze direction, significantly impact interpersonal communication. However, little is known about the behind multi-brain neural correlates and whether it could affect high-level creative group communication. Here, we provided a new, scalable, and neuro-based approach to explore the effects of non-verbal factors on different communication tasks, and revealed the underlying multi-brain neural correlates using fNIRS-based hyperscanning technique. Across two experiments, we found that closer spatial distance and more direct gaze angle could promote collaborative behaviors, improve both creative and non-creative communication outcomes, and enhance inter-brain neural synchronization. Moreover, compared to the non-creative communication task, participants’ inter-brain network was more intertwined when performing the creative communication task. These findings suggest that close spatial distance and direct gaze serve as positive social cues, bringing interacting brains into alignment and optimizing inter-brain information transfer, thus improving communication outcomes.

The presence of adjacent others facilitates interpersonal neural synchronization in the left prefrontal cortex during a simple addition task

The hyperscanning technique, that is, simultaneous measurement of neural signals in more than one person, is a powerful research tool for understanding humans’ social interactions. In recent years, many studies have investigated interpersonal neural synchronization during various types of communication processes. However, there has been little focus on the impact of the presence of others without explicit social interaction, despite the mere presence of others having been suggested as influencing behavior. In this study, we clarify whether neural signals during a self-paced, repeated, addition task are synchronized when another individual is adjacent without direct interaction. Twenty pairs of participants were measured using a hyperscanning approach with near-infrared spectroscopy. The results show that interpersonal neural synchronization of the task-related signal in the left forehead region was enhanced under the condition of being adjacent to another participant. By contrast, a significant decrease in neural synchronization in the center of the forehead region, where increased neural synchronization is often reported in explicit communication, was observed. Thus, the results indicate that the adjacency of others modulates interpersonal neural synchronization in the task-related signal, and the effect on cognitive processing is different from that of explicit social interaction.

Distinct neural couplings to shared goal and action coordination in joint action: evidence based on fNIRS hyperscanning.

Joint action is central to human nature, enabling individuals to coordinate in time and space to achieve a joint outcome. Such interaction typically involves two key elements: shared goal and action coordination. Yet, the substrates entrained to these two components in joint action remained unclear. In the current study, dyads performed two tasks involving both sharing goal and action coordination, i.e. complementary joint action and imitative joint action, a task only involving shared goal and a task only involving action coordination, while their brain activities were recorded by the functional near-infrared spectroscopy hyperscanning technique. The results showed that both complementary and imitative joint action (i.e. involving shared goal and action coordination) elicited better behavioral performance than the task only involving shared goal/action coordination. We observed that the interbrain synchronization (IBS) at the right inferior frontal cortex (IFC) entrained more to shared goal, while left-IFC IBS entrained more to action coordination. We also observed that the right-IFC IBS was greater during completing a complementary action than an imitative action. Our results suggest that IFC plays an important role in joint action, with distinct lateralization for the sub-components of joint action.

Inter-brain neural mechanism underlying turn-based interaction under acute stress in women: a hyperscanning study using functional near-infrared spectroscopy.

With the ever-changing social environment, stress has exerted a substantial influence on social interaction. The present study examined the underlying cognitive and neural mechanism on how acute stress affected the real-time cooperative and competitive interaction with four hypothesized path models. We used the hyperscanning technique based on functional near-infrared spectroscopy (fNIRS) device to examine brain-to-brain coherence within the dyads engaging Pattern Game under acute stress manipulated through Trier Social Stress Test for Groups. Behavioral results showed stressed dyads exhibited better cooperative performance and higher self-other overlap level during the cooperative session than dyads in the control group. The fNIRS results identified higher interpersonal brain synchronization in the right temporal-parietal junction (r-TPJ) stronger Granger causality from partner-to-builder during the cooperative session in the stress group when compared with the control group. Our results corroborated better performance in the cooperative context and further identified that brain-to-brain coherence in r-TPJ and self-other overlap serially mediated the effect of acute stress on cooperative performance.

Suffer together, bond together: Brain-to-brain synchronization and mutual affective empathy when sharing painful experiences

Previous behavioral studies have shown that sharing painful experiences can strengthen social bonds and promote mutual prosociality, yet the neural mechanisms underlying this phenomenon remain unclear. We hypothesized that sharing a painful experience induces brain-to-brain synchronization and mutual empathy for each other's pain between pain-takers and pain-observers, which then leads to enhanced social bonding. To test this hypothesis, we adopted an electroencephalographic (EEG) hyper-scanning technique to assess neuronal and behavioral activity during a Pain-Sharing task in which high- or low-intensity pain stimulation was randomly delivered to one participant of a dyad on different experimental trials. Single-brain analysis showed that sensorimotor α-oscillation power was suppressed more when expecting high-intensity pain than when expecting low-intensity pain similarly for self-directed or partner-directed pain. Dual-brain analysis revealed that expecting high-intensity pain induced greater brain-to-brain synchronization of sensorimotor α-oscillation phases between pain-takers and pain-observers than did expecting low-intensity pain. Mediation analysis further revealed that brain-to-brain synchronization of sensorimotor α-oscillations mediated the effects of pain-stimulation intensity on mutual affective sharing for partner-directed pain. This mutual affective empathy during the task predicted the social bonding, as indexed by prosocial inclinations measured after the task. These results support the hypothesis that sharing a painful experience triggers emotional resonance between pairs of individuals through brain-to-brain synchronization of neuronal α-oscillations recorded over the sensorimotor cortex, and this emotional resonance further strengthens social bonds and motivates prosocial behavior within pairs of individuals.

Inter-brain amplitude correlation differentiates cooperation from competition in a motion-sensing sports game.

Cooperation and competition are two basic modes of human interaction. Their underlying neural mechanisms, especially from an interpersonal perspective, have not been fully explored. Using the electroencephalograph-based hyperscanning technique, the present study investigated the neural correlates of both cooperation and competition within the same ecological paradigm using a classic motion-sensing tennis game. Both the inter-brain coupling (the inter-brain amplitude correlation and inter-brain phase-locking) and the intra-brain spectral power were analyzed. Only the inter-brain amplitude correlation showed a significant difference between cooperation and competition, with different spatial patterns at theta, alpha and beta frequency bands. Further inspection revealed distinct inter-brain coupling patterns for cooperation and competition; cooperation elicited positive inter-brain amplitude correlation at the delta and theta bands in extensive brain regions, while competition was associated with negative occipital inter-brain amplitude correlation at the alpha and beta bands. These findings add to our knowledge of the neural mechanisms of cooperation and competition and suggest the significance of adopting an inter-brain perspective in exploring the neural underpinnings of social interaction in ecological contexts.

Interpersonal brain synchronization under bluffing in strategic games.

People commonly use bluffing as a strategy to manipulate other people’s beliefs about them for gain. Although bluffing is an important part of successful strategic thinking, the inter-brain mechanisms underlying bluffing remain unclear. Here, we employed a functional near-infrared spectroscopy hyperscanning technique to simultaneously record the brain activity in the right temporal-parietal junction in 32 pairs of participants when they played a bluffing game against each other or with computer opponents separately. We also manipulated the penalty for bluffing (high vs low). Under the condition of high relative to low penalty, results showed a higher bluffing rate and a higher calling rate in human-to-human as compared to human-to-computer pairing. At the neural level, high relative to low penalty condition increased the interpersonal brain synchronization (IBS) in the right angular gyrus (rAG) during human-to-human as compared to human-to-computer interaction. Importantly, bluffing relative to non-bluffing, under the high penalty and human-to-human condition, resulted in an increase in response time and enhanced IBS in the rAG. Participants who bluffed more frequently also elicited stronger IBS. Our findings support the view that regions associated with mentalizing become synchronized during bluffing games, especially under the high penalty and human-to-human condition.

EEG-based approach for recognizing human social emotion perception

Social emotion perception plays an important role in our daily social interactions and is involved in the treatments for mental disorders. Hyper-scanning technique enables to measure brain activities simultaneously from two or more persons, which was employed in this study to explore social emotion perception. We analyzed the recorded electroencephalogram (EEG) to explore emotion perception in terms of event related potential (ERP) and phase synchronization, and classified emotion categories based on convolutional neural network (CNN). The results showed that (1) ERP was significantly different among four emotion categories (i.e., anger, disgust, neutral, and happy), but there was no significant difference for ERP in the comparison of rating orders (the order of rating actions of the paired participants); (2) the intra-brain phase lag index (PLI) was higher than the inter-brain PLI but its number of connections exhibiting significant difference was less in all typical frequency bands (from delta to gamma); (3) the emotion classification accuracy of inter-PLI-Conv outperformed that of intra-PLI-Conv for all cases of using each frequency band (five frequency bands totally). In particular, the classification accuracies averaged across all participants in the alpha band were 65.55% and 50.77% (much higher than the chance level) for the inter-PLI-Conv and intra-PLI-Conv, respectively. According to our results, the emotion category of happiness can be classified with a higher performance compared to the other categories.

Coordination Elicits Synchronous Brain Activity Between Co-actors: Frequency Ratio Matters.

People could behave in two different ways when engaging in interpersonal coordination activities: moving at the same frequency (isofrequency pattern, IP; the movement frequency ratio is 1:1) or at different frequencies (multifrequency pattern, MP; the movement frequency ratio is non 1:1). However, how the interpersonal coordination pattern modulates coordination outcome and the related brain-to-brain connectivity is not fully understood. Here, we adopted a continuous joint drawing task in which two participants co-drew parallelogram shapes according to two coordination patterns (i.e., IP vs. MP) while their brain activities were simultaneously recorded by the functional near-infrared spectroscopy (fNIRS) based hyperscanning technique. Dyads showed better coordination performance, as well as relatively greater interpersonal brain synchronization (IBS) at the left frontopolar area, in the MP condition compared to the IP condition. Granger causality analyses further disclosed the bidirectional influences between the brains of the coordinating individuals. Such interpersonal influences were enhanced when individuals coordinated in the MP condition. Finally, the IBS during coordination was related to the dyadic self-control level. Taken together, our study revealed that interpersonal multifrequency coordination pattern facilitates the coordination efficiency, which was associated with the enhanced brain-to-brain connectivity. Our work also suggests the potentially positive role of self-control during the interpersonal coordination process.

Autism Symptoms Modulate Interpersonal Neural Synchronization in Children with Autism Spectrum Disorder in Cooperative Interactions

Previous neuroscience studies exploring the neural mechanisms of social deficits of individuals with autism spectrum disorder (ASD) have mainly examined single participants' brain responses to pictures or video-clips displayed on a monitor from the perspective of a passive observer. The present study examined inter-brain communication between children with ASD and their parents in a socio-interactive context. We used a functional near-infrared spectroscopy (fNIRS)-based hyperscanning technique to simultaneously measure the prefrontal activations in 16 pairs of children with ASD and their parents in a two-person key-press task. The children's task was to press a key together with their parents in a cooperation condition when a "go" signal was present or to press a key as fast as possible under the observation by their parents in a single-person condition. We also measured children's severity of autism symptoms.We found that children with ASD showed increased interpersonal neural synchronization in the frontal cortex when engaging in cooperative interactions with their parents than when performing solo and non-interactive behaviors. Furthermore, this neural synchronization was modulated by the children's autism symptoms, which also covaried with their cooperation task performance. That is, children with severer autism symptoms showed lower level of action and neural synchronization with their parents during cooperation. Our study moved a major step forward in understanding the neural correlates underlying social deficits in ASD and provided important implications for the treatment and behavioral training of ASD.

Exploring the Connected Brain by fNIRS: Human-to-Human Interactions Engineering

Functional Near Infrared Spectroscopy (fNIRS) is a relatively new neuroimagingtechnique adequate and useful for exploring neural activity in social contexts involving humaninteractions. Compared to functional Magnetic Resonance Imaging (fMRI), fNIRS is easy-to-usesafe, noninvasive, silent, relatively low cost and portable, and applicable to subjects of all ages, thusresulting in a good option for ecological studies involving humans in their real-life context.Moreover, by using hyperscanning technique, fNIRS allows recording the hemodynamic cerebralactivity of two interacting subjects in an ecological context or during a shared performance. Thus,moving from a simple analysis about each subject’s neural response during joint actions towardsmore complex computations makes possible to investigate brain synchrony, that is the if and howone’s brain activity is related to that of another interacting partner simultaneously recorded. Here,we discuss how connectivity analyses, with respect to both time and frequency domain procedures,permitted to deepen some aspects of inter-brain synchrony in relation to emotional closeness, and tohighlight how concurrent, cooperative actions can lead to interpersonal synchrony and bondconstruction.

Praising or keeping silent on partner’s ideas: Leading brainstorming in particular ways

This study aimed to investigate how different feedback affect group creative performance, and reveal the underlying interpersonal neural correlates using the functional near-infrared spectroscopy (fNIRS)-based hyperscanning technique. Participants solved one creativity task with two strangers in conditions with positive/negative/no feedback. Results revealed that performance in the negative condition was lower than in the other conditions. Moreover, results showed the highest ‘index of convergence’/collective flexibility in the positive/control condition respectively. The fNIRS results demonstrated IBS increment in the frontopolar and bilateral dorsolateral prefrontal cortex, which was stronger in the positive and negative conditions. The IBS increment in the frontopolar and bilateral DLPFC covaried with group creative performance in the positive condition. The findings indicated that negative feedback suppressed the group creative performance; whereas no feedback facilitated collective flexibility and positive feedback promoted interpersonal interaction, these two feedback conditions both benefited group creative performance.