Gaze Perception Research Articles

Cognitive Mechanisms of Aberrant Self-Referential Social Perception in Psychosis and Bipolar Disorder: Insights From Computational Modeling.

Individuals with schizophrenia (SZ) and bipolar disorder (BD) show disruptions in self-referential gaze perception-a social perceptual process related to symptoms and functioning. However, our current mechanistic understanding of these dysfunctions and relationships is imprecise. The present study used mathematical modeling to uncover cognitive processes driving gaze perception abnormalities in SZ and BD, and how they relate to cognition, symptoms, and social functioning. We modeled the behavior of 28 SZ, 38 BD, and 34 controls (HC) in a self-referential gaze perception task using drift-diffusion models parameterized to index key cognitive components: drift rate (evidence accumulation efficiency), drift bias (perceptual bias), start point (expectation bias), threshold separation (response caution), and nondecision time (encoding/motor processes). Results revealed that aberrant gaze perception in SZ and BD was driven by less efficient evidence accumulation, perceptual biases predisposing self-referential responses, and greater caution (SZ only). Across SZ and HC, poorer social functioning was related to greater expectation biases. Within SZ, perceptual and expectancy biases were associated with hallucination and delusion severity, respectively. These findings indicate that diminished evidence accumulation and perceptual biases may underlie altered gaze perception in patients and that SZ may engage in compensatory cautiousness, sacrificing response speed to preserve accuracy. Moreover, biases at the belief and perceptual levels may relate to symptoms and functioning. Computational modeling can, therefore, be used to achieve a more nuanced, cognitive process-level understanding of the mechanisms of social cognitive difficulties, including gaze perception, in individuals with SZ and BD.

Enhancing social robot's direct gaze expression through vestibulo-ocular movements

Non-verbal communication, especially eye contact, plays a crucial role in human interaction. Integrating eye contact capabilities into robotic behavior presents challenges that require comprehensive interdisciplinary collaboration from the fields of vision science and robotics. This study introduces an innovative methodology to improve the user's perception of the tabletop social robot Haru's direct gaze by explaining how eye and head movements can be managed through simulating visual perception processing. By considering the mechanisms of eye and head movements during visual perception processing, the robot's eye and head gaze behaviors are intended to replicate human gaze movements with Vestibulo-Ocular Movement (VOM) during face-to-face interaction, ultimately enhancing the user's perception of the robot's direct gaze.

Cognitive Mechanisms of Aberrant Self-Referential Social Perception in Psychosis and Bipolar Disorder: Insights from Computational Modeling.

Individuals with schizophrenia (SZ) and bipolar disorder (BD) show disruptions in self-referential gaze perception-a social perceptual process related to symptoms and functioning. However, our current mechanistic understanding of these dysfunctions and relationships is imprecise. The present study used mathematical modeling to uncover cognitive processes driving gaze perception abnormalities in SZ and BD, and how they relate to cognition, symptoms, and social functioning. We modeled the behavior of 28 SZ, 38 BD, and 34 controls (HC) in a self-referential gaze perception task using drift-diffusion models (DDM) parameterized to index key cognitive components: drift rate (evidence accumulation efficiency), drift bias (perceptual bias), start point (expectation bias), threshold separation (response caution), and non- decision time (encoding/motor processes). Results revealed that aberrant gaze perception in SZ and BD was driven by less efficient evidence accumulation, perceptual biases predisposing self-referential responses, and greater caution (SZ only). Across SZ and HC, poorer social functioning was related to greater expectation biases. Within SZ, perceptual and expectancy biases were associated with hallucination and delusion severity, respectively. These findings indicate that diminished evidence accumulation and perceptual biases may underlie altered gaze perception in patients and that SZ may engage in compensatory cautiousness, sacrificing response speed to preserve accuracy. Moreover, biases at the belief and perceptual levels may relate to symptoms and functioning. Computational modeling can, therefore, be used to achieve a more nuanced, cognitive process-level understanding of the mechanisms of social cognitive difficulties, including gaze perception, in individuals with SZ and BD.

Mentalizing impairments and hypermentalizing bias in individuals with first-episode schizophrenia-spectrum disorder and at-risk mental state: the differential roles of neurocognition and social anxiety.

Mentalizing, or theory of mind (ToM), impairments and self-referential hypermentalizing bias are well-evident in schizophrenia. However, findings compared to individuals with at-risk mental states (ARMS) are inconsistent, and investigations into the relationship between social cognitive impairments and social anxiety in the two populations are scarce. This study aimed to examine and compare these deficits in first-episode schizophrenia-spectrum disorder (FES) and ARMS, and to explore potential specific associations with neurocognition and symptomatology. Forty patients with FES, 40 individuals with ARMS, and 40 healthy controls (HC) completed clinical assessments, a battery of neurocognitive tasks, and three social cognitive tasks. The comic strip and hinting tasks were used to measure non-verbal and verbal mentalizing abilities, and the gaze perception task was employed to assess self-referential hypermentalizing bias. FES and ARMS showed comparable mentalizing impairments and self-referential hypermentalizing bias compared to HC. However, only ambiguous self-referential gaze perception (SRGP) bias remained significantly different between three groups after controlling for covariates. Findings suggested that self-referential hypermentalizing bias could be a specific deficit and may be considered a potential behavioral indicator in early-stage and prodromal psychosis. Moreover, working memory and social anxiety were related to the social cognitive impairments in ARMS, whereas higher-order executive functions and positive symptoms were associated with the impairments in FES. The current study indicates the presence of stage-specific mechanisms of mentalizing impairments and self-referential hypermentalizing bias, providing insights into the importance of personalized interventions to improve specific neurocognitive domains, social cognition, and clinical outcomes for FES and ARMS.

Gazing Heads: Investigating Gaze Perception in Video-Mediated Communication

Videoconferencing has become a ubiquitous medium for collaborative work. It does suffer however from various drawbacks such as zoom fatigue. This paper addresses the quality of user experience by exploring an enhanced system concept with the capability of conveying gaze and attention. Gazing Heads is a round-table virtual meeting concept that uses only a single screen per participant. It enables direct eye contact, and signals gaze via controlled head rotation. The technology to realise this novel concept is not quite mature though, so we built a camera-based simulation for four simultaneous videoconference users. We conducted a user study comparing Gazing Heads with a conventional “Tiled View” video conferencing system, for 20 groups of 4 people, on each of two tasks. The study found that head rotation clearly conveys gaze and strongly enhances the perception of attention. Measurements of turn-taking behaviour did not differ decisively between the two systems (though there were significant differences between the two tasks). A novel insight in comparison to prior studies is that there was a significant increase in mutual eye contact with Gazing Heads, and that users clearly felt more engaged, encouraged to participate and more socially present. Overall, participants expressed a clear preference for Gazing Heads. These results suggest that fully implementing the Gazing Heads concept, using modern computer vision technology as it matures, could significantly enhance the experience of videoconferencing.

The modulation effect of cognition on mentalization in late-life depression: a study of gaze perception-a potential screening tool for high-risk group of late-life depression.

Impairment in mentalization is implicated in the development and maintenance of depression. Major depressive disorders showed significant impairment in social cognition and such impairment appears to be positively associated with the severity of depression. Self-referential gaze perception (SRGP), a measurement of mentalization, was predominantly measured in patients with psychosis but rarely examined in late-life depression (LLD). To assess the effect of cognition on the interpretation bias of mentalization, 29 LLD patients and 29 healthy controls were asked to judge if various gaze directions were directed to self in SRGP. Patients with better cognition showed less unambiguous-SRGP bias than those with worse cognitive scores; this difference was not found in healthy controls. Global cognition and executive function contributed to the SRGP rate in patients. The current study is the first study to explore the relationship between cognition and SRGP in the LLD population. Our study findings suggested that the cognitive function of LLD patients may contribute to the modulation of interpretation bias, which in turn underlie the role of SRGP bias. Greater SRGP bias in patients may reflect social cognition deterioration, impairing the social interaction and functioning of LLD patients. This highlights the need for early intervention and cognitive decline identification to facilitate better prognosis and treatment effectiveness; thus, further studies could navigate the potential of SRGP task as a screening tool for high-risk group of LLD likely to develop dementia.

The Effect of Eye Contact in Multi-Party Conversations with Virtual Humans and Mitigating the Mona Lisa Effect

The demand for kiosk systems with embodied conversational agents has increased with the development of artificial intelligence. There have been attempts to utilize non-verbal cues, particularly virtual human (VH) eye contact, to enable human-like interaction. Eye contact with VHs can affect satisfaction with the system and the perception of VHs. However, when rendered in 2D kiosks, the gaze direction of a VH can be incorrectly perceived, due to a lack of stereo cues. A user study was conducted to examine the effects of the gaze behavior of VHs in multi-party conversations in a 2D display setting. The results showed that looking at actual speakers affects the perceived interpersonal skills, social presence, attention, co-presence, and competence in conversations with VHs. In a second study, the gaze perception was further examined with consideration of the Mona Lisa effect, which can lead users to believe that a VH rendered on a 2D display is gazing at them, regardless of the actual direction, within a narrow range. We also proposed the camera rotation angle fine tuning (CRAFT) method to enhance the users’ perceptual accuracy regarding the direction of the VH’s gaze.The results showed that the perceptual accuracy for the VH gaze decreased in a narrow range and that CRAFT could increase the perceptual accuracy.

Social visual attention as a treatment outcome: evaluating the social games for autistic adolescents (SAGA) intervention

A core feature of autism involves difficulty perceiving and interpreting eye gaze shifts as nonverbal communicative signals. A hypothesis about the origins of this phenotype is that it emerges from developmentally different social visual attention (SVA). We developed Social Games for Autistic Adolescents (SAGA; Scherf et al. BMJ Open 8(9):e023682, 2018) as a serious game intervention for autistic individuals to discover the significance of eye gaze cues. Previously, we demonstrated the effectiveness of SAGA to improve the perception and understanding of eye gaze cues and social skills for autistic adolescents (Griffin et al. JCPP Adv 1(3):e12041, 2021). Here, we determine whether increases in social visual attention to faces and/or target gazed-at objects, as measured via eye tracking during the same Gaze Perception task in the same study sample, moderated this improvement. In contrast to predictions, SVA to faces did not differentially increase for the treatment group. Instead, both groups evinced a small increase in SVA to faces over time. Second, Prior to the SAGA intervention, attention to faces failed to predict performance in the Gaze Perception task for both the treatment and standard care control groups. However, at post-test, autistic adolescents in the treatment group were more likely to identify the object of directed gaze when they attended longer to faces and longer to target objects. Importantly, this is the first study to measure social visual attention via eye tracking as a treatment response in an RCT for autism. NCT02968225

Differences in the perception of direct gaze between the externally and internally rotated eye.

The perception of direct gaze provides multiple benefits for the observer. Previous studies have investigated how the information from both eyes is used to estimate gaze direction, showing that the perception of gaze direction differs when only the externally rotated eye versus only the internally rotated eye is visible. We examined the width and center of the area of direct gaze by presenting the observers with either the externally or internally rotated eye, or both eyes with the task to judge whether a computer avatar is looking at them. Two experiments yield evidence for a wider area of direct gaze for the externally rotated eye (around 6°) than for the internally rotated eye (around 4°). The area of direct gaze for both eyes was found to be the same as for the internally rotated eye, but smaller than for the externally rotated eye. When both eyes were present, our results indicate that the perception of direct gaze is more likely to follow the internally rotated eye. The discussion substantiates a new generalization that observers base their judgments on the more rotated eye, which can differ due to angle kappa and vergence, in our study it is the internally rotated eye.

Are the directions of both eyes integrated before or after the perception of direct gaze? Evidence from simulated mild strabismus.

The cone of gaze is a looker's range of gaze directions that is accepted as direct by an observer. The present research asks how the condition of mild strabismus, that is, when the two eyes point in slightly different directions, influences the cone of gaze. Normally, both eyes are rotated in a coordinated manner such that both eyes are directed to the same fixation point. With strabismus, there are two fixation points, and, therefore, two directions into which the two eyes point. This raises the question of the direction and the shape (i.e., width) of the gaze cone. Two experiments are conducted with simulated mild strabismus. Three conditions are tested, the two strabismic conditions of esotropia, and exotropia and one orthotropic (nonstrabismic) condition. Results show that the direction of the gaze cone is roughly the average of the directions of the two eyes. Furthermore, the width of the gaze cone is not affected by simulated strabismus and is thus the same for the strabismic and the orthotropic conditions. The results imply a model where at first the direction of gaze based on both eyes is perceived, and where the gaze cone is implied on the basis of the combined gaze direction.

Neurobehavioral indices of gaze perception are associated with social cognition across schizophrenia patients and healthy controls.

Gaze perception is a basic building block of social cognition, which is impaired in schizophrenia (SZ) and contributes to functional outcomes. Few studies, however, have investigated neural underpinnings of gaze perception and their relation to social cognition. We address this gap. We recruited 77 SZ patients and 71 healthy controls, who completed various social-cognition tasks. During functional magnetic resonance imaging, participants (62 SZ, 54 controls) completed a gaze-perception task, where they judged whether faces with varying gaze angles were self-directed or averted; as a control condition, participants identified stimulus gender. Activation estimates were extracted based on (a) task versus baseline, (b) gaze-perception versus gender-identification, (c) parametric modulation by perception of stimuli as self-directed versus averted, and (d) parametric modulation by stimulus gaze angle. We used latent variable analysis to test associations among diagnostic group, brain activation, gaze perception, and social cognition. Preferential activation to gaze perception was observed throughout dorsomedial prefrontal cortex, superior temporal sulcus, and insula. Activation was modulated by stimulus gaze angle and perception of stimuli as self-directed versus averted. More precise gaze perception and higher task-related activation were associated with better social cognition. Patients with SZ showed hyperactivation within left pre-/postcentral gyrus, which was associated with more precise gaze perception and fewer symptoms and thus may be a compensatory mechanism. Neural and behavioral indices of gaze perception were related to social cognition, across patients and controls. This suggests gaze perception is an important perceptual building block for more complex social cognition. Results are discussed in the context of dimensional psychopathology and clinical heterogeneity. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The Overestimation Effect in Gaze Perception Reduces with Distance

The Overestimation Effect in Gaze Perception Reduces with Distance

Body Form Modulates the Prediction of Human and Artificial Behaviour from Gaze Observation

The future of human–robot collaboration relies on people’s ability to understand and predict robots' actions. The machine-like appearance of robots, as well as contextual information, may influence people’s ability to anticipate the behaviour of robots. We conducted six separate experiments to investigate how spatial cues and task instructions modulate people’s ability to understand what a robot is doing. Participants observed goal-directed and non-goal directed gaze shifts made by human and robot agents, as well as directional cues displayed by a triangle. We report that biasing an observer's attention, by showing just one object an agent can interact with, can improve people’s ability to understand what humanoid robots will do. Crucially, this cue had no impact on people’s ability to predict the upcoming behaviour of the triangle. Moreover, task instructions that focus on the visual and motor consequences of the observed gaze were found to influence mentalising abilities. We suggest that the human-like shape of an agent and its physical capabilities facilitate the prediction of an upcoming action. The reported findings expand current models of gaze perception and may have important implications for human–human and human–robot collaboration.

Supplemental Material for Neurobehavioral Indices of Gaze Perception Are Associated With Social Cognition Across Schizophrenia Patients and Healthy Controls

Supplemental Material for Neurobehavioral Indices of Gaze Perception Are Associated With Social Cognition Across Schizophrenia Patients and Healthy Controls

Face masks have a limited effect on the feeling of being looked at.

Wearing face masks has been promoted as an effective measure to reduce the spread of COVID-19. Because face masks cover a major part of the face, they have detrimental effects on various aspects of social cognition. Yet, a highly important feature of the face is not occluded by face masks: the eyes. The eyes play an important role in social interactions: knowing where another person is looking is of central importance when interacting with others. Recent research has reported an attentional shift toward the eye region as a consequence of the widespread exposure to face masks. However, no study has yet investigated the influence of face masks on the perception of eye gaze direction. Here we investigated whether face masks have an effect on the feeling of being looked at. Assuming an attentional shift toward the eyes, we might expect more accurate gaze perception in faces wearing face masks. Sixty-five participants decided for a series of realistic avatar faces whether each face was making eye contact or not. Half of the faces wore face masks, the other half did not. For each participant and separately for each condition (mask vs. no mask), we calculated the cone of direct gaze (CoDG), a commonly used measure to quantify the range of gaze angles within which an observer assumes mutual gaze. Contrary to our expectations, results show that mutual gaze is not recognized more accurately in masked faces. Rather, the CoDG was, on average, slightly wider for faces wearing masks compared to faces without masks. Notwithstanding the relatively small effect of face mask, these findings potentially have implications on our social interactions. If we inadvertently feel looked at by an onlooker, we may react inappropriately by reciprocating the alleged approach orientation.

Predictability modulates neural response to eye contact in ASD

BackgroundDeficits in establishing and maintaining eye-contact are early and persistent vulnerabilities of autism spectrum disorder (ASD), and the neural bases of these deficits remain elusive. A promising hypothesis is that social features of autism may reflect difficulties in making predictions about the social world under conditions of uncertainty. However, no research in ASD has examined how predictability impacts the neural processing of eye-contact in naturalistic interpersonal interactions.MethodWe used eye tracking to facilitate an interactive social simulation wherein onscreen faces would establish eye-contact when the participant looked at them. In Experiment One, receipt of eye-contact was unpredictable; in Experiment Two, receipt of eye-contact was predictable. Neural response to eye-contact was measured via the N170 and P300 event-related potentials (ERPs). Experiment One included 23 ASD and 46 typically developing (TD) adult participants. Experiment Two included 25 ASD and 43 TD adult participants.ResultsWhen receipt of eye-contact was unpredictable, individuals with ASD showed increased N170 and increased, but non-specific, P300 responses. The magnitude of the N170 responses correlated with measures of sensory and anxiety symptomology, such that increased response to eye-contact was associated with increased symptomology. However, when receipt of eye-contact was predictable, individuals with ASD, relative to controls, exhibited slower N170s and no differences in the amplitude of N170 or P300.LimitationsOur ASD sample was composed of adults with IQ > 70 and included only four autistic women. Thus, further research is needed to evaluate how these results generalize across the spectrum of age, sex, and cognitive ability. Additionally, as analyses were exploratory, some findings failed to survive false-discovery rate adjustment.ConclusionsNeural response to eye-contact in ASD ranged from attenuated to hypersensitive depending on the predictability of the social context. These findings suggest that the vulnerabilities in eye-contact during social interactions in ASD may arise from differences in anticipation and expectation of eye-contact in addition to the perception of gaze alone.

Gaze perception from head and pupil rotations in 2D and 3D: Typical development and the impact of autism spectrum disorder.

The study of gaze perception has largely focused on a single cue (the eyes) in two-dimensional settings. While this literature suggests that 2D gaze perception is shaped by atypical development, as in Autism Spectrum Disorder (ASD), gaze perception is in reality contextually-sensitive, perceived as an emergent feature conveyed by the rotation of the pupils and head. We examined gaze perception in this integrative context, across development, among children and adolescents developing typically or with ASD with both 2D and 3D stimuli. We found that both groups utilized head and pupil rotations to judge gaze on a 2D face. But when evaluating the gaze of a physically-present, 3D robot, the same ASD observers used eye cues less than their typically-developing peers. This demonstrates that emergent gaze perception is a slowly developing process that is surprisingly intact, albeit weakened in ASD, and illustrates how new technology can bridge visual and clinical science.

Designing Perennial Landscapes: Plant Form and Species Richness Influence the Gaze Perception Associated with Aesthetic Preference

The up-close experience of perennial landscapes has been shown to enhance residents’ perception of naturalness in the context of increasing small-scale vegetation landscapes. This study explored how formal aesthetic characteristics were related to landscape perception and whether landscape preference correlated with eye movements. We created a series of photomontages showing perennial combinations that contained different plant forms, degrees of species richness, and plant arrangements and recorded 73 participants’ eye movements during 10 s of free viewing in Experiment A and task-oriented viewing in Experiment B and ratings of landscape preference collected through rating scales. We found that the effects of plant form and species richness were significant for gaze behavior, while arrangement showed no significant effect. We also found that landscape preference was positively correlated with fixation count but negatively correlated with mean fixation duration and total fixation duration. Additionally, women had more but shorter fixations than men while viewing these photomontages, and the difference in aesthetic preferences between men and women was not significant. Concerning the different professional background groups, no significant gaze behavior difference between professionals and nonprofessionals was detected, but compared with professionals, nonprofessionals tended to give strongly higher preference ratings. The outcomes shed light on the influence of formal aesthetic characteristics on gaze behavior and advanced the application of eye-tracking technology in perennial landscape studies. Our findings also confirmed the efficiency of vegetation landscapes designed based on public preferences for providing restoration from stress or fatigue.

Age-Related Differences in the Perception of Robotic Referential Gaze in Human-Robot Interaction

There is an increased interest in using social robots to assist older adults during their daily life activities. As social robots are designed to interact with older users, it becomes relevant to study these interactions under the lens of social cognition. Gaze following, the social ability to infer where other people are looking at, deteriorates with older age. Therefore, the referential gaze from robots might not be an effective social cue to indicate spatial locations to older users. In this study, we explored the performance of older adults, middle-aged adults, and younger controls in a task assisted by the referential gaze of a Pepper robot. We examined age-related differences in task performance, and in self-reported social perception of the robot. Our main findings show that referential gaze from a robot benefited task performance, although the magnitude of this facilitation was lower for older participants. Moreover, perceived anthropomorphism of the robot varied less as a result of its referential gaze in older adults. This research supports that social robots, even if limited in their gazing capabilities, can be effectively perceived as social entities. Additionally, this research suggests that robotic social cues, usually validated with young participants, might be less optimal signs for older adults.

Do gaze and non-gaze stimuli trigger different spatial interference effects? It depends on stimulus perceivability.

Among the studies on the perception of gaze vs. non-gaze stimuli, some have shown that the two types of stimuli trigger different patterns of attentional effects, while others have reported no such differences. In three experiments, we investigated the role of stimulus perceivability in spatial interference effects when the targets were gaze vs. non-gaze stimuli. We used a spatial Stroop task that required participants to make a speeded response to the direction indicated by the targets located on the left or right side of fixation. In different experiments, the targets consisted of eyes, symbols, and/or arrows. The results showed that the magnitude of the spatial congruency effect differed between the types of targets when stimulus perceivability was not controlled. However, when the perceivability of the task relevant parts was comparable between the different types of targets, similar congruency effects were found regardless of target type. These results underscore the importance of controlling for stimulus perceivability, which is closely linked to the attentional zoom required to perform a task, when making inferences about the attentional mechanisms in the processing of gaze vs. non-gaze stimuli.