Gaze Orientation Research Articles

It Is Written in the Eyes: Inferences From Pupil Size and Gaze Orientation Shape Interpersonal Liking

Research has shown that pupil size shapes interpersonal impressions: Individuals with dilated pupils tend to be perceived more positively than those with constricted pupils. Untested so far is the role of cognitive processes in shaping the effects of pupil size. Two preregistered studies investigated whether the effect of pupil size was qualified by partner's attention allocation inferred from gaze orientation. In Experiment 1 (N = 50) partners with dilated pupils were more liked when gazing toward the participant, but less liked when gazing toward a disliked other. Experiment 2 (N = 50) unveiled the underlying mechanism of the pupil-gaze interplay. Pupillary changes led to inferences about the feelings held by the partner toward the gazed target: Larger pupils signaled positive feelings. Crucially, target identity moderated the response of the participants (i.e., liking toward the partner). This work shows the importance of considering the interplay of affective and cognitive eye-signals when studying person perception.

Influence of the Location of a Decision Cue on the Dynamics of Pupillary Light Response.

The pupils of the eyes reflexively constrict in light and dilate in dark to optimize retinal illumination. Non-visual cognitive factors, like attention, arousal, decision-making, etc., also influence pupillary light response (PLR). During passive viewing, the eccentricity of a stimulus modulates the pupillary aperture size driven by spatially weighted corneal flux density (CFD), which is the product of luminance and the area of the stimulus. Whether the scope of attention also influences PLR remains unclear. In this study, we contrasted the pupil dynamics between diffused and focused attentional conditions during decision-making, while the global CFD remained the same in the two conditions. A population of 20 healthy humans participated in a pair of forced choice tasks. They distributed attention to the peripheral decision cue in one task, and concentrated at the center in the other to select the target from four alternatives for gaze orientation. The location of this cue did not influence participants’ reaction time (RT). However, the magnitude of constriction was significantly less in the task that warranted attention to be deployed at the center than on the periphery. We observed similar pupil dynamics when participants either elicited or canceled a saccadic eye movement, which ruled out pre-saccadic obligatory attentional orientation contributing to PLR. We further addressed how the location of attentional deployment might have influenced PLR. We simulated a biomechanical model of PLR with visual stimulation of different strengths as inputs corresponding to the two attentional conditions. In this homeomorphic model, the computational characteristic of each element was derived from the physiological and/or mechanical properties of the corresponding biological element. The simulation of this model successfully mimicked the observed data. In contrast to common belief that the global ambient luminosity drives pupillary response, the results of our study suggest that the effective CFD (eCFD) determined via the luminance multiplied by the size of the stimulus at the location of deployed attention in the visual space is critical for the magnitude of pupillary constriction.

Anatomical and neurophysiological basis of face recognition

Face recognition is a highly developed and efficient human function that involves multiple neural networks. A main pathway links the occipital cortex, where an occipital face area (OFA) has been identified, to a fusiform face area (FFA) in the fusiform gyrus, which plays a critical role in face recognition. This core pathway deals with invariant aspects of the face. Another pathway, including the superior temporal sulcus, is involved in the perception of more changeable aspects of the face such as gaze orientation, face expression and lip movements. It has been defined by some authors as a “third pathway of visual recognition”, i.e. a lateral pathway in addition to the “what” and “where” pathways. It deals with sociocognitive aspects of face perception. Many other accessory functional systems are connected to the core system of visual recognition to act in concert with it: the intraparietal sulcus (for the management of spatial attention), the primary auditory cortex (prelexical perception of speech), the amygdala, the insula and the limbic system (perception of emotions), the anterior temporal pole (access to the identity of the individual, his name, biographical information), etc. Functional brain imaging has made remarkable progress in the understanding of face perception, which in the early years was limited to the description of single cases of brain-damaged patients. This progress has made it possible to better analyse the many face recognition disorders, sometimes subtle, other times confusing, observed in human pathology.

Robotically aligned optical coherence tomography with 5 degree of freedom eye tracking for subject motion and gaze compensation.

Optical coherence tomography (OCT) has revolutionized diagnostics in ophthalmology. However, OCT requires a trained operator and patient cooperation to carefully align a scanner with the subject's eye and orient it in such a way that it images a desired region of interest at the retina. With the goal of automating this process of orienting and aligning the scanner, we developed a robot-mounted OCT scanner that automatically aligned with the pupil while matching its optical axis with the target region of interest at the retina. The system used two 3D cameras for face tracking and three high-resolution 2D cameras for pupil and gaze tracking. The tracking software identified 5 degrees of freedom for robot alignment and ray aiming through the ocular pupil: 3 degrees of translation (x, y, z) and 2 degrees of orientation (yaw, pitch). We evaluated the accuracy, precision, and range of our tracking system and demonstrated imaging performance on free-standing human subjects. Our results demonstrate that the system stabilized images and that the addition of gaze tracking and aiming allowed for region-of-interest specific alignment at any gaze orientation within a 28° range.

The effects of gaze and head orientation on the speed of face categorization during RSVP

The effects of gaze and head orientation on the speed of face categorization during RSVP

Human-Machine Interface: Multiclass Classification by Machine Learning on 1D EOG Signals for the Control of an Omnidirectional Robot.

People with severe disabilities require assistance to perform their routine activities; a Human–Machine Interface (HMI) will allow them to activate devices that respond according to their needs. In this work, an HMI based on electrooculography (EOG) is presented, the instrumentation is placed on portable glasses that have the task of acquiring both horizontal and vertical EOG signals. The registration of each eye movement is identified by a class and categorized using the one hot encoding technique to test precision and sensitivity of different machine learning classification algorithms capable of identifying new data from the eye registration; the algorithm allows to discriminate blinks in order not to disturb the acquisition of the eyeball position commands. The implementation of the classifier consists of the control of a three-wheeled omnidirectional robot to validate the response of the interface. This work proposes the classification of signals in real time and the customization of the interface, minimizing the user’s learning curve. Preliminary results showed that it is possible to generate trajectories to control an omnidirectional robot to implement in the future assistance system to control position through gaze orientation.

Serial or parallel proactive control of components of task-set? A task-switching investigation with concurrent EEG and eye-tracking

Among the issues examined by studies of cognitive control in multitasking is whether processes underlying performance in the different tasks occur serially or in parallel. Here we ask a similar question about processes that pro-actively control task-set. In task-switching experiments, several indices of task-set preparation have been extensively documented, including anticipatory orientation of gaze to the task-relevant location (an unambiguous marker of reorientation of attention), and a positive polarity brain potential over the posterior cortex (whose functional significance is less well understood). We examine whether these markers of preparation occur in parallel or serially, and in what order. On each trial a cue required participants to make a semantic classification of one of three digits presented simultaneously, with the location of each digit consistently associated with one of three classification tasks (e.g., if the task was odd/even, the digit at the top of the display was relevant). The EEG positivity emerged following, and appeared time-locked to, the anticipatory fixation on the task-relevant location, which might suggest serial organisation. However, the fixation-locked positivity was not better defined than the cue-locked positivity; in fact, for the trials with the earliest fixations the positivity was better time-locked to the cue onset. This is more consistent with (re)orientation of spatial attention occurring in parallel with, but slightly before, the reconfiguration of other task-set components indexed by the EEG positivity.

Collaborative construction of turn constructional units in responsive positions of question-answer sequences in Mandarin conversation

AbstractThis paper explores the features and interactional functions of collaboratively constructed TCUs (CCTs) in responsive positions of question-answer sequences in Mandarin daily conversations. Adopting the methodologies of Conversation Analysis, Interactional Linguistics and Multimodal Analysis, the study explores the sequential features of the CCTs and bodily-visual resources co-occurring with the CCTs, such as gaze orientations and gestures. Two categories have been identified based on the participants’ roles in the question-answer sequences. First, the answerer initiates the response to the question, and the questioner collaboratively completes the response. The analysis shows that the questioners are not conveying the action of answering the question but assuming the answer to the question. Second, one answerer initiates the response to the question, and another one collaboratively completes the response. The data demonstrates that this type of CCTs usually involves the two question-recipients with more or less equal epistemic access to the referent.

Electrophysiological Evidence of Space-Number Associations in 9-Month-Old Infants.

Infant research is providing accumulating evidence that number‐space mappings appear early in development. Here, a Posner cueing paradigm was used to investigate the neural mechanisms underpinning the attentional bias induced by nonsymbolic numerical cues in 9‐month‐old infants (N = 32). Event‐related potentials and saccadic reaction time were measured to the onset of a peripheral target flashing right after the offset of a centered small or large numerical cue, with the location of the target being either congruent or incongruent with the number’s relative position on a left‐to‐right oriented representational continuum. Results indicated that the cueing effect induced by numbers on infants’ orienting of eye gaze brings about sensory facilitation in processing visual information at the cued location.

Early Visual Saliency Based on Isolated Optimal Features.

Under fast viewing conditions, the visual system extracts salient and simplified representations of complex visual scenes. Saccadic eye movements optimize such visual analysis through the dynamic sampling of the most informative and salient regions in the scene. However, a general definition of saliency, as well as its role for natural active vision, is still a matter for discussion. Following the general idea that visual saliency may be based on the amount of local information, a recent constrained maximum-entropy model of early vision, applied to natural images, extracts a set of local optimal information-carriers, as candidate salient features. These optimal features proved to be more informative than others in fast vision, when embedded in simplified sketches of natural images. In the present study, for the first time, these features were presented in isolation, to investigate whether they can be visually more salient than other non-optimal features, even in the absence of any meaningful global arrangement (contour, line, etc.). In four psychophysics experiments, fast discriminability of a compound of optimal features (target) in comparison with a similar compound of non-optimal features (distractor) was measured as a function of their number and contrast. Results showed that the saliency predictions from the constrained maximum-entropy model are well verified in the data, even when the optimal features are presented in smaller numbers or at lower contrast. In the eye movements experiment, the target and the distractor compounds were presented in the periphery at different angles. Participants were asked to perform a simple choice-saccade task. Results showed that saccades can select informative optimal features spatially interleaved with non-optimal features even at the shortest latencies. Saccades’ choice accuracy and landing position precision improved with SNR. In conclusion, the optimal features predicted by the reference model, turn out to be more salient than others, despite the lack of any clues coming from a global meaningful structure, suggesting that they get preferential treatment during fast image analysis. Also, peripheral fast visual processing of these informative local features is able to guide gaze orientation. We speculate that active vision is efficiently adapted to maximize information in natural visual scenes.

Does gaze direction of fearful faces facilitate the processing of threat? An ERP study of spatial precuing effects.

Eye gaze is very important for attentional orienting in social life. By adopting the event-related potential (ERP) technique, we explored whether attentional orienting of eye gaze is modulated by emotional congruency between facial expressions and the targets in a spatial cuing task. Faces with different emotional expressions (fearful/angry/happy/neutral) directing their eye gaze to the left or right were used as cues, indicating the possible location of subsequent targets. Targets were line drawings of animals, which could be either threatening or neutral. Participants indicated by choice responses whether the animal would fit inside a shoebox in real life or not. Reaction times to targets were faster after valid compared with invalid cues, showing the typical eye gaze cuing effect. Analyses of the late positive potential (LPP) elicited by targets revealed a significant modulation of the gaze cuing effect by emotional congruency. Threatening targets elicited larger LPPs when validly cued by gaze in faces with negative (fearful and angry) expressions. Similarly, neutral targets showed larger LPPs when validly cued by faces with neutral expressions. Such effects were not present after happy face cues. Source localization in the LPP time window revealed that for threatening targets, the activity of right medial frontal gyrus could be related to a larger gaze-orienting effect for the fearful than the angry condition. Our findings provide electrophysiological evidence for the modulation of gaze cuing effects by emotional congruency.

Effect of Prism Adaptation Therapy on the Activities of Daily Living and Awareness for Spatial Neglect: A Secondary Analysis of the Randomized, Controlled Trial.

Background: Rehabilitation for unilateral spatial neglect (USN) using prism adaptation (PA) is one of the most widely used methods, and the effectiveness of PA is well-evidenced. Although the effect of PA generalized various neglect symptoms, the effectiveness for some aspects of neglect is not fully proven. The Catherine Bergego Scale (CBS) was developed to identify problems with the activities of daily living (ADL) caused by USN. The CBS is composed of 10 observation assessments and a self-assessment questionnaire. To assess the self-awareness of USN, the anosognosia score is calculated as the difference between the observational scores and the self-assessment scores. To investigate how PA affects ADL and self-awareness in subacute USN patients during rehabilitation, we analyzed each item of the CBS and self-awareness from a randomized, controlled trial (RCT) that we previously conducted (Mizuno et al., 2011). Methods: A double-masked randomized, controlled trial was conducted to evaluate the effects of a 2-week PA therapy on USN in 8 hospitals in Japan. We compared each item of the CBS, anosognosia score, and absolute value of the anosognosia score between the prism group and the control group. Results: Two of ten items (gaze orientation and exploration of personal belongings) were significantly improved in the prism group compared with those in the control group. The absolute value of the anosognosia score was significantly improved by PA. Conclusions: Improvement of oculomotor exploration by PA may generalize the behavioral level in a daily living environment. This study suggested that PA could accelerate the self-awareness of neglect during subacute rehabilitation.

Optimizing turn-taking in interpreter-mediated therapy: on the importance of the interpreter’s speaking space

This paper highlights two types of turn-taking problems that can occur in dialogue interpreting within the context of mental healthcare. Although interpreting in mental health care has received some scholarly attention over the past two decades, the multimodal dimension of such encounters has not been investigated in detail so far.Based on a dataset of video recorded psychotherapeutic sessions with refugees, the study aims to show how interpreters deal with turn-taking issues during the conversation and how this affectsboth their ownrolein the encounter and the interaction itself. Both verbal and nonverbal behavior (gaze orientation and gestures) were taken into account. The data were analyzed qualitatively by drawing on the insights from Conversation Analysis (CA). The analysis suggests that problems may arise when the interpreter is not able to negotiate the moment of turn transfer or his/her turn space during the talk.Such problems in the coordination of turn-taking with the interpreter can even result in loss of information. We argue that turn-taking in therapeutic counseling with an onsite interpreter is a collaborative achievement between both speakers and the interpreter, and that acknowledging the interpreter as a co-participant with rights for speaking space supports the interpreting process.

Effects of Reinforcement Learning on Gaze Following of Gaze and Head Direction in Early Infancy: An Interactive Eye-Tracking Study.

The current four experiments investigated gaze following behavior in response to gaze and head turns in 4-month-olds and how reinforcement learning influences this behavior (N=99). Using interactive eye tracking, infants' gaze elicited an animation whenever infants followed a person's head or gaze orientation (Experiment 1.1, 2.1 and 2.2) or looked at the opposite side (Experiment 1.2). Infants spontaneously followed the direction of a turning head with and without simultaneously shifted gaze direction (Cohen's d: 0.93-1.05) but not the direction of isolated gaze shifts. We only found a weak effect of reinforcement on gaze following in one of the four experiments. Results will be discussed with regard to the impact of reinforcement on the maintenance of already existing gaze following behavior.

Effects of Emotional Expressions, Gaze, and Head Orientation on Person Perception in Social Situations

When an observer perceives and judges two persons next to each other, different types of social cues simultaneously arise from both perceived faces. Using a controlled stimulus set depicting this scenario (with two persons identified respectively as “target face” and “looking face”), we explored how emotional expressions, gaze, and head direction of the looking face affect the observers’ eye movements and judgments of the target face. The target face always displayed a neutral expression, gazing directly at the observer (“direct gaze”). The looking face showed either a direct gaze, looked toward the target face, or averted it. A total of 52 undergraduate students (25 males) freely viewed these scenes for 5 s while their eye movements were recorded, which was followed by collecting ratings of attractiveness and trustworthiness. Dwell times on target faces were longer when accompanied by a looking face with direct gaze, regardless of its emotional expression. However, participants looked longer on faces looking toward the target in the approach condition and fixated more often on target faces that were either next to an angry-looking face directly looking at them or to a happy-looking averted face. We found no gaze effect on faces that were looked at by another face and no significant correlation between observers’ dwell time and attractiveness or trustworthiness ratings of the target and looking face, indicating dissociated perception and judgment processes. Irrespective of the gaze direction, as expected, happy faces were judged as more attractive and trustworthy than angry faces. Future studies will need to examine this dynamic interplay of social cues in triadic scenes.

Eye Tracking as a Tool to Identify Mood in Aphasia: A Feasibility Study

Background. Persons with aphasia often present with low mood/depression, which can negatively affect their quality of life. The validity and reliability of existing depression measures for aphasia have been called into question. Eye tracking in nonstroke populations is reliable in identifying low mood/depression. Depressed persons are biased to negative emotions compared with nondepressed persons and have an absence of bias to positive emotions. However, nondepressed persons may be biased to positive emotions. Objective. To examine the feasibility of using eye tracking to measure mood in persons with aphasia. Methods. We recruited 22 persons with chronic aphasia and 12 healthy controls. Participants completed 2 self-report measures of mood. They also viewed faces that showed happy, sad, and neutral facial expressions during eye tracking. We analyzed 2 eye tracking indices: initial gaze orientation and gaze maintenance to happy, sad, and neutral faces. Results. For initial gaze orientation, participants with aphasia fixated faster on emotional faces compared with healthy controls but directed their gaze less often to happy faces compared with healthy controls. For gaze maintenance components, the duration of first fixation and total fixation duration were shorter on sad faces for participants with aphasia compared with healthy controls. Conclusion. Use of eye tracking with faces representing different mood states is feasible in persons with aphasia. Although there were some similarities, participants with aphasia had different gaze patterns to emotional faces compared with healthy controls. Further research is needed to establish whether this is a valid and reliable method of mood assessment.

Perception of Gaze Direction in Glaucoma: A Study on Social Cognition.

Discriminating quickly where another person's gaze is directed is a key component of social interaction, as gaze direction conveys information about others' intentions (approach or avoidance) and shift in gaze is used in group conversation. This study shows that patients with glaucoma are delayed in their discrimination of gaze direction. The purpose of this study was to investigate whether glaucoma affects the perception of gaze direction. Twenty-four patients with open-angle glaucoma, 24 age-matched controls, and 20 young normally sighted controls were presented with faces displayed centrally for 200 milliseconds. The gaze could either be direct or averted, and the head could be a frontal view or a rotated view. Half of the participants in each group were asked to press a key only for faces with a direct gaze. The other half responded for a face with an averted gaze. The orientation of the head had to be ignored. Accuracy was greater than 90% correct for all three groups. We found no difference in performance between young and older controls, except for shorter response times for the frontal view than for the rotated face in young participants. Patients with glaucoma needed on average 140 milliseconds longer to decide if the gaze was averted than for the direct gaze, and they were less accurate than controls in perceiving the gaze as direct when the head was rotated. Patients with glaucoma often experience impaired vision due to a reduced sensitivity in central vision. Although lower central sensitivity had little effect on their ability to discriminate gaze orientation, they required on average 140 milliseconds longer than age-matched controls to perceive an averted gaze.

Vestibular cognition: State-of-the-art and future directions

ABSTRACT Vestibular information has been traditionally considered as a specialized input for basic orienting behaviours, such as oculo-motor adjustments, postural control and gaze orientation. However, in the past two decades a widespread vestibular network in the human brain has been identified, that goes far beyond the low-level reflex circuits emphasized by earlier work. Because this vestibular cortical network is so widely distributed, it could, in principle, impact multiple neurocognitive functions in health and disease. This paper focuses on the relations between vestibular input, vestibular networks, and vestibular interventions by providing the authors’ personal viewpoint on the state-of-the-art of vestibular cognitive neuropsychology, and its potential relevance for neurorehabilitation.

The effect of head and gaze orientation on spine kinematics during forward flexion

Compound, or awkward, spine postures have been suggested as a biomechanical risk factor for low back injury. This experiment investigates the influence of head (i.e. head-on-torso) and gaze (i.e. eye-in-head) orientation on three-dimensional (3D) neck and spine range of motion (ROM) during forward flexion movements. To emulate previous experimental protocols and replicate real-world scenarios, a sample of ten young, healthy males (mean ± standard deviation: age: 20.8 ± 1.03 years, height: 180.2 ± 7.36 cm, and mass: 81.9 ± 6.47 kg) completed forward flexion movements with a constrained and unconstrained pelvis, respectively. Surface kinematics were gathered from the head and spine (C7-S1). Movements were completed under a baseline condition as well as upward, downward, leftward, and rightward head and gaze orientations. For each condition, mean neck angle and inter-segmental spine (C7T1 through L5S1) ROM were evaluated. The results demonstrate that directed head and gaze orientations can influence the ROM of specific spine regions during a forward flexion task. With leftward and rightward directed head and gaze orientations, the neck became increasingly twisted and superior thoracic segments (i.e. C7T1-T2T3) were significantly more twisted during the leftward head orientation condition than the baseline condition. With upward and downward directed head and gaze orientations, a similar effect was observed for neck and superior thoracic (i.e. C7T1-T4T5) flexion-extension. Interestingly, it was also demonstrated that changes in upward/downward head orientation can also change flexion-extension kinematics of the thoracolumbar region as well (i.e. T7T8-L1L2), suggesting that head postures requiring neck extension may also promote extension throughout these spine regions. These findings provide evidence for a functional link between changes in neck flexion-extension posture and flexion-extension movement of the thoracolumbar region of the spine.

Gaze cueing in naturalistic scenes under top-down modulation – Effects on gaze behaviour and memory performance

ABSTRACT Humans as social beings rely on information provided by conspecifics. One important signal in social communication is eye gaze. The current study (n = 93) sought to replicate and extend previous findings of attentional guidance by eye gaze in complex everyday scenes. In line with previous studies, longer, more and earlier fixations for objects cued by gaze compared to objects that were not cued were observed in free viewing conditions. To investigate how robust this prioritization is against top-down modulation, half of the participants received a memory task that required scanning the whole scene instead of exclusively focusing on cued objects. Interestingly, similar gaze cueing effects occurred in this group. Moreover, the human beings depicted in the scene received a large amount of attention, especially during early phases of visual attention, even though they were irrelevant to the current task. These results indicate that the mere presence of other human beings, as well as their gaze orientation, have a strong impact on attentional exploration. Data and analysis scripts are available at https://osf.io/jk9s4.