Eye Gaze Direction Research Articles

Drivers’ Visual Distraction Detection Using Facial Landmarks and Head Pose

Drivers’ distraction has been widely studied in the field of naturalistic driving studies. However, it is difficult to use traditional variables, such as speed, acceleration, and yaw rate to detect drivers’ distraction in real time. Emerging technologies have obtained features from human faces, such as eye gaze, to detect drivers’ visual distraction. However, eye gaze is hard to detect in naturalistic driving situations, because of low-resolution cameras, drivers wearing sunglasses, and so forth. Instead, head pose is easier to detect, and has correlation with eye gaze direction. In this study, city-wide videos are collected using onboard cameras from over 289 drivers representing 423 events. Head pose (pitch, yaw, and roll rates) are derived and fed into a convolutional neural network to detect drivers’ distraction. The experiment results show that the proposed model can achieve recall value of 0.938 and area under the receiver operating characteristic curve value of 0.931, with variables from five time slices (1.25 s) used as input. The study proves that head pose can be used to detect drivers’ distraction. The study offers insights for detecting drivers’ distraction and can be used for the development of advanced driver assistance systems.

Social Cognition Theory and Physical Activity among Local Population: An Overview

Social cognition concerns the various psychological processes that enable individuals to take advantageof being part of a social group. Of major importance to social cognition are the various social signalsthat enable us to learn about the world. Such signals include facial expressions, such as fear and disgust,which warn us of danger, and eye gaze direction, which indicate where interesting things can be found.Such signals are particularly important in infant development. Social referencing, for example, refers tothe phenomenon in which infants refer to their mothers’ facial expressions to determine whether or notto approach a novel object. We can learn a great deal simply by observing others. Much of this signallingseems to happen automatically and unconsciously on the part of both the sender and the receiver. Wecan learn to fear a stimulus by observing the response of another, in the absence of awareness ofthat stimulus. By contrast, learning by instruction, rather than observation, does seem to depend uponawareness of the stimulus, since such learning does not generalize to situations where the stimulus ispresented subliminally. Learning by instruction depends upon a meta-cognitive process through whichboth the sender and the receiver recognize that signals are intended to be signals. An example wouldbe the ‘ostensive’ signals that indicate that what follows are intentional communications. Infants learnmore from signals that they recognize to be instructive. I speculate that it is this ability to recognize andlearn from instructions rather than mere observation which permitted that advanced ability to benefitfrom cultural learning that seems to be unique to the human race.

Direct eye gaze enhances the ventriloquism effect

The “ventriloquism effect” describes an illusory phenomenon where the perceived location of an auditory stimulus is pulled toward the location of a visual stimulus. Ventriloquists use this phenomenon to create an illusion where an inanimate puppet is perceived to speak. Ventriloquists use the expression and suppression of their own and the puppet’s mouth movements as well the direction of their respective eye gaze to maximize the illusion. While the puppet’s often exaggerated mouth movements have been demonstrated to enhance the ventriloquism effect, the contribution of direct eye gaze remains unknown. In Experiment 1, participants viewed an image of a person’s face while hearing a temporally synchronous recording of a voice originating from different locations on the azimuthal plane. The eyes of the facial stimuli were either looking directly at participants or were closed. Participants were more likely to misperceive the location of a range of voice locations as coming from a central position when the eye gaze of the facial stimuli were directed toward them. Thus, direct gaze enhances the ventriloquist effect by attracting participants’ perception of the voice locations toward the location of the face. In an exploratory analysis, we furthermore found no evidence for an other-race effect between White vs Asian listeners. In Experiment 2, we replicated the effect of direct eye gaze on the ventriloquism effect, also showing that faces per se attract perceived sound locations compared with audio-only sound localization. Showing a modulation of the ventriloquism effect by socially-salient eye gaze information thus adds to previous findings reporting top-down influences on this effect.

Experimental evidence that uniformly white sclera enhances the visibility of eye-gaze direction in humans and chimpanzees.

Hallmark social activities of humans, such as cooperation and cultural learning, involve eye-gaze signaling through joint attentional interaction and ostensive communication. The gaze-signaling and related cooperative-eye hypotheses posit that humans evolved unique external eye morphologies, including uniformly white sclera (the whites of the eye), to enhance the visibility of eye-gaze for conspecifics. However, experimental evidence is still lacking. This study tested the ability of human and chimpanzee participants to discriminate the eye-gaze directions of human and chimpanzee images in computerized tasks. We varied the level of brightness and size in the stimulus images to examine the robustness of the eye-gaze directional signal against simulated shading and distancing. We found that both humans and chimpanzees discriminated eye-gaze directions of humans better than those of chimpanzees, particularly in visually challenging conditions. Also, participants of both species discriminated the eye-gaze directions of chimpanzees better when the contrast polarity of the chimpanzee eye was reversed compared to when it was normal; namely, when the chimpanzee eye has human-like white sclera and a darker iris. Uniform whiteness in the sclera thus facilitates the visibility of eye-gaze direction even across species. Our findings thus support but also critically update the central premises of the gaze-signaling hypothesis.

Improving Haptic Response for Contextual Human Robot Interaction.

For haptic interaction, a user in a virtual environment needs to interact with proxies attached to a robot. The device must be at the exact location defined in the virtual environment in time. However, due to device limitations, delays are always unavoidable. One of the solutions to improve the device response is to infer human intended motion and move the robot at the earliest time possible to the desired goal. This paper presents an experimental study to improve the prediction time and reduce the robot time taken to reach the desired position. We developed motion strategies based on the hand motion and eye-gaze direction to determine the point of user interaction in a virtual environment. To assess the performance of the strategies, we conducted a subject-based experiment using an exergame for reach and grab tasks designed for upper limb rehabilitation training. The experimental results in this study revealed that eye-gaze-based prediction significantly improved the detection time by 37% and the robot time taken to reach the target by 27%. Further analysis provided more insight on the effect of the eye-gaze window and the hand threshold on the device response for the experimental task.

Integration of Facial Expression and Gaze Direction in Individuals with a High Level of Autistic Traits.

Background. We investigated whether individuals with high levels of autistic traits integrate relevant communicative signals, such as facial expression, when decoding eye-gaze direction. Methods. Students with high vs. low scores on the Autism Spectrum Quotient (AQ) performed a task in which they responded to the eye directions of faces, presented on the left or the right side of a screen, portraying different emotional expressions. Results. In both groups, the identification of gaze direction was faster when the eyes were directed towards the center of the scene. However, in the low AQ group, this effect was larger for happy faces than for neutral faces or faces showing other emotional expressions, whereas participants from high AQ group were not affected by emotional expressions. Conclusions. These results suggest that individuals with more autistic traits may not integrate multiple communicative signals based on their emotional value.

Explicit vs. implicit spatial processing in arrow vs. eye-gaze spatial congruency effects

Arrows and gaze stimuli lead to opposite spatial congruency effects. While standard congruency effects are observed for arrows (faster responses for congruent conditions), responses are faster when eye-gaze stimuli are presented on the opposite side of the gazed-at location (incongruent trials), leading to a reversed congruency effect (RCE). Here, we explored the effects of implicit vs. explicit processing of arrows and eye-gaze direction. Participants were required to identify the direction (explicit task) or the colour (implicit task) of left or right looking/pointing gaze or arrows, presented to either the left or right of the fixation point. When participants responded to the direction of stimuli, standard congruency effects for arrows and RCE for eye-gaze stimuli were observed. However, when participants responded to the colour of stimuli, no congruency effects were observed. These results suggest that it is necessary to explicitly pay attention to the direction of eye-gaze and arrows for the congruency effect to occur. The same pattern of data was observed when participants responded either manually or verbally, demonstrating that manual motor components are not responsible for the results observed. These findings are not consistent with some hypotheses previously proposed to explain the RCE observed with eye-gaze stimuli and, therefore, call for an alternative plausible hypothesis.

Multimodal Evidence of Atypical Processing of Eye Gaze and Facial Emotion in Children With Autistic Traits.

According to the shared signal hypothesis (SSH) the impact of facial expressions on emotion processing partially depends on whether the gaze is directed toward or away from the observer. In autism spectrum disorder (ASD) several aspects of face processing have been found to be atypical, including attention to eye gaze and the identification of emotional expressions. However, there is little research on how gaze direction affects emotional expression processing in typically developing (TD) individuals and in those with ASD. This question is investigated here in two multimodal experiments. Experiment 1 required processing eye gaze direction while faces differed in emotional expression. Forty-seven children (aged 9–12 years) participated. Their Autism Diagnostic Observation Schedule (ADOS) scores ranged from 0 to 6 in the experiment. Event-related potentials (ERPs) were sensitive to gaze direction and emotion, but emotion processing did not depend on gaze direction. However, for angry faces the gaze direction effect on the N170 amplitude, as typically observed in TD individuals, diminished with increasing ADOS score. For neutral expressions this correlation was not significant. Experiment 2 required explicit emotion classifications in a facial emotion composite task while eye gaze was manipulated incidentally. A group of 22 children with ASD was compared to a propensity score-matched group of TD children (mean age = 13 years). The same comparison was carried out for a subgroup of nine children with ASD who were less trained in social cognition, according to clinician’s report. The ASD group performed overall worse in emotion recognition than the TD group, independently of emotion or gaze direction. However, for disgust expressions, eye tracking data revealed that TD children fixated relatively longer on the eyes of the stimulus face with a direct gaze as compared with averted gaze. In children with ASD we observed no such modulation of fixation behavior as a function of gaze direction. Overall, the present findings from ERPs and eye tracking confirm the hypothesis of an impaired sensitivity to gaze direction in children with ASD or elevated autistic traits, at least for specific emotions. Therefore, we conclude that multimodal investigations of the interaction between emotional processing and stimulus gaze direction are promising to understand the characteristics of individuals differing along the autism trait dimension.

Part-based processing in eye-gaze direction produces the reversed spatial Stroop effect

Part-based processing in eye-gaze direction produces the reversed spatial Stroop effect

When Preschoolers Interact with an Educational Robot, Does Robot Feedback Influence Engagement?

In this paper, we examine to what degree children of 3–4 years old engage with a task and with a social robot during a second-language tutoring lesson. We specifically investigated whether children’s task engagement and robot engagement were influenced by three different feedback types by the robot: adult-like feedback, peer-like feedback and no feedback. Additionally, we investigated the relation between children’s eye gaze fixations and their task engagement and robot engagement. Fifty-eight Dutch children participated in an English counting task with a social robot and physical blocks. We found that, overall, children in the three conditions showed similar task engagement and robot engagement; however, within each condition, they showed large individual differences. Additionally, regression analyses revealed that there is a relation between children’s eye-gaze direction and engagement. Our findings showed that although eye gaze plays a significant role in measuring engagement and can be used to model children’s task engagement and robot engagement, it does not account for the full concept and engagement still comprises more than just eye gaze.

Improving sensitivity to eye gaze cues in adolescents on the autism spectrum using serious game technology: A randomized controlled trial.

Perceiving and interpreting eye gaze cues is foundational for social cognition and social interactions because it involves the ability to use eye gaze direction to predict the actions and intentions of others. Autism is a disability that impacts social interactions. A diagnostic symptom of autism is difficulty understanding eye gaze cues as social signals. This deficit has long-term consequences for understanding goal-directed behavior, language learning, and social communication. We hypothesize that targeted intervention methods designed to improve sensitivity to eye gaze cues may begin to treat core symptoms of autism and potentially alter multiple aspects of social functioning. Social Games for Autistic Adolescents (SAGA) is a serious computer game intervention designed to improve sensitivity to eye gaze cues. Serious games improve targeted skills with the goal of enhancing real life outcomes. In SAGA, participants progress through a narrative storyline and interact with animated characters. In so doing, they implicitly discover that eye gaze cues are useful for guiding their own goal-directed behavior to solve problems in the game. We evaluated the feasibility and effectiveness of SAGA in a hybrid phase 1/2, randomized controlled trial. Forty adolescents on the autism spectrum were randomized to either the treatment or standard care control group. Adolescents in the treatment group were asked to play SAGA for 30-minute sessions at home 3 times a week over 10 weeks. A group × time interaction revealed that the treatment group developed increasing sensitivity to human eye gaze cues, whereas the standard care group did not. Participants who experienced a sufficient dose of gameplay showed larger treatment-related improvements. Critically, increases in sensitivity to human eye gaze cues were associated with improvements in social skills. This accessible, scalable, and affordable intervention shows promise as an effective tool for improving the ability to interpret and understand eye gaze cues and social skills in adolescents on the autism spectrum.

A survey on Deep Learning Based Eye Gaze Estimation Methods

In recent years, deep-learning systems have made great progress, particularly in the disciplines of computer vision and pattern recognition. Deep-learning technology can be used to enable inference models to do real-time object detection and recognition. Using deep-learning-based designs, eye tracking systems could determine the position of eyes or pupils, regardless of whether visible-light or near-infrared image sensors were utilized. For growing electronic vehicle systems, such as driver monitoring systems and new touch screens, accurate and successful eye gaze estimates are critical. In demanding, unregulated, low-power situations, such systems must operate efficiently and at a reasonable cost. A thorough examination of the different deep learning approaches is required to take into consideration all of the limitations and opportunities of eye gaze tracking. The goal of this research is to learn more about the history of eye gaze tracking, as well as how deep learning contributed to computer vision-based tracking. Finally, this research presents a generalized system model for deep learning-driven eye gaze direction diagnostics, as well as a comparison of several approaches.

Social attention triggered by eye gaze and walking direction is resistant to temporal decay.

The attentional orienting induced by social cues, such as eye gaze and walking direction of biological motion, plays a vital role in human survival and interpersonal interactions. It has long been debated whether this indispensable ability is unique and intrinsically distinct from nonsocial attention. In the current study, we characterized the temporal profiles of the attentional orienting triggered by social cues (i.e., eye gaze and walking direction) and compared them with those induced by nonsocial cues (i.e., arrows) and exogenous cues using a covert orienting task. We calculated the attentional cuing effects in the early and the late periods of the task and further carried out a time course analysis to characterize their dynamic changes over trials. Whereas the cuing effect induced by nonsocial cues exhibited a significant trend of temporal decay, the cuing effects induced by the two different social cues were similar and remained stable throughout the task, resembling that induced by reflexive exogenous cues. These results clearly demonstrate that the socially coordinated attentional orienting is a highly reflexive and temporally stable response, which is less susceptible to top-down cognitive control and substantially distinguished from the attentional orienting induced by nonsocial cues. These findings extend our understandings of the distinction between social and nonsocial attention and further substantiate the specificity of social attentional orienting from a temporal-stability perspective. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

On the determination of eye gaze and arrow direction: Automaticity reconsidered.

It is a widely held view that the determination of eye gaze direction is "automatic" in various senses (e.g., innate; informationally encapsulated; triggered without intent). The determination of arrow direction is also held to be automatic (following a certain amount of learning) despite not being innate. The present experiments evaluate the automaticity assumption of both eyes and arrows in terms of an interference criterion. The results of 10 experiments support the inference that explicit judgements of eye gaze direction, when participants respond with a lateralized key press, are (a) neither automatic in the strong sense (they are interfered with by an uninformative, incongruent arrow in the display) and (b) nor are they are automatic in a weaker sense (uninformative, incongruent arrows interfere more strongly with the determination of eye gaze direction than uninformative, incongruent eyes interfere with the arrow direction task). However, the determination of arrow direction is also not strongly automatic, given that it is interfered with by irrelevant eyes. At least with respect to an interference criterion, the determination of eye gaze direction appears less prepotent than the determination of arrow direction, which itself is only weakly automatic. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A Human-Computer Control System Based on Intelligent Recognition of Eye Movements and Its Application in Wheelchair Driving

This paper presents a practical human-computer interaction system for wheelchair motion through eye tracking and eye blink detection. In this system, the pupil in the eye image has been extracted after binarization, and the center of the pupil was localized to capture the trajectory of eye movement and determine the direction of eye gaze. Meanwhile, convolutional neural networks for feature extraction and classification of open-eye and closed-eye images have been built, and machine learning was performed by extracting features from multiple individual images of open-eye and closed-eye states for input to the system. As an application of this human-computer interaction control system, experimental validation was carried out on a modified wheelchair and the proposed method proved to be effective and reliable based on the experimental results.

Abnormal visual attention to simple social stimuli in 4-month-old infants at high risk for Autism

Despite an increasing interest in detecting early signs of Autism Spectrum Disorders (ASD), the pathogenesis of the social impairments characterizing ASD is still largely unknown. Atypical visual attention to social stimuli is a potential early marker of the social and communicative deficits of ASD. Some authors hypothesized that such impairments are present from birth, leading to a decline in the subsequent typical functioning of the learning-mechanisms. Others suggested that these early deficits emerge during the transition from subcortically to cortically mediated mechanisms, happening around 2–3 months of age. The present study aimed to provide additional evidence on the origin of the early visual attention disturbance that seems to characterize infants at high risk (HR) for ASD. Four visual preference tasks were used to investigate social attention in 4-month-old HR, compared to low-risk (LR) infants of the same age. Visual attention differences between HR and LR infants emerged only for stimuli depicting a direct eye-gaze, compared to an adverted eye-gaze. Specifically, HR infants showed a significant visual preference for the direct eye-gaze stimulus compared to LR infants, which may indicate a delayed development of the visual preferences normally observed at birth in typically developing infants. No other differences were found between groups. Results are discussed in the light of the hypotheses on the origins of early social visual attention impairments in infants at risk for ASD.

The Cone of Direct Gaze: A Stable Trait.

Direct eye gaze is a potent stimulus in social interactions and is often associated with interest and approach orientation. Yet, there is remarkable variability in the range of gaze lines that people accept as being direct. A measure that is frequently used to quantify the range of gaze angles within which an observer assumes mutual gaze is the cone of direct gaze (CoDG). While individual differences in CoDG have often been examined, studies that systematically investigate the stability of an observers' CoDG over time are scarce. In two experiments, we measured the CoDG using an established paradigm and repeated the measurement after 5 min and/or after 1 week. We found high inter-individual variation, but high agreement within participants (ICCs between 0.649 and 0.855). We conclude that the CoDG can be seen as a rather stable measure, much like a personality trait.

The role of biases in the judgement processing of (un)attractive faces in body dysmorphic symptomatology

The current study tested the novel hypothesis that elevated facial appearance concern reflects a heightened tendency to judge facial appearance in terms of attractiveness, rather than in terms of attractiveness-irrelevant dimensions. Seventy-three females, reporting either high or low facial appearance concerns, were exposed to an individualised stimulus development protocol which involved a photoshoot and the subsequent selection of photographs perceived to be most and least attractive. Participants were then required to judge photographs in terms of either attractiveness or the eye gaze direction of the individual in the photograph. Results revealed that when judging attractiveness, low facial concern participants were faster to classify photographs as belonging to the attractive category rather than the unattractive category. We termed this potentially protective bias the General Attractiveness Recognition Advantage. Moreover, regardless of judgement required, high facial concern participants were faster to make responses to unattractive relative to attractive photographs of themselves. We termed this potential vulnerability bias the Efficient Processing of Unattractive Self. Further analyses revealed that the former bias associated with broader body dysmorphic symptomatology. The current findings, although inconsistent with the hypothesis under test, were novel in revealing two distinct mechanisms which distinguished high and low facial appearance concern individuals.

Age-related Differences in Expression Recognition of Faces with Direct and Averted Gaze Using Dynamic Stimuli

ABSTRACT Background: It is still an open to what extent the ecological validity of face stimuli modulates age-related differences in the recognition of facial expression; and to what extent eye gaze direction may play a role in this process. The present study tested whether age effects in facial expression recognition, also as a function of eye gaze direction, would be less pronounced in dynamic than static face displays. Method: Healthy younger and older adults were asked to recognize emotional expressions of faces with direct or averted eye gaze presented in static and dynamic format. Results: While there were no differences between the age groups in facial expression recognition ability across emotions, when considering individual expressions, age-related differences in the recognition of angry facial expressions were attenuated for dynamic compared to static stimuli. Conclusion: Our findings suggest a moderation effect of dynamic vs. static stimulus format on age-related deficits in the identification of angry facial expressions, suggesting that older adults may be less disadvantaged when recognizing angry facial expressions in more naturalistic displays. Eye gaze direction did not further modulate this effect. Findings from this study qualify and extend previous research and theory on age-related differences in facial expression recognition and have practical impact on study design by supporting the use of dynamic faces in aging research.

How the Eyes Connect to the Heart: The Influence of Eye Gaze Direction on Advertising Effectiveness

AbstractA model’s eyes are a powerful and ubiquitous visual feature in virtually any advertisement depicting a person. But does where the ad model’s eyes look matter? Integrating insights from social psychology and performance and visual art theory, we demonstrate that when the ad model’s gaze is averted (looking away from the viewer), the viewer is more readily transported into the ad narrative and responds more favorably to the ad than when the ad model’s gaze is direct (looking directly at the viewer). Five multi-method experiments (field and lab studies) illustrate that averted gaze (direct gaze) enhances narrative transportation (spokesperson credibility) to boost the effectiveness of emotional (informative) ads. Study 1 is a Facebook field study that demonstrates the effect of averted (vs. direct) gaze direction on advertising effectiveness using a real brand. Studies 2a and 2b implicate enhanced narrative transportation as the underlying process mechanism by measuring (study 2a) and manipulating (study 2b) narrative transportation. Studies 3a and 3b examine ad contexts in which direct gaze can enhance ad effectiveness: when the ad has informational (vs. emotional) appeal (study 3a), and when the viewer prefers not to identify with the negative emotional content of the ad (study 3b).