Eye Movement Behavior Research Articles

The dynamics of eye movement behavior during a contour integration task

The dynamics of eye movement behavior during a contour integration task

Algorithmic Complexity of EEG for Prognosis of Neurodegeneration in Idiopathic Rapid Eye Movement Behavior Disorder (RBD).

Idiopathic rapid eye movement sleep behavior disorder (RBD) is a serious risk factor for neurodegenerative processes such as Parkinson's disease (PD). We investigate the use of EEG algorithmic complexity derived metrics for its prognosis. We analyzed resting state EEG data collected from 114 idiopathic RBD patients and 83 healthy controls in a longitudinal study forming a cohort in which several RBD patients developed PD or dementia with Lewy bodies. Multichannel data from ~ 3 min recordings was converted to spectrograms and their algorithmic complexity estimated using Lempel-Ziv-Welch compression. Complexity measures and entropy rate displayed statistically significant differences between groups. Results are compared to those using the ratio of slow to fast frequency power, which they are seen to complement by displaying increased sensitivity even when using a few EEG channels. Poor prognosis in RBD appears to be associated with decreased complexity of EEG spectrograms stemming in part from frequency power imbalances and cross-frequency amplitude algorithmic coupling. Algorithmic complexity metrics provide a robust, powerful and complementary way to quantify the dynamics of EEG signals in RBD with links to emerging theories of brain function stemming from algorithmic information theory.

The rhythm of cognition - Effects of an auditory beat on oculomotor control in reading and sequential scanning.

Eye-movement behavior is inherently rhythmic. Even without cognitive input, the eyes never rest, as saccades are generated 3 to 4 times per second. Based on an embodied view of cognition, we asked whether mental processing in visual cognitive tasks is also rhythmic in nature by studying the effects of an external auditory beat (rhythmic background music) on saccade generation in exemplary cognitive tasks (reading and sequential scanning). While in applied settings background music has been demonstrated to impair reading comprehension, the effect of musical tempo on eye-movement control during reading or scanning has not been investigated so far. We implemented a tempo manipulation in four steps as well as a silent baseline condition, while participants completed a text reading or a sequential scanning task that differed from each other in terms of underlying cognitive processing requirements. The results revealed that increased tempo of the musical beat sped up fixations in text reading, while the presence (vs. absence) of the auditory stimulus generally reduced overall reading time. In contrast, sequential scanning was unaffected by the auditory pacemaker. These results were supported by additionally applying Bayesian inference statistics. Our study provides evidence against a cognitive load account (i.e., that spare resources during low-demand sequential scanning allow for enhanced processing of the external beat). Instead, the data suggest an interpretation in favor of a modulation of the oculomotor saccade timer by irrelevant background music in cases involving highly automatized oculomotor control routines (here: in text reading).

Eye movements in scene perception while listening to slow and fast music.

To date, there is insufficient knowledge of how visual exploration of outdoor scenes may be influenced by the simultaneous processing of music. Eye movements during viewing various outdoor scenes while listening to music at either a slow or fast tempo or in silence were measured. Significantly shorter fixations were found for viewing urban scenes com-pared with natural scenes, but there was no interaction between the type of scene and the acoustic conditions. The results revealed shorter fixation durations in the silent control condition in the range 30 ms, compared to both music conditions but, in contrast to previ-ous studies, these differences were non-significant. Moreover, we did not find differences in eye movements between music conditions with a slow or fast tempo. It is supposed that the type of musical stimuli, the specific tempo, the specific experimental procedure, and the engagement of participants in listening to background music while processing visual information may be important factors that influence attentional processes, which are mani-fested in eye-movement behavior.

Eye movements in scene perception while listening to slow and fast music

To date, there is insufficient knowledge of how visual exploration of outdoor scenes may be influenced by the simultaneous processing of music. Eye movements during viewing various outdoor scenes while listening to music at either a slow or fast tempo or in silence were measured. Significantly shorter fixations were found for viewing urban scenes compared with natural scenes, but there was no interaction between the type of scene and the acoustic conditions. The results revealed shorter fixation durations in the silent control condition in the range 30 ms, compared to both music conditions but, in contrast to previous studies, these differences were non-significant. Moreover, we did not find differences in eye movements between music conditions with a slow or fast tempo. It is supposed that the type of musical stimuli, the specific tempo, the specific experimental procedure, and the engagement of participants in listening to background music while processing visual information may be important factors that influence attentional processes, which are manifested in eye-movement behavior.

Detrimental effects of carryover of eye movement behaviour on hazard perception accuracy: Effects of driver experience, difficulty of task, and hazardousness of road

Novice drivers are more likely to be involved in road accidents than experienced drivers and this relates to their lower performance in hazard perception tasks. Hazard perception performed under dual task conditions is also affected differentially due to driver experience. In this study, we explore the detrimental effect of vertical eye-movement carryover from one task to a second task in drivers of different levels of experience, whilst accounting for road conditions. Participants searched letters presented horizontally, vertically, or in a random array. Following this, they identified a hazard in a driving scene. Carryover of eye movements from the letter search to the driving scene was observed and participants were quicker and more accurate when responding to a hazard following horizontal scanning, compared to following vertical and random scanning. Furthermore, while carryover of eye movements was equivalent for all participants, the negative effect it had on hazard identification accuracy was less prominent in experienced drivers, especially when viewing the most hazardous of images. These results indicate that carryover of eye movements is another potentially distracting effect that can impact on the ability and safety of novice drivers.

A Validation of Automatically-Generated Areas-of-Interest in Videos of a Face for Eye-Tracking Research.

When mapping eye-movement behavior to the visual information presented to an observer, Areas of Interest (AOIs) are commonly employed. For static stimuli (screen without moving elements), this requires that one AOI set is constructed for each stimulus, a possibility in most eye-tracker manufacturers' software. For moving stimuli (screens with moving elements), however, it is often a time-consuming process, as AOIs have to be constructed for each video frame. A popular use-case for such moving AOIs is to study gaze behavior to moving faces. Although it is technically possible to construct AOIs automatically, the standard in this field is still manual AOI construction. This is likely due to the fact that automatic AOI-construction methods are (1) technically complex, or (2) not effective enough for empirical research. To aid researchers in this field, we present and validate a method that automatically achieves AOI construction for videos containing a face. The fully-automatic method uses an open-source toolbox for facial landmark detection, and a Voronoi-based AOI-construction method. We compared the position of AOIs obtained using our new method, and the eye-tracking measures derived from it, to a recently published semi-automatic method. The differences between the two methods were negligible. The presented method is therefore both effective (as effective as previous methods), and efficient; no researcher time is needed for AOI construction. The software is freely available from https://osf.io/zgmch/.

Effects of Point of Aim on the Accuracy and Eye Movement Behavior in Bowling: A Pilot Study

Background: A common research question in far-target aiming has been the importance and significance of the final visual fixation before movement initiation. In rolling tasks, such as 10-pin bowling, location of point of aim needs not be at the final target, the pins, but may be located at any point along the trajectory of the ball. Objective: Specific interest in the present experiment has focused on the relationship between visual point of aim and performance accuracy, and the relationship of visual control strategies utilized by expert performers. Methods: Skilled bowlers (N=7) performed 20 trials per condition concentrating on visual targets in different distances along the bowling lane (20, 40, 60 feet, and self-selected). Ball trajectory was tracked using a video based system and eye movement was measured using an eye tracking system. Results: Deviation of the ball from the visual target increased with visual target distance, while deviation of the ball from the pins was the lowest in the self-selected visual targets, followed by aiming at the pins. The final fixation duration before movement initiation was not associated with ball accuracy regardless of visual target locations. However, results demonstrated the association between final fixation duration task difficulty, that is, longer final fixation duration with increased visual target distance. Conclusion: The results indicate that visual fixations before movement initiation are uncharacteristically long while visual fixations just before the completion of the movement are relatively short.

Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning

Metacomprehension is key to successful learning of complex topics when using multimedia materials. The goal of this study was to determine if eye-movement dyads could be: (1) identified by sequence mining techniques, and (2) aligned with self-reported metacognitive judgments during learning with multimedia materials that contain conceptual discrepancies designed to interfere with participants' metacomprehension. Thirty-two undergraduate students' metacognitive judgments were examined with RM-MANOVAs, and sequential pattern mining and differential sequence mining were conducted on their eye movements as they learned with complex multimedia materials. Additionally, we distinguished between event- (i.e., if participants looked at specific areas of the content) and duration-based (i.e., if participants looked at areas of interest [AOIs] for a medium or long amount of time) eye-movement dyads to assess if qualitative and quantitative differences existed in their eye-movement behaviors. For content with text and graph discrepancies, results indicated participants' metacognitive judgments were lower and less accurate, and more fixation dyads were found between the text and graph. Furthermore, specific dyads of different length (i.e., long fixations on the graph to medium fixations on the text) fixations may align with lowered and inaccurate metacognitive judgments for content with text and graph discrepancies. This study begins to address how to identify behavioral indices of metacomprehension processes during multimedia learning.

Eye Tracking as a Tool for the Detection of Simulated Memory Impairment

Effects of feigned memory impairment on eye-movement behavior were investigated. Participants encoded scene–face pairs and were tested with displays containing three studied faces preceded by a studied scene. Half of these displays contained the face that had previously been associated with the scene cue, while the remainders did not. Participants made presence/absence judgments while eye movements were recorded and either attempted to perform optimally (controls) or feign impairment (simulators). While explicit recognition was at chance among simulators, both groups looked disproportionately at associates early in the trial. The magnitude of this effect was matched across groups and significant even when simulators made incorrect recognition responses. Eye tracking may have potential as a tool for the detection of concealed recognition and warrants further research into its efficacy and underlying mechanisms.

Influence of visual cues on head and eye movements during listening tasks in multi-talker audiovisual environments with animated characters

Recent studies of hearing aid benefits indicate that head movement behavior influences performance. To systematically assess these effects, movement behavior must be measured in realistic communication conditions. For this, the use of virtual audiovisual environments with animated characters as visual stimuli has been proposed. It is unclear, however, how these animations influence the head- and eye-movement behavior of subjects. Here, two listening tasks were carried out with a group of 14 young normal hearing subjects to investigate the influence of visual cues on head- and eye-movement behavior; on combined localization and speech intelligibility task performance; as well as on perceived speech intelligibility, perceived listening effort and the general impression of the audiovisual environments. Animated characters with different lip-syncing and gaze patterns were compared to an audio-only condition and to a video of real persons. Results show that movement behavior, task performance, and perception were all influenced by visual cues. The movement behavior of young normal hearing listeners in animation conditions with lip-syncing was similar to that in the video condition. These results in young normal hearing listeners are a first step towards using the animated characters to assess the influence of head movement behavior on hearing aid performance.

Effects of Deep Brain Stimulation on Eye Movements and Vestibular Function.

Discovery of inter-latching circuits in the basal ganglia and invention of deep brain stimulation (DBS) for their modulation is a breakthrough in basic and clinical neuroscience. The DBS not only changes the quality of life of hundreds of thousands of people with intractable movement disorders, but it also offers a unique opportunity to understand how the basal ganglia interacts with other neural structures. An attractive yet less explored area is the study of DBS on eye movements and vestibular function. From the clinical perspective such studies provide valuable guidance in efficient programming of stimulation profile leading to optimal motor outcome. From the scientific standpoint such studies offer the ability to assess the outcomes of basal ganglia stimulation on eye movement behavior in cognitive as well as in motor domains. Understanding the influence of DBS on ocular motor function also leads to analogies to interpret its effects on complex appendicular and axial motor function. This review focuses on the influence of globus pallidus, subthalamic nucleus, and thalamus DBS on ocular motor and vestibular functions. The anatomy and physiology of basal ganglia, pertinent to the principles of DBS and ocular motility, is discussed. Interpretation of the effects of electrical stimulation of the basal ganglia in Parkinson's disease requires understanding of baseline ocular motor function in the diseased brain. Therefore we have also discussed the baseline ocular motor deficits in these patients and how the DBS changes such functions.

How task demands influence scanpath similarity in a sequential number-search task

More and more researchers are considering the omnibus eye movement sequence—the scanpath—in their studies of visual and cognitive processing (e.g. Hayes, Petrov, & Sederberg, 2011; Madsen, Larson, Loschky, & Rebello, 2012; Ni et al., 2011; von der Malsburg & Vasishth, 2011). However, it remains unclear how recent methods for comparing scanpaths perform in experiments producing variable scanpaths, and whether these methods supplement more traditional analyses of individual oculomotor statistics. We address this problem for MultiMatch (Jarodzka et al., 2010; Dewhurst et al., 2012), evaluating its performance with a visual search-like task in which participants must fixate a series of target numbers in a prescribed order. This task should produce predictable sequences of fixations and thus provide a testing ground for scanpath measures. Task difficulty was manipulated by making the targets more or less visible through changes in font and the presence of distractors or visual noise. These changes in task demands led to slower search and more fixations. Importantly, they also resulted in a reduction in the between-subjects scanpath similarity, demonstrating that participants’ gaze patterns became more heterogenous in terms of saccade length and angle, and fixation position. This implies a divergent strategy or random component to eye-movement behaviour which increases as the task becomes more difficult. Interestingly, the duration of fixations along aligned vectors showed the opposite pattern, becoming more similar between observers in 2 of the 3 difficulty manipulations. This provides important information for vision scientists who may wish to use scanpath metrics to quantify variations in gaze across a spectrum of perceptual and cognitive tasks.

Eye movements and postural control in dyslexic children performing different visual tasks.

The aim of this study was to examine eye movements and postural control performance among dyslexic children while reading a text and performing the Landolt reading task. Fifteen dyslexic and 15 non-dyslexic children were asked to stand upright while performing two experimental visual tasks: text reading and Landolt reading. In the text reading task, children were asked to silently read a text displayed on a monitor, while in the Landolt reading task, the letters in the text were replaced by closed circles and Landolt rings, and children were asked to scan each circle/ring in a reading-like fashion, from left to right, and to count the number of Landolt rings. Eye movements (Mobile T2®, SuriCog) and center of pressure excursions (Framiral®, Grasse, France) were recorded. Visual performance variables were total reading time, mean duration of fixation, number of pro- and retro-saccades, and amplitude of pro-saccades. Postural performance variable was the center of pressure area. The results showed that dyslexic children spent more time reading the text and had a longer duration of fixation than non-dyslexic children. However, no difference was observed between dyslexic and non-dyslexic children in the Landolt reading task. Dyslexic children performed a higher number of pro- and retro-saccades than non-dyslexic children in both text reading and Landolt reading tasks. Dyslexic children had smaller pro-saccade amplitude than non-dyslexic children in the text reading task. Finally, postural performance was poorer in dyslexic children than in non-dyslexic children. Reading difficulties in dyslexic children are related to eye movement strategies required to scan and obtain lexical and semantic meaning. However, postural control performance, which was poor in dyslexic children, is not related to lexical and semantic reading requirements and might not also be related to different eye movement behavior.

Toward Improved Child–Robot Interaction by Understanding Eye Movements

Globally, 1 in 160 children has an autism spectrum disorder (ASD). Problems with joint attention (JA) are core features of ASDs. Here, we investigate how typically developing (TD) children and children with ASD initiate JA with a gaze contingent avatar. Thirty-one participants with ASD and 33 TD matched controls directed an avatar’s gaze to a series of referent images. Observing pupil diameter and gaze location data, we explore how distinguishing the two groups as well as their different eye-movement behaviors could be used to improve child–robot interaction. With a sequence to sequence neural network we distinguish if a child is TD or has an ASD, then using ${K}$ -means clustering, we group pupil diameters and gaze locations independently to determine the child’s attention level as well as to refine the classification process. Using these metrics, we could trigger appropriate responses from the robot to increase the level of attention from the child toward the robot. Results show significant differences between the eye behaviors of individuals with ASDs and those without. Further to this, we achieve a 79.76% classification accuracy when using pupil diameter data to distinguish the two groups.

SoTCM: a scene-oriented task complexity metric for gaze-supported teleoperation tasks

Recent developments in human–robot interaction (HRI) research have heightened the need to incorporate indirect human signals to implicitly facilitate intuitive human–guided interactions. Eye-gaze has been widely used nowadays as an input interface in multi-modal teleoperation scenarios due to their advantage in revealing human intentions and forthcoming actions. However, to date, there has been no discussion about how the structure of the environment, that the human is interacting with, could affect the complexity of the teleoperation task. In this paper, a new metric named “Scene-oriented Task Complexity Metric” (SoTCM) is proposed to estimate the complexity of a certain scene that is involved in eye-gaze-supported teleoperation tasks. The proposed SoTCM objectively estimates the effort that could be exerted by the human operator in terms of the expected time required to point at all the informative locations retrieved from the scene under discussion. The developed SoTCM depends on both the density and distribution of the informative locations in the scene, while incorporates the eye movement behavior found in the psychology literature. The proposed SoTCM is subjectively validated by using the time-to-complete index in addition to the standard (NASA-TLX) workload measure in eight varying structure scenes. Results confirmed a significant relation between SoTCM and the measured task workload which endorses the applicability of using SoTCM in predicting scene complexities and subsequently the task workload in advance.

Differences in eye movements while viewing images with various levels of restorativeness

Previous studies have suggested that analysis of eye movements can provide new insights into the research of features of restorative environments in terms of the Attention Restoration Theory. Humans prefer images of nature over urban scenes, and eye-movement behavior seems be different across these categories. To enhance these findings, we investigated differences in eye movements while viewing nature scenes versus two categories of urban scenes. Apart from ordinary urban scenes, we also included the category of scenic images of old cities. The analysis of eye movements indicated lower eye movement activity while viewing nature scenes relative to ordinary urban scenes and also minor differences in eye movements between urban scenes and old city scenes. The results are explained in terms of lower cognitive effort while processing nature scenes in contrast to images of built environments.

Cognition-Cognizant Sentiment Analysis With Multitask Subjectivity Summarization Based on Annotators' Gaze Behavior

For document level sentiment analysis (SA), Subjectivity Extraction, ie., extracting the relevant subjective portions of the text that cover the overall sentiment expressed in the document, is an important step. Subjectivity Extraction, however, is a hard problem for systems, as it demands a great deal of world knowledge and reasoning. Humans, on the other hand, are good at extracting relevant subjective summaries from an opinionated document (say, a movie review), while inferring the sentiment expressed in it. This capability is manifested in their eye-movement behavior while reading: words pertaining to the subjective summary of the text attract a lot more attention in the form of gaze-fixations and/or saccadic patterns. We propose a multi-task deep neural framework for document level sentiment analysis that learns to predict the overall sentiment expressed in the given input document, by simultaneously learning to predict human gaze behavior and auxiliary linguistic tasks like part-of-speech and syntactic properties of words in the document. For this, a multi-task learning algorithm based on multi-layer shared LSTM augmented with task specific classifiers is proposed. With this composite multi-task network, we obtain performance competitive with or better than state-of-the-art approaches in SA. Moreover, the availability of gaze predictions as an auxiliary output helps interpret the system better; for instance, gaze predictions reveal that the system indeed performs subjectivity extraction better, which accounts for improvement in document level sentiment analysis performance.

New approaches to the analysis of eye movement behaviour across expertise while viewing brain MRIs

Brain tumour detection and diagnosis requires clinicians to inspect and analyse brain magnetic resonance images. Eye-tracking is commonly used to examine observers’ gaze behaviour during such medical image interpretation tasks, but analysis of eye movement sequences is limited. We therefore used ScanMatch, a novel technique that compares saccadic eye movement sequences, to examine the effect of expertise and diagnosis on the similarity of scanning patterns. Diagnostic accuracy was also recorded. Thirty-five participants were classified as Novices, Medics and Experts based on their level of expertise. Participants completed two brain tumour detection tasks. The first was a whole-brain task, which consisted of 60 consecutively presented slices from one patient; the second was an independent-slice detection task, which consisted of 32 independent slices from five different patients. Experts displayed the highest accuracy and sensitivity followed by Medics and then Novices in the independent-slice task. Experts showed the highest level of scanning pattern similarity, with medics engaging in the least similar scanning patterns, for both the whole-brain and independent-slice task. In the independent-slice task, scanning patterns were the least similar for false negatives across all expertise levels and most similar for experts when they responded correctly. These results demonstrate the value of using ScanMatch in the medical image perception literature. Future research adopting this tool could, for example, identify cases that yield low scanning similarity and so provide insight into why diagnostic errors occur and ultimately help in training radiologists.

Visual Processing during Short-Term Memory Binding in Mild Alzheimer's Disease.

Patients with Alzheimer's disease (AD) typically present with attentional and oculomotor abnormalities that can have an impact on visual processing and associated cognitive functions. Over the last few years, we have witnessed a shift toward the analyses of eye movement behaviors as a means to further our understanding of the pathophysiology of common disorders such as AD. However, little work has been done to unveil the link between eye moment abnormalities and poor performance on cognitive tasks known to be markers for AD patients, such as the short-term memory-binding task. We analyzed eye movement fixation behaviors of thirteen healthy older adults (Controls) and thirteen patients with probable mild AD while they performed the visual short-term memory binding task. The short-term memory binding task asks participants to detect changes across two consecutive arrays of two bicolored object whose features (i.e., colors) have to be remembered separately (i.e., Unbound Colors), or combined within integrated objects (i.e., Bound Colors). Patients with mild AD showed the well-known pattern of selective memory binding impairments. This was accompanied by significant impairments in their eye movements only when they processed Bound Colors. Patients with mild AD remarkably decreased their mean gaze duration during the encoding of color-color bindings. These findings open new windows of research into the pathophysiological mechanisms of memory deficits in AD patients and the link between its phenotypic expressions (i.e., oculomotor and cognitive disorders). We discuss these findings considering current trends regarding clinical assessment, neural correlates, and potential avenues for robust biomarkers.