Simulated Eye Movements Research Articles

An open-source software for the simulation of fixational eye movements

Abstract Laser-scanning confocal microscopy of the human cornea acquires in vivoimages of corneal tissues with cellular resolution, which offers diagnostic potential for a variety of diseases. However, involuntary fixational eye movements induce motion artefacts that pose a challenge for accurate morphometric analysis and particularly for preceding image data fusion steps. Different image registration algorithms promise to eliminate such artefacts, but objective, quantitative evaluation of the registration results is difficult because of the lack of ground truth data. Simulation approaches offer a possibility to close this gap. Existing open-source software for the simulation of fixational eye movements is currently limited to creating drift movement. The present contribution proposes complementary models for microsaccade and tremor components to provide complete simulated fixational eye movement paths. In addition, it reports results from an extensive examination of different model parameter combinations. It is shown that the microsaccade model is capable of reproducing intermicrosaccade intervals that closely resemble experimental data. A software implementation is provided as open-source Python modules.

Language models outperform cloze predictability in a cognitive model of reading.

Although word predictability is commonly considered an important factor in reading, sophisticated accounts of predictability in theories of reading are lacking. Computational models of reading traditionally use cloze norming as a proxy of word predictability, but what cloze norms precisely capture remains unclear. This study investigates whether large language models (LLMs) can fill this gap. Contextual predictions are implemented via a novel parallel-graded mechanism, where all predicted words at a given position are pre-activated as a function of contextual certainty, which varies dynamically as text processing unfolds. Through reading simulations with OB1-reader, a cognitive model of word recognition and eye-movement control in reading, we compare the model's fit to eye-movement data when using predictability values derived from a cloze task against those derived from LLMs (GPT-2 and LLaMA). Root Mean Square Error between simulated and human eye movements indicates that LLM predictability provides a better fit than cloze. This is the first study to use LLMs to augment a cognitive model of reading with higher-order language processing while proposing a mechanism on the interplay between word predictability and eye movements.

Eye gaze as a biomarker in the recognition of autism spectrum disorder using virtual reality and machine learning: A proof of concept for diagnosis.

The core symptoms of autism spectrum disorder (ASD) mainly relate to social communication and interactions. ASD assessment involves expert observations in neutral settings, which introduces limitations and biases related to lack of objectivity and does not capture performance in real-world settings. To overcome these limitations, advances in technologies (e.g., virtual reality) and sensors (e.g., eye-tracking tools) have been used to create realistic simulated environments and track eye movements, enriching assessments with more objective data than can be obtained via traditional measures. This study aimed to distinguish between autistic and typically developing children using visual attention behaviors through an eye-tracking paradigm in a virtual environment as a measure of attunement to and extraction of socially relevant information. The 55 children participated. Autistic children presented a higher number of frames, both overall and per scenario, and showed higher visual preferences for adults over children, as well as specific preferences for adults' rather than children's faces on which looked more at bodies. A set of multivariate supervised machine learning models were developed using recursive feature selection to recognize ASD based on extracted eye gaze features. The models achieved up to 86% accuracy (sensitivity=91%) in recognizing autistic children. Our results should be taken as preliminary due to the relatively small sample size and the lack of an external replication dataset. However, to our knowledge, this constitutes a first proof of concept in the combined use of virtual reality, eye-tracking tools, and machine learning for ASD recognition. LAY SUMMARY: Core symptoms in children with ASD involve social communication and interaction. ASD assessment includes expert observations in neutral settings, which show limitations and biases related to lack of objectivity and do not capture performance in real settings. To overcome these limitations, this work aimed to distinguish between autistic and typically developing children in visual attention behaviors through an eye-tracking paradigm in a virtual environment as a measure of attunement to, and extraction of, socially relevant information.

Neural sensitization improves encoding fidelity in the primate retina

An animal’s motion through the environment can induce large and frequent fluctuations in light intensity on the retina. These fluctuations pose a major challenge to neural circuits tasked with encoding visual information, as they can cause cells to adapt and lose sensitivity. Here, we report that sensitization, a short-term plasticity mechanism, solves this difficult computational problem by maintaining neuronal sensitivity in the face of these fluctuations. The numerically dominant output pathway in the macaque monkey retina, the midget (parvocellular-projecting) pathway, undergoes sensitization under specific conditions, including simulated eye movements. Sensitization is present in the excitatory synaptic inputs from midget bipolar cells and is mediated by presynaptic disinhibition from a wide-field mechanism extending >0.5 mm along the retinal surface. Direct physiological recordings and a computational model indicate that sensitization in the midget pathway supports accurate sensory encoding and prevents a loss of responsiveness during dynamic visual processing.

Amphiphilic additives in silicone oil tamponade and emulsification: an eye-on-a-chip study.

Recently, chemically modified silicone oil has been demonstrated as a reservoir for sustained release of intraocular drugs, many of which might be amphiphilic in nature. In this work, we study the effect of amphiphilic additives in silicone oil on emulsification under eye-like movements. Three silicone-oil-soluble surfactants, namely DC749, MQ1640 and FZ2233, were used as model amphiphilic additives. The change of viscosity was measured by a rheometer in the cone-and-plate geometry. The interfacial tension (IFT) between silicone oil and model aqueous phase was measured by pendant drop tensiometry. Emulsification of silicone oil was induced by simulated saccadic eye movements on a cell-coated eye-on-a-chip platform for 4 days. The number of emulsified silicone oil droplets observed in the aqueous phase was assessed daily by optical microscopy. Significantly more emulsified droplets were formed in silicone oil with DC749 or MQ1640 (P<0.05). However, such increase was not directly related to the change in IFT nor viscosity. Moreover, water droplets were also found in the silicone oil, but not in the control silicone oil without additive. The amphiphilic substances in silicone oil promoted emulsification. Besides typical oil-in-water drops that normally affect the eye, water-in-oil drops were also formed. Before silicone oil could be considered as a vehicle for drug delivery, the nature of the drug and its possible effect on emulsification and therefore on the pharmacokinetics needs to be investigated. An additional concern is that water-in-oil droplets in the eye would affect the optical clarity of silicone oil and might cause visual symptoms.

A Geometrical Approach to Human Saccade Simulation.

Modeling and simulation of human eye movement have a wide range of applications in many domains. Various attempts have been made to model and simulate eye movements in a physically accurate manner. All the existing models show limitations and problems in simulating secondary and tertiary eye movements. Recent investigation of pulley models (passive and active hypotheses) in representing human eye motion has recognized mathematical complexity in modeling eye behavior. Sophisticated techniques of modeling are required to investigate eye movements. This paper presents a procedure for eye movement simulation through geometrical modeling (an OpenSim script with its recent MATLAB binding) for binocular vision. First order neural dynamics with Millard's muscle model are used to actuate six Extra Ocular Muscles (EOMs). Pulse-step inputs are used to generate the muscle forces around the eye globe. The implemented model is successful in simulating horizontal and vertical movements of the human eye with respect to the prescribed activation. The developed technique is evaluated using responses from lumped parameter models and EOG recordings.

Adhesion of silicone oil and emulsification: an invitro assessment using a microfluidic device and 'Eye-on-a-Chip'.

Silicone oil (SiO) with additives of high-molecular-weight (HMW) SiO molecules, eases both the injection and removal. When used inside an eye, it remains unclear how increasing extensional viscosity of SiO might reduce emulsification. Using cell-lined models, this study aims to understand the reason why SiO with HMW is less prone to emulsification. The adhesion of SiO was studied and recorded in a cell-coated microchannel by optical microscopy. The resistance of SiO against emulsification was tested on another cell-coated eye-on-a-chip platform, which was subject to simulated saccadic eye movements, for 4days. Silicone oil (SiO) candidates with HMW, SiOHMW 2000 and SiOHMW 5000 , and their counterparts SiO2000 and SiO5000 without HMW, were tested. The quantity of the SiO emulsified droplets formed was assessed daily by optical microscopy. When flowing in the microchannel, SiO adheres on the cell-coated substrate. The number of droplets is generally lower in SiO with HMW than their counterparts. At the end of the experiment, the average numbers of droplets in SiO2000 (29.1±41.0) and SiO5000 (9.1±19.5) are higher than those in SiOHMW 2000 (6.0±4.5) and SiOHMW 5000 (5.6±4.1). A new mechanism of emulsification of SiO is proposed: SiO adheres to ocular tissue to form emulsified droplets. The presence of HMW, which increases the extensional viscosity, may resist the break-up of SiO from the substrate to form emulsified droplets. When tested in a physiologically representative platform, the use of HMW in SiO generally reduces the number of droplets formed invitro.

Computational model of brain-stem circuit for state-dependent control of hypoglossal motoneurons.

In patients with obstructive sleep apnea (OSA), the pharyngeal muscles become relaxed during sleep, which leads to a partial or complete closure of upper airway. Experimental studies suggest that withdrawal of noradrenergic and serotonergic drives importantly contributes to depression of hypoglossal motoneurons and, therefore, may contribute to OSA pathophysiology; however, specific cellular and synaptic mechanisms remain unknown. In this new study, we developed a biophysical network model to test the hypothesis that, to explain experimental observations, the neuronal network for monoaminergic control of excitability of hypoglossal motoneurons needs to include excitatory and inhibitory perihypoglossal interneurons that mediate noradrenergic and serotonergic drives to hypoglossal motoneurons. In the model, the state-dependent activation of the hypoglossal motoneurons was in qualitative agreement with in vivo data during simulated rapid eye movement (REM) and non-REM sleep. The model was applied to test the mechanisms of action of noradrenergic and serotonergic drugs during REM sleep as observed in vivo. We conclude that the proposed minimal neuronal circuit is sufficient to explain in vivo data and supports the hypothesis that perihypoglossal interneurons may mediate state-dependent monoaminergic drive to hypoglossal motoneurons. The population of the hypothesized perihypoglossal interneurons may serve as novel targets for pharmacological treatment of OSA. NEW & NOTEWORTHY In vivo studies suggest that during rapid eye movement sleep, withdrawal of noradrenergic and serotonergic drives critically contributes to depression of hypoglossal motoneurons (HMs), which innervate the tongue muscles. By means of a biophysical model, which is consistent with a broad range of empirical data, we demonstrate that the neuronal network controlling the excitability of HMs needs to include excitatory and inhibitory interneurons that mediate noradrenergic and serotonergic drives to HMs.

Intraoperative head drift and eye movement: two under addressed challenges during cataract surgery.

To objectively measure head drift during cataract surgery, and subjectively simulate eye movements and assess impact on surgical technique. Twelve consecutively recorded routine cataract operations in the Tennent Institute of Ophthalmology, Gartnavel General Hospital, Glasgow, were reviewed. The speculum was used as a fixed point and correlated with a superimposed virtual ruler to measure maximum head drift in each direction throughout the operations. To simulate intraoperative eye movement, we attached string to the cataract surgical simulator (Eyesi) eye and manually induced abduction and adduction. A calibrated scale secured to the Eyesi head ensured 5 mm eye movements were consistently created. Ophthalmology trainees performed the continuous curvilinear capsulorhexis (CCC) exercise without and with sequential eye movements. Movements were induced every three seconds. Scores were compared using a paired Student's T-test. Mean head drift in the surgical recordings was 3.1 mm medially (range 2-7 mm), 2.9 mm laterally (range 2-4 mm), 2.6 mm superiorly (range 1-5 mm), and 1.9 mm inferiorly (range 1-4 mm). In 11 of 12 cases, the operating microscope had to be adjusted for head drift. Six junior trainees completed the CCC module on the Eyesi without then with eye movements. After introducing eye movements the mean Eyesi score reduced from 92.7 to 76.9 (P = 0.014), 'roundness of rhexis' score reduced from 89.4 to 57.5 (P = 0.020), and trainees operated 17 s faster (P = 0.016). This study objectively demonstrates the under-reported clinical scenario of head drift during cataract surgery. By manipulating the Eyesi we have shown that eye movements reduce the quality of cataract surgery.

Distribution of fluorescein sodium and triamcinolone acetonide in the simulated liquefied and vitrectomized Vitreous Model with simulated eye movements

Intravitreal administration is the method of choice for drug delivery to the posterior segment of the eye with special emphasis on the vitreous body and its surrounding retinal vasculature. In order to gain a better understanding of the underlying distribution processes, an in vitro model simulating the vitreous body (Vitreous Model, VM) and a system simulating the impact of movement on the VM (Eye Movement System, EyeMoS) was previously developed. In the study reported here, these systems were modified in regard to a standardized injection procedure, the diversity of simulated eye movements, extended periods of investigation, the opportunity to simulate the state after vitrectomy and in considering the physiological temperature. Fluorescein sodium (FS) and triamcinolone acetonide (TA) were used as (model) drugs to examine the drug distribution within the VM. Vitrectomy was simulated by replacing half the volume of the polyacrylamide gel that was used as vitreous substitute with the clinically used Siluron® 5000 whereas for a simulated liquefaction half the volume of the gel was replaced by buffer. A simulated liquefaction caused a 12-fold faster distribution of FS compared to the simulated juvenile VM, which was most likely caused by convective forces and mass transfer. Also, the injection technique (injection into the gel or into the buffer compartment) influenced the resulting distribution pattern. Without any liquefaction, the previously described initial injection channel occurred with both (model) drugs and, in the case of TA, remained almost unchanged during the investigation period of 72h. Simulating vitrectomized eyes, TA did not spread uniformly, but either remained in the depot or strongly sedimented within the VM suggesting that a homogenous distribution of a TA suspension is highly unlikely in vitrectomized eyes. High variabilities were observed with ex vivo animal eyes, demonstrating the limited benefit of explanted tissues for such distribution studies. The combination of the modified VM and EyeMoS seems a valuable tool for characterizing intravitreal dosage forms in a reproducible simulation of diversified eye movements and a partially liquefied or vitrectomized vitreous body.

Where are you heading? Flexible integration of retinal and extra-retinal cues during self-motion perception.

As we move forward in the environment, we experience a radial expansion of the retinal image, wherein the center corresponds to the instantaneous direction of self-motion. Humans can precisely perceive their heading direction even when the retinal motion is distorted by gaze shifts due to eye/body rotations. Previous studies have suggested that both retinal and extra-retinal strategies can compensate for the retinal image distortion. However, the relative contributions of each strategy remain unclear. To address this issue, we devised a two-alternative-headings discrimination task, in which participants had either real or simulated pursuit eye movements. The two conditions had the same retinal input but either with or without extra-retinal eye movement signals. Thus, the behavioral difference between conditions served as a metric of extra-retinal contribution. We systematically and independently manipulated pursuit speed, heading speed, and the reliability of retinal signals. We found that the levels of extra-retinal contributions increased with increasing pursuit speed (stronger extra-retinal signal), and with decreasing heading speed (weaker retinal signal). In addition, extra-retinal contributions also increased as we corrupted retinal signals with noise. Our results revealed that the relative magnitude of retinal and extra-retinal contributions was not fixed but rather flexibly adjusted to each specific task condition. This task-dependent, flexible integration appears to take the form of a reliability-based weighting scheme that maximizes heading performance.

The effect of saccadic eye movements on the sensor-level magnetoencephalogram.

We used a combination of simulation and recordings from human subjects to characterize how saccadic eye movements affect the magnetoencephalogram (MEG). We used simulated saccadic eye movements to generate simulated MEG signals. We also recorded the MEG signals from three healthy adults to 5° magnitude saccades that were vertical up and down, and horizontal left and right. The signal elicited by the rotating eye dipoles is highly dependent on saccade direction, can cover a large area, can sometimes have a non-intuitive trajectory, but does not significantly extend above approximately 30Hz in the frequency domain. In contrast, the saccadic spikes (which are primarily monophasic pulses, but can be biphasic) are highly localized to the lateral frontal regions for all saccade directions, and in the frequency domain extend up past 60Hz. Gamma band saccadic artifact is spatially localized to small regions regardless of saccade direction, but beta band and lower frequency saccadic artifact have broader spatial extents that vary strongly as a function of saccade direction. We have here characterized the MEG saccadic artifact in both the spatial and the frequency domains for saccades of different directions. This could be important in ruling in or ruling out artifact in MEG recordings.

Influence of Siluron® insertion on model drug distribution in the simulated vitreous body

Abstract Biorelevant in vitro test systems may be helpful to understand the in vivo behaviour of modern intravitreal dosage forms such as implants and injections. The already presented Vitreous Model (VM) in combination with the Eye Movement System (EyeMoS) was used to simulate the situation after a vitrectomy in combination with Siluron® silicone oil (SO) insertion in vitro and to investigate the distribution of the model drug fluorescein sodium (FS) within the modified VM. The state after a vitrectomy was simulated in vitro by replacing half the volume of the gelled vitreous substitute by SO. Under consideration of simulated eye movements the position of SO towards the simulated vitreous body was examined. Furthermore, the influence of two different injection techniques was studied. On the one hand, FS was injected directly into the gel and on the other hand the injection was set through the gel in order to directly reach the SO. Independent of the injection technique, it was shown that the model drug distributed almost exclusively into the gel and not into the SO. This can be explained with the backflow of FS into the gel and the lack of solubility in the SO. Using the modified VM and EyeMoS, the in vitro characterization of drug release and distribution behaviour of intravitreal injections can be performed under consideration of a simulated vitrectomy.

Joint representation of translational and rotational components of optic flow in parietal cortex

Terrestrial navigation naturally involves translations within the horizontal plane and eye rotations about a vertical (yaw) axis to track and fixate targets of interest. Neurons in the macaque ventral intraparietal (VIP) area are known to represent heading (the direction of self-translation) from optic flow in a manner that is tolerant to rotational visual cues generated during pursuit eye movements. Previous studies have also reported that eye rotations modulate the response gain of heading tuning curves in VIP neurons. We tested the hypothesis that VIP neurons simultaneously represent both heading and horizontal (yaw) eye rotation velocity by measuring heading tuning curves for a range of rotational velocities of either real or simulated eye movements. Three findings support the hypothesis of a joint representation. First, we show that rotation velocity selectivity based on gain modulations of visual heading tuning is similar to that measured during pure rotations. Second, gain modulations of heading tuning are similar for self-generated eye rotations and visually simulated rotations, indicating that the representation of rotation velocity in VIP is multimodal, driven by both visual and extraretinal signals. Third, we show that roughly one-half of VIP neurons jointly represent heading and rotation velocity in a multiplicatively separable manner. These results provide the first evidence, to our knowledge, for a joint representation of translation direction and rotation velocity in parietal cortex and show that rotation velocity can be represented based on visual cues, even in the absence of efference copy signals.

Non-linear retinal processing supports invariance during fixational eye movements

Fixational eye movements can rapidly shift the retinal image, but typically remain unnoticed. We identify and simulate a model mechanism for the suppression of erroneous motion signals under fixational eye movements. This mechanism exploits the non-linearities common to many classes of large retinal ganglion cells in the mammalian retina, and negates the need for extra-retinal signals or explicit gaze information. When tested using natural images undergoing simulated fixational eye movements, our model successfully distinguishes “real world” motion from retinal motion induced by eye movements. In addition, this model suggests a possible explanation for several fixational eye movement related visual illusions such as the Ouchi–Spillmann and “Out-of-focus” illusions.

In Vitro Modeling of Emulsification of Silicone Oil as Intraocular Tamponade Using Microengineered Eye-on-a-Chip.

There is a lack of a standardized methodology or a physiologically realistic in vitro model to investigate silicone oil (SO) emulsification. In this study, we replicated the SO-aqueous interface within a microfluidic chip to study the formation of SO emulsion droplets in the eye cavity. A chip made of poly(methylmethacrylate) was used to represent a cross-section of the posterior eye chamber. A retinal ganglion cell line was coated on the inner surface of the chamber to mimic the surface property of the retina. Silicone oil of different viscosities were tested. The SO-aqueous interface was created inside the chip, which, in turn, was affixed to a stepper-motor-driven platform and subjected to simulated saccadic eye movement for four days. Optical microscopy was used to quantify the count and size of SO emulsified droplets. Among SO of different viscosities, SO 5 centistokes (cSt) emulsifies readily, and a high number of droplets formed inside the chip. Silicone oil 100 cSt led to fewer droplets than 5 cSt, but the droplet count was still significantly higher than other SO of higher viscosities. There were no significant differences in the number of droplets among SO with viscosities of 500, 1000, and 5000 cSt. In all SOs tested, the number of droplets increased, whereas their size decreased with longer duration of simulated saccades. The study platform allows quantification of the number and size of emulsified SO droplets in situ. More importantly, this platform demonstrates the potential of microtechnology for constructing a more physiologically realistic in vitro eye model. Eye-on-a-chip technology presents exciting opportunities to study emulsification and potentially other phenomena in the human eye.

Visual selectivity for heading in the macaque ventral intraparietal area.

The patterns of optic flow seen during self-motion can be used to determine the direction of one's own heading. Tracking eye movements which typically occur during everyday life alter this task since they add further retinal image motion and (predictably) distort the retinal flow pattern. Humans employ both visual and nonvisual (extraretinal) information to solve a heading task in such case. Likewise, it has been shown that neurons in the monkey medial superior temporal area (area MST) use both signals during the processing of self-motion information. In this article we report that neurons in the macaque ventral intraparietal area (area VIP) use visual information derived from the distorted flow patterns to encode heading during (simulated) eye movements. We recorded responses of VIP neurons to simple radial flow fields and to distorted flow fields that simulated self-motion plus eye movements. In 59% of the cases, cell responses compensated for the distortion and kept the same heading selectivity irrespective of different simulated eye movements. In addition, response modulations during real compared with simulated eye movements were smaller, being consistent with reafferent signaling involved in the processing of the visual consequences of eye movements in area VIP. We conclude that the motion selectivities found in area VIP, like those in area MST, provide a way to successfully analyze and use flow fields during self-motion and simultaneous tracking movements.

How active perception and attractor dynamics shape perceptual categorization: A computational model

We propose a computational model of perceptual categorization that fuses elements of grounded and sensorimotor theories of cognition with dynamic models of decision-making. We assume that category information consists in anticipated patterns of agent–environment interactions that can be elicited through overt or covert (simulated) eye movements, object manipulation, etc. This information is firstly encoded when category information is acquired, and then re-enacted during perceptual categorization. The perceptual categorization consists in a dynamic competition between attractors that encode the sensorimotor patterns typical of each category; action prediction success counts as “evidence” for a given category and contributes to falling into the corresponding attractor. The evidence accumulation process is guided by an active perception loop, and the active exploration of objects (e.g., visual exploration) aims at eliciting expected sensorimotor patterns that count as evidence for the object category. We present a computational model incorporating these elements and describing action prediction, active perception, and attractor dynamics as key elements of perceptual categorizations. We test the model in three simulated perceptual categorization tasks, and we discuss its relevance for grounded and sensorimotor theories of cognition.

Using E-Z Reader to simulate eye movements in nonreading tasks: A unified framework for understanding the eye–mind link.

Nonreading tasks that share some (but not all) of the task demands of reading have often been used to make inferences about how cognition influences when the eyes move during reading. In this article, we use variants of the E-Z Reader model of eye-movement control in reading to simulate eye-movement behavior in several of these tasks, including z-string reading, target-word search, and visual search of Landolt Cs arranged in both linear and circular arrays. These simulations demonstrate that a single computational framework is sufficient to simulate eye movements in both reading and nonreading tasks but also suggest that there are task-specific differences in both saccadic targeting (i.e., decisions about where to move the eyes) and the coupling between saccadic programming and the movement of attention (i.e., decisions about when to move the eyes). These findings suggest that some aspects of the eye-mind link are flexible and can be configured in a manner that supports efficient task performance.

Supplemental Material for Using E-Z Reader to Simulate Eye Movements in Nonreading Tasks: A Unified Framework for Understanding the Eye–Mind Link

Supplemental Material for Using E-Z Reader to Simulate Eye Movements in Nonreading Tasks: A Unified Framework for Understanding the Eye–Mind Link