Eye Guidance Research Articles

Policy advice and best practices on bias and fairness in AI

The literature addressing bias and fairness in AI models (fair-AI) is growing at a fast pace, making it difficult for novel researchers and practitioners to have a bird’s-eye view picture of the field. In particular, many policy initiatives, standards, and best practices in fair-AI have been proposed for setting principles, procedures, and knowledge bases to guide and operationalize the management of bias and fairness. The first objective of this paper is to concisely survey the state-of-the-art of fair-AI methods and resources, and the main policies on bias in AI, with the aim of providing such a bird’s-eye guidance for both researchers and practitioners. The second objective of the paper is to contribute to the policy advice and best practices state-of-the-art by leveraging from the results of the NoBIAS research project. We present and discuss a few relevant topics organized around the NoBIAS architecture, which is made up of a Legal Layer, focusing on the European Union context, and a Bias Management Layer, focusing on understanding, mitigating, and accounting for bias.

A Bibliographic Review and Meta-Analysis of the Effect of Teaching Behaviors on Learners’ Performance in Videotaped Lectures

Although the discussion about the influence of the instructor-present videos has become a hot issue in recent years, the potential moderators on the effectiveness of an on-screen instructor have not been thoroughly synthesized. The present review systematically retrieves 47 empirical studies on how the instructors’ behaviors moderated online education quality as measured by learning performance via a bibliographic study using VOSviewer and meta-analysis using Stata/MP 14.0. The bibliographic networks illustrate instructors’ eye gaze, gestures, and facial expressions attract more researchers’ attention. The meta-analysis results further reveal that better learning performance can be realized by integrating the instructor’s gestures, eye guidance, and expressive faces with their speech in video lectures. Future studies can further explore the impact of instructors’ other characteristics on learning perception and visual attention including voice, gender, age, etc. The underlying neural mechanism should also be considered via more objective technologies.

Targeted Radiotherapy Using Contact X-ray Brachytherapy 50 kV.

Simple SummaryOrgan preservation is becoming a topic of great interest for rectal cancer. Radiotherapy, often in association with chemotherapy, is playing a major role in achieving tumor sterilization and long-term local control. In order to achieve this goal, a high dose (>70 Gy) of radiotherapy is necessary. To avoid radiation toxicity of such a high dose, an endocavitary approach using contact X-ray 50 kV brachytherapy (CXB) is an attractive method. Historical series and two randomized trials in Europe give good evidence of the merit of CXB. The selection of early tumors is a key prognostic factor to achieve good results. A planned organ preservation treatment using CXB followed by chemoradiotherapy can be proposed to patients with T2–T3 a–b < 3 cm in diameter. At 3 years, the chance of local control should be close to 90% with good bowel function.Rectal adenocarcinoma is a quite radioresistant tumor. In order to achieve non-operative management (NOM) radiotherapy plays a major role. Targeted radiotherapy aiming at high precision 3D radiotherapy uses stereotactic image-guided external beam radiotherapy machines. To further safely increase the tumor dose, endocavitary brachytherapy (ECB) is an original approach. There are two different ways to perform such an ECB: contact X-ray brachytherapy (CXB) using a 50 kV X-ray generator with an X-ray tube positioned under eye guidance into the rectal cavity and high-dose-rate brachytherapy (HDRB) using iridium-192 sources positioned into the rectal cavity under image guidance. This study focused on CXB. CXB uses a small mobile generator that produces 50 kV X-rays with limited penetration. This technique is well adapted to accessible tumors of limited size and especially needs a high dose rate (≥15 Gy/minutes) for rectal tumors. It is performed on an ambulatory basis. A total dose between 80–110 Gy is delivered in 3–4 fractions over 3 to 6 weeks into a small volume (5 cm3). CXB was pioneered in the 1970s by Papillon using the Philips RT 50TM. Since 2009, the Papillon P50TM has been used in 11 institutions in Europe. The OPERA Phase III trial tested the hypothesis that a CXB boost (90 Gy/3 fr) compared to an EBRT boost (9 Gy/5 fr) for T2–T3 ab < 5 cm and N0–N1 < 8 mm will increase the 3-year organ preservation (OP) rate when combined with 45 Gy/5 weeks with concomitant capecitabine. Out of more than 300 patients with tumors < 3 cm (1962–1992), Papillon reported a long-term local control close to 85%. Similar results were published in Europe and USA at that time. The Lyon R96-2 Phase III trial (2004) demonstrated that, when combined with preoperative EBRT, a CXB boost (90 Gy/3 fr) significantly increased the rate of clinical complete response (cCR) and sphincter preservation, with some patients having OP at 10 years. With more than 2000 patients treated in Europe (2010–2020) using the Papillon 50TM, organ preservation appears possible in close to 80% of cases in selected early T2–T3. The OPERA trial closed after 141 inclusions (2015–2020) after an independent data monitoring committee recommendation because of promising results. At the 2-year follow-up (blinded data), the rate of cCR and OP were 77% and 72%, respectively, for the 141 tumors, and for T < 3 cm (61 pts), they were 86% and 85%, respectively, with good bowel function. The final results should be available in 2022. Organ preservation using NOM appears to be a promising approach for rectal cancer. A CXB boost with chemoradiotherapy in selected early T2–T3 could become an attractive option to achieve a planned OP. This approach should be proposed to well-informed patients after discussion in an MDT.

The effect of target salience and size in visual search within naturalistic scenes under degraded vision.

We address two questions concerning eye guidance during visual search in naturalistic scenes. First, search has been described as a task in which visual salience is unimportant. Here, we revisit this question by using a letter-in-scene search task that minimizes any confounding effects that may arise from scene guidance. Second, we investigate how important the different regions of the visual field are for different subprocesses of search (target localization, verification). In Experiment 1, we manipulated both the salience (low vs. high) and the size (small vs. large) of the target letter (a “T”), and we implemented a foveal scotoma (radius: 1°) in half of the trials. In Experiment 2, observers searched for high- and low-salience targets either with full vision or with a central or peripheral scotoma (radius: 2.5°). In both experiments, we found main effects of salience with better performance for high-salience targets. In Experiment 1, search was faster for large than for small targets, and high-salience helped more for small targets. When searching with a foveal scotoma, performance was relatively unimpaired regardless of the target's salience and size. In Experiment 2, both visual-field manipulations led to search time costs, but the peripheral scotoma was much more detrimental than the central scotoma. Peripheral vision proved to be important for target localization, and central vision for target verification. Salience affected eye movement guidance to the target in both central and peripheral vision. Collectively, the results lend support for search models that incorporate salience for predicting eye-movement behavior.

On the 'Where' and 'When' of Eye Guidance in Real-World Scenes.

Keynote at the 20th European Conference on Eye Movement Research (ECEM) in Alicante, 22.8.2019Video stream: https://vimeo.com/361729502

Investigating word length effects in Chinese reading.

A word’s length in English is fundamental in determining whether readers fixate it, and how long they spend processing it during reading. Chinese is unspaced, and most words are two characters long: Is word length an important cue to eye guidance in Chinese reading? Eye movements were recorded as participants read sentences containing a one-, two-, or three-character word matched for frequency. Results showed that longer words took longer to process (primarily driven by refixations). Furthermore, skips were fewer, incoming saccades longer, and landing positions further to the right of long than short words. Additional analyses of a three-character region (matched stroke number) showed an incremental processing cost when character(s) belonged to different, rather than the same, word. These results demonstrate that word length affects both lexical identification and saccade target selection in Chinese reading.

PERCEPTUAL GUIDANCE IMPROVES VISUAL SEARCH IN AGING AND IN ALZHEIMER’S DISEASE: EVIDENCE FROM EYE-MOVEMENT BEHAVIOR WITHIN REAL-WORLD SCENES

Elderly people and, especially, patients with Alzheimer's disease (AD) show impaired performance in visual search. This seems to arise from inefficient strategies, with an increased number of eye fixations and longer fixation durations. As many activities of daily living require efficient search of objects within complex environments, visual search deficits of AD patients may have a key role in reducing their functional autonomy. However, while previous studies have examined patients’ search in simple object arrays, little is know about their search behaviour in real-world scenes. Using an EyeLink 1000, we recorded eye movements of 12 mild AD patients, 24 healthy age-matched individuals (“elderly controls”) and 32 young individuals (“young controls”) during search within real-world scenes. We examined whether any group difference in eye guidance during search depends on target template specificity (name vs. precise picture), target visual salience (low vs. high) and reliability of scene semantics (consistent vs. inconsistent targets). We analysed data using linear mixed effect models, considering search initiation (first search saccade), scene scanning (until first target fixations) and target verification (dwell time on target). Aging (elderly vs. young controls) and AD (patients vs. elderly controls) led to less effective behaviour in all search phases, with fewer first saccades directed toward the target, longer scanning time and longer verification time. While these patterns were not affected by scene semantic guidance, a greater amount of target perceptual guidance was especially beneficial for the elderly controls and, even more, for the AD patients. In particular, higher target salience reduced performance gaps between elderly and young controls during scene scanning and target verification, and – when combined with a pictorial template – between patients and elderly controls during scene scanning. Eye-movement analysis enables to examine the time course of fine impairments during real-world scene search. Enhancing target visual salience and providing a priori knowledge of the perceptual appearance of the specific target objects seem promising ways of improving visual search behaviour in healthy elderly individuals and in AD patients. This understanding may contribute to the development of more appropriate information systems and living places, promoting individual's everyday functional autonomy.

Effects of word length on eye guidance differ for young and older Chinese readers.

Effects of word length on where and for how long readers fixate within text are preserved in older age for alphabetic languages like English that use spaces to demarcate word boundaries. However, word length effects for older readers of naturally unspaced, character-based languages like Chinese are unknown. Accordingly, we examined age differences in eye movements for short (2-character) and long (4-character) words during Chinese reading. Word length effects on eye-fixation times were greater for older than younger adults. We suggest this age difference is due to older adults’ saccades landing more rarely at optimal intraword locations, especially in longer words.

Word segmentation by alternating colors facilitates eye guidance in Chinese reading.

During sentence reading, low spatial frequency information afforded by spaces between words is the primary factor for eye guidance in spaced writing systems, whereas saccade generation for unspaced writing systems is less clear and under debate. In the present study, we investigated whether word-boundary information, provided by alternating colors (consistent or inconsistent with word-boundary information) influences saccade-target selection in Chinese. In Experiment 1, as compared to a baseline (i.e., uniform color) condition, word segmentation with alternating color shifted fixation location towards the center of words. In contrast, incorrect word segmentation shifted fixation location towards the beginning of words. In Experiment 2, we used a gaze-contingent paradigm to restrict the color manipulation only to the upcoming parafoveal words and replicated the results, including fixation location effects, as observed in Experiment 1. These results indicate that Chinese readers are capable of making use of parafoveal word-boundary knowledge for saccade generation, even if such information is unfamiliar to them. The present study provides novel support for the hypothesis that word segmentation is involved in the decision about where to fixate next during Chinese reading.

Morphological structure influences the initial landing position in words during reading Finnish.

The preferred viewing location in words [Rayner, K. (1979). Eye guidance in reading: Fixation locations within words. Perception, 8, 21-30] during reading is near the word centre. Parafoveal word length information is utilized to guide the eyes toward it. A recent study by Yan and colleagues [Yan, M., Zhou, W., Shu, H., Yusupu, R., Miao, D., Krügel, A., & Kliegl, R. (2014). Eye movements guided by morphological structure: Evidence from the Uighur language. Cognition, 132, 181-215] demonstrated that the word's morphological structure may also be used in saccadic targeting. The study was conducted in a morphologically rich language, Uighur. The present study aimed at replicating their main findings in another morphologically rich language, Finnish. Similarly to Yan et al., it was found that the initial fixation landed closer to the word beginning for morphologically complex than for monomorphemic words. Word frequency, saccade launch site, and word length were also found to influence the initial landing position. It is concluded that in addition to low-level factors (word length and saccade launch site), also higher level factors related to the word's morphological structure and frequency may be utilized in saccade programming during reading.

Supplemental Material for Effects of Word Length on Eye Guidance Differ for Young and Older Chinese Readers

Supplemental Material for Effects of Word Length on Eye Guidance Differ for Young and Older Chinese Readers

Temporal evolution of the central fixation bias in scene viewing.

When watching the image of a natural scene on a computer screen, observers initially move their eyes toward the center of the image-a reliable experimental finding termed central fixation bias. This systematic tendency in eye guidance likely masks attentional selection driven by image properties and top-down cognitive processes. Here, we show that the central fixation bias can be reduced by delaying the initial saccade relative to image onset. In four scene-viewing experiments we manipulated observers' initial gaze position and delayed their first saccade by a specific time interval relative to the onset of an image. We analyzed the distance to image center over time and show that the central fixation bias of initial fixations was significantly reduced after delayed saccade onsets. We additionally show that selection of the initial saccade target strongly depended on the first saccade latency. A previously published model of saccade generation was extended with a central activation map on the initial fixation whose influence declined with increasing saccade latency. This extension was sufficient to replicate the central fixation bias from our experiments. Our results suggest that the central fixation bias is generated by default activation as a response to the sudden image onset and that this default activation pattern decreases over time. Thus, it may often be preferable to use a modified version of the scene viewing paradigm that decouples image onset from the start signal for scene exploration to explicitly reduce the central fixation bias.

Parafoveal preview effects in reading unspaced text.

In English reading, eye guidance relies heavily on the spaces between words for demarcating word boundaries. In an eye tracking experiment, we examined the impact of removing spaces on parafoveal processing. Using the gaze-contingent boundary paradigm (Rayner, 1975), a high or low frequency preboundary word was followed by a postboundary preview presented either normally (i.e., identical to the postboundary word), or with letters replaced creating an orthographically illegal preview. The spaces between words were either retained or removed. Results replicate previous findings of increased reading times during unspaced reading (Rayner, Fischer, & Pollatsek, 1998) and indicate rather limited evidence for more distributed processing: Observations of processing of the previous word (spill-over effects) or processing of the next word (parafoveal-on-foveal effects) influencing fixation durations on the currently fixated word were limited. Spill-over effects were only observed in the unspaced layout when the postboundary preview was correct, presumably because the orthographically illegal, incorrect preview was visually salient enough to allow for relatively easy word segmentation and therefore more focused processing of the preboundary word. As such, results points toward a system that prefers narrowly focused processing of a single word, at least when means for easy word segmentation are available. (PsycINFO Database Record

The saccadic flow baseline: Accounting for image-independent biases in fixation behavior

Much effort has been made to explain eye guidance during natural scene viewing. However, a substantial component of fixation placement appears to be a set of consistent biases in eye movement behavior. We introduce the concept of saccadic flow, a generalization of the central bias that describes the image-independent conditional probability of making a saccade to (xi+1, yi+1), given a fixation at (xi, yi). We suggest that saccadic flow can be a useful prior when carrying out analyses of fixation locations, and can be used as a submodule in models of eye movements during scene viewing. We demonstrate the utility of this idea by presenting bias-weighted gaze landscapes, and show that there is a link between the likelihood of a saccade under the flow model, and the salience of the following fixation. We also present a minor improvement to our central bias model (based on using a multivariate truncated Gaussian), and investigate the leftwards and coarse-to-fine biases in scene viewing.

Word Skipping in Reading English as a Foreign Language: Evidence from Eye Tracking

Word Skipping in Reading English as a Foreign Language: Evidence from Eye Tracking

Eye guidance during real-world scene search: The role color plays in central and peripheral vision.

The visual system utilizes environmental features to direct gaze efficiently when locating objects. While previous research has isolated various features' contributions to gaze guidance, these studies generally used sparse displays and did not investigate how features facilitated search as a function of their location on the visual field. The current study investigated how features across the visual field--particularly color--facilitate gaze guidance during real-world search. A gaze-contingent window followed participants' eye movements, restricting color information to specified regions. Scene images were presented in full color, with color in the periphery and gray in central vision or gray in the periphery and color in central vision, or in grayscale. Color conditions were crossed with a search cue manipulation, with the target cued either with a word label or an exact picture. Search times increased as color information in the scene decreased. A gaze-data based decomposition of search time revealed color-mediated effects on specific subprocesses of search. Color in peripheral vision facilitated target localization, whereas color in central vision facilitated target verification. Picture cues facilitated search, with the effects of cue specificity and scene color combining additively. When available, the visual system utilizes the environment's color information to facilitate different real-world visual search behaviors based on the location within the visual field.

Disentangling the effects of spatial inconsistency of targets and distractors when searching in realistic scenes.

Previous research has suggested that correctly placed objects facilitate eye guidance, but also that objects violating spatial associations within scenes may be prioritized for selection and subsequent inspection. We analyzed the respective eye guidance of spatial expectations and target template (precise picture or verbal label) in visual search, while taking into account any impact of object spatial inconsistency on extrafoveal or foveal processing. Moreover, we isolated search disruption due to misleading spatial expectations about the target from the influence of spatial inconsistency within the scene upon search behavior. Reliable spatial expectations and precise target template improved oculomotor efficiency across all search phases. Spatial inconsistency resulted in preferential saccadic selection when guidance by template was insufficient to ensure effective search from the outset and the misplaced object was bigger than the objects consistently placed in the same scene region. This prioritization emerged principally during early inspection of the region, but the inconsistent object also tended to be preferentially fixated overall across region viewing. These results suggest that objects are first selected covertly on the basis of their relative size and that subsequent overt selection is made considering object-context associations processed in extrafoveal vision. Once the object was fixated, inconsistency resulted in longer first fixation duration and longer total dwell time. As a whole, our findings indicate that observed impairment of oculomotor behavior when searching for an implausibly placed target is the combined product of disruption due to unreliable spatial expectations and prioritization of inconsistent objects before and during object fixation.

The maturation of eye movement behavior: scene viewing characteristics in children and adults.

While the close link between eye movements and visual attention has often been demonstrated, recently distinct attentional modes have been associated with specific eye movement patterns. The ambient mode-serving the localization of objects and dominating early scene inspection-is expressed by short fixations and large saccade amplitudes. The focal mode-associated with the identification of object details and dominating later stages of scene exploration-is indicated by longer fixations embedded in short saccades. The relationship between these processing modes and eye movement characteristics has so far only been examined in adults. While studies in children revealed a maturation of oculomotor behavior up to adolescence, developmental aspects of the processing modes are still unknown. Here we explored these mechanisms by comparing eye movements during the inspection of naturalistic scenes. Therefore, gaze behavior from adults and children in four different age groups (2, 4-6, 6-8, 8-10years old) was examined. We found a general effect of age, revealing that with age fixation durations decrease and saccade amplitudes increase. However, in all age groups fixations were shorter and saccades were longer at the beginning of scene inspection but fixations became longer and saccades became shorter over time. While saliency influenced eye guidance in the two youngest groups over the full inspection period, for the older groups this influence was found only at the beginning of scene inspection. The results reveal indications for ambient and focal processing strategies for as early as 2 years of age.

Deriving an appropriate baseline for describing fixation behaviour.

Humans display image-independent viewing biases when inspecting complex scenes. One of the strongest such bias is the central tendency in scene viewing: observers favour making fixations towards the centre of an image, irrespective of its content. Characterising these biases accurately is important for three reasons: (1) they provide a necessary baseline for quantifying the association between visual features in scenes and fixation selection; (2) they provide a benchmark for evaluating models of fixation behaviour when viewing scenes; and (3) they can be included as a component of generative models of eye guidance. In the present study we compare four commonly used approaches to describing image-independent biases and report their ability to describe observed data and correctly classify fixations across 10 eye movement datasets. We propose an anisotropic Gaussian function that can serve as an effective and appropriate baseline for describing image-independent biases without the need to fit functions to individual datasets or subjects.

Modelling Task-Dependent Eye Guidance to Objects in Pictures

We introduce a model of attentional eye guidance based on the rationale that the deployment of gaze is to be considered in the context of a general action-perception loop relying on two strictly intertwined processes: sensory processing, depending on current gaze position, identifies sources of information that are most valuable under the given task; motor processing links such information with the oculomotor act by sampling the next gaze position and thus performing the gaze shift. In such a framework, the choice of where to look next is task-dependent and oriented to classes of objects embedded within pictures of complex scenes. The dependence on task is taken into account by exploiting the value and the payoff of gazing at certain image patches or proto-objects that provide a sparse representation of the scene objects. The different levels of the action-perception loop are represented in probabilistic form and eventually give rise to a stochastic process that generates the gaze sequence. This way the model also accounts for statistical properties of gaze shifts such as individual scan path variability. Results of the simulations are compared either with experimental data derived from publicly available datasets and from our own experiments.