Visual Scanning Strategies Research Articles

How flight experience impacts pilots’ decision-making and visual scanning pattern in low-visibility approaches: preliminary evidence from eye tracking

The visual approach is the most accident-prone phase of a flight, especially in low-visibility conditions. This preliminary study aimed to examine the effects of flight experience on pilots’ decision-making and visual scanning pattern in low-visibility approaches. Twenty pilots were separated into two groups based on their flight experience and completed the high- and low-visibility approaches in balanced order using a high-fidelity flight simulator. Pilots’ mental workload and visual scanning patterns were recorded via an eye tracker. The results showed that, compared to less flight-experienced pilots (20%, 3/15), experienced pilots (80%, 4/5) were more likely to make go-around decisions in the low-visibility approaches. Furthermore, they exhibited a more flexible and adaptable visual scanning pattern by quickly shifting their attention, as evidenced by decreased fixations and increased saccades. These findings suggest that the integration of visual scanning strategy and training solution with a marginally meteorological approach may enhance decision-making safety for novice pilots.

The Real-World Effects of Route Familiarity on Drivers' Eye Fixations at Urban Intersections in Changsha, China.

A crucial factor, route familiarity, can affect traffic safety. Nevertheless, focus on the influence of route familiarity on drivers’ eye fixations at urban intersections has received less attention. Identifying the real-world effect of route familiarity on drivers’ eye fixations at urban intersections in Changsha, China, was the objective of this study. Their visual fixation indicators were recorded while unfamiliar drivers and familiar drivers drove a 9 km-long route with nine intersections in an urban environment, but their effectiveness was indicated by the data collected 150 m before the lane stop and 50 m after the lane stop at these intersections. From the analysis of the extracted data, the results indicated that route familiarity could influence drivers’ processing times in the left window (LW) and other areas (OT). Compared with familiar drivers, unfamiliar drivers had longer processing times and higher mental workloads for the right front (RF). For the vehicle’s front (RF, FL, FR), the sampling rates and mental workloads of unfamiliar drivers were higher than those of familiar drivers, but it was the opposite for the driver’s sides (LW, RW) and rear (LM, RM, ReM). It was also indicated that the phenomenon said to increase familiarity with the route and make drivers more likely to be distracted in urban intersections had not been found. From the present findings, the effect of route familiarity on drivers’ eye fixations at urban intersections was confirmed. The high accident risk of familiar drivers could be partly explained by the decrement in drivers’ eye fixation strategies. However, the strategies could not account for the phenomenon that more familiar drivers are involved in rear-end accidents. Therefore, the reason can be investigated based on drivers’ visual scanning strategies, their physiological signals and driving behavior in the future.

The facial emotion recognition deficit in Parkinson's disease: Implications of a visual scanning strategy.

We explored the relationship between a visual scanning strategy and a facial emotion recognition deficit in Parkinson's disease (PD). Thirty nondemented PD patients (balanced for symptom side at onset) and 20 age, education and gender-matched healthy controls (HC) were enrolled. The PD group underwent a comprehensive neuropsychological battery also exploring the executive functions. In both groups, eye movements were recorded while subjects categorized facial emotion from Ekman's 60-faces test. We were particularly interested in the location of fixations on facial pictures (top vs. bottom) and in emotional valence (positive vs. negative). We also compared performance of the two groups on a verbal emotion attribution task. Compared to HC, PD patients performed worse on visual recognition of negative emotions such as anger, fear, and sadness (where the upper part of the face is more informative than the lower part); the two groups did not differ on the verbal emotion attribution task. HC modified their visual scanning strategy (both number and overall time duration of fixations) according to the valence of the emotion; by contrast, PD showed the same pattern regardless of the valence. In the PD group, accuracy in the visual recognition of negative emotions and fixation pattern correlated with performance on tasks exploring executive functions; however, no associations were observed with severity of motor state. Our results suggest that visual scanning strategy contributes significantly to the facial emotion recognition deficit of PD patients, especially at a "high level" related to cognitive control of eye movements. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Theory of mind, empathy and eye gaze strategies during observation of a work of art: neurodegenerative pathologies.

Theory of mind (ToM) and empathy are severely impaired in the behavioural variant of frontotemporal dementia (bvFTD) and more mildly in Alzheimer's (AD) and Parkinson's diseases (PD). Such impairments are associated with behavioural disorders. Modification of visual scanning strategies of complex visual scenes is also found in these pathologies. We hypothesised that these patients applied atypical gaze strategies when observing social events, which does not allow them to properly process social cues and results in the production of erroneous inferences and lack of empathy towards others. Fifty-five participants were divided into four groups: five bvFTD subjects, 19 AD subjects, 17 PD subjects and 14 matched control subjects. ToM and empathy were assessed by eye movement recording (eye-tracking) and by a questionnaire during a painting observation. Scores obtained were compared between each group and to social cognition reference tests and correlated to the NeuroPsychiatric Inventory. Our paradigm was suitable for assessing cognitive ToM while it lacked sensitivity for empathy assessment. Severe ToM impairment was highlighted in bvFTD, while milder difficulties were observed in AD and for PD. Subjects with bvFTD and AD produced erroneous inferences from cognitive mental states. ToM performances were linked to visual exploration strategies of the painting. Atypical visual observation was highlighted in the bvFTD and AD groups causing a time delay in taking the perspective of the other person. Finally, we highlighted that social cognition performances, gaze strategies and behavioural disorders were correlated. Observing a painting in association with eye-tracking technology can be a good support for social cognition assessment. We highlighted a link between atypical visual scanning strategies, ToM impairment and behavioural disorders in these pathologies. ToM skills could be improved by training in the search for visual social cues. Therefore, this kind of remediation could have positive effects on behavioural disorders.

Theory of mind, empathy and eye gaze strategies during an artwork observation in neurodegenerative pathologies

Theory of mind (ToM) and empathy are severely impaired in the behavioral-variant of frontotemporal dementia (bvFTD) and more mildly in Alzheimer's (AD) and Parkinson's diseases (PD). Such impairments are associated with behavioral disorders (BD). Modification of visual scanning strategies of complex visual scenes is also found in these pathologies. We hypothesized that these patients applied atypical gaze strategies when observing social events, which would not allow to properly process social cues and would result in the production of erroneous inferences and lack of empathy towards others. Fifty-five participants were divided into four groups: five bvFTD, 19AD, 17PD and 14matched controls subjects. ToM and empathy were assessed by eye movements recording (eye-tracking) and by a questionnaire during a painting observation. Scores obtained were compared between each group and to social cognition reference tests, and correlated to the NeuroPsychiatric Inventory. Our paradigm was suitable for assessing cognitive ToM while it lacked sensitivity for empathy assessment. Severe ToM impairment was highlighted in bvFTD while milder difficulties were observed in AD and for PD. bvFTD and AD groups produced erroneous inferences from cognitive mental states. ToM performances were linked to visual exploration strategies of the painting. Atypical visual observation was highlighted in bvFTD and AD groups causing a time shift in perspective taking of the character. Finally, we have highlighted that social cognition performances, gaze strategies and BD were correlated. The observation of a painting in association with eye-tracking technology can be a good support for social cognition assessment. We highlighted a link between atypical visual scanning strategies, ToM impairment and BD in these pathologies. ToM skills could be improved by training in the search for visual social cues. Therefore, this kind of remediation could have positive effects on BD.

Improving Airline Pilots’ Visual Scanning and Manual Flight Performance through Training on Skilled Eye Gaze Strategies

Poor cockpit monitoring has been identified as an important contributor to aviation accidents. Improving pilots’ monitoring strategies could therefore help to enhance flight safety. During two different sessions, we analyzed the flight performance and eye movements of professional airline pilots in a full-flight simulator. In a pre-training session, 20 pilots performed a manual approach scenario as pilot flying (PFs) and were classified into three groups according to their flight performance: unstabilized, standard, and most accurate. The unstabilized pilots either under- or over-focused various instruments. Their number of visual scanning patterns was lower than those of pilots who managed to stabilize their approach. The most accurate pilots showed a higher perceptual efficiency with shorter fixation times and more fixations on important primary flight instruments. Approximately 10 months later, fourteen pilots returned for a post-training session. They received a short training program and performed a similar manual approach as during the pre-training session. Seven of them, the experimental group, received individual feedback on their own performance and visual behavior (i.e., during the pre-training session) and a variety of data obtained from the most accurate pilots, including an eye-tracking video showing efficient visual scanning strategies from one of the most accurate pilots. The other seven, the control group, received general guidelines on cockpit monitoring. During the post-training session, the experimental group had better flight performance (compared to the control group), and its visual scanning strategies became more similar to those of the most accurate pilots. In summary, our results suggest that cockpit monitoring underlies manual flight performance and that it can be improved using a training program based mainly on exposure to eye movement examples from highly accurate pilots.

Visual Search and Conflict Mitigation Strategies Used by Expert en Route Air Traffic Controllers

The role of the en route air traffic control specialist (ATCS) is vital to maintaining safety and efficiency within the National Airspace System (NAS). ATCSs must vigilantly scan the airspace under their control and adjacent airspaces using an En Route Automation Modernization (ERAM) radar display. The intent of this research is to provide an understanding of the expert controller visual search and aircraft conflict mitigation strategies that could be used as scaffolding methods during ATCS training. Interviews and experiments were conducted to elicit visual scanning and conflict mitigation strategies from the retired controllers who were employed as air traffic control instructors. The interview results were characterized and classified using various heuristics. In particular, representative visual scanpaths were identified, which accord with the interview results of the visual search strategies. The highlights of our findings include: (1) participants used systematic search patterns, such as circular, spiral, linear or quadrant-based, to extract operation-relevant information; (2) participants applied an information hierarchy when aircraft information was cognitively processed (altitude -> direction -> speed); (3) altitude or direction changes were generally preferred over speed changes when imminent potential conflicts were mitigated. Potential applications exist in the implementation of the findings into the training curriculum of candidates.

Automation Type and Reliability Impact on Visual Automation Monitoring and Human Performance

ABSTRACT We compared automation monitoring evolution of static or adaptive automation for four different reliability levels over 90 minutes. Previous studies have demonstrated degraded human performance when monitoring automation and that it is possible to mitigate this monitoring performance drop by using adaptive automation. We used the Open Multi-Attribute Task Battery to manipulate two type of automation (static automation without manual take-over sessions and adaptive automation with planned take-over sessions) and four levels of reliability. Participants performed three simultaneous tasks, one of which was automated. Our results suggest that a perfectly reliable or a totally unreliable automation led to different strategies by the participants in terms of visual allocation policy. Under static automation, the time spent looking at the automated task in the 0% reliability level increased over the duration of the experiment; however, the opposite was observed for the 100% reliability level. Although similar, the magnitude of this pattern of results was largely diminished under adaptive automation. For static automation, the reported data also showed a direct link between trust in automation and visual scanning strategies. The more the trust increased, the less the automated task was looked at.

Visual scanning strategies in the cockpit are modulated by pilots' expertise: A flight simulator study.

During a flight, pilots must rigorously monitor their flight instruments since it is one of the critical activities that contribute to update their situation awareness. The monitoring is cognitively demanding, but is necessary for timely intervention in the event of a parameter deviation. Many studies have shown that a large part of commercial aviation accidents involved poor cockpit monitoring from the crew. Research in eye-tracking has developed numerous metrics to examine visual strategies in fields such as art viewing, sports, chess, reading, aviation, and space. In this article, we propose to use both basic and advanced eye metrics to study visual information acquisition, gaze dispersion, and gaze patterning among novices and pilots. The experiment involved a group of sixteen certified professional pilots and a group of sixteen novice during a manual landing task scenario performed in a flight simulator. The two groups landed three times with different levels of difficulty (manipulated via a double task paradigm). Compared to novices, professional pilots had a higher perceptual efficiency (more numerous and shorter dwells), a better distribution of attention, an ambient mode of visual attention, and more complex and elaborate visual scanning patterns. We classified pilot's profiles (novices-experts) by machine learning based on Cosine KNN (K-Nearest Neighbors) using transition matrices. Several eye metrics were also sensitive to the landing difficulty. Our results can benefit the aviation domain by helping to assess the monitoring performance of the crews, improve initial and recurrent training and ultimately reduce incidents, and accidents due to human error.

Dynamic Advances in Emotion Processing: Differential Attention towards the Critical Features of Dynamic Emotional Expressions in 7-Month-Old Infants.

Infants’ visual processing of emotion undergoes significant development across the first year of life, yet our knowledge regarding the mechanisms underlying these advances is limited. Additionally, infant emotion processing is commonly examined using static faces, which do not accurately depict real-world emotional displays. The goal of this study was to characterize 7-month-olds’ visual scanning strategies when passively viewing dynamic emotional expressions to examine whether infants modify their scanning patterns depending on the emotion. Eye-tracking measures revealed differential attention towards the critical features (eyes, mouth) of expressions. The eyes captured the greatest attention for angry and neutral faces, and the mouth captured the greatest attention for happy faces. A time-course analysis further elucidated at what point during the trial differential scanning patterns emerged. The current results suggest that 7-month-olds are sensitive to the critical features of emotional expressions and scan them differently depending on the emotion. The scanning patterns presented in this study may serve as a link to understanding how infants begin to differentiate between expressions in the context of emotion recognition.

Exploring eye movement data with image-based clustering

In this article, we describe a new feature for exploring eye movement data based on image-based clustering. To reach this goal, visual attention is taken into account to compute a list of thumbnail images from the presented stimulus. These thumbnails carry information about visual scanning strategies, but showing them just in a space-filling and unordered fashion does not support the detection of patterns over space, time, or study participants. In this article, we present an enhancement of the EyeCloud approach that is based on standard word cloud layouts adapted to image thumbnails by exploiting image information to cluster and group the thumbnails that are visually attended.To also indicate the temporal sequence of the thumbnails, we add color-coded links and further visual features to dig deeper in the visual attention data. The usefulness of the technique is illustrated by applying it to eye movement data from a formerly conducted eye tracking experiment investigating route finding tasks in public transport maps. Finally, we discuss limitations and scalability issues of the approach.Graphic abstract

Analyzing pilots’ fatigue for prolonged flight missions: Multimodal analysis approach using vigilance test and eye tracking

Performance decrement associated with pilot fatigue is considered a leading contributor to aviation accidents and fatalities. The output of prevalent pilot fatigue methodologies (both subjective & objective) either suffer from human judgement bias or require complex data processing. Moreover, studies catering to long duration flight missions have not been performed. Presently, we investigate the impact of fatigue on pilot performance for long duration of a flight mission composed of multiple take-offs and landings. We propose a new multimodal approach that integrates traditional fatigue metrics with eye tracking methodology. The effect of fatigue on the pilots’ eye movements was evaluated using information theory-based entropy measures. Results showed an increase in the fatigue level (measured by mean reaction times and the number of lapses) with increase in flight duration. The entropy measures showed that visual attention distribution and scanning strategy both became random in nature as fatigue level increased in pilots. Obtained results suggest fatigue decreases both information searching and processing capability in pilots. The proposed method can show which aspect of the pilot performance becomes impaired by fatigue and thus can be applied to evaluate fatigue onset in real time, which enables timely recovery interventions.

The visual search patterns of drivers with Autism Spectrum Disorders in complex driving scenarios

BackgroundDriving is a highly demanding task which presents itself with various unpredictable and potentially hazardous situations. The failure to visually scan the driving environment and strategically search for potential road hazards, can be considered as unsafe driving practices. Little is known about how licensed drivers with Autism Spectrum Disorder (ASD) visually scan the roads while driving. The present study assessed the visual scanning and fixation patterns of drivers with and without ASD during a simulated drive. MethodsTwenty-eight licensed drivers between the age of 18–40 years old, including 14 drivers with ASD (male = 13) driving at least 2 h per week participated in a simulated drive with 14 matched controls. Psychometric profiles and visual scanning patterns on various objects of interest were analysed between groups. ResultsDrivers with ASD were found to fixate and spend significantly more time focusing on the central visual field and less time scanning where hazards potentially emerge. They also tended to allocate less visual attention on social stimuli (i.e., involving a person), and failed to stop in time at the red lights. Psychometric profiles confirmed poorer visual scanning and motor processing speed but less risk-taking behaviour in drivers with ASD. ConclusionLicensed drivers with ASD were found to allocate visual attention differently compared to licensed drivers without ASD. Poor scanning patterns with an over-focus on the road ahead and less scanning of the road side and periphery may possibly result in unsafe driving. However, risk-taking behaviour was not prevalent in these drivers. Effective visual scanning strategies could be incorporated in the driver training of individuals with ASD.

Effects of an in-vehicle eco-safe driving system on drivers’ glance behaviour

We have designed a new in-vehicle eco-safe driving system and shown its effectiveness in prompting drivers to execute a fuel-saving and safe driving style (Vaezipour et al., 2018, submitted for publication). However, the system could also bring potential negative outcomes, i.e. driver distraction. This simulator study investigated drivers’ glance behaviours as indicators of driver distraction when using our Eco-Safe Human-Machine-Interface (HMI). Four types of eco-safe information display conditions (baseline, advice only, feedback only, both advice and feedback) were tested on different traffic situations with varied road traffic complexity. Results showed that the eco-safe HMI system did not cause visual distraction. In contrast, the advice only or feedback only information improved forward gazing on the roadway. In addition, drivers tended to adapt their visual scanning strategies according to the traffic situations. In the car-following situation they paid longer glances to the forward roadway, while in the intersections they spent more time to look at the HMI system. The findings indicated that our eco-safe driving system improved drivers’ eco-safe behaviours and meanwhile enhanced their visual attention on road while no evidence showed that drivers were distracted by it.

Changes in Visual Scanning Strategies Accompany the Acquisition of Perceptual Expertise

Changes in Visual Scanning Strategies Accompany the Acquisition of Perceptual Expertise

Eye Movement Correlates of Expertise in Visual Arts.

The aim of this study was to search for oculomotor correlates of expertise in visual arts, in particular with regard to paintings. Achieving this goal was possible by gathering data on eye movements of two groups of participants: experts and non-experts in visual arts who viewed and appreciated the aesthetics of paintings. In particular, we were interested in whether visual arts experts more accurately recognize a balanced composition in one of the two paintings being compared simultaneously, and whether people who correctly recognize harmonious paintings are characterized by a different visual scanning strategy than those who do not recognize them. For the purposes of this study, 25 paintings with an almost ideal balanced composition have been chosen. Some of these paintings are masterpieces of the world cultural heritage, and some of them are unknown. Using Photoshop, the artist developed three additional versions of each of these paintings, differing from the original in the degree of destruction of its harmonious composition: slight, moderate, or significant. The task of the participants was to look at all versions of the same painting in pairs (including the original) and decide which of them looked more pleasing. The study involved 23 experts in art, students of art history, art education or the Academy of Fine Arts, and 19 non-experts, students in the social sciences and the humanities. The experimental manipulation of comparing pairs of paintings, whose composition is at different levels of harmony, has proved to be an effective tool for differentiating people because of their ability to distinguish paintings with balanced composition from an unbalanced one. It turned out that this ability only partly coincides with expertise understood as the effect of education in the field of visual arts. We also found that the eye movements of people who more accurately appreciated paintings with balanced composition differ from those who more liked their altered versions due to dwell time, first and average fixation duration and number of fixations. The familiarity of paintings turned out to be the factor significantly affects both the aesthetic evaluation of paintings and eye movement.

Driver’s emotional state and detection of vulnerable road users: Towards a better understanding of how emotions affect drivers’ perception using cardiac and ocular metrics

Traditionally, anger has been considered to have a detrimental effect on driving. However, recent studies suggest that this statement should be qualified, especially where vulnerable road user detection is concerned. One primary factor which may determine anger’s effect on a driver’s attention is its intensity. In the present study, different intensities of anger were elicited via film clips, then performances in vulnerable road user detection (i.e., of cyclists) were assessed while participants drove a car in a simulated environment. Cardiac and ocular measurements and self-reported data were used in order to accurately assess emotional state and attention management throughout the experiment. Results suggested that participants resorted to reappraisal strategies when they were exposed to the emotional film clips. This phenomenon did not directly affect cyclist detection performances, but evidence of different visual scanning strategies between groups emerged. The contribution of cardiac and ocular measurements to emotional assessment and the advantages of appraisal approaches of emotion were also discussed.

Protocole d’évaluation interdisciplinaire du regard, des fonctions perceptives et praxiques chez l’enfant

This article provides a protocol for an interdisciplinary assessment of perception and praxis for children. Understanding the term “dyspraxia” was a key point to our approach, with the purpose of providing better help for the children we work with. Most of these children have cerebral palsy with relatively preserved intellectual capacities. First of all, we have elabored a theoretical model based on recent scientific data. This has served as a guideline in selecting standardized tests and developing the protocol. Which brought out in this way, multiple fields of perceptual functions and praxis. We will also discuss the testing of “vision” (central and peripheral vision, oculomotricity, eye-hand coordination, visual scanning and exploration strategies) which is essential in measuring perceptual and praxis functions. Then, a reflection on the assessment of visualspatial, visual-perceptual and visual-constructive processing is proposed. Finally, we will discuss ideomotor and ideatory aspects of the praxis functions as well as the relevance of assessing bucco-linguo-facial praxis functions in this context. The benefit of this protocol is the fact that it combines diverse scientific data and interdisciplinary clinical experience (neuropsychologist, occupational therapist, speech therapist and physiotherapist) to make a precise diagnosis for to target and efficient intervention.

Rearview camera system use among older drivers: A naturalistic observation study

With rearview camera systems becoming more widely available in the automobile market, there is a need to understand how older drivers use them when reversing. The rearview camera display can reduce a driver’s blind spot at the rear of the vehicle; however, it is unclear how seniors, who have a high risk of reversing collisions, incorporate the display into their visual scanning strategies when reversing their personal vehicles. Moreover, it remains to be demonstrated how aware they are of their reversing habits or how those habits relate to the usability or familiarity with the technology. This study explored these issues using a naturalistic observation approach that captured the on-road behaviour of fourteen seniors aged 69–82years who owned vehicles equipped with rearview camera systems, followed by an interview about system usability. Results indicate that older drivers incorporate the display alongside other areas of interest (e.g., the rearview mirror) into their scanning strategies when reversing, especially in the presence of other vehicles, but least of all when at home. While older drivers appear to be aware of their reversing habits, they overestimate how often they turn to look directly behind through the rear window. Our sample rated system usability positively, but duration of vehicle ownership may affect perceptions and actual use of the technology. Our data show that seniors incorporate this technology into their reversing routines, which has the potential to make the driving task safer and easier.

Eye-Tracking as a Tool to Evaluate Functional Ability in Everyday Tasks in Glaucoma.

To date, few studies have investigated the eye movement patterns of individuals with glaucoma while they undertake everyday tasks in real-world settings. While some of these studies have reported possible compensatory gaze patterns in those with glaucoma who demonstrated good task performance despite their visual field loss, little is known about the complex interaction between field loss and visual scanning strategies and the impact on task performance and, consequently, on quality of life. We review existing approaches that have quantified the effect of glaucomatous visual field defects on the ability to undertake everyday activities through the use of eye movement analysis. Furthermore, we discuss current developments in eye-tracking technology and the potential for combining eye-tracking with virtual reality and advanced analytical approaches. Recent technological developments suggest that systems based on eye-tracking have the potential to assist individuals with glaucomatous loss to maintain or even improve their performance on everyday tasks and hence enhance their long-term quality of life. We discuss novel approaches for studying the visual search behavior of individuals with glaucoma that have the potential to assist individuals with glaucoma, through the use of personalized programs that take into consideration the individual characteristics of their remaining visual field and visual search behavior.