Visual Exploration Patterns Research Articles

Thinking eyes: visual thinking strategies and the social brain.

The foundation of art processes in the social brain can guide the scientific study of how human beings perceive and interact with their environment. Here, we applied the theoretical frameworks of the social and artistic brain connectomes to an eye-tracking paradigm with the aim to elucidate how different viewing conditions and social cues influence gaze patterns and personal resonance with artworks and complex imagery in healthy adults. We compared two viewing conditions that encourage personal or social perspective taking-modeled on the well-known Visual Thinking Strategies (VTS) method-to a viewing condition during which only contextual information about the image was provided. Our findings showed that the viewing conditions that used VTS techniques directed the gaze more toward highly salient social cues (Animate elements) in artworks and complex imagery, compared to when only contextual information was provided. We furthermore found that audio cues also directed visual attention, whereby listening to a personal reflection by another person (VTS) had a stronger effect than contextual information. However, we found no effect of viewing condition on the personal resonance with the artworks and complex images when taking the random effects of the image selection into account. Our study provides a neurobiological grounding of the VTS method in the social brain, revealing that this pedagogical method of engaging viewers with artworks measurably shapes people's visual exploration patterns. This is not only of relevance to (art) education but also has implications for art-based diagnostic and therapeutic applications.

Did You Commit a Crime There? Investigating the Visual Exploration Patterns of Guilty, Innocent, Honest, and Dishonest Subjects When Viewing a Complex Mock Crime Scene

Previous studies with the eye-tracking technology have predominantly tracked eye parameters in response to a single simple stimulus, and have generated interesting - sometimes inconsistent - results in research on deceptive behavior. The present study analyzed visual patterns in response to a complex image, to investigate potential differences in eye fixation between guilty versus innocent, and honest versus dishonest participants.One hundred and sixty participants were assigned to one of four experimental groups, defined by the parameters of honesty (dishonesty) and guilt (innocence), and asked to complete a computer-based task, looking at neutral and target images (i.e., images of the mock crime scene). RealEye software was used to capture participants’ eye movements when viewing the images.The findings revealed significant differences in eye movements between the four experimental groups in the pictures in which the area where the crime took place was clearly visible. Dishonest and guilty participants recorded fewer and shorter fixations in the area of the image where the crime took place than those who entered the crime scene but did not commit the crime. No differences between groups emerged in the visual patterns in response to neutral images, confirming that the number and duration of fixations in response to the target images may be attributed to the experimental condition.

Individual patterns of visual exploration predict the extent of fear generalization in humans.

Generalization of fear is considered an important mechanism contributing to the etiology and maintenance of anxiety disorders. Although previous studies have identified the importance of stimulus discrimination for fear generalization, it is still unclear to what degree overt attention to relevant stimulus features might mediate its magnitude. To test the prediction that visual preferences for distinguishing stimulus aspects are associated with reduced fear generalization, we developed a set of facial stimuli that was meticulously manipulated such that pairs of faces could either be distinguished by looking into the eyes or into the region around mouth and nose, respectively. These pairs were then employed as CS + and CS- in a differential fear conditioning paradigm followed by a generalization test with morphs in steps of 20%. Shock expectancy ratings indicated a moderately curved fear generalization gradient that is typical for healthy samples, but its shape was altered depending on individual attentional deployment: Particpants who dwelled on the distinguishing facial features faster and for longer periods of time exhibited less fear generalization. Although both pupil and heart rate responses also showed a generalization gradient, with pupil diameter and heart rate deceleration increasing as a function of threat, these responses were not significantly related to visual exploration. In total, the current results indicate that the extent of explicit fear generalization depends on individual patterns of attentional deployment. Future studies evaluating the efficacy of perceptual trainings that aim to augment stimulus discriminability in order to reduce (over)generalization seem desirable. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Gaze-contingent display technology can help to reduce the ipsilesional attention bias in hemispatial neglect following stroke

BackgroundHemispatial neglect results from unilateral brain damage and represents a disabling unawareness for objects in the hemispace opposite the brain lesion (contralesional). The patients’ attentional bias for ipsilesional hemispace represents a hallmark of neglect, which results from an imbalanced attentional priority map in the brain. The aim of this study was to investigate whether gaze-contingent display (GCD) technology, reducing the visual salience of objects in ipsilesional hemispace, is able to rebalance this map and increase awareness and exploration of objects in the neglected contralesional hemispace.MethodsUsing remote eye-tracking, we recorded gaze positions in 19 patients with left hemispatial neglect following right-hemisphere stroke and 22 healthy control subjects, while they were watching static naturalistic scenes. There were two task conditions, free viewing (FV) or goal-directed visual search (VS), and four modification conditions including the unmodified original picture, a purely static modification and two differently strong modifications with an additional gaze-contingent mask (GC-LOW, GC-HIGH), that continuously reduced color saturation and contrast of objects in the right hemispace.ResultsThe patients’ median gaze position (Center of Fixation) in the original pictures was markedly deviated to the right in both tasks (FV: 6.8° ± 0.8; VS: 5.5° ± 0.7), reflecting the neglect-typical ipsilesional attention bias. GC modification significantly reduced this bias in FV (GC-HIGH: d = − 3.2 ± 0.4°; p < 0.001). Furthermore, in FV and VS, GC modification increased the likelihood to start visual exploration in the (neglected) left hemifield by about 20%. This alleviation of the ipsilesional fixation bias was not associated with an improvement in detecting left-side targets, in contrast, the GC mask even decreased and slowed the detection of right-side targets. Subjectively, patients found the intervention pleasant and most of the patients did not notice any modification.ConclusionsGCD technology can be used to positively influence visual exploration patterns in patients with hemispatial neglect. Despite an alleviation of the neglect-related ipsilesional fixation bias, a concomitant functional benefit (improved detection of contralesional targets) was not achieved. Future studies may investigate individualized GCD-based modifications as augmented reality applications during the activities of daily living.

Did You Commit a Crime There? Investigating the Visual Exploration Patterns of Guilty, Innocent, Honest, and Dishonest Subjects when Viewing a Complex Crime Scene

Did You Commit a Crime There? Investigating the Visual Exploration Patterns of Guilty, Innocent, Honest, and Dishonest Subjects when Viewing a Complex Crime Scene

Active vision in sight recovery individuals with a history of long-lasting congenital blindness.

What we see is intimately linked to how we actively and systematically explore the world through eye movements. However, it is unknown to what degree visual experience during early development is necessary for such systematic visual exploration to emerge. The present study investigated visual exploration behavior in 10 human participants whose sight had been restored only in childhood or adulthood, after a period of congenital blindness because of dense bilateral congenital cataracts. Participants freely explored real-world images while their eye movements were recorded. Despite severe residual visual impairments and gaze instability (nystagmus), visual exploration patterns were preserved in individuals with reversed congenital cataract. Modeling analyses indicated that, similar to healthy control subjects, visual exploration in individuals with reversed congenital cataract was based on the low-level (luminance contrast) and high-level (object components) visual content of the images. Moreover, participants used visual short-term memory representations for narrowing down the exploration space. More systematic visual exploration in individuals with reversed congenital cataract was associated with better object recognition, suggesting that active vision might be a driving force for visual system development and recovery. The present results argue against a sensitive period for the development of neural mechanisms associated with visual exploration.

Supplemental Material for Individual Patterns of Visual Exploration Predict the Extent of Fear Generalization in Humans

Supplemental Material for Individual Patterns of Visual Exploration Predict the Extent of Fear Generalization in Humans

Comparison of Visual Fixation Trajectories in Toddlers with Autism Spectrum Disorder and Typical Development: A Markov Chain Model.

Autism spectrum disorder (ASD) is a neurodevelopmental condition in which visual attention and visual search strategies are altered. Eye-tracking paradigms have been used to detect these changes. In our study, 18 toddlers with ASD and 18 toddlers with typical development (TD; age range 12–36 months) underwent an eye-tracking paradigm where a face was shown together with a series of objects. Eye gaze was coded according to three areas of interest (AOIs) indicating where the toddlers’ gaze was directed: ‘Face’, ‘Object’, and ‘No-stimulus fixation’. The fixation sequence for the ASD and TD groups was modelled with a Markov chain model, obtaining transition probabilities between AOIs. Our results indicate that the transition between AOIs could differentiate between toddlers with ASD or TD, highlighting different visual exploration patterns between the groups. The sequence of exploration is strictly conditioned based on previous fixations, among which ‘No-stimulus fixation’ has a critical role in differentiating the two groups. Furthermore, our analyses underline difficulties of individuals with ASD to engage in stimulus exploration. These results could improve clinical and interventional practice by considering this dimension among the evaluation process.

Similarity of gaze patterns across physical and virtual versions of an installation artwork

An experiment was conducted to compare museum visitors’ gaze patterns using mobile eye-trackers, whilst they were engaging with a physical and a virtual reality (VR) installation of Piet Mondrian’s Neo-plasticist room design. Visitors’ eye movements produced approximately 25,000 fixations and were analysed using linear mixed-effects models. Absolute and area-normalized dwell time analyses yielded mostly non-significant main effects of the environment, indicating similarity of visual exploration patterns between physical and VR settings. One major difference observed was the decrease of average fixation duration in VR, where visitors tended to more rapidly switch focus in this environment with shorter bursts of attentional focus. The experiment demonstrated the ability to compare gaze data between physical and virtual environments as a proxy to measure the similarity of aesthetic experience. Similarity of viewing patterns along with questionnaire results suggested that virtual galleries can be treated as ecologically valid environments that are parallel to physical art galleries.

Your Eyes Do Not Lie! Dissecting Humor Effects in Health Messages Using Eye Tracker Technology.

In the past decade, humor in scientific research has become more and more popular providing an increase of data identifying the context in which humor is a promising communication strategy in preventive health messages. To avoid the limits of declarative responses usually recorded in past studies, eye tracker technology offers the possibility to assess and dissect the effects of humor on visual attention. In this brief report, we first attempt to extend the results of previous studies by recording eye movements while participants were exposed to humorous and nonhumorous print health ads dealing with tobacco and alcohol consumption. A secondary purpose is specifically to test the visual attention French women devoted to humorous tobacco preventive ads, the worrying results of recent studies urging to find a way to improve tobacco preventive campaigns. Based on three complementary eye-tracking measures (i.e., total dwell time, fixation count, and revisits), the results showed that humorous health messages were scanned longer and more frequently and revisited more often compared to nonhumorous ones. In addition, humor appeared to reduce smokers' avoidance of preventive tobacco messages. The different pattern of visual exploration confirms that humor is a good strategy to grab attention even of individuals who are involved in the health topic addressed. In short, this paper argues for introducing lightness into a very serious subject, health communication, based on the analysis of eye movement evidence.

The impact of social complexity on the visual exploration of others' actions in preschoolers with autism spectrum disorder

BackgroundTypical development of socio-communicative skills relies on keen observation of others. It thus follows that decreased social attention negatively impacts the subsequent development of socio-communicative abilities in children with autism spectrum disorders (ASD). In addition, studies indicate that social attention is modulated by context and that greater social difficulties are observed in more socially demanding situations. Our study aims to investigate the effect of social complexity on visual exploration of others’ actions in preschoolers.MethodsTo investigate the impact of social complexity, we used an eye-tracking paradigm with 26 typically developing preschoolers (TD, age = 3.60 ± 1.55) and 37 preschoolers with ASD (age = 3.55 ± 1.21). Participants were shown videos of two children engaging in socially simple play (parallel) versus socially complex play (interactive). We subsequently quantified the time spent and fixation duration on faces, objects, bodies, as well as the background and the number of spontaneous gaze shifts between socially relevant areas of interest.ResultsIn the ASD group, we observed decreased time spent on faces. Social complexity (interactive play) elicited changes in visual exploration patterns in both groups. From the parallel to the interactive condition, we observed a shift towards socially relevant parts of the scene, a decrease in fixation duration, as well as an increase in spontaneous gaze shifts between faces and objects though there were fewer in the ASD group.LimitationsOur results need to be interpreted cautiously due to relatively small sample sizes and may be relevant to male preschoolers, given our male-only sample and reported phenotypic differences between males and females.ConclusionOur results suggest that similar to TD children, though to a lesser extent, visual exploration patterns in ASD are modulated by context. Children with ASD that were less sensitive to context modulation showed decreased socio-communicative skills or higher levels of symptoms. Our findings support using naturalistic designs to capture socio-communicative deficits in ASD.

The role of labels and motions in infant category learning

We investigated the impact of two highly salient transient features, labels and motions, on novel visual category learning in 10-month-old infants. In three eye-tracking experiments, infants were presented with exemplars from two novel categories either accompanied by category-specific labels, accompanied by category-specific motions, or in silence. Labels (Experiment 1) and motions (Experiment 2) were presented using a gaze-contingent design in which these transient features were triggered by infants’ fixations. Gaze-contingent transient features, despite being redundant, had a strong impact on categorization. The results revealed that both labels and motions support infants’ category formation. Furthermore, both labels and motions promoted similarity-focused exploration, whereas no such pattern was found when infants learned the categories in silence. Analyses of visual exploration patterns revealed that infants readily form expectations about motion properties of categories and that these expectations drive their looking behavior.

Long-lasting effects of a gaze-contingent intervention on change detection in healthy participants – Implications for neglect rehabilitation

Patients with spatial neglect show an ipsilesional exploration bias. We developed a gaze-contingent intervention that aims at reducing this bias and tested its effects on visual exploration in healthy participants: During a visual search, stimuli in one half of the search display are removed when the gaze moves into this half. This leads to a relative increase in the exploration of the other half of the search display – the one that can be explored without impediments. In the first experiment, we tested whether this effect transferred to visual exploration during a change detection task (under change blindness conditions), which was the case. In a second experiment, we modified the intervention (to an intermittent application) but the original version yielded more promising results. Thus, in the third experiment, the original version was used to test the longevity of its effects and whether its repeated application produced even stronger results. To this aim, we compared two groups: the first group received the intervention once, the second group repeatedly on three consecutive days. The change detection task was administered before the intervention and at four points in time after the last intervention (directly afterwards, + 1 hour, + 1 day, and +4 days). The results showed long-lasting effects of the intervention, most pronounced in the second group. Here the intervention changed the bias in the visual exploration pattern significantly until the last follow-up. We conclude that the intervention shows promise for the successful application in neglect patients.

How Older Adults Remember the World Depends On How They See It.

Age-related changes in visual exploration and memory have typically been studied separately. However, recent evidence suggests that mnemonic processes both affect, and are affected by, eye movements (EMs). Thus, by relating older adults' memory deficits to age-specific visual exploration patterns, we can improve upon models of cognitive aging.

Look up to the body: An eye-tracking investigation of 7-months-old infants’ visual exploration of emotional body expressions

The human body is an important source of information to infer a person’s emotional state. Research with adult observers indicate that the posture of the torso, arms and hands provide important perceptual cues for recognising anger, fear and happy expressions. Much less is known about whether infants process body regions differently for different body expressions. To address this issue, we used eye tracking to investigate whether infants’ visual exploration patterns differed when viewing body expressions. Forty-eight 7-months-old infants were randomly presented with static images of adult female bodies expressing anger, fear and happiness, as well as an emotionally-neutral posture. Facial cues to emotional state were removed by masking the faces. We measured the proportion of looking time, proportion and number of fixations, and duration of fixations on the head, upper body and lower body regions for the different expressions. We showed that infants explored the upper body more than the lower body. Importantly, infants at this age fixated differently on different body regions depending on the expression of the body posture. In particular, infants spent a larger proportion of their looking times and had longer fixation durations on the upper body for fear relative to the other expressions. These results extend and replicate the information about infant processing of emotional expressions displayed by human bodies, and they support the hypothesis that infants’ visual exploration of human bodies is driven by the upper body.

Visual Exploration at Higher Fixation Frequency Increases Subsequent Memory Recall.

Only a small proportion of what we see can later be recalled. Up to date it is unknown how far differences in visual exploration during encoding affect the strength of episodic memories. Here, we identified individual gaze characteristics by analyzing eye tracking data in a picture encoding task performed by 967 healthy subjects during fMRI. We found a positive correlation between fixation frequency during visual exploration and subsequent free recall performance. Brain imaging results showed a positive correlation of fixation frequency with activations in regions related to vision and memory, including the medial temporal lobe. To investigate if higher fixation frequency is causally linked to better memory, we experimentally manipulated visual exploration patterns in an independent population of 64 subjects. Doubling the number of fixations within a given exploration time increased subsequent free recall performance by 19%. Our findings provide evidence for a causal relationship between fixation frequency and episodic memory for visual information.

Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Alongside language and bipedal locomotion, tool use is a characterizing activity of human beings. Current theories in the field embrace two contrasting approaches: “manipulation-based” theories, which are anchored in the embodied-cognition view, explain tool use as deriving from past sensorimotor experiences, whereas “reasoning-based” theories suggest that people reason about object properties to solve everyday-life problems. Here, we present results from two eye-tracking experiments in which we manipulated the visuo-perceptual context (thematically consistent vs. inconsistent object-tool pairs) and the goal of the task (free observation or looking to recognise). We found that participants exhibited reversed tools’ visual-exploration patterns, focusing on the tool’s manipulation area under thematically consistent conditions and on its functional area under thematically inconsistent conditions. Crucially, looking at the tools with the aim of recognising them produced longer fixations on the tools’ functional areas irrespective of thematic consistency. In addition, tools (but not objects) were recognised faster in the thematically consistent conditions. These results strongly support reasoning-based theories of tool use, as they indicate that people primarily process semantic rather than sensorimotor information to interact with the environment in an agent’s consistent-with-goal way. Such a pre-eminence of semantic processing challenges the mainstream embodied-cognition view of human tool use.

Visual exploration differences during relational memory encoding in early psychosis

Relational memory, or the ability to form contextual associations among items encountered closely in time, is impaired in schizophrenia. The ability to bind items into a relational memory is dependent on the hippocampus, a region that is abnormal in schizophrenia. However, the hippocampus is also involved in exploratory behavior, leaving open the question whether relational memory deficits in schizophrenia are due to failure of relational binding or diminished visual exploration of individual items during encoding. We studied visual exploration patterns during the encoding of face-scene pairs in 66 healthy control subjects and 69 early psychosis patients, to test the hypothesis that differences in visual exploration during the encoding phase can explain task accuracy differences between the two groups. Psychosis patients had lower explicit test accuracy and were less likely to transition from mouth to eyes during encoding. The visual exploration pattern differences between groups did not mediate the relationship between group and explicit test accuracy. We conclude that early psychosis patients have an abnormal pattern of binding items together during encoding that warrants further research.

Sensory Processing Issues and Their Association with Social Difficulties in Children with Autism Spectrum Disorders.

Sensory processing issues have been frequently reported in individuals with Autism Spectrum Disorders (ASD), but their relationship with social and overall adaptive functioning has not been extensively characterized to date. Here, we investigate how sensory processing atypicalities relate with deficits in social skills, impaired social cognition, and general adaptive functioning in a group of preschoolers with ASD. Sixty-four children with ASD aged 3 to 6 were included in this study, along with 36 age-matched typically-developing (TD) peers. Parent-reported measures of sensory processing, social difficulties and overall adaptive functioning were collected for all children. We also obtained precise measures of social attention deployment using a custom-design eye-tracking task depicting naturalistic social scenes. Within the group of children with ASD, higher intensities of sensory issues were associated with more prominent social difficulties and lower adaptive functioning. We also found that children with ASD who had more sensory issues showed visual exploration patterns of social scenes that strongly deviated from the one seen in the TD group. The association of sensory processing atypicalities with “higher-order” functional domains such as social and adaptive functioning in children with ASD stresses the importance of further research on sensory symptoms in autism.

Coevolution of visual behaviour, the material world and social complexity, depicted by the eye-tracking of archaeological objects in humans

We live in a cluttered visual world that is overflowing with information, the continuous processing of which would be a truly daunting task. Nevertheless, our brains have evolved to select which part of a visual scene is to be prioritized and analysed in detail, and which parts can be discarded or analysed at a later stage. This selection is in part determined by the visual stimuli themselves, and is known as “selective attention”, which, in turn, determines how we explore and interact with our environment, including the distinct human artefacts produced in different socio-cultural contexts. Here we hypothesize that visual responses and material objects should therefore co-evolve to reflect changes in social complexity and culture throughout history. Using eye-tracking, we analysed the eye scan paths in response to prehistoric pottery ranging from the Neolithic through to the Iron Age (ca 6000–2000 BP), finding that each ceramic style caused a particular pattern of visual exploration. Horizontal movements become dominant in earlier periods, while vertical movements are more frequent in later periods that were marked by greater social complexity.