Visual Scene Research Articles

Context-dependent modulation of spatial attention: prioritizing behaviourally relevant stimuli

Human attention can be guided by semantic information conveyed by individual objects in the environment. Over time, we learn to allocate attention resources towards stimuli that are behaviourally relevant to ongoing action, leading to attention capture by meaningful peripheral stimuli. A common example includes, while driving, stimuli that imply a possibly hazardous scenario (e.g. a pedestrian about to cross the road) warrant attentional prioritization to ensure safe proceedings. In the current study, we report a novel phenomenon in which the guidance of attention is dependent on the stimuli appearing in a behaviourally relevant context. Using a driving simulator, we simulated a real-world driving task representing an overlearned behaviour for licensed drivers. While driving, participants underwent a peripheral cue-target paradigm where a roadside pedestrian avatar (target) appeared following a cylinder cue. Results revealed that, during simulated driving conditions, participants (all with driver’s licenses) showed greater attentional facilitation when pedestrians were oriented towards the road compared to away. This orientation-specific selectivity was not seen if the 3-D context was removed (Experiment 1) or the same visual scene was presented, but participants’ viewpoints remained stationary (Experiment 2), or an inanimate object served as a target during simulated driving (Experiment 3). This context-specific attention modulation likely reflects drivers’ expertise in automatically attending to behaviourally relevant information in a context-dependent manner.

Increased brightness assimilation in rod vision.

Our visual system uses contextual cues to estimate the brightness of surfaces: brightness can shift toward (assimilation) or away from (contrast) the brightness of the surroundings. We investigated brightness induction at different light levels and found a potential influence of rod photoreceptors on brightness induction. We then used a novel tetrachromatic display to generate stimuli differentially exciting rods or cones at a fixed light adaptation level. Under rod vision, brightness assimilation was enhanced while brightness contrast was not altered in comparison to cone vision. We ruled out that this effect was mediated by the low resolution of night vision. Our findings suggest that rod vision affects the high-level interpretation of visual scenes that results in differences in brightness assimilation but not contrast. Our results imply that the visual system employs more perceptual inferences under rod vision than under cone vision to solve visual ambiguities in complex spatial displays.

Is this scenery worth exploring? Insight into the visual encoding of navigating ants.

Solitary foraging insects like desert ants rely heavily on vision for navigation. While ants can learn visual scenes, it is unclear what cues they use to decide if a scene is worth exploring at the first place. To investigate this, we recorded the motor behavior of Cataglyphis velox ants navigating in a virtual reality set-up (VR) and measured their lateral oscillations in response to various unfamiliar visual scenes under both closed-loop and open-loop conditions. In naturalistic-looking panorama, ants display regular oscillations as observed outdoors, allowing them to efficiently scan the scenery. Manipulations of the virtual environment revealed distinct functions served by dynamic and static cues. Dynamic cues, mainly rotational optic flow, regulated the amplitude of oscillations but not their regularity. Conversely, static cues had little impact on the amplitude but were essential for producing regular oscillations. Regularity of oscillations decreased in scenes with only horizontal, only vertical or no edges but was restored in scenes with both edge types together. The actual number of edges, the visual pattern heterogeneity across azimuths, the light intensity or the relative elevation of brighter regions did not affect oscillations. We conclude that ants use a simple but functional heuristic to determine if the visual world is worth exploring, relying on the presence of at least two different edge orientations in the scene.

Effects of AAC Intervention on School Readiness Skills from Young Children with Intellectual and Developmental Disabilities.

Introduction Young children with intellectual and developmental disabilities (IDD) who have limited speech and language require access to augmentative and alternative communication (AAC) intervention, including technology and instruction. While research shows that AAC intervention can effectively support communication from children with IDD, research on its impact on school readiness skills is limited. Methods Two preschool aged children with IDD participated in this study. Using a single case, multiple baseline across responses design for each participant, this study evaluated the effectiveness of an AAC intervention, including a high-tech AAC with a personalized color photo visual scene display with modeling, time delay, prompting, and social praise. The targeted responses were three questions from the participants' state kindergarten readiness standards, including: "What is your first name?"; "What is your last name?"; and "How old are you?". Results In baseline, both participants demonstrated little success communicating answers to questions. During intervention, both participants mastered the school readiness skill of responding accurately and independently to the targeted questions. Conclusion The current study shows that with early access to AAC intervention, children with IDD may begin to build communication skills that align with school readiness standards, providing them increased opportunity to participate fully and meaningfully in general education curricula, upon entering kindergarten. Keywords: school readiness, augmentative and alternative communication, intellectual and developmental disability.

Topography of scene memory and perception activity in posterior cortex - a publicly available resource.

Adaptive behavior in complex environments requires integrating visual perception with memory of our spatial environment. Recent work has implicated three brain areas in posterior cerebral cortex - the place memory areas (PMAs) that are anterior to the three visual scene perception areas (SPAs) - in this function. However, PMAs' relationship to the broader cortical hierarchy remains unclear due to limited group-level characterization. Here, we examined the PMA and SPA locations across three fMRI datasets (44 participants, 29 female). SPAs were identified using a standard visual localizer where participants viewed scenes versus faces. PMAs were identified by contrasting activity when participants recalled personally familiar places versus familiar faces (Datasets 1-2) or places versus multiple categories (familiar faces, bodies, and objects, and famous faces; Dataset 3). Across datasets, the PMAs were located anterior to the SPAs on the ventral and lateral cortical surfaces. The anterior displacement between PMAs and SPAs was highly reproducible. Compared to public atlases, the PMAs fell at the boundary between externally-oriented networks (dorsal attention) and internally-oriented networks (default mode). Additionally, while SPAs overlapped with retinotopic maps, the PMAs were consistently located anterior to mapped visual cortex. These results establish the anatomical position of the PMAs at inflection points along the cortical hierarchy between unimodal sensory and transmodal, apical regions, which informs broader theories of how the brain integrates perception and memory for scenes. We have released probabilistic parcels of these regions to facilitate future research into their roles in spatial cognition.

Eye posture and screen alignment with simulated see-through head-mounted displays.

When rendering the visual scene for near-eye head-mounted displays, accurate knowledge of the geometry of the displays, scene objects, and eyes is required for the correct generation of the binocular images. Despite possible design and calibration efforts, these quantities are subject to positional and measurement errors, resulting in some misalignment of the images projected to each eye. Previous research investigated the effects in virtual reality (VR) setups that triggered such symptoms as eye strain and nausea. This work aimed at investigating the effects of binocular vertical misalignment (BVM) in see-through augmented reality (AR). In such devices, two conflicting environments coexist. One environment corresponds to the real world, which lies in the background and forms geometrically aligned images on the retinas. The other environment corresponds to the augmented content, which stands out as foreground and might be subject to misalignment. We simulated a see-through AR environment using a standard three-dimensional (3D) stereoscopic display to have full control and high accuracy of the real and augmented contents. Participants were involved in a visual search task that forced them to alternatively interact with the real and the augmented contents while being exposed to different amounts of BVM. The measured eye posture indicated that the compensation for vertical misalignment is equally shared by the sensory (binocular fusion) and the motor (vertical vergence) components of binocular vision. The sensitivity of each participant varied, both in terms of perceived discomfort and misalignment tolerance, suggesting that a per-user calibration might be useful for a comfortable visual experience.

Manipulable Semantic Components: a Computational Representation of Data Visualization Scenes.

Various data visualization applications such as reverse engineering and interactive authoring require a vocabulary that describes the structure of visualization scenes and the procedure to manipulate them. A few scene abstractions have been proposed, but they are restricted to specific applications for a limited set of visualization types. A unified and expressive model of data visualization scenes for different applications has been missing. To fill this gap, we present Manipulable Semantic Components (MSC), a computational representation of data visualization scenes, to support applications in scene understanding and augmentation. MSC consists of two parts: a unified object model describing the structure of a visualization scene in terms of semantic components, and a set of operations to generate and modify the scene components. We demonstrate the benefits of MSC in three applications: visualization authoring, visualization deconstruction and reuse, and animation specification.

Multisensory naturalistic decoding with high-density diffuse optical tomography.

Decoding naturalistic content from brain activity has important neuroscience and clinical implications. Information about visual scenes and intelligible speech has been decoded from cortical activity using functional magnetic resonance imaging (fMRI) and electrocorticography, but widespread applications are limited by the logistics of these technologies. High-density diffuse optical tomography (HD-DOT) offers image quality approaching that of fMRI but with the silent, open scanning environment afforded by optical methods, thus opening the door to more naturalistic research and applications. Although early visual decoding studies with HD-DOT have been promising, decoding of naturalistic auditory and multisensory stimulus information from HD-DOT data has not been established. Audiovisual decoding was investigated using HD-DOT data collected from participants who viewed a library of movie clips. A template-matching strategy was used to decode which movie clip a participant viewed based on their HD-DOT data. Factors affecting decoding performance-including trial duration and number of decoding choices-were systematically evaluated. Decoding accuracy was 94.2% for four-way decoding utilizing 4min of data per trial as a starting point. As parameters were made more stringent, decoding performance remained significantly above chance with strong effect sizes down to 15-s trials and up to 32 choices. Comparable decoding accuracies were obtained when cortical sampling was confined to visual and auditory regions and when participants were presented with purely auditory or visual clips. HD-DOT data sample cortical hemodynamics with sufficient resolution and fidelity to support decoding complex, naturalistic, multisensory stimuli via template matching. These results provide a foundation for future studies on more intricate decoding algorithms to reconstruct diverse features of novel naturalistic stimuli from HD-DOT data.

Cultural Differences in Visual Attention Emerge in Infancy.

East Asians are more likely than North Americans to attend to visual scenes holistically, focusing on the relations between objects and their background rather than isolating components. This cultural difference in context sensitivity-greater attentional allocation to the background of an image or scene-has been attributed to socialization, yet it is unknown how early in development it appears, and whether it is moderated by social information. We employed eye-tracking to investigate context-sensitivity in 15-month-olds in Japan (n=45) and the United States (n=52). Viewing faces, Japanese infants were more attentive and studied the background longer than U.S. infants. Viewing cartoon videos, Japanese infants looked at the background twice as long as U.S. infants, particularly for objects with eyes. In parent-child book reading, Japanese parents referred to the background significantly more than U.S. parents, although this was uncorrelated with infant behavior on the preceding tasks. These findings illustrated that cultural differences in attention are detectable in infancy, and that sustained attention may be an important foundation upon which culturally-specific attentional styles are built. Overall, results were consistent with the view that a context-sensitive orientation first emerges for social information and later generalizes to non-social situations.

Distinct retinal ganglion cell types in strictly subterranean, naturally microphthalmic mammals.

African mole-rats (Bathyergidae, Rodentia) are subterranean rodents that live in extensive dark underground tunnel systems and rarely emerge aboveground. They can discriminate between light and dark but show no overt visually driven behaviours except for light-avoidance responses. Their eyes and central visual system are strongly reduced but not degenerated. Here, we focus on retinal ganglion cells (RGCs). Sighted mammals have numerous RGC types with distinct morphological and functional properties that encode different aspects of a visual scene. We analysed the morphological diversity of 216 intracellularly dye-injected RGCs in the giant mole-rat (Fukomys mechowii) and 48 RGCs in Ansell's mole-rat (Fukomys anselli). Using a hierarchical cluster analysis on 11 morphological parameters, we show that both species possess at least five RGC types with distinct dendritic field sizes and branching patterns. These resemble some RGC types of the mouse and rat, but mole-rat RGCs feature overall sparser and more asymmetric branching patterns. The dendritic trees of most RGCs in all clusters are monostratified in the inner plexiform layer, but bistratified and multistratified/diffuse cells also exist. Thus, although RGC morphologies have become disorganized, the basic retinal organization principle of parallel information processing by distinct RGC types is retained.

Luminance invariant encoding in mouse primary visual cortex.

The visual system adapts to maintain sensitivity and selectivity over a large range of luminance intensities. One way that the retina maintains sensitivity across night and day is by switching between rod and cone photoreceptors, which alters the receptive fields and interneuronal correlations of retinal ganglion cells (RGCs). While these adaptations allow the retina to transmit visual information to the brain across environmental conditions, the code used for that transmission varies. To determine how downstream targets encode visual scenes across light levels, we measured the effects of luminance adaptation on thalamic and cortical population activity. While changes in the retinal output are evident in the lateral geniculate nucleus (LGN), selectivity in the primary visual cortex (V1) is largely invariant to the changes in luminance. We show that the visual system could maintain sensitivity across environmental conditions without altering cortical selectivity through the convergence of parallel functional pathways from the thalamus to the cortex.

The N400 component reflecting semantic and repetition priming of visual scenes is suppressed during the attentional blink.

In the attentional blink paradigm, participants attempt to identify two targets appearing in a rapidly presented stream of distractors. Report accuracy is typically high for the first target (T1) while identification of the second target (T2) is impaired when it follows within about 200-400ms of T1. An important question is whether T2 is processed to a semantic level even when participants are unaware of its identity. We examined this issue in three studies that used natural scenes as stimuli and the N400 component of the event-related potential (ERP) as a measure of semantic priming. In the first experiment, the prime (e.g., a doghouse in a yard) was presented at the beginning of the trial and a test picture that was related (e.g., a dog standing in the kitchen) or unrelated (e.g., a coffee mug on a table) appeared as T2. In the second experiment, the prime was presented as T2 and the test picture appeared at the end of the picture sequence. In both experiments, we found robust semantic priming when participants were aware of the identity of the blinked picture and an absence of priming when they were unaware. In Experiment 3, we used identity priming to assess whether earlier representations preceding semantics were preserved, and again found that priming critically depended on awareness of the prime's identity. These results suggest that semantic priming in scenes, as measured with the N400, is a higher-level process that critically depends on attention and awareness.

A Scene Knowledge Integrating Network for Transmission Line Multi-Fitting Detection.

Aiming at the severe occlusion problem and the tiny-scale object problem in the multi-fitting detection task, the Scene Knowledge Integrating Network (SKIN), including the scene filter module (SFM) and scene structure information module (SSIM) is proposed. Firstly, the particularity of the scene in the multi-fitting detection task is analyzed. Hence, the aggregation of the fittings is defined as the scene according to the professional knowledge of the power field and the habit of the operators in identifying the fittings. So, the scene knowledge will include global context information, fitting fine-grained visual information and scene structure information. Then, a scene filter module is designed to learn the global context information and fitting fine-grained visual information, and a scene structure module is designed to learn the scene structure information. Finally, the scene semantic features are used as the carrier to integrate three categories of information into the relative scene features, which can assist in the recognition of the occluded fittings and the tiny-scale fittings after feature mining and feature integration. The experiments show that the proposed network can effectively improve the performance of the multi-fitting detection task compared with the Faster R-CNN and other state-of-the-art models. In particular, the detection performances of the occluded and tiny-scale fittings are significantly improved.

Vection in Individuals with and without Concussion: Associations with Postural Responses and Visual Dependence

Background We investigated whether everyday situations that trigger post-concussion symptoms (i.e., dynamic visual scenes), induce vection (illusory self-motion) and/or affect postural stability. Materials & Methods Concussed and control participants were moved through a virtual grocery store, and rated their vection intensity. Postural sway during visual motion was measured. Baseline tests assessed concussion symptoms and sensory functioning, including visual dependence. Results Vection ratings were higher in concussed individuals than controls, and were predicted by faster visual speeds. Vection and visual speed also predicted postural sway in the concussion group. Visual dependence was positively associated with vection intensity and all postural measures. Conclusion These findings provide valuable insights for the development of future symptom-screening tools and rehabilitation strategies.

Catch-Up Saccades in Vestibulo-Ocular Reflex Deficit: Contribution of Visual Information?

Catch-up saccades help to compensate for loss of gaze stabilization during rapid head rotation in case of vestibular deficit. While overt saccades observed after head rotation are obviously visually guided, some of these catch-up saccades occur with shorter latency while the head is still moving, anticipating the needed final eye position. These covert saccades seem to be generated based on the integration of multisensory inputs. Vision could be one of these inputs, but the known delay for triggering visually guided saccades questions this possibility. The main objective of this study is to evaluate the potential role of visual information for controlling (triggering and guiding) the first catch-up saccades in patients suffering from bilateral vestibulopathy. To investigate this, we used head impulse test in a virtual reality setting allowing to create different visuo-vestibular mismatch conditions. Twelve patients with bilateral vestibulopathy were recruited. We assessed in our patient group the validity of our virtual reality head impulse testing approach by comparing recorded eye and head movement to classical video head impulse test. Then, using the virtual reality system, we tested head impulse test under both normal and three visuo-vestibular mismatch conditions. In these mismatch conditions, the movement of the visual scene relative to the head movement was altered: decreased in amplitude by 50% (half), nullified (freeze), or inverted in direction (inverse). Recorded eye and head movements during these different conditions were then analyzed, more specifically the characteristics of the first catch-up saccade. Impaired vestibulo-ocular reflex required subjects to systematically perform catch-up saccades, which could be covert or overt. The latency of the first catch-up saccade increased along with the amount of visuo-vestibular mismatch between the four conditions (i.e., from normal to half to freeze to inverse) and, consequently, the mean percentage of covert saccades decreased with increasing visual feedback error. However, the freeze and inverse conditions allowed us to reveal the existence of many saccades performed in the wrong direction relative to visual feedback. These visually discordant saccades were present in over half of the trials, they were mainly covert and their percentage was inversely correlated with residual vestibulo-ocular reflex gain. Visual information significantly impacts catch-up saccade latency and the relative number of covert saccades during head impulse testing in vestibular deficit. However, in more than 50% of trials involving a visuo-vestibular mismatch, catch-up saccades remained directed in the compensatory direction relative to head movement, that is, they were visually discordant. Therefore, contrary to previously published proposals, visual information does not appear to be the primary component of the multisensory inputs required for the production of catch-up covert saccades in vestibular deficit. Finally, we discuss a new nomenclature of catch-up saccades in case of vestibular deficit introducing the terms of open and closed loop saccades.

A perceptual cue-based mechanism for automatic assignment of thematic agent and patient roles.

Understanding social events requires assigning the participating entities to roles such as agent and patient, a mental operation that is reportedly effortless. We investigated whether, in processing visual scenes, role assignment is accomplished automatically (i.e., when the task does not require it), based on visuospatial information, without requiring semantic or linguistic encoding of the stimuli. Human adults saw a series of images featuring the same male and female actors next to each other, one in an agentlike (more dynamic/leaning forward) and the other in a patientlike (static/less dynamic) posture. Participants indicated the side (left/right) of a target actor (i.e., the woman). From trial to trial, body postures changed, but the roles, defined by the type of posture, sometimes changed, sometimes not. We predicted that if participants spontaneously saw the actors as agent and patient, they should be slower to respond when roles switched from trial n-1 to trial n, than when they stayed the same (role switch cost). Results confirmed this hypothesis (Experiments 1-3). A role switch cost was also found when roles were defined by another visual relational cue, the relative positioning (where one actor stands relative to another), but not when actors were presented in isolation (Experiments 4-6). These findings reveal a mechanism for automatic role assignment based on encoding of visual relational information in social (multiple-person) scenes. Since we found that roles in one trial affected the processing of the subsequent trial despite variations in postures and spatial relations, this mechanism must be one that assigns entities in a scene, to the abstract categories of agent and patient. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Analysis of Tidal Cycle Wave Breaking Distribution Characteristics on a Low-Tide Terrace Beach Using Video Imagery Segmentation

Wave breaking is a fundamental process in ocean energy dissipation and plays a crucial role in the exchange between ocean and nearshore sediments. Foam, the primary visible feature of wave breaking areas, serves as a direct indicator of wave breaking processes. Monitoring the distribution of foam via remote sensing can reveal the spatiotemporal patterns of nearshore wave breaking. Existing studies on wave breaking processes primarily focus on individual wave events or short timescales, limiting their effectiveness for nearshore regions where hydrodynamic processes are often represented at tidal cycles. In this study, video imagery from a typical low-tide terrace (LTT) beach was segmented into four categories, including the wave breaking foam, using the DeepLabv3+ architecture, a convolutional neural networks (CNNs)-based model suitable for semantic segmentation in complex visual scenes. After training and testing on a manually labelled dataset, which was divided into training, validation, and testing sets based on different time periods, the overall classification accuracy of the model was 96.4%, with an accuracy of 96.2% for detecting wave breaking foam. Subsequently, a heatmap of the wave breaking foam distribution over a tidal cycle on the LTT beach was generated. During the tidal cycle, the foam distribution density exhibited both alongshore variability, and a pronounced bimodal structure in the cross-shore direction. Analysis of morphodynamical data collected in the field indicated that the bimodal structure is primarily driven by tidal variations. The wave breaking process is a key factor in shaping the profile morphology of LTT beaches. High-frequency video monitoring further showed the wave breaking patterns vary significantly with tidal levels, leading to diverse geomorphological features at various cross-shore locations.

YOLOv10-based Model for Player and Football Detection

This study presents an advanced YOLOv10n-based method for the automatic detection of football players and balls directly from match videos. We enhance the YOLOv10 architecture with several significant improvements, including additional detection heads, the integration of C2f_faster and C3_faster modules for enhanced processing speed and accuracy, and the inclusion of BotNet modules with self-attention mechanisms for managing complex visual scenes. Further, we incorporate GhostConv modules to reduce computational overhead while maintaining effective feature extraction. These architectural modifications ensure robust detection capabilities in real-time sports environments, addressing challenges such as high-speed movements, frequent occlusions, and variable lighting conditions typical of both indoor and outdoor stadiums. Validation on internet-sourced images from football matches demonstrates the practicality and effectiveness of our model.

When abstract becomes concrete, naturalistic encoding of concepts in the brain.

Language is acquired and processed in complex and dynamic naturalistic contexts, involving the simultaneous processing of connected speech, faces, bodies, objects, etc. How words and their associated concepts are encoded in the brain during real-world processing is still unknown. Here, the representational structure of concrete and abstract concepts was investigated during movie watching to address the extent to which brain responses dynamically change depending on visual context. First, across contexts, concrete and abstract concepts are shown to encode different experience-based information in separable sets of brain regions. However, these differences are reduced when multimodal context is considered. Specifically, the response profile of abstract words becomes more concrete-like when these are processed in visual scenes highly related to their meaning. Conversely, when the visual context is unrelated to a given concrete word, the activation pattern resembles more that of abstract conceptual processing. These results suggest that while concepts generally encode habitual experiences, the underlying neurobiological organisation is not fixed but depends dynamically on available contextual information.

The underestimation of speed perception while walking in virtual reality with bodyweight unloading

As space agencies prepare for long-duration missions, such as establishing a permanent moon base, maintaining astronauts’ physical and mental health becomes crucial. Exercise is known to counteract the adverse effects of space travel, and virtual reality (VR) has the potential to enhance the psychological well-being of crew members. This study aimed to investigate the impact of bodyweight unloading on speed perception during treadmill walking in a VR environment. Eighteen participants walked on an AlterG treadmill while wearing a VR headset that displayed a moving virtual outdoor environment. The treadmill simulated different bodyweight conditions (100%, 60%, and 20%) to represent Earth’s gravity, intermediate, and lunar gravity, respectively. A staircase method was used to determine participants’ estimated threshold speed for perceiving visual speed equivalence. The results revealed that in all conditions, participants consistently underestimated their walking speed compared to the visual scene speed. No significant differences were found in estimated threshold speeds between the unloading conditions. Individual responses varied, possibly due to biomechanical differences and personal preferences. Participants reported no motion sickness symptoms, likely attributed to the short duration of speed mismatches and the use of high-resolution and high-refresh-rate VR equipment. This study provides insights into the combination of VR and treadmill walking, suggesting the importance of individual customization to enhance user experience and prevent motion sickness.