Complex Visual Environments Research Articles

Brain-like border ownership signals support prediction of natural videos.

To make sense of visual scenes, the brain must segment foreground from background. This is thought to be facilitated by neurons in the primate visual system that encode border ownership (BOS), i.e. whether a local border is part of an object on one or the other side of the border. It is unclear how these signals emerge in neural networks without a teaching signal of what is foreground and background. In this study, we investigated whether BOS signals exist in PredNet, a self-supervised artificial neural network trained to predict the next image frame of natural video sequences. We found that a significant number of units in PredNet are selective for BOS. Moreover these units share several other properties with the BOS neurons in the brain, including robustness to scene variations that constitute common object transformations in natural videos, and hysteresis of BOS signals. Finally, we performed ablation experiments and found that BOS units contribute more to prediction than non-BOS units for videos with moving objects. Our findings indicate that BOS units are especially useful to predict future input in natural videos, even when networks are not required to segment foreground from background. This suggests that BOS neurons in the brain might be the result of evolutionary or developmental pressure to predict future input in natural, complex dynamic visual environments.

Introduction to the assessment and management of persistent postural-perceptual dizziness.

Persistent postural-perceptual dizziness classifies patients with chronic dizziness, often triggered by an acute episode of vestibular dysfunction or threat to balance. Unsteadiness and spatial disorientation vary in intensity but persist for over three months, exacerbated by complex visual environments. Literature suggests diagnosis relies on a clinical history of persistent subjective dizziness and normal vestibular and neurological examination findings. Behavioural diagnostic biomarkers have been proposed, to facilitate diagnosis. Research has focused on understanding the neural mechanisms that underpin this perceptual disorder, with imaging data supporting altered connectivity between neural brain networks that process vision, motion and emotion. Behavioural research identified the perceptual and motor responses to a heightened perception of imbalance. Management utilises head and body motion detection, and downregulation of visual motion excitability, reducing postural hypervigilance and anxiety. Combinations of physical and cognitive therapies, with antidepressant medications, help if the condition is associated with mood disorder.

AI Enhancement Automated Movement Detection

This paper introduces an innovative artificial intelligence (AI) method aimed at tackling the challenges associated with detecting and tracking moving objects in video surveillance systems. By utilizing self-organization through artificial neural networks, our approach effectively manages scenes with dynamic backgrounds and gradual changes in lighting, ensuring robust detection across different types of videos recorded by stationary cameras. In the realm of moving object detection, our method leverages the adaptability of neural networks, enabling precise detection in complex visual environments. For object tracking, we propose a combination of Kalman filtering techniques and a sophisticated matching model based on Multiple Hypothesis Testing, ensuring accurate and consistent tracking across frames. Through experimental validation using various color video sequences, we demonstrate the effectiveness and reliability of our approach, highlighting its potential to enhance the performance of surveillance systems in real-world scenarios.

Gaze-contingent processing improves mobility, scene recognition and visual search in simulated head-steered prosthetic vision.

Objective.The enabling technology of visual prosthetics for the blind is making rapid progress. However, there are still uncertainties regarding the functional outcomes, which can depend on many design choices in the development. In visual prostheses with a head-mounted camera, a particularly challenging question is how to deal with the gaze-locked visual percept associated with spatial updating conflicts in the brain. The current study investigates a recently proposed compensation strategy based on gaze-contingent image processing with eye-tracking. Gaze-contingent processing is expected to reinforce natural-like visual scanning and reestablished spatial updating based on eye movements. The beneficial effects remain to be investigated for daily life activities in complex visual environments.Approach.The current study evaluates the benefits of gaze-contingent processing versus gaze-locked and gaze-ignored simulations in the context of mobility, scene recognition and visual search, using a virtual reality simulated prosthetic vision paradigm with sighted subjects.Main results.Compared to gaze-locked vision, gaze-contingent processing was consistently found to improve the speed in all experimental tasks, as well as the subjective quality of vision. Similar or further improvements were found in a control condition that ignores gaze-dependent effects, a simulation that is unattainable in the clinical reality.Significance.Our results suggest that gaze-locked vision and spatial updating conflicts can be debilitating for complex visually-guided activities of daily living such as mobility and orientation. Therefore, for prospective users of head-steered prostheses with an unimpaired oculomotor system, the inclusion of a compensatory eye-tracking system is strongly endorsed.

TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation

Multi-spectral object Re-identification (ReID) aims to retrieve specific objects by leveraging complementary information from different image spectra. It delivers great advantages over traditional single-spectral ReID in complex visual environment. However, the significant distribution gap among different image spectra poses great challenges for effective multi-spectral feature representations. In addition, most of current Transformer-based ReID methods only utilize the global feature of class tokens to achieve the holistic retrieval, ignoring the local discriminative ones. To address the above issues, we step further to utilize all the tokens of Transformers and propose a cyclic token permutation framework for multi-spectral object ReID, dubbled TOP-ReID. More specifically, we first deploy a multi-stream deep network based on vision Transformers to preserve distinct information from different image spectra. Then, we propose a Token Permutation Module (TPM) for cyclic multi-spectral feature aggregation. It not only facilitates the spatial feature alignment across different image spectra, but also allows the class token of each spectrum to perceive the local details of other spectra. Meanwhile, we propose a Complementary Reconstruction Module (CRM), which introduces dense token-level reconstruction constraints to reduce the distribution gap across different image spectra. With the above modules, our proposed framework can generate more discriminative multi-spectral features for robust object ReID. Extensive experiments on three ReID benchmarks (i.e., RGBNT201, RGBNT100 and MSVR310) verify the effectiveness of our methods. The code is available at https://github.com/924973292/TOP-ReID.

An investigation into the correlation between visual performance in simulated complex environments and academic attainment among primary school students

Traditional vision screenings in schools are limited to simple visual tasks, yet students in their daily learning face more complex visual environments. Binocular rivalry tasks can partially simulate the visual challenges of real visual environments and activate advanced visual processing mechanisms that simple visual tasks cannot. Therefore, by superimposing binocular rivalry-state tasks onto simple visual tasks, we have developed an innovative vision screening program to rapidly and extensively assess students’ visual performance in complex environments. This is a cross-sectional study in which we investigated the performance of 1126 grade 1–6 students from a primary school in Wuxi, China, in rivalry-state stereoscopic vision tasks. The correlation between the screening results of 1044 students and their academic achievements was also statistically analyzed. The study results revealed pass rates of 53.5–60.5% across various visual tests. Specifically, for first-grade students, there was a statistically significant difference in standardized Chinese scores between the group that failed and the group that passed the rivalry-state stereoscopic vision test (− 0.49 ± 3.42 vs. 0.22 ± 0.58, t = − 2.081, P = 0.04). This result underscores the importance of focusing on the visual adaptability of first graders in complex environments.Trail registration: Ethics Committee of Affiliated Children’s Hospital of Jiangnan University-Certificate number: WXCH2022-04-027

Cortical recurrence supports resilience to sensory variance in the primary visual cortex

Our daily endeavors occur in a complex visual environment, whose intrinsic variability challenges the way we integrate information to make decisions. By processing myriads of parallel sensory inputs, our brain is theoretically able to compute the variance of its environment, a cue known to guide our behavior. Yet, the neurobiological and computational basis of such variance computations are still poorly understood. Here, we quantify the dynamics of sensory variance modulations of cat primary visual cortex neurons. We report two archetypal neuronal responses, one of which is resilient to changes in variance and co-encodes the sensory feature and its variance, improving the population encoding of orientation. The existence of these variance-specific responses can be accounted for by a model of intracortical recurrent connectivity. We thus propose that local recurrent circuits process uncertainty as a generic computation, advancing our understanding of how the brain handles naturalistic inputs.

Head-mounted display versus computer monitor for visual attention screening: A comparative study

Visual attention is crucial to many tasks during working. When it is impaired, the risk of occupational accidents is increased. A potential accident prevention would be the tracking of employees' attentional states to construct break regimes. There is a promising visual attention test administered on a computer monitor (CM) that has several advantages over widely used continuous performance tests in detecting inattentiveness in occupational environments. However, as the setup with a CM is impractical for the use in particular working environments (e.g., lack of space or disturbing exposure to light), the test was implemented into a head-mounted display (HMD). This study aimed to investigate whether the HMD version of the test is a suitable alternative to the CM version. For this purpose, participants (N = 30; 20–29 y) performed both tests. The performance on the HMD was significantly lower than on the CM. Moreover, the performances were compared with normative data recorded with a CM in a previous study. These data significantly differ from the data recorded with the CM in the present study. This emphasizes the importance of a standardized test environment, which could be provided by an HMD. Conclusively, this study revealed that the new VR tool, based on a previous test designed to assess visual skills in a complex visual environment, exhibited good psychometric property regarding the reliability. In additional, no problems were revealed regarding the functionality and usability of the HMD.

A Time-Delay Feedback Neural Network for Discriminating Small, Fast-Moving Targets in Complex Dynamic Environments.

Discriminating small moving objects within complex visual environments is a significant challenge for autonomous micro-robots that are generally limited in computational power. By exploiting their highly evolved visual systems, flying insects can effectively detect mates and track prey during rapid pursuits, even though the small targets equate to only a few pixels in their visual field. The high degree of sensitivity to small target movement is supported by a class of specialized neurons called small target motion detectors (STMDs). Existing STMD-based computational models normally comprise four sequentially arranged neural layers interconnected via feedforward loops to extract information on small target motion from raw visual inputs. However, feedback, another important regulatory circuit for motion perception, has not been investigated in the STMD pathway and its functional roles for small target motion detection are not clear. In this article, we propose an STMD-based neural network with feedback connection (feedback STMD), where the network output is temporally delayed, then fed back to the lower layers to mediate neural responses. We compare the properties of the model with and without the time-delay feedback loop and find that it shows a preference for high-velocity objects. Extensive experiments suggest that the feedback STMD achieves superior detection performance for fast-moving small targets, while significantly suppressing background false positive movements which display lower velocities. The proposed feedback model provides an effective solution in robotic visual systems for detecting fast-moving small targets that are always salient and potentially threatening.

Neural mechanisms of background and velocity effects in smooth pursuit eye movements.

Smooth pursuit eye movements (SPEM) are essential to guide behaviour in complex visual environments. SPEM accuracy is known to be degraded by the presence of a structured visual background and at higher target velocities. The aim of this preregistered study was to investigate the neural mechanisms of these robust behavioural effects. N=33 participants performed a SPEM task with two background conditions (present and absent) at two target velocities (0.4 and 0.6Hz). Eye movement and BOLD data were collected simultaneously. Both the presence of a structured background and faster target velocity decreased pursuit gain and increased catch-up saccade rate. Faster targets additionally increased position error. Higher BOLD response with background was found in extensive clusters in visual, parietal, and frontal areas (including the medial frontal eye fields; FEF) partially overlapping with the known SPEM network. Faster targets were associated with higher BOLD response in visual cortex and left lateral FEF. Task-based functional connectivity analyses (psychophysiological interactions; PPI) largely replicated previous results in the basic SPEM network but did not yield additional information regarding the neural underpinnings of the background and velocity effects. The results show that the presentation of visual background stimuli during SPEM induces activity in a widespread visuo-parieto-frontal network including areas contributing to cognitive aspects of oculomotor control such as medial FEF, whereas the response to higher target velocity involves visual and motor areas such as lateral FEF. Therefore, we were able to propose for the first time different functions of the medial and lateral FEF during SPEM.

Introduction to Emerging Research for Informing the Design and Use of Wayfinding Systems

This issue of the Interdisciplinary Journal of Signage and Wayfinding provides findings from emerging research informing the design and use of technology and traditional signage for enhanced wayfinding in a diverse array of complex visual environments. Some of the results reported here will likely have important implications for the design and use of wayfinding systems in interior, as well as outdoors environments.

Vestibular/ocular motor symptoms in concussed adolescents are linked to retrosplenial activation.

Following concussion, adolescents often experience vestibular and ocular motor symptoms as well as working memory deficits that may affect their cognitive, academic and social well-being. Complex visual environments including school activities, playing sports, or socializing with friends may be overwhelming for concussed adolescents suffering from headache, dizziness, nausea and fogginess, thus imposing heightened requirements on working memory to adequately function in such environments. While understanding the relationship between working memory and vestibular/ocular motor symptoms is critically important, no previous study has examined how an increase in working memory task difficulty affects the relationship between severity of vestibular/ocular motor symptoms and brain and behavioural responses in a working memory task. To address this question, we examined 80 adolescents (53 concussed, 27 non-concussed) using functional MRI while performing a 1-back (easy) and 2-back (difficult) working memory tasks with angry, happy, neutral and sad face distractors. Concussed adolescents completed the vestibular/ocular motor screening and were scanned within 10 days of injury. We found that all participants showed lower accuracy and slower reaction time on difficult (2-back) versus easy (1-back) tasks (P-values < 0.05). Concussed adolescents were significantly slower than controls across all conditions (P < 0.05). In concussed adolescents, higher vestibular/ocular motor screening total scores were associated with significantly greater differences in reaction time between 1-back and 2-back across all distractor conditions and significantly greater differences in retrosplenial cortex activation for the 1-back versus 2-back condition with neutral face distractors (P-values < 0.05). Our findings suggest that processing of emotionally ambiguous information (e.g. neutral faces) additionally increases the task difficulty for concussed adolescents. Post-concussion vestibular/ocular motor symptoms may reduce the ability to inhibit emotionally ambiguous information during working memory tasks, potentially affecting cognitive, academic and social functioning in concussed adolescents.

Benefits of Endogenous Spatial Attention During Visual Double-Training in Cortically-Blinded Fields.

Recovery of visual discrimination thresholds inside cortically-blinded (CB) fields is most commonly attained at a single, trained location at a time, with iterative progress deeper into the blind field as performance improves over several months. As such, training is slow, inefficient, burdensome, and often frustrating for patients. Here, we investigated whether double-location training, coupled with a covert spatial-attention (SA) pre-cue, could improve the efficiency of training. Nine CB participants completed a randomized, training assignment with either a spatial attention or neutral pre-cue. All trained for a similar length of time on a fine direction discrimination task at two blind field locations simultaneously. Training stimuli and tasks for both cohorts were identical, save for the presence of a central pre-cue, to manipulate endogenous (voluntary) SA, or a Neutral pre-cue. Participants in the SA training cohort demonstrated marked improvements in direction discrimination thresholds, albeit not to normal/intact-field levels; participants in the Neutral training cohort remained impaired. Thus, double-training within cortically blind fields, when coupled with SA pre-cues can significantly improve direction discrimination thresholds at two locations simultaneously, offering a new method to improve performance and reduce the training burden for CB patients. Double-training without SA pre-cues revealed a hitherto unrecognized limitation of cortically-blind visual systems’ ability to improve while processing two stimuli simultaneously. These data could potentially explain why exposure to the typically complex visual environments encountered in everyday life is insufficient to induce visual recovery in CB patients. It is hoped that these new insights will direct both research and therapeutic developments toward methods that can attain better, faster recovery of vision in CB fields.

Ecology drives patterns of spectral transmission in the ocular lenses of frogs and salamanders

Abstract The spectral characteristics of vertebrate ocular lenses affect the image of the world that is projected onto the retina, and thus help shape diverse visual capabilities. Here, we tested whether amphibian lens transmission is driven by adaptation to diurnal activity (bright light) and/or scansorial habits (complex visual environments). Spectral transmission through the lenses of 79 species of frogs and six species of salamanders was measured, and data for 29 additional frog species compiled from published literature. Phylogenetic comparative methods were used to test ecological explanations of variation in lens transmission and to test for selection across traits. Lenses of diurnal (day‐active) and scansorial (climbing) frogs transmitted significantly less shortwave light than those of non‐diurnal or non‐scansorial amphibians, and evolutionary modelling suggested that these differences have resulted from differential selection. The presence of shortwave‐transparent lenses was common among the sampled amphibians, which implies that many are sensitive to shortwave light to some degree even in the absence of visual pigments maximally sensitive in the UV. This suggests that shortwave light, including UV, could play an important role in amphibian behaviour and ecology. Shortwave‐absorbing lens pigments likely provide higher visual acuity to diurnally active frogs of multiple ecologies and to nocturnally active scansorial frogs. This new mechanistic understanding of amphibian visual systems suggests that shortwave‐filtering lenses are adaptive not only in daylight conditions but also in those scotopic conditions where high acuity is advantageous. Read the free Plain Language Summary for this article on the Journal blog.

Factors governing the assignment of visual consequence to the corresponding action.

To achieve fast feedback control of voluntary movements, the visual consequences of our motor commands need to be quickly identified and analyzed by the motor control processes in the brain. These processes work remarkably well even in complex visual environments and in the face of discrepancies between physical actuator and visually perceived effect, e.g. when moving a computer mouse on a visually crowded screen. Here, we use an ambiguous situation in which a single cursor could be controlled by either the left or the right hand to determine the visual and cognitive factors that determine the assignment of a visual stimulus to the corresponding motor command. Our results demonstrate that the visuomotor system is exquisitely sensitive to the spatio-temporal correlation between cursor and hands, learning the appropriate mapping implicitly within several minutes. In contrast, spatial proximity between end effector and visual consequence has an immediate but only transient effect on the assignment process. Finally, an explicit instruction about which hand controls the cursor only has a minor influence when the instruction is presented first. These findings provide insight into the relative importance of the factors that determine the binding of visual information to the corresponding motor structures to enable fast feedback control.NEW & NOTEWORTHY For efficient visuomotor online control, the brain needs to solve the correspondence problem between an ongoing movement and its visual consequences. Here, we challenge the visuomotor system with an ambiguous reaching task, in which the visual feedback was controlled by either hand or by a combination of both. Our findings characterize the properties of a flexible assignment process that quickly takes into account the spatio-temporal properties of movements and the visual scene.

Visual fixations and visually induced dizziness: An exploratory study

BackgroundVisually induced dizziness can develop as a sequala of a vestibular disorder and is characterized by symptoms of nausea, dizziness, and imbalance in rich visual environments such as supermarkets and shopping malls. To date the mechanisms underlying visually induced dizziness are poorly understood. Research questionWhat are the characteristics of visual fixations and postural sway in adults with visually induced dizziness compared to healthy adults when exposed to increasingly complex visual environments? MethodsWe recruited 20 adults with visually induced dizziness and 20 healthy adults to this cross-sectional exploratory study. Participants were instructed to maintain gaze on letters projected on a large screen with backgrounds of differing visual complexity. The number of visual refixations, movement of the centre of pressure, and movement of the head and body centres of mass were recorded. ResultsAdults with visually induced dizziness showed a significantly higher number of visual refixations (F= 10.592, p < 0.01), and increased mean velocity of head and body centres of mass movement (F= 14.034, p < 0.01 and F= 6.553, p < 0.05 respectively) compared to healthy adults. SignificanceAdults with visually induced dizziness exhibited visual fixational instability and increased postural and head sway compared to healthy adults. This was mainly observed in conditions with complex and moving backgrounds. This may account for reports from adults with visually induced dizziness of worsening symptoms in busy environments. The results from the study may assist in guiding intervention development to reduce symptoms of visually induced dizziness.

"Blue Sky Effect": Contextual Influences on Pupil Size During Naturalistic Visual Search.

Pupil size is influenced by cognitive and non-cognitive factors. One of the strongest modulators of pupil size is scene luminance, which complicates studies of cognitive pupillometry in environments with complex patterns of visual stimulation. To help understand how dynamic visual scene statistics influence pupil size during an active visual search task in a visually rich 3D virtual environment (VE), we analyzed the correlation between pupil size and intensity changes of image pixels in the red, green, and blue (RGB) channels within a large window (~14 degrees) surrounding the gaze position over time. Overall, blue and green channels had a stronger influence on pupil size than the red channel. The correlation maps were not consistent with the hypothesis of a foveal bias for luminance, instead revealing a significant contextual effect, whereby pixels above the gaze point in the green/blue channels had a disproportionate impact on pupil size. We hypothesized this differential sensitivity of pupil responsiveness to blue light from above as a “blue sky effect,” and confirmed this finding with a follow-on experiment with a controlled laboratory task. Pupillary constrictions were significantly stronger when blue was presented above fixation (paired with luminance-matched gray on bottom) compared to below fixation. This effect was specific for the blue color channel and this stimulus orientation. These results highlight the differential sensitivity of pupillary responses to scene statistics in studies or applications that involve complex visual environments and suggest blue light as a predominant factor influencing pupil size.

Habitat complexity and complex signal function – exploring the role of ornamentation

AbstractAnimals often communicate in complex, heterogeneous environments, leading to hypothesized selection for increased detectability or discriminability in signaling traits. The extent to which secondary sexual ornaments have evolved to overcome the challenges of signaling in complex environments, however, remains understudied, especially in comparison to their role as indicator traits. This study tested the hypothesis that the condition-dependent secondary sexual ornamentation in the wolf spider Rabidosa rabida functions to increase detectability/discriminability in visually complex environments. We predicted that male ornamentation would interact with the complexity of the signaling environment to affect male mating success. In particular, we expected ornaments to confer a greater mating advantage when males courted in visually complex environments. To test this, we artificially manipulated male foreleg ornamentation (present/absent) and ran repeated-measures mating trials across laboratory microcosms that represented simple versus complex visual signaling environments. Microcosm visual complexity differed in their background pattern, grass stem color, and grass stem placement. We found that ornamented males mated more often and more quickly than unornamented males across both environments, but we found no support for an ornament-by-environment interaction. Male courtship rate, however, did interact with the signaling environment. Despite achieving the same level of mating success across signaling environments, ornamented males courted less rapidly in complex versus simple environments, although environmental complexity had no influence on unornamented male courtship rates. Our results suggest that the visual complexity of the signaling environment influences the interactive influence of ornamentation and dynamic visual courtship on female mate choice.

P.0438 Saccadik eye movements in simple and complex visual environments differenciate mild cognitive impairment: preliminary clinical results

P.0438 Saccadik eye movements in simple and complex visual environments differenciate mild cognitive impairment: preliminary clinical results

People with persistent postural-perceptual dizziness demonstrate altered postural strategies in complex visual and cognitive environments.

People with PPPD report imbalance, increase in symptoms and impaired function within complex visual environments, but understanding of the mechanism for these behaviors is still lacking. To investigate postural control in PPPD we compared changes in center of pressure (COP) and head kinematics of people with PPPD (N = 22) and healthy controls (N = 20) in response to different combinations of visual and cognitive perturbations during a challenging balance task. Participants stood in a tandem position. Static or moving stars (0.2 Hz, 5 mm or 32 mm amplitude, anterior-posterior direction) were displayed through a head-mounted display (HTC Vive). On half the trials, participants performed a serial-3 subtraction task. We measured medio-lateral and anterior-posterior path and acceleration of COP and head. Controls significantly increased all COP and head parameters with the cognitive task whereas PPPD increased only COP ML path and acceleration. Only controls significantly increased head anterior-posterior & medio-lateral acceleration with moving visual load. Cognitive task performance was similar between groups. We observed altered postural strategies in people with PPPD, in the form of reduced movement with challenge, particularly around the head segment. The potential of this simple and portable head-mounted display setup for differential diagnosis of vestibular disorders should be further explored.