Early Stages Of Visual Processing Research Articles

Decoding Trans-Saccadic Prediction Error.

We are constantly sampling our environment by moving our eyes, but our subjective experience of the world is stable and constant. Stimulus displacement during or shortly after a saccade often goes unnoticed, a phenomenon called the saccadic suppression of displacement. Although we fail to notice such displacements, our oculomotor system computes the prediction errors and adequately adjusts the gaze and future saccadic execution, a phenomenon known as saccadic adaptation. In the present study, we aimed to find a brain signature of the trans-saccadic prediction error that informs the motor system but not explicit perception. We asked participants (either sex) to report whether a visual target was displaced during a saccade while recording electroencephalography (EEG). Using multivariate pattern analysis, we were able to differentiate displacements from no displacements, even when participants failed to report the displacement. In other words, we found that trans-saccadic prediction error is represented in the EEG signal 100 ms after the displacement presentation, mainly in occipital and parieto-occipital channels, even in the absence of explicit perception of the displacement.SIGNIFICANCE STATEMENT Stability in vision occurs even while performing saccades. One suggested mechanism for this counterintuitive visual phenomenon is that external displacement is suppressed during the retinal remapping caused by a saccade. Here, we shed light on the mechanisms of trans-saccadic stability by showing that displacement information is not entirely suppressed and specifically present in the early stages of visual processing. Such a signal is relevant and computed for oculomotor adjustment despite being neglected for perception.

Fast discrimination of fragmentary images: the role of local optimal information.

In naturalistic conditions, objects in the scene may be partly occluded and the visual system has to recognize the whole image based on the little information contained in some visible fragments. Previous studies demonstrated that humans can successfully recognize severely occluded images, but the underlying mechanisms occurring in the early stages of visual processing are still poorly understood. The main objective of this work is to investigate the contribution of local information contained in a few visible fragments to image discrimination in fast vision. It has been already shown that a specific set of features, predicted by a constrained maximum-entropy model to be optimal carriers of information (optimal features), are used to build simplified early visual representations (primal sketch) that are sufficient for fast image discrimination. These features are also considered salient by the visual system and can guide visual attention when presented isolated in artificial stimuli. Here, we explore whether these local features also play a significant role in more natural settings, where all existing features are kept, but the overall available information is drastically reduced. Indeed, the task requires discrimination of naturalistic images based on a very brief presentation (25 ms) of a few small visible image fragments. In the main experiment, we reduced the possibility to perform the task based on global-luminance positional cues by presenting randomly inverted-contrast images, and we measured how much observers' performance relies on the local features contained in the fragments or on global information. The size and the number of fragments were determined in two preliminary experiments. Results show that observers are very skilled in fast image discrimination, even when a drastic occlusion is applied. When observers cannot rely on the position of global-luminance information, the probability of correct discrimination increases when the visible fragments contain a high number of optimal features. These results suggest that such optimal local information contributes to the successful reconstruction of naturalistic images even in challenging conditions.

Brain State-Dependent Modulation of Thalamic Visual Processing by Cortico-Thalamic Feedback.

The behavioral state of a mammal impacts how the brain responds to visual stimuli as early as in the dorsolateral geniculate nucleus of the thalamus (dLGN), the primary relay of visual information to the cortex. A clear example of this is the markedly stronger response of dLGN neurons to higher temporal frequencies of the visual stimulus in alert as compared with quiescent animals. The dLGN receives strong feedback from the visual cortex, yet whether this feedback contributes to these state-dependent responses to visual stimuli is poorly understood. Here, we show that in male and female mice, silencing cortico-thalamic feedback profoundly reduces state-dependent differences in the response of dLGN neurons to visual stimuli. This holds true for dLGN responses to both temporal and spatial features of the visual stimulus. These results reveal that the state-dependent shift of the response to visual stimuli in an early stage of visual processing depends on cortico-thalamic feedback.SIGNIFICANCE STATEMENT Brain state affects even the earliest stages of sensory processing. A clear example of this phenomenon is the change in thalamic responses to visual stimuli depending on whether the animal's brain is in an alert or quiescent state. Despite the radical impact that brain state has on sensory processing, the underlying circuits are still poorly understood. Here, we show that both the temporal and spatial response properties of thalamic neurons to visual stimuli depend on the state of the animal and, crucially, that this state-dependent shift relies on the feedback projection from visual cortex to thalamus.

Learned value modulates the access to visual awareness during continuous flash suppression

Monetary value enhances visual perception and attention and boosts activity in the primary visual cortex, however, it is still unclear whether monetary value can modulate the conscious access to rewarding stimuli. Here we investigate this issue by employing a breaking continuous flash suppression (b-CFS) paradigm. We measured suppression durations of sinusoidal gratings having orthogonal orientations under CFS in adult volunteers before and after a short session of Pavlovian associative learning in which each orientation was arbitrarily associated either with high or low monetary reward. We found that monetary value accelerated the access to visual awareness during CFS. Specifically, after the associative learning, suppression durations of the visual stimulus associated with high monetary value were shorter compared to the visual stimulus associated with low monetary value. Critically, the effect was replicated in a second experiment using a detection task for b-CFS that was orthogonal to the reward associative learning. These results indicate that monetary reward facilitates the access to awareness of visual stimuli associated with monetary value probably by boosting their representation at the early stages of visual processing in the brain.

Peculiarities of the Activity of the Brain Structures of People with Schizophrenia During the Categorization of Objects of Animate and Inanimate Nature

One of the features of the work of the brain of people suffering from schizophrenia is changes in the activity of their brain during visual categorization of animate and inanimate objects. The purpose of this study was to analyze the brain activity of people with schizophrenia and their visual categorization of objects with different semantic and physical characteristics. It was assumed that the patterns of brain activity in individuals with schizophrenia would differ from the group of healthy individuals both in the early and late stages of visual processing. Using the method of visual evoked potentials, we studied the features of brain activity in 25 people suffering from schizophrenia from 1 to 7 years old, when they categorized images of animate and inanimate nature, low and high spatial frequency. It was found that the amplitudes of P170 (N170) in the left and right posterior and central leads, as well as the amplitudes of P300 in the central lead in people with schizophrenia do not differ during categorization of animate and inanimate objects, which does not correspond to the data obtained earlier from the people without mental health abnormalities. The revealed result is important for a better understanding of the restructuring of the brain during visual perception of objects of different categories, which occurs during the development of schizophrenia.

Computational framework of the visual sensory system based on neuroscientific evidence of the ventral pathway

The visual system provides with relevant information to create an internal representation of the environment and with this information to make decisions. Visual information is primarily processed in the visual cortex, where neurons react to certain visual features. In computational neuroscience and artificial vision, the study and modeling of vision processes is a topic of great interest since the human visual system can process a great diversity of stimuli in a wide variety of conditions.In this work, we propose a computational framework inspired by the human visual system, that integrates models obtained from neuroscience to describe aspects of vision such as color processing and shape processing and thus contribute to formulating a general vision system and incorporating it into artificial entities in order to approximate human behavior.This system covers the early stages of visual sensory processing, in addition to intermediate processing stages, and considers connections with other cognitive functions within a cognitive architecture. In this proposal, the model is formally described, and the results obtained are analyzed qualitatively to later weigh the considerations for the expansion of this framework.

Layer-specific, retinotopically-diffuse modulation in human visual cortex in response to viewing emotionally expressive faces

Viewing faces that are perceived as emotionally expressive evokes enhanced neural responses in multiple brain regions, a phenomenon thought to depend critically on the amygdala. This emotion-related modulation is evident even in primary visual cortex (V1), providing a potential neural substrate by which emotionally salient stimuli can affect perception. How does emotional valence information, computed in the amygdala, reach V1? Here we use high-resolution functional MRI to investigate the layer profile and retinotopic distribution of neural activity specific to emotional facial expressions. Across three experiments, human participants viewed centrally presented face stimuli varying in emotional expression and performed a gender judgment task. We found that facial valence sensitivity was evident only in superficial cortical layers and was not restricted to the retinotopic location of the stimuli, consistent with diffuse feedback-like projections from the amygdala. Together, our results provide a feedback mechanism by which the amygdala directly modulates activity at the earliest stage of visual processing.

Attentional Enhancement of Tracked Stimuli in Early Visual Cortex Has Limited Capacity.

Keeping track of the location of multiple moving objects is one of the well documented functions of visual attention. However, the mechanism of attentional selection that supports such continuous tracking is unclear. In particular, it has been proposed that target selection in early visual cortex occurs in parallel, with tracking errors arising because of attentional limitations at later processing stages. Here, we examine whether, instead, total attentional capacity for enhancement of early visual processing of tracked targets is shared between all attended stimuli. If the magnitude of attentional facilitation of multiple tracked targets was a key limiting factor of tracking ability, then one should expect it to drop systematically with increasing set-size of tracked targets. Human observers (male and female) were instructed to track two, four, or six moving objects among a pool of identical distractors. Steady-state visual evoked potentials (SSVEPs) recorded during the tracking period revealed that the processing of tracked targets was consistently amplified compared with the processing of the distractors. The magnitude of this amplification decreased with increasing set size, and at lateral occipital electrodes it closely followed inverse proportionality to the number of tracked items, suggesting that limited attentional resources must be shared among the tracked stimuli. Accordingly, the magnitude of attentional facilitation predicted the behavioral outcome at the end of the trial. Together, these findings demonstrate that the limitations of multiple object tracking (MOT) across set-sizes stem from the limitations of top-down selective attention already at the early stages of visual processing.SIGNIFICANCE STATEMENT The ability to selectively attend to relevant features or objects is the key to flexibility of perception and action in the continuously changing environment. This ability is demonstrated in the multiple object tracking (MOT) task where observers monitor multiple independently moving objects at different locations in the visual field. The role of early attentional enhancement in tracking was previously acknowledged in the literature, however, the limitations on tracking were thought to arise during later stages of processing. Here, we demonstrate that the strength of attentional facilitation depends on the number of tracked objects and predicts successful tracking performance. Thus, it is the limitations of attentional enhancement at the early stages of visual processing that determine behavioral performance limits.

Neural coding: Looking up and down the visual thalamus

Integrating sensory and postural information is essential for perception and behavior. A new study shows that information about whether mice are looking up or down is combined with visual information in the primary visual thalamus, an early sensory stage of visual processing.

Utilization of passive visual perception task indetecting patients with major depressive disorder for active health.

Depression is a common emotional and mental disease. At present, doctors' diagnosis mainly depends on the existing evaluation scales and their accumulated experience, lack of objective electrophysiological quantitative evaluation indicators. This study explores the difference in event-related potential (ERP) between patients with depression and healthy controls under the stimulation of multi-dimensional tasks, extracts the characteristic data, and uses a t-test for statistical analysis to provide an objective evaluation index for the clinical diagnosis of depression. Ninety-nine patients in the major depression group (MDD) and thirty patients in the healthy control group (HC) were used to compare the responses to positive, negative, and neutral stimulation, the results showed that there were significant differences between the left and right occipital lobes and one frontal lobe, and the frontal lobe showed lateralization; There were significant differences between the depression group and the healthy control group under multi-dimensional stimulation (P<0.01), and the depression patients were significantly lower than the healthy patients. The results showed that emotional information was processed differently in patients with depression in the early stages of visual face processing. Therefore, EEG indices could be used as an objective index for the early detection of depression. Moreover, according to the instructions before the test, the subject is only asked to recognize visual pictures instead report emotional feelings. Stigma about psychiatric disorders could thus be reduced in this way. The explorations above facilitate designing more accurate and implicit active mental health techniques.

Spatial Attention Modulates Spike Count Correlations and Granger Causality in the Primary Visual Cortex.

The influence of spatial attention on neural interactions has been revealed even in early visual information processing stages. It resolves the process of competing for sensory information about objects perceived as targets and distractors. However, the attentional modulation of the interaction between pairs of neurons with non-overlapping receptive fields (RFs) is not well known. Here, we investigated the activity of anatomically distant neurons in two behaving monkeys’ primary visual cortex (V1), when they performed a spatial attention task detecting color change. We compared attentional modulation from the perspective of spike count correlations and Granger causality among simple and complex cells. An attention-related increase in spike count correlations and a decrease in Granger causality were found. The results showed that spatial attention significantly influenced only the interactions between rather than within simple and complex cells. Furthermore, we found that the attentional modulation of neuronal interactions changed with neuronal pairs’ preferred directions differences. Thus, we found that spatial attention increased the functional communications and competing connectivities when attending to the neurons’ RFs, which impacts the interactions only between simple and complex cells. Our findings enrich the model of simple and complex cells and further understand the way that attention influences the neurons’ activities.

Restriction of orientation variability and spatial frequency on the perception of average orientation

During the observation of a single object, orientation and spatial frequency are jointly coded in an early stage of visual processing, as is evident from studies on the aftereffects of specific combinations of both these features. However, they become independent in the decision-making stage because observers can identify one feature while ignoring the other. Does this separability expand into the perception of ensemble representations? This study investigated the effect of the spatial frequencies of Gabor patches on orientation averaging. In the experiment, the average orientations of all eight patches composed of either homogeneous (i.e., eight 3 cycles/degree or 0.8 cycles/degree patches) or heterogeneous (i.e., four 3 cycles/degree and four 0.8 cycles/degree patches) spatial frequencies were stably estimated if the orientation varied within the range of ±7.7° around the true mean. However, when the range was extended to ±14°, we found that the averaging performance was better in the homogeneous lower spatial frequency than in the homogeneous higher spatial frequency and heterogeneous spatial frequency conditions. These results suggest that an ensemble perception of orientation is modulated by spatial frequency components.

Inter-trial theta phase consistency during face processing in infants is associated with later emerging autism.

A growing body of research suggests that consistency in cortical activity may be a promising neurophysiological marker of autism spectrum disorder (ASD). In the current study we examined inter-trial coherence, a measure of phase consistency across trials, in the theta range (t-ITC: 3-6Hz), as theta has been implicated in the processing of social and emotional stimuli in infants and adults. The sample included infants who had an older sibling with a confirmed ASD diagnosis and typically developing (TD) infants with no family history of ASD. The data were collected as part of the British Autism Study of Infant Siblings (BASIS) study. Infants between 6 and 10months of age (Mage = 7.34, SDage = 1.21) performed a visual face processing task that included faces and scrambled, "face noise", stimuli. Follow-up assessments in higher likelihood infants were completed at 24 and again at 36months to determine diagnostic outcomes. Analysis focused on posterior t-ITC during early (0-200ms) and late (200-500ms) visual processing stages commonly investigated in infant studies. t-ITC over posterior scalp regions during late stage face processing was significantly higher in TD and higher likelihood infants without ASD (HRA-), indicating reduced consistency in theta-band responses in higher likelihood infants who eventually receive a diagnosis of ASD (HRA+). These findings indicate that the temporal dynamics of theta during face processing relate to ASD outcomes. Reduced consistency of oscillatory dynamics at basic levels of infant sensory processing could have downstream effects on learning and social communication. LAY SUMMARY: We examined the consistency in brain responses to faces in infants at lower or higher familial likelihood for autism. Our results show that the consistency of EEG responses was lower during face processing in higher likelihood infants who eventually received a diagnosis of autism. These findings highlight that reduced consistency in brain activity during face processing in the first year of life is related to emerging autism.

Enhanced early ERP responses to looming angry faces

Although the brain is known to process threatening emotional stimuli and looming motion rapidly, little is known about how the emotion and motion interact. To address this question, two experiments were carried out which presented angry and neutral emotional faces on a depth-cued background that induced the perception of distance, or a non-cued background. Furthermore, faces either expanded or contracted in size such that they appeared to approach or recede from the viewer. EEG/ERP measures were used to identify the time course of brain activity for these looming and receding, angry and neutral emotional faces. The results of both experiments revealed that the P1 was enhanced by looming angry faces on the depth-cued background, compared to neutral approaching faces, as well as all receding faces, indicating an early interaction of emotion and motion within 100 ms of presentation. Angry expressions were also found to enhance the N170 regardless of movement. These findings suggest that processing of threat and looming motion interact at the very early stages of visual processing. Furthermore, as the modulating effect of looming motion on angry expressions only arose on the depth-cued background, the findings highlight the importance of approaching movements rather than sole increases in the retinal size of the stimuli.

Level of Orthographic Knowledge Helps to Reveal Automatic Predictions in Visual Word Processing.

The brain generates predictions about visual word forms to support efficient reading. The “interactive account” suggests that the predictions in visual word processing can be strategic or automatic (non-strategic). Strategic predictions are frequently demonstrated in studies that manipulated task demands, however, few studies have investigated automatic predictions. Orthographic knowledge varies greatly among individuals and it offers a unique opportunity in revealing automatic predictions. The present study grouped the participants by level of orthographic knowledge and recorded EEGs in a non-linguistic color matching task. The visual word-selective N170 response was much stronger to pseudo than to real characters in participants with low orthographic knowledge, but not in those with high orthographic knowledge. Previous work on predictive coding has demonstrated that N170 is a good index for prediction errors, i.e., the mismatches between predictions and visual inputs. The present findings provide unambiguous evidence that automatic predictions modulate the early stage of visual word processing.

The interplay between audiovisual temporal synchrony and semantic congruency in the cross-modal boost of the visual target discrimination during the attentional blink.

The visual attentional blink can be substantially reduced by delivering a task‐irrelevant sound synchronously with the second visual target (T2), and this effect is further modulated by the semantic congruency between the sound and T2. However, whether the cross‐modal benefit originates from audiovisual interactions or sound‐induced alertness remains controversial, and whether the semantic congruency effect is contingent on audiovisual temporal synchrony needs further investigation. The current study investigated these questions by recording event‐related potentials (ERPs) in a visual attentional blink task wherein a sound could either synchronize with T2, precede T2 by 200 ms, be delayed by 100 ms, or be absent, and could be either semantically congruent or incongruent with T2 when delivered. The behavioral data showed that both the cross‐modal boost of T2 discrimination and the further semantic modulation were the largest when the sound synchronized with T2. In parallel, the ERP data yielded that both the early occipital cross‐modal P195 component (192–228 ms after T2 onset) and late parietal cross‐modal N440 component (424–448 ms) were prominent only when the sound synchronized with T2, with the former being elicited solely when the sound was further semantically congruent whereas the latter occurring only when that sound was incongruent. These findings demonstrate not only that the cross‐modal boost of T2 discrimination during the attentional blink stems from early audiovisual interactions and the semantic congruency effect depends on audiovisual temporal synchrony, but also that the semantic modulation can unfold at the early stage of visual discrimination processing.

Contributed Session II: Evidence for a dipper effect in the perception of global form: findings from psychophysics and EEG.

Decades of psychophysical experiments have shown that the perception of low luminance contrast violates Webers' law: contrast discrimination is best at low (but non-zero) pedestal intensities. This "dipper effect" is thought to be the product of a sigmoidal neuronal transducer function in the early stages of visual processing which is expansive at low contrasts. Here, we ask 1) whether the transducer that governs the perception of global form is subject to a similar nonlinearity and 2) if sensitivity is biased towards a certain axis of pattern alignment (translational, radial, concentric). Using a combination of psychophysics and steady-state VEPs, we examined observers' sensitivity to global form by manipulating the dipole orientation coherence of Glass patterns. Our psychophysical data indicate a Glass pattern "dipper effect" that is strongest for concentric patterns, while our EEG results show some evidence of nonlinear transduction but do not have the form predicted by psychophysical thresholds. Our findings indicate that, like low-level contrast, mid-level form discrimination is subject to mild facilitation (relative to detection) at low global form coherences, and suggest that this could be driven by neurons with nonlinear transducer functions.

Binocularly Asymmetric Crowding in Glaucoma and a Lack of Binocular Summation in Crowding.

PurposeGlaucoma is associated with progressive loss of retinal ganglion cells. Here we investigated the impact of glaucomatous damage on monocular and binocular crowding in parafoveal vision. We also examined the binocular summation of crowding to see if crowding is alleviated under binocular viewing.MethodsThe study design included 40 individuals with glaucoma and 24 age-similar normal cohorts. For each subject, the magnitude of crowding was determined by the extent of crowding zone. Crowding zone measurements were made binocularly in parafoveal vision (i.e., at 2° and 4° retinal eccentricities) visual field. For a subgroup of glaucoma subjects (n = 17), crowding zone was also measured monocularly for each eye.ResultsOur results showed that, compared with normal cohorts, individuals with glaucoma exhibited significantly larger crowding—enlargement of crowding zone (an increase by 21%; P < 0.01). Moreover, we also observed a lack of binocular summation (i.e., a binocular ratio of 1): binocular crowding was determined by the better eye. Hence, our results did not provide evidence supporting binocular summation of crowding in glaucomatous vision.ConclusionsOur findings show that crowding is exacerbated in parafoveal vision in glaucoma and binocularly asymmetric glaucoma seems to induce binocularly asymmetric crowding. Furthermore, the lack of binocular summation for crowding observed in glaucomatous vision combined with the lack of binocular summation reported in a previous study on normal healthy vision support the view that crowding may start in the early stages of visual processing, at least before the process of binocular integration takes place.

Impact of axonal delay on structure development in a multi-layered network

The mechanisms underlying how activity in the visual pathway gives rise through neural plasticity to many features observed experimentally in early stages of visual processing was provided by Linsker in a seminal, three-paper series. Owing to the complexity of multi-layer models, an implicit assumption in Linsker’s and subsequent papers has been that propagation delay is homogeneous, playing little functional role in neural behavior. In this paper, we relax this assumption to examine the impact of distance-dependent axonal propagation delay on neural learning. We show that propagation delay induces low-pass filtering by dispersing arrival times of spikes from presynaptic neurons, providing a natural correlation cancellation mechanism for distal connections. The cut-off frequency decreases as radial propagation delay within a layer increases relative to propagation delay between layers, introducing an upper limit on temporal resolution. Given that the postsynaptic potential acts as a low-pass filter, we show that the effective time constant of each should enable processing of similar scales of temporal information. This has implications for the visual system, in which receptive field size and, thus, propagation delay, increases with eccentricity. Furthermore, network response is frequency dependent since higher frequencies require increased input amplitude to compensate for attenuation. This concords with frequency-dependent contrast sensitivity, which changes with eccentricity and receptive field size. We further show that the proportion of inhibition relative to excitation is larger where radial propagation delay is long relative to inter-laminar delay, and that delay reduces the range in on-center size, providing stability to variations in homeostatic parameters.

The time course of processing handwritten words: An ERP investigation

Behavioral studies have shown that the legibility of handwritten script hinders visual word recognition. Furthermore, when compared with printed words, lexical effects (e.g., word-frequency effect) are magnified for less intelligible (difficult) handwriting (Barnhart and Goldinger, 2010; Perea et al., 2016). This boost has been interpreted in terms of greater influence of top-down mechanisms during visual word recognition. In the present experiment, we registered the participants' ERPs to uncover top-down processing effects on early perceptual encoding. Participants' behavioral and EEG responses were recorded to high- and low-frequency words that varied in script's legibility (printed, easy handwritten, difficult handwritten) in a lexical decision experiment. Behavioral results replicated previous findings: word-frequency effects were larger in difficult handwriting than in easy handwritten or printed conditions. Critically, the ERP data showed an early effect of word-frequency in the N170 that was restricted to the difficult-to-read handwritten condition. These results are interpreted in terms of increased attentional deployment when the bottom-up signal is weak (difficult handwritten stimuli). This attentional boost would enhance top-down effects (e.g., lexical effects) in the early stages of visual word processing.