Early Visual Information Processing Research Articles

The Gaze of Schizophrenia Patients Captured by Bottom-up Saliency

Schizophrenia (SCHZ) notably impacts various human perceptual modalities, including vision. Prior research has identified marked abnormalities in perceptual organization in SCHZ, predominantly attributed to deficits in bottom-up processing. Our study introduces a novel paradigm to differentiate the roles of top-down and bottom-up processes in visual perception in SCHZ. We analysed eye-tracking fixation ground truth maps from 28 SCHZ patients and 25 healthy controls (HC), comparing these with two mathematical models of visual saliency: one bottom-up, based on the physical attributes of images, and the other top-down, incorporating machine learning. While the bottom-up (GBVS) model revealed no significant overall differences between groups (beta = 0.01, p = 0.281, with a marginal increase in SCHZ patients), it did show enhanced performance by SCHZ patients with highly salient images. Conversely, the top-down (EML-Net) model indicated no general group difference (beta = −0.03, p = 0.206, lower in SCHZ patients) but highlighted significantly reduced performance in SCHZ patients for images depicting social interactions (beta = −0.06, p < 0.001). Over time, the disparity between the groups diminished for both models. The previously reported bottom-up bias in SCHZ patients was apparent only during the initial stages of visual exploration and corresponded with progressively shorter fixation durations in this group. Our research proposes an innovative approach to understanding early visual information processing in SCHZ patients, shedding light on the interplay between bottom-up perception and top-down cognition.

Local field potentials, spiking activity, and receptive fields in human visual cortex.

The concept of receptive field (RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals, while those in humans remain nearly unexplored. Here, we measured neuronal RFs with intracranial local field potentials (LFPs) and spiking activity in human visual cortex (V1/V2/V3). We recorded LFPs via macro-contacts and discovered that RF sizes estimated from low-frequency activity (LFA, 0.5-30 Hz) were larger than those estimated from low-gamma activity (LGA, 30-60 Hz) and high-gamma activity (HGA, 60-150 Hz). We then took a rare opportunity to record LFPs and spiking activity via microwires in V1 simultaneously. We found that RF sizes and temporal profiles measured from LGA and HGA closely matched those from spiking activity. In sum, this study reveals that spiking activity of neurons in human visual cortex could be well approximated by LGA and HGA in RF estimation and temporal profile measurement, implying the pivotal functions of LGA and HGA in early visual information processing.

Spatial Attention Modulates Spike Count Correlations and Granger Causality in the Primary Visual Cortex.

The influence of spatial attention on neural interactions has been revealed even in early visual information processing stages. It resolves the process of competing for sensory information about objects perceived as targets and distractors. However, the attentional modulation of the interaction between pairs of neurons with non-overlapping receptive fields (RFs) is not well known. Here, we investigated the activity of anatomically distant neurons in two behaving monkeys’ primary visual cortex (V1), when they performed a spatial attention task detecting color change. We compared attentional modulation from the perspective of spike count correlations and Granger causality among simple and complex cells. An attention-related increase in spike count correlations and a decrease in Granger causality were found. The results showed that spatial attention significantly influenced only the interactions between rather than within simple and complex cells. Furthermore, we found that the attentional modulation of neuronal interactions changed with neuronal pairs’ preferred directions differences. Thus, we found that spatial attention increased the functional communications and competing connectivities when attending to the neurons’ RFs, which impacts the interactions only between simple and complex cells. Our findings enrich the model of simple and complex cells and further understand the way that attention influences the neurons’ activities.

Visible-Infrared Image Fusion Based on Early Visual Information Processing Mechanisms

In this work, we simulate the early visual information processing mechanisms in biological visual system (BVS) to solve the visible-infrared image fusion (VIIF) task. Concretely, both infrared and visible images are first processed with a dynamic receptive field (DRF), which is imitated by a Difference of Gaussian (DoG) function whose parameters are adjusted according to local image statistics (e.g., local edge responses). The DRF processing produces two components for each source image (e.g., the visible image or the infrared image) that respectively represent the results of On-center based DRF and Off-center based DRF. Then, the results of On-center based DRF for visible image are fused with the results of Off-center based DRF for infrared image and the results of On-center based DRF for infrared image are fused with the results of Off-center based DRF for visible image, according to the mechanisms of cortex-based center-surround fusion. Algorithmically, this step fuses the visible and infrared images processed by DRF with On-center and Off-center according to the different levels of local homogeneity. Moreover, a feedward signal acting as the sub-cortical flow is also introduced to adjust the results during the fusion. The final output image is simply obtained by the summation of two components after the cortex and sub-cortex based fusion. Qualitative and quantitative tests on four datasets demonstrate that our approach can fuse the infrared and visible images effectively with the good background details and discernible salient areas. We emphasize the importance of corticothalamic feedback, cortex and sub-cortex-based fusion, and the interactions between On-center pathway and Off-center pathway that are ubiquitous in BVS for producing the high-quality visual signals.

Dysfunction of Magnocellular/dorsal Processing Stream in Schizophrenia

Background:Patients with schizophrenia show not only cognitive, but also perceptual deficits. Perceptual deficits may affect different sensory modalities. Among these, the impairment of visual information processing is of particular relevance as demonstrated by the high incidence of visual disturbances. In recent years, the study of neurophysiological mechanisms that underlie visuo-perceptual, -spatial and -motor disorders in schizophrenia has increasingly attracted the interest of researchers.Objective:The study aims to review the existent literature on magnocellular/dorsal (occipitoparietal) visual processing stream impairment in schizophrenia. The impairment of relatively early stages of visual information processing was examined using experimental paradigms such as backward masking, contrast sensitivity, contour detection, and perceptual closure. The deficits of late processing stages were detected by examining visuo-spatial and -motor abilities.Results:Neurophysiological and behavioral studies support the existence of deficits in the processing of visual information along the magnocellular/dorsal pathway. These deficits appear to affect both early and late stages of visual information processing.Conclusion:The existence of disturbances in the early processing of visual information along the magnocellular/dorsal pathway is strongly supported by neurophysiological and behavioral observations. Early magnocellular dysfunction may provide a substrate for late dorsal processing impairment as well as higher-level cognition deficits.

Investigation of Emotional Expression Processing Following Cognitive Behavioural Therapy for Patients with Schizophrenia: An Event-Related Potentials Study

Growing evidence supports the use of cognitive behavioural therapy (CBT) for psychosis, including CBT for voices (CBTv), which targets auditory verbal hallucinations (AVH). The present study observed the effects of CBTv on electrophysiological measures of facial expression processing in patients with schizophrenia with AVH. Twenty-five patients with schizophrenia were randomly assigned to a treatment group (TG; n = 14) or a treatment as usual (TAU) group (n = 11). The TG received group CBTv for five-six months in addition to their TAU. The matched waitlist group received TAU for the five-six months. The CBTv treatment showed shorter P100 latency in response to facial expressions following treatment compared with baseline, but not the TAU group. Amount of negative content of voices and “omnipotence” of voices were modified following CBTv treatment, but not following TAU. This study provides evidence that CBTv decreases early visual information processing time as indexed by the P100 latency.

AB069. Defining the perceptual profiles of children and adolescents with a neurodevelopmental condition

Background: Perceptual profiles, or the performance on visual-perceptual tasks that reflect early visual information processing, have been used to suggest condition-specific visuo-perceptual abilities across neurodevelopmental conditions (NDCs). The complexity-specific hypothesis (Bertone et al., 2010) was based on perceptual profiles defined by a selective decrease in sensitivity to more complex, texture-defined information in adults with autism and fragile-x syndrome, suggesting the atypical development of neural networks underlying early perception in NDCs. The aim of this study was to evaluate whether the complexity-specific hypothesis is applicable to children and adolescents with different NDCs by defining and comparing their perceptual profiles.

Attentional Modulation of Neuronal Activity Depends on Neuronal Feature Selectivity

Attention exerts a powerful influence on visual perception. The impact of attention on neuronal activity manifests at early visual information processing stages and progressively increases throughout the visual cortical hierarchy. However, the neuronal mechanisms of attention are unresolved. In particular, the rules governing attentional modulation of individual neurons, whether they are facilitated by or suppressed by attention, are not known. To obtain a more granular or neuron- and circuit-level understanding of the mechanisms of attention and to directly test the feature similarity gain model in V1, we compared attentional modulation with neuronal feature selectivity across a large population of V1 neurons in alert and behaving macaque monkeys trained on an attention-demanding contrast-change detection task. We utilized emerging multi-electrode array technology to record simultaneously from V1 neurons spanning all six cortical layers so that we could characterize the laminar position and physiological response properties of diverse V1 neuronal populations. We found significant relationships between attentional modulation and neuronal position within the cortical hierarchy, neuronal physiology, and neuronal feature selectivity. Our results support the feature similarity gain model and further suggest that attentional modulation depends critically upon the match between neuronal feature selectivity and the features required for the task.

Gunslinger Effect and Müller-Lyer Illusion: Examining Early Visual Information Processing for Late Limb-Target Control.

The multiple process model contends that there are two forms of online control for manual aiming: impulse regulation and limb-target control. This study examined the impact of visual information processing for limb-target control. We amalgamated the Gunslinger protocol (i.e., faster movements following a reaction to an external trigger compared with the spontaneous initiation of movement) and Müller-Lyer target configurations into the same aiming protocol. The results showed the Gunslinger effect was isolated at the early portions of the movement (peak acceleration and peak velocity). Reacted aims reached a longer displacement at peak deceleration, but no differences for movement termination. The target configurations manifested terminal biases consistent with the illusion. We suggest the visual information processing demands imposed by reacted aims can be adapted by integrating early feedforward information for limb-target control.

Post-Decision Wagering Affects Metacognitive Awareness of Emotional Stimuli: An Event Related Potential Study.

The present research investigated metacognitive awareness of emotional stimuli and its psychophysiological correlates. We used a backward masking task presenting participants with fearful or neutral faces. We asked participants for face discrimination and then probed their metacognitive awareness with confidence rating (CR) and post-decision wagering (PDW) scales. We also analysed psychophysiological correlates of awareness with event-related potential (ERP) components: P1, N170, early posterior negativity (EPN), and P3. We have not observed any differences between PDW and CR conditions in the emotion identification task. However, the "aware" ratings were associated with increased accuracy performance. This effect was more pronounced in PDW, especially for fearful faces, suggesting that emotional stimuli awareness may be enhanced by monetary incentives. EEG analysis showed larger N170, EPN and P3 amplitudes in aware compared to unaware trials. It also appeared that both EPN and P3 ERP components were more pronounced in the PDW condition, especially when emotional faces were presented. Taken together, our ERP findings suggest that metacognitive awareness of emotional stimuli depends on the effectiveness of both early and late visual information processing. Our study also indicates that awareness of emotional stimuli can be enhanced by the motivation induced by wagering.

Fusiform Gyrus Dysfunction is Associated with Perceptual Processing Efficiency to Emotional Faces in Adolescent Depression: A Model-Based Approach.

While the extant literature has focused on major depressive disorder (MDD) as being characterized by abnormalities in processing affective stimuli (e.g., facial expressions), little is known regarding which specific aspects of cognition influence the evaluation of affective stimuli, and what are the underlying neural correlates. To investigate these issues, we assessed 26 adolescents diagnosed with MDD and 37 well-matched healthy controls (HCL) who completed an emotion identification task of dynamically morphing faces during functional magnetic resonance imaging (fMRI). We analyzed the behavioral data using a sequential sampling model of response time (RT) commonly used to elucidate aspects of cognition in binary perceptual decision making tasks: the Linear Ballistic Accumulator (LBA) model. Using a hierarchical Bayesian estimation method, we obtained group-level and individual-level estimates of LBA parameters on the facial emotion identification task. While the MDD and HCL groups did not differ in mean RT, accuracy, or group-level estimates of perceptual processing efficiency (i.e., drift rate parameter of the LBA), the MDD group showed significantly reduced responses in left fusiform gyrus compared to the HCL group during the facial emotion identification task. Furthermore, within the MDD group, fMRI signal in the left fusiform gyrus during affective face processing was significantly associated with greater individual-level estimates of perceptual processing efficiency. Our results therefore suggest that affective processing biases in adolescents with MDD are characterized by greater perceptual processing efficiency of affective visual information in sensory brain regions responsible for the early processing of visual information. The theoretical, methodological, and clinical implications of our results are discussed.

Prophylactic lithium treatment and cognitive performance in patients with a long history of bipolar illness: no simple answers in complex disease-treatment interplay.

Cognitive impairment in patients with bipolar disorder (BD) is not restricted to symptomatic phases. It is also present in euthymia. There is evidence of differences in the brain’s structure between bipolar patients and healthy individuals, as well as changes over time in patients. Lithium constitutes the gold standard in long-term prophylactic treatment. Appropriate therapy that prevents new episodes improves the disease’s course and reduces the frequency of harmful outcomes. Interestingly, preclinical data suggest that lithium has a (additional) neuroprotective effect. There is limited data on its related effects in humans and even less on its long-term application. In this multi-center cross-sectional study from the International Group for the Study of Lithium-treated Patients (IGSLi), we compared three groups: bipolar patients without long-term lithium treatment (non-Li group; <3 months cumulative lithium exposure, ≥24 months ago), bipolar patients with long-term lithium treatment (Li group, ongoing treatment ≥24 months), and healthy subjects (controls). Strict inclusion and exclusion criteria were defined; the inclusion criteria for patients were diagnosis of BD types I or II, duration of illness ≥10 years, ≥5 episodes in patient’s history and a euthymic mood state. Neurocognitive functioning was assessed using the Wechsler Adult Intelligence Scale-Revised (WAIS-R), the California Verbal Learning Test (CVLT), and a visual backward masking (VBM) task. A total of 142 subjects were included, 31 in the non-Li and 58 in the Li group, as well as 53 healthy controls. Treated patients with long-standing BD and controls did not differ significantly in overall cognitive functioning and verbal learning, recall, and recognition; regardless of whether lithium had been part of the treatment. Patients, however, demonstrated poorer early visual information processing than healthy controls, with the lithium-treated patients performing worse than those without. Our data suggest that bipolar patients with a long illness history and effective prophylactic treatment do not reveal significantly impaired general cognitive functioning or verbal learning and memory. However, they are worse at processing early visual information. Accompanying volumetric and spectroscopic data suggest cell loss in patients not treated with lithium that may be counterbalanced by long-term lithium treatment.

A visual motion detecting module for dragonfly-controlled robots.

When imitating biological sensors, we have not completely understood the early processing of the input to reproduce artificially. Building hybrid systems with both artificial and real biological components is a promising solution. For example, when a dragonfly is used as a living sensor, the early processing of visual information is performed fully in the brain of the dragonfly. The only significant remaining tasks are recording and processing neural signals in software and/or hardware. Based on existing works which focused on recording neural signals, this paper proposes a software application of neural information processing to design a visual processing module for dragonfly hybrid bio-robots. After a neural signal is recorded in real-time, the action potentials can be detected and matched with predefined templates to detect when and which descending neurons fire. The output of the proposed system will be used to control other parts of the robot platform.

Early event-related potentials indicate context-specific target processing for eye and hand motor systems

Concurrent eye and hand movements toward a common visual target require different motor programs based on identical visual input. We used event-related brain potentials (ERP) to determine if and when the processing of the visual target differs for the two motor systems.The N2, an index for target evaluation, was more negative for the target of a hand than of an eye movement in two experiments. A possible interpretation for this finding is different visual target processing. Targets for hand movements require a different weighting of visual information, for example concerning features such as surface structure which are important for hand but not for eye movements.In experiment 2, the early C1-component, which had an average maximum at 67ms following target onset, was significantly more negative when subjects pointed at the stimuli. Traditionally, the C1 has been regarded as a sensory component, but recent studies have linked it to higher order processing, such as attention and expectations. Thus, the present data indicate that target processing for eye or hand movements is already context-specific during early visual information processing. We suggest that differences in a target's relevance for upcoming movements modify target processing as well as sensory expectations.

Association of the Nicotinic Receptor α7 Subunit Gene (CHRNA7) with Schizophrenia and Visual Backward Masking.

The nicotinic system is involved in the pathophysiology of schizophrenia. However, very little is known about its genetic basis and how it relates to clinical symptoms and potentially pharmacological intervention. Here, we investigated five single nucleotide polymorphisms (SNPs) [rs3826029] [rs2337506] [rs982574] [rs904952] [rs2337980] of the cholinergic nicotinic receptor gene, alpha 7 subunit (CHRNA7) and their association to schizophrenia. We found an association with rs904952 (p = 0.009) in a German sample of 224 schizophrenic patients and 224 healthy control subjects. The same trend was shown in an independent Georgian sample of 50 schizophrenic patients, 57 first order unaffected relatives, and 51 healthy controls. In addition, visual backward masking (VBM), a sensitive test for early visual information processing, was assessed in the Georgian sample. In line with prior studies, VBM performance deficits were much more pronounced in schizophrenic patients and their unaffected relatives compared to healthy controls (schizophrenic patients: 156 ms; unaffected relatives: 60 ms; healthy controls: 33 ms). VBM was strongly correlated with SNP rs904952 (H[2] = 7.3, p = 0.026). Our results further support the notion that changes in the nicotinic system are involved in schizophrenia and open the avenue for pharmacological intervention.

Influence of Highly Distinctive Structural Properties on the Excitability of Pyramidal Neurons in Monkey Visual and Prefrontal Cortices

Whole-cell patch-clamp recordings and high-resolution 3D morphometric analyses of layer 3 pyramidal neurons in in vitro slices of monkey primary visual cortex (V1) and dorsolateral granular prefrontal cortex (dlPFC) revealed that neurons in these two brain areas possess highly distinctive structural and functional properties. Area V1 pyramidal neurons are much smaller than dlPFC neurons, with significantly less extensive dendritic arbors and far fewer dendritic spines. Relative to dlPFC neurons, V1 neurons have a significantly higher input resistance, depolarized resting membrane potential, and higher action potential (AP) firing rates. Most V1 neurons exhibit both phasic and regular-spiking tonic AP firing patterns, while dlPFC neurons exhibit only tonic firing. Spontaneous postsynaptic currents are lower in amplitude and have faster kinetics in V1 than in dlPFC neurons, but are no different in frequency. Three-dimensional reconstructions of V1 and dlPFC neurons were incorporated into computational models containing Hodgkin-Huxley and AMPA receptor and GABA(A) receptor gated channels. Morphology alone largely accounted for observed passive physiological properties, but led to AP firing rates that differed more than observed empirically, and to synaptic responses that opposed empirical results. Accordingly, modeling predicts that active channel conductances differ between V1 and dlPFC neurons. The unique features of V1 and dlPFC neurons are likely fundamental determinants of area-specific network behavior. The compact electrotonic arbor and increased excitability of V1 neurons support the rapid signal integration required for early processing of visual information. The greater connectivity and dendritic complexity of dlPFC neurons likely support higher level cognitive functions including working memory and planning.

Core dysfunction in schizophrenia: electrophysiology trait biomarkers

Core symptoms of schizophrenia, particularly in the cognitive domain are hypothesized to be due to an abnormality in neural connectivity. Biomarkers of connectivity may therefore be a promising tool in exploring the aetiology of schizophrenia. We used electrophysiological methods to demonstrate abnormal visual information processing during in patients performing a simple cognitive task. Electrophysiological recordings were acquired from 20 chronically ill, medicated patients diagnosed with either schizophrenia or schizo-affective disorder and 20 healthy volunteers while they conducted a working memory (WM) task. The patient group had significantly lower accuracy on the WM task and a trend for slower responses. An early visual evoked response potential was reduced in patients. Analysis of the electroencephalographic oscillations showed a decreased phase-locking factor (in the theta, beta and gamma bands) and signal power (theta frequency band). The beta and gamma oscillatory abnormalities were confined to two sets of correlated fronto and occipital electrodes. The findings of event-related potential and oscillatory abnormalities in patients with schizophrenia confirm the sensitivity of early visual information processing measurements for identification of schizophrenia phenotype. The fronto-occipital distribution of the oscillatory abnormalities replicates our findings from a schizotypal sample and implicates a possible top-down dysfunction as a vulnerability trait.

Estimating size fraction categories of coal particles on conveyor belts using image texture modeling methods

Motivation: Physical properties of coal such as particle size distribution have a large influence on the stability and operational behavior of fluidized bed reactors and metallurgical furnaces. In particular, the presence of large amounts of “fine” particles invariably has a drastic effect on plant performance as a result of impaired gas permeability characteristics of the coal or ore burden. Therefore, monitoring and control of particle size distribution profiles of such aggregate material on reactor feed streams, such as moving conveyor belts, is critical for predictable operation of these processes. Traditionally, the method of sieve analysis using stock or belt cut samples has been widely used in industry. Unfortunately, the reliability and usefulness of belt cut techniques are constrained by frequency of sampling as well as laboratory analysis turnaround times. For real-time monitoring and control purposes, automated sampling and analysis methods are more desirable. Methods: In this study, the problem of estimating the particle size distribution profile of material on a moving conveyor belt is formulated within a texture classification framework, which has its basis in machine vision and incorporates elements from statistical pattern recognition. Using exemplar images of coal particles taken on a process stream, a set of local features that compactly describes the textural properties of each image are expressed in terms of localized nonlinear features called textons. Representation of image information using textons is primarily motivated by insights from neuroscience research on the optimality of linear oriented basis functions as models of perception in early processing of visual information in the cortex regions of the human brain. Using these representations for different textures, nearest neighbor and support vector machine classification models are subsequently used to classify test images. Results: Using a comprehensive evaluation, it is shown that the use of texton representation obtained from decomposing images with linear oriented basis functions can be sufficiently discriminative compared to the use of the widely used second-order statistical features or features from other baseline models. In particular, model performance obtained with appropriately tuned filters suggest the importance of including shape and spatial structure information in an image representation for texture classification of coal particles. Furthermore, using nonlinear support vector machines rather than nearest neighbor classifiers significantly improved classification performance. A texture classification approach to particle size profile estimation has potential applications in the online monitoring of the proportion of “fines” in coal material on moving conveyor belts.

Threat and trait anxiety affect stability of gaze fixation

Threat accelerates early visual information processing, as shown by shorter P100 latencies of pattern Visual Evoked Potentials in subjects with low trait anxiety, but the opposite is true for high anxious subjects. We sought to determine if, and how, threat and trait anxiety interact to affect stability of gaze fixation. We used video oculography to record gaze position in the presence and in the absence of a fixational stimulus, in a safe and a verbal threat condition in subjects characterised for their trait anxiety. Trait anxiety significantly predicted fixational instability in the threat condition. An extreme tertile analysis revealed that fixation was less stable in the high anxiety group, especially under threat or in the absence of a stimulus. The effects of anxiety extend to perceptual and sensorimotor processes. These results have implications for the understanding of individual differences in occulomotor planning and visually guided behavior.

Early information processing among infants with and without spina bifida

This study focuses on the development of early visual information processing among infants with spina bifida (SB) compared to typically developing infants using the habituation–dishabituation paradigm. Analyses were conducted in two stages. First infants were evaluated to determine if 18-month old infants (SB = 47; Control = 40) differed in their ability to shift attention and habituate to two female faces, as well as their responses to composite and novel stimuli. Second, relations between these variables and infant motor and mental functioning were evaluated. The results of the study indicated that difficulties with visual attention skills can be detected as early as 18 months of age among infants with SB. Infants with SB differed significantly from controls on attention getting. Although there were no differences found on habituation and composite tasks, infants with SB differed significantly from controls on their ability to dishabituate. Implications are discussed.