Visual Information Processing Mechanisms Research Articles

Bidirectional Feature Aggregation Network for Stereo Image Quality Assessment Considering Parallax Attention-Based Binocular Fusion

Inspired by the two-path visual information processing mechanism (i.e., a bottom-up path and a top-down path), we propose a bidirectional binocular feature aggregation based stereo image quality assessment (SIQA) network, which considers a two-path visual mechanism and realizes the binocular fusion based on parallax information. To better aggregate binocular features from different levels, a two-path feature aggregation structure, which simulates the bottom-up and top-down mechanism in human visual system (HVS), is proposed. It not only realizes the supplement of low-level detail information to high-level semantic in the bottom-up path, but also realizes the supplement of high-level semantic information to low-level detail in the top-down path. Simultaneously, because feature misalignment exists in binocular features of adjacent levels, a feature alignment module (FAM) based on deformable convolution is designed to integrate the binocular fusion features of adjacent levels. In addition, considering the importance role of parallax in guiding binocular fusion, a binocular fusion module (BFM) based on parallax attention mechanism, which is different with existing binocular fusion methods, is explicitly proposed to achieve the binocular fusion between the left and right view features. Extensive experiments are conducted on LIVE I, LIVE II, WIVC I and WIVC II databases to demonstrate the effectiveness of the proposed method.

IdeNet: Making Neural Network Identify Camouflaged Objects Like Creatures.

Camouflaged objects often blend in with their surroundings, making the perception of a camouflaged object a more complex procedure. However, most neural-network-based methods that simulate the visual information processing pathway of creatures only roughly define the general process, which deficiently reproduces the process of identifying camouflaged objects. How to make modeled neural networks perceive camouflaged objects as effectively as creatures is a significant topic that deserves further consideration. After meticulous analysis of biological visual information processing, we propose an end-to-end prudent and comprehensive neural network, termed IdeNet, to model the critical information processing. Specifically, IdeNet divides the entire perception process into five stages: information collection, information augmentation, information filtering, information localization, and information correction and object identification. In addition, we design tailored visual information processing mechanisms for each stage, including the information augmentation module (IAM), the information filtering module (IFM), the information localization module (ILM), and the information correction module (ICM), to model the critical visual information processing and establish the inextricable association of biological behavior and visual information processing. The extensive experiments show that IdeNet outperforms state-of-the-art methods in all benchmarks, demonstrating the effectiveness of the five-stage partitioning of visual information processing pathway and the tailored visual information processing mechanisms for camouflaged object detection. Our code is publicly available at: https://github.com/whyandbecause/IdeNet.

Driver's Visual Attention Characteristics and Their Emotional Influencing Mechanism under Different Cognitive Tasks.

The visual attention system is the gateway to the human information processing system, and emotion is an important part of the human perceptual system. In this paper, the driver’s visual attention characteristics and the influences of typical driving emotions on those were explored through analyzing driver’s fixation time and identification accuracy to different visual cognitive tasks during driving. The results showed that: the increasing complexity of the cognitive object led to the improvement of visual identification speed. The memory and recall process increased drivers’ fixation time to cognitive objects, and the recall accuracy decreased with the increase in time interval. The increase in the number of cognitive objects resulted in the driver improving the visual identification speed for the cognitive object at the end of the sequence consciously. The results also showed that: the visual cognitive efficiency was improved in the emotional states of anger and contempt, and was decreased in the emotional states of surprise, fear, anxiety, helplessness and pleasure, and the emotional state of relief had no significant effect on the visual cognitive efficiency. The findings reveal the driver’s visual information processing mechanism to a certain extent, which are of great significance to understand the inner micro-psychology of driver’s cognition.

Motor imagery practice and feedback effects on functional connectivity

Objective. The use of motor imagery (MI) in motor rehabilitation protocols has been increasingly investigated as a potential technique for enhancing traditional treatments, yielding better clinical outcomes. However, since MI performance can be challenging, practice is usually required. This demands appropriate training, actively engaging the MI-related brain areas, consequently enabling the user to properly benefit from it. The role of feedback is central for MI practice. Yet, assessing which underlying neural changes are feedback-specific or purely due to MI practice is still a challenging effort, mainly due to the difficulty in isolating their contributions. In this work, we aimed to assess functional connectivity (FC) changes following MI practice that are either extrinsic or specific to feedback. Approach. To achieve this, we investigated FC, using graph theory, in electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data, during MI performance and at resting-state (rs), respectively. Thirty healthy subjects were divided into three groups, receiving no feedback (control), ‘false’ feedback (sham) or actual neurofeedback (active). Participants underwent 12–13 hands-MI EEG sessions and pre- and post-MI training fMRI exams. Main results. Following MI practice, control participants presented significant increases in degree and in eigenvector centrality for occipital nodes at rs-fMRI scans, whereas sham-feedback produced similar effects, but to a lesser extent. Therefore, MI practice, by itself, seems to stimulate visual information processing mechanisms that become apparent during basal brain activity. Additionally, only the active group displayed decreases in inter-subject FC patterns, both during MI performance and at rs-fMRI. Significance. Hence, actual neurofeedback impacted FC by disrupting common inter-subject patterns, suggesting that subject-specific neural plasticity mechanisms become important. Future studies should consider this when designing experimental NFBT protocols and analyses.

Visible-Infrared Image Fusion Based on Early Visual Information Processing Mechanisms

In this work, we simulate the early visual information processing mechanisms in biological visual system (BVS) to solve the visible-infrared image fusion (VIIF) task. Concretely, both infrared and visible images are first processed with a dynamic receptive field (DRF), which is imitated by a Difference of Gaussian (DoG) function whose parameters are adjusted according to local image statistics (e.g., local edge responses). The DRF processing produces two components for each source image (e.g., the visible image or the infrared image) that respectively represent the results of On-center based DRF and Off-center based DRF. Then, the results of On-center based DRF for visible image are fused with the results of Off-center based DRF for infrared image and the results of On-center based DRF for infrared image are fused with the results of Off-center based DRF for visible image, according to the mechanisms of cortex-based center-surround fusion. Algorithmically, this step fuses the visible and infrared images processed by DRF with On-center and Off-center according to the different levels of local homogeneity. Moreover, a feedward signal acting as the sub-cortical flow is also introduced to adjust the results during the fusion. The final output image is simply obtained by the summation of two components after the cortex and sub-cortex based fusion. Qualitative and quantitative tests on four datasets demonstrate that our approach can fuse the infrared and visible images effectively with the good background details and discernible salient areas. We emphasize the importance of corticothalamic feedback, cortex and sub-cortex-based fusion, and the interactions between On-center pathway and Off-center pathway that are ubiquitous in BVS for producing the high-quality visual signals.

Small target detection based on bird’s visual information processing mechanism

Detecting small targets in large fields of view is a challenging task. Nowadays, many targets detection models based on the convolutional neural network (CNN) achieve excellent performance. However, these CNN-based detectors are inefficient when applied to tasks of real-time detection of small targets. This paper proposes a small-target detection model in large fields of view based on the tectofugal–thalamofugal–accessory optic system of birds. Within this model, first, we design an unsupervised saliency algorithm to generate saliency regions to suppress background information according to the visual information processing mechanism of the tectofugal pathway of birds. Second, we design a super-resolution (SR) analysis method to enlarge small targets and improve image resolution by the information processing mechanism of the accessory optic system of birds. Then, according to the information processing mechanism of the thalamofugal pathway, we propose a CNN-based method to detect small targets. We further test our model on two public datasets (the VEDAI dataset and DLR 3 K dataset), and the experimental results demonstrate that the proposed detection model outperforms the state-of-the-art methods on small-target detection.

Optogenetics Advances in Primate Visual Pathway

In this issue of Neuron, Klein etal. (2016) used cell-type-specific optogenetics and electrical microstimulation to characterize the koniocellular geniculocortical projections in nonhuman primates. Their work offers a powerful platform for refining our understanding of the mechanisms of visual information processing in the lateral geniculate nucleus and primary visual cortex.

Objects Classification by Learning-Based Visual Saliency Model and Convolutional Neural Network.

Humans can easily classify different kinds of objects whereas it is quite difficult for computers. As a hot and difficult problem, objects classification has been receiving extensive interests with broad prospects. Inspired by neuroscience, deep learning concept is proposed. Convolutional neural network (CNN) as one of the methods of deep learning can be used to solve classification problem. But most of deep learning methods, including CNN, all ignore the human visual information processing mechanism when a person is classifying objects. Therefore, in this paper, inspiring the completed processing that humans classify different kinds of objects, we bring forth a new classification method which combines visual attention model and CNN. Firstly, we use the visual attention model to simulate the processing of human visual selection mechanism. Secondly, we use CNN to simulate the processing of how humans select features and extract the local features of those selected areas. Finally, not only does our classification method depend on those local features, but also it adds the human semantic features to classify objects. Our classification method has apparently advantages in biology. Experimental results demonstrated that our method made the efficiency of classification improve significantly.

Contextual influence on the tilt after-effect in foveal and para-foveal vision

A sensory stimulus can only be properly interpreted in light of the stimuli that surround it in space and time. The tilt illusion (TI) and tilt after-effect (TAE) provide good evidence that the perception of a target depends strongly on both its spatial and temporal context. In previous studies, the TI and TAE have typically been investigated separately, so little is known about their co-effects on visual perception and information processing mechanisms. Here, we considered the influence of the spatial context and the temporal effect together and asked how center-surround context affects the TAE in foveal and para-foveal vision. Our results showed that different center-surround spatial patterns significantly affected the TAE for both foveal and para-foveal vision. In the fovea, the TAE was mainly produced by central adaptive gratings. Cross-oriented surroundings significantly inhibited the TAE, and iso-oriented surroundings slightly facilitated it; surround inhibition was much stronger than surround facilitation. In the para-fovea, the TAE was mainly decided by the surrounding patches. Likewise, a cross-oriented central patch inhibited the TAE, and an iso-oriented one facilitated it, but there was no significant difference between inhibition and facilitation. Our findings demonstrated, at the perceptual level, that our visual system adopts different mechanisms to process consistent or inconsistent central-surround orientation information and that the unequal magnitude of surround inhibition and facilitation is vitally important for the visual system to improve the detectability or discriminability of novel or incongruent stimuli.

Shift in vertical stance parameters in humans viewing different images

The type of an image shown to a subject could be an important factor in traditional stabilometric tests or tests with biofeedback that affects the results. This aspect is important for practical areas, including medicine and sports. The results obtained in experiments on nine healthy volunteers show that the stance is more stabilized when viewing clear simple visual images, which is probably related to the basic mechanisms of visual information processing.

A novel biologically inspired local feature descriptor

Local feature descriptor is a fundamental representation for image patch which has been extensively used in many computer vision applications. In this paper, different from state-of-the-art features, a novel biologically inspired local descriptor (BILD) is proposed based on the visual information processing mechanism of ventral pathway in human brain. The local features used for constructing BILD are extracted by a two-layer network, which corresponds to the simple-to-complex cell hierarchy in the primary visual cortex (V1). It works in a similar way as the simple cell and complex cell do to get responses by applying the lateral inhibition from different orientations and operating an improved cortical pooling. To enhance the distinctiveness of BILD, we combine the local features from different orientations. Extensive evaluations have been performed for image matching and object recognition. Experimental results reveal that our proposed BILD outperforms many widely used descriptors such as SIFT and SURF, which demonstrate its efficiency for representing local regions.

Orientation Coding by Population of Neurons in Rats' Primary Visual Cortex

Research on Primate visual cortex (V1 area) neurons orientation coding mechanism is the base of revealing the whole visual cortex information processing mechanism. Firstly, this paper adopted different orientation grating to stimulate visually on rats. Meanwhile, gather response signals of population neurons from V1 area using multi-electrode arrays. Then, screen effective response channels according to the orientation selection of different neurons in different channels. Besides, extract Spike average fire rate and LFPγ band power feature in every effective channel signals within specific stimulus response time to construct population response joint features. Finally, taking Lasso regression model as coding model, use joint features to differentiate grating orientation, in order to research on V1 areas population neurons orientation coding. The consequences indicate that the results of population response joint features coding for six different orientation are superior to the results of any single feature of population response coding, and remarkably better than the results of single channel response feature coding.

Transient and cumulative memory impairments induced by GSM 1.8 GHz cell phone signal in a mouse model

This study was designed to investigate the transient and cumulative impairments in spatial and non-spatial memory of C57Bl/6J mice exposed to GSM 1.8 GHz signal for 90 min daily by a typical cellular (mobile) phone at a specific absorption rate value of 0.11 W/kg. Free-moving male mice 2 months old were irradiated in two experimental protocols, lasting for 66 and for 148 days respectively. Each protocol used three groups of animals (n = 8 each for exposed, sham exposed and controls) in combination with two behavioural paradigms, the object recognition task and the object location task sequentially applied at different time points. One-way analysis of variance revealed statistically significant impairments of both types of memory gradually accumulating, with more pronounced effects on the spatial memory. The impairments persisted even 2 weeks after interruption of the 8 weeks daily exposure, whereas the memory of mice as detected by both tasks showed a full recovery approximately 1 month later. Intermittent every other day exposure for 1 month had no effect on both types of memory. The data suggest that visual information processing mechanisms in hippocampus, perirhinal and entorhinal cortex are gradually malfunctioning upon long-term daily exposure, a phenotype that persists for at least 2 weeks after interruption of radiation, returning to normal memory performance levels 4 weeks later. It is postulated that cellular repair mechanisms are operating to eliminate the memory affecting molecules. The overall contribution of several possible mechanisms to the observed cumulative and transient impairments in spatial and non-spatial memory is discussed.

A Model of Human Behavior Based on Visual Information Processing Mechanism for Reliability-Based System Design

The aim of this study is to discuss the relation between the reliability of structural systems and the performance of human perception in human–machine systems. In this paper, we would like to evaluate the effect of performance deterioration on structural safety from the viewpoint of uncertainties in human behavior. The sensitivity of structural safety to such uncertainties is discussed on the basis of a stress–strength model by considering that uncertainties in structural systems can be divided into objective and subjective uncertainties in applied stress and strength. There are two types of subjective uncertainties, one associated with the designer and the other associated with the operator. In this context, the performance of the operator in terms of visual perception was investigated through visual tasks. Concretely, two kinds of experiments such as response time and useful field of view were carried out. Based on experimental results and analysis, it was shown that the relations between subjective uncertainty of machine and human performance affect the final probability of failure of structural systems, including human-machine systems.

Visual attention in a complex search task differs between honeybees and bumblebees

Mechanisms of spatial attention are used when the amount of gathered information exceeds processing capacity. Such mechanisms have been proposed in bees, but have not yet been experimentally demonstrated. We provide evidence that selective attention influences the foraging performance of two social bee species, the honeybee Apis mellifera and the bumblebee Bombus terrestris. Visual search tasks, originally developed for application in human psychology, were adapted for behavioural experiments on bees. We examined the impact of distracting visual information on search performance, which we measured as error rate and decision time. We found that bumblebees were significantly less affected by distracting objects than honeybees. Based on the results, we conclude that the search mechanism in honeybees is serial like, whereas in bumblebees it shows the characteristics of a restricted parallel-like search. Furthermore, the bees differed in their strategy to solve the speed-accuracy trade-off. Whereas bumblebees displayed slow but correct decision-making, honeybees exhibited fast and inaccurate decision-making. We propose two neuronal mechanisms of visual information processing that account for the different responses between honeybees and bumblebees, and we correlate species-specific features of the search behaviour to differences in habitat and life history.

Dynamics of spatial frequency tuning in mouse visual cortex

Neuronal spatial frequency tuning in primary visual cortex (V1) substantially changes over time. In both primates and cats, a shift of the neuron's preferred spatial frequency has been observed from low frequencies early in the response to higher frequencies later in the response. In most cases, this shift is accompanied by a decreased tuning bandwidth. Recently, the mouse has gained attention as a suitable animal model to study the basic mechanisms of visual information processing, demonstrating similarities in basic neuronal response properties between rodents and highly visual mammals. Here we report the results of extracellular single-unit recordings in the anesthetized mouse where we analyzed the dynamics of spatial frequency tuning in V1 and the lateromedial area LM within the lateral extrastriate area V2L. We used a reverse-correlation technique to demonstrate that, as in monkeys and cats, the preferred spatial frequency of mouse V1 neurons shifted from low to higher frequencies later in the response. However, this was not correlated with a clear selectivity increase or enhanced suppression of responses to low spatial frequencies. These results suggest that the neuronal connections responsible for the temporal shift in spatial frequency tuning may considerably differ between mice and monkeys.

Reduction of Image Complexity Explains Aesthetic Preference for Symmetry

Symmetric patterns are more appealing to human observers than asymmetric ones. Here, we investigate the visual information processing mechanisms underlying this aesthetic preference. All stimuli were derived from phase scrambled versions of forty face or nature images. In addition to the scrambled images, there were four other types of test image: symmetric, in which one part of the image was a reflection of another around an axis; repetitive, in which one part of the image was a copy of the other; anti-symmetric, similar to symmetric but with the contrast of one side reversed; and interleaved patterns, in which half of the symmetric pattern was replaced by a scrambled image. The number of axes ranged from 1 to 16 for all image types. The task of our 20 observers was to give a preference rating to each image on a 6-point Lickert scale. The preference rating increased with the number of axes for all stimulus types. The observers showed a similar preference for symmetric and repetitive patterns and slightly less preference for anti-symmetric patterns. The preference for interleaved patterns was much less than for other types of stimuli. Preference for an image cannot be explained by either the ecological significance of its content or the slope of its amplitude spectrum. Instead, preference can be accounted for by the complexity of the image.

Separating physiological from pathological effects of drugs tested for retinal toxicity in animal models

Abstract Purpose Drugs tested for retinal toxicity in animal models frequently induce physiological effects with or without pathological effects. The physiological effects may be interpreted to reflect toxic action. Here, different experimental approaches and modes of data analysis are described to separate physiological from pathological effects of drugs on retinal function and structure in laboratory animals. Methods The flash electroretinogram (ERG) and the visual evoked potential (VEP) were used to assess respectively the functional integrity of the distal and proximal parts of the retina following intravitreal administration of drugs. Histology at the light microscopic level and immunocytochemistry were used to evaluate structural damage and to localize the site/s of action of the drugs. Results The ERG responses and the VEP were recorded at different time intervals after intravitreal injection of different drugs in order to assess short‐term and long‐term retinal effects. In some cases severe functional deficit was identified, but it was transient in nature, and partial or even complete recovery was observed during a follow‐up period of at least 4 weeks. Different modes of data analysis were applied in order to separate physiological effects from pathological ones. Conclusion Drugs, used in the ophthalmic clinic, can be retinotoxic and induce permanent retinal damage, or may exert transient changes that can reflect drug interference with the normal physiological function of the retina. Data are analyzed to separate these possibilities in order to allow accurate assessment of drug toxicity. Furthermore, drugs exerting transient physiological effects can be used to study the normal mechanisms of visual information processing in the retina.

The Cognitive Informatics Theory and Mathematical Models of Visual Information Processing in the Brain

It is recognized that the internal mechanisms for visual information processing are based on semantic inferences where visual information is represented and processed as visual semantic objects rather than direct images or episode pictures in the long-term memory. This article presents a cognitive informatics theory of visual information and knowledge processing in the brain. A set of cognitive principles of visual perception is reviewed particularly the classic gestalt principles, the cognitive informatics principles, and the hypercolumn theory. A visual frame theory is developed to explain the visual information processing mechanisms of human vision, where the size of a unit visual frame is tested and calibrated based on vision experiments. The framework of human visual information processing is established in order to elaborate mechanisms of visual information processing and the compatibility of internal representations between visual and abstract information and knowledge in the brain.

Complex Temporal Response Patterns With a Simple Retinal Circuit

The retina can respond to a wide array of features in the visual input. It was recently reported that the retina can even recognize complicated temporal input patterns and signal violations in the patterns. When a sequence of flashes was presented, ganglion cells exhibited a variety of firing profiles and many cells showed an "omitted stimulus response" (OSR), in which they fired strongly if a flash in the sequence was omitted. We examined the synaptic origins of the OSR by recording excitatory synaptic currents from ganglion cells in the salamander retina in response to periodic flash sequences. Consistent with previous spike recordings, ganglion cells exhibited an OSR in their current response and the OSR shifted in time with a change in flash frequency such that it could predict when the next flash should have occurred. Although the behavior may seem sophisticated, we show that a simple linear-nonlinear model with a spike threshold can account for the OSR in on ganglion cells and that the variety of complex firing profiles seen in other ganglion cells can be explained by adding contributions from the off pathway. We discuss the physiological and simulation results and their implications for understanding retinal mechanisms of visual information processing.