Orientation-selective Mechanisms Research Articles

Temporal crowding with central vision reveals the fragility of visual representations.

This study examined whether temporal crowding-the impaired object identification when distracting objects precede and succeed it-occurs at the fovea and if so whether its magnitude is reduced. We presented a central sequence of three oriented items separated by relatively long intervals (200/400 ms) and used an orientation estimation task with mixture-model analyses. We found clear evidence of temporal crowding with central vision, even with 400 ms intervals. Critically, reduced encoding precision surfaced as a robust and unique characteristic of temporal crowding. The magnitude of central and peripheral temporal crowding was similar suggesting the involvement of higher visual areas. Precision impairment emerged even when only the target contained orientation information, excluding "response competition" as the sole interference mechanism; yet it was larger when all items included orientation information, underscoring the importance of orientation-selective mechanisms. Overall, we show that even with central simple stimuli, the formation of a stable visual representation is surprisingly slow. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Poster Session II: Alternating orientation of the chromatic pattern VEP improves signal even in the absence of contrast adaptation.

The visual evoked potential (VEP) to chromatic pattern reversal is greatly reduced compared to VEPs to pattern onsets. Chromatic pattern onsets produce large and stereotypical waveforms that reliably differ from standard achromatic pattern reversal VEP waveforms used in clinical applications. Rapid contrast adaptation for sustained chromatic but not transient achromatic mechanisms has been suggested as one explanation for these observations. Here we first examined changes in the magnitude of response during recordings to reversing and onset grating patterns that preferentially modulate the L-M, S, and achromatic pathways. Given the evidence for both chromatic and achromatic orientation-selective mechanisms, we then hypothesized that contrast adaptation may be reduced by changing the orientation of the pattern for each reversal or onset. VEPs were recorded for 60 s with 2 onsets/reversals per second using both fixed and alternating (horizontal/vertical) orientations. FFT amplitudes for 6-second windows did not reveal evidence of adaptation for chromatic or achromatic onsets or reversal patterns over the 60-second recording period. Despite this, alternating pattern orientation increased the signal for all chromatic but not achromatic conditions. Although alternating the orientation for reversals increased the signal, the onset responses were still larger, even for non-alternating orientations. Mechanisms other than contrast adaptation must be invoked to explain the results.

Exploiting sublimated deep features for image retrieval

Deep learning techniques can be used to describe image content, which is a good way to reduce the semantic gap between low-level and high-level features. However, convolutional neural networks (CNNs) exhibit texture bias and largely ignore global object shape. This does not conform to human perception and can fail to gain the advantages of both low-level features and deep features. To address this problem, in this work, the main focus was shifted from using simple deep features to the use of sublimated deep features, which incorporate global object shape and color features. Along these lines, in this work, a novel image retrieval method named the sublimated deep feature histogram (SDFH) was proposed. The main highlights are: 1) An effect orientation feature was introduced, namely, orientation-selective feature, to mimic the orientation-selection mechanism. This provides a good representation of global object shape and reduces the side-effects of texture bias. 2) A new concept was developed, namely color perceptual feature, to address the shortcoming of deep features—that they discard color features. This includes color cues in the deep features and provides a more discriminating representation. 3) The orientation selective and color perception mechanisms were effectively mimicked to provide a compact yet efficient representation, and propose an effective transfer learning method called gain whitening learning. By carrying out comparative experiments, it was demonstrated that sublimated deep features can provide highly competitive retrieval performance (in terms of mean average precision) using a pre-trained CNN model applied to well-known benchmark datasets. Our results provide new insights into image retrieval based on the mechanisms of the primary visual cortex (V1). Furthermore, the method is more in line with human perception than other methods.

Fixational eye movements enable robust edge detection.

Human vision relies on mechanisms that respond to luminance edges in space and time. Most edge models use orientation-selective mechanisms on multiple spatial scales and operate on static inputs assuming that edge processing occurs within a single fixational instance. Recent studies, however, demonstrate functionally relevant temporal modulations of the sensory input due to fixational eye movements. Here we propose a spatiotemporal model of human edge detection that combines elements of spatial and active vision. The model augments a spatial vision model by temporal filtering and shifts the input images over time, mimicking an active sampling scheme via fixational eye movements. The first model test was White's illusion, a lightness effect that has been shown to depend on edges. The model reproduced the spatial-frequency-specific interference with the edges by superimposing narrowband noise (1–5 cpd), similar to the psychophysical interference observed in White's effect. Second, we compare the model's edge detection performance in natural images in the presence and absence of Gaussian white noise with human-labeled contours for the same (noise-free) images. Notably, the model detects edges robustly against noise in both test cases without relying on orientation-selective processes. Eliminating model components, we demonstrate the relevance of multiscale spatiotemporal filtering and scale-specific normalization for edge detection. The proposed model facilitates efficient edge detection in (artificial) vision systems and challenges the notion that orientation-selective mechanisms are required for edge detection.

Pattern Recognition of Spiking Neural Networks Based on Visual Mechanism and Supervised Synaptic Learning.

Electrophysiological studies have shown that mammalian primary visual cortex are selective for the orientations of visual stimuli. Inspired by this mechanism, we propose a hierarchical spiking neural network (SNN) for image classification. Grayscale input images are fed through a feed-forward network consisting of orientation-selective neurons, which then projected to a layer of downstream classifier neurons through the spiking-based supervised tempotron learning rule. Based on the orientation-selective mechanism of the visual cortex and tempotron learning rule, the network can effectively classify images of the extensively studied MNIST database of handwritten digits, which achieves 96% classification accuracy based on only 2000 training samples (traditional training set is 60000). Compared with other classification methods, our model not only guarantees the biological plausibility and the accuracy of image classification but also significantly reduces the needed training samples. Considering the fact that the most commonly used deep learning neural networks need big data samples and high power consumption in image recognition, this brain-inspired computational neural network model based on the layer-by-layer hierarchical image processing mechanism of the visual cortex may provide a basis for the wide application of spiking neural networks in the field of intelligent computing.

Neural mechanism of orientation selectivity for distinct gamma oscillations in cat V1

Neural mechanism of orientation selectivity for distinct gamma oscillations in cat V1

Characterizing the orientation selectivity in V1 and V4 of macaques by quadratic phase coupling

Objective. Orientation selectivity is one of the significant characteristics of neurons in the primary visual cortex (V1). Some neurons in extrastriate visual cortical areas also exhibit certain orientation selectivity. But it is still not well understood that how the orientation selectivity generates. Most previous studies about the orientation selectivity are based on the spike firing rate. However, the spikes are prone to be biased by the detection and sorting algorithms. Then, in this paper, the local field potential (LFP) is adopted to investigate the mechanism of orientation selectivity. Approach. We used the quadratic phase coupling (QPC), which was calculated by wavelet bicoherence, to describe the characteristics of orientation selectivity available in V1 and V4. The raw wideband neural signals were recorded by two chronically implanted multi-electrode arrays, which were placed in V1 and V4 respectively in two macaques performing a selective visual attention task. Main results. There is a strong correlation between the total bicoherence (TotalBic), which is a quantization for the overall QPC of frequency pairs in gamma band, and the grating orientation. Furthermore, the QPC distribution at the non-preferred orientation is mainly concentrated in the low frequencies (30–40 Hz) of gamma; while the QPC distribution at the preferred orientation concentrates in both the low frequencies and high frequencies (60–80 Hz) of gamma. In addition, the TotalBic of the gamma-band LFP between V1 and V4 varies with the grating orientations, indicating that the QPC is available in the feedforward link and the gamma-band LFP in V1 modulates the QPC in V4. Significance. The QPC reflects the orientations of the sinusoidal grating and describes the interaction of gamma-band LFP between different brain regions. Our results suggest that the QPC is an alternative avenue to explore the mechanism for generating orientation selectivity of visual neurons effectively.

Image Retrieval Using the Intensity Variation Descriptor

Variations between image pixel characteristics contain a wealth of information. Extraction of such cues can be used to describe image content. In this paper, we propose a novel descriptor, called the intensity variation descriptor (IVD), to represent variations in colour, edges, and intensity and apply it to image retrieval. The highlights of the proposed method are as follows. (1) The IVD combines the advantages of the HSV and RGB colour spaces. (2) It can simulate the lateral inhibition mechanism and orientation-selective mechanism to determine an optimal direction and spatial layout. (3) An extended weighted L1 distance metric is proposed to calculate the similarity of images. It does not require complex operations such as square or square root and leads to good performance. Comparative experiments on two Corel datasets containing 15,000 images show that the proposed method performs better than the SoC-GMM, CPV-THF, and STH methods and provides good matching of texture, colour, and shape.

Spatial and Temporal Feature-Based Reduced Reference Quality Assessment for Rate-Varying Videos in Wireless Networks

For the impact of the bitrate change of video streaming services according to the available bandwidth on user satisfaction, in this paper, we propose a spatial and temporal feature-based reduced reference (RR) quality assessment for rate-varying videos in wireless networks called STRQAW. First, simulating the orientation selectivity mechanism of the human visual system (HVS), the histogram of the orientation selectivity-based visual pattern in each frame is extracted as the spatial feature. The histogram similarity between the rate-varying video and the original video is computed as the spatial metric. Second, we extract the temporal variation of the DCT coefficients of the consecutive frame differences as the temporal feature. The temporal variation similarity between the rate-varying video and the original video is calculated as the temporal metric. Finally, we take into account the recency effect and assess the overall quality by combining the temporal and spatial metric. The experimental results using the Laboratory for Image and Video Engineering (LIVE) mobile video quality assessment (VQA) database show that STRQAW is consistent with the subjective assessment results, which means it reflects human subjective feelings well and it provides an evaluation for adjusting compression-coding rates in real time. STRQAW can be used to guide video application providers and network operators working towards satisfying end-user experiences.

Blind image blur metric based on orientation-aware local patterns

We develop an effective blind image blur assessment model based on a novel orientation-aware local pattern operator. The resulting metric first proposes an orientation-aware local pattern operator that fully considers the impact of anisotropy of orientation selectivity mechanism and the gradient orientation effect on visual perception. Our results indicate that the proposed descriptor is sensitive to image distortion and can effectively represent orientation information. We thus use it to extract image structure information. In order to enhance features’ representation capability for blur image, we extract edge information by a Toggle operator and use it as weight of local patterns to optimize the computed structural statistical features. Finally, a support vector regression method is used to train a predictive model with optimized features and subjective scores. Experimental results obtained on six public databases show that our proposed model performs better than state-of-the-art image blur assessment models.

Quality assessment for view synthesis using low-level and mid-level structural representation

View synthesis is the most important technique in multi-view and free-viewpoint videos. The whole view synthesis includes the acquisition and processing of texture and depth images, and the virtual view rendering stage. Existing quality metrics for view synthesis have limited ability for the whole synthesis process for the following reasons. First, they are dedicated to a single stage of view synthesis, overlooking the commonality of all the possible distortions introduced in the whole process. Moreover, they only extract low-level features for quality assessment, ignoring the perceptual degradation caused by the mid-level contours that are destructed by heavy distortions in texture/depth images and the imperfect view rendering, which represent the spatial distribution/connection of adjacent contour pixels. Inspired by the above facts, this paper presents a quality metric for view synthesis using both Low-level and Mid-level Structural representation (LMS), aiming to accurately evaluate the distortions in the whole view synthesis process. Specifically, the scale space is first constructed to mimic the hierarchical property of the human visual system. Then, the statistics of gradient orientation is integrated with the statistics of gradient intensity for the low-level structural representation, which is motivated by the importance of the orientation selectivity mechanism to visual perception. Further, the mid-level structure is represented using bag of words for contour description based on the sparse coding of the primary visual cortex. Then the distances of both the low-level and mid-level features between the synthesized and reference images are calculated. Finally, two distances are integrated to generate the whole quality score. Extensive experiments on two public view synthesis databases demonstrate the superiority of the proposed method to the state-of-the-arts in evaluating the quality of the whole view synthesis.

Blind image quality assessment with hierarchy: Degradation from local structure to deep semantics

Though blind image quality assessment (BIQA) is highly desired in perceptual-oriented image processing systems, it is extremely difficult to design a reliable BIQA method. With the help of the prior knowledge, the human visual system (HVS) hierarchically perceives the quality degradation during the visual recognition. Inspired by this, we suggest different levels of distortion generate individual degradations on hierarchical features, and propose to consider the degradations on both low and high level features for quality prediction. By mimicking the orientation selectivity (OS) mechanism in the primary visual cortex, an OS based local structure is designed for low-level visual information representation. At the meantime, the deep residual network, which possesses multiple levels for feature integration, is employed to extract the deep semantics for high-level visual content representation. By fusing the local structure and the deep semantics, a hierarchical feature set is acquired. Next, the correlations between the degradations of image qualities and their corresponding hierarchical feature sets are analyzed, and a novel hierarchical feature degradation (HFD) based BIQA (HFD-BIQA) method is built. Experimental results on the legacy and wild image quality assessment databases demonstrate the prediction accuracy of the proposed HFD-BIQA method, and verify that the HFD-BIQA performs highly consistent with the subjective perception.

Blind quality assessment for screen content images by orientation selectivity mechanism

Currently, most objective image quality assessment (IQA) methods designed for screen content images (SCIs) require reference information, and existing blind IQA metrics cannot obtain consistent results with subjective scores. In this study, we propose a novel blind IQA method for SCIs based on orientation selectivity mechanism. First, we extract orientation features to compute the visual distortion of degraded SCIs by orientation selectivity mechanism. The statistical orientation features are further obtained by the histogram of orientation features. Second, the statistical structure features are calculated as the complementary information of orientation features for quality prediction of degraded SCIs. Finally, we employ support vector regression (SVR) as the mapping function from these extracted statistical features to quality scores. Experimental results show that the proposed method can obtain better performance of visual quality prediction for SCIs than other existing related methods.

Enhanced Just Noticeable Difference Model for Images With Pattern Complexity.

The just noticeable difference (JND) in an image, which reveals the visibility limitation of the human visual system (HVS), is widely used for visual redundancy estimation in signal processing. To determine the JND threshold with the current schemes, the spatial masking effect is estimated as the contrast masking, and this cannot accurately account for the complicated interaction among visual contents. Research on cognitive science indicates that the HVS is highly adapted to extract the repeated patterns for visual content representation. Inspired by this, we formulate the pattern complexity as another factor to determine the total masking effect: the interaction is relatively straightforward with a limited masking effect in a regular pattern, and is complicated with a strong masking effect in an irregular pattern. From the orientation selectivity mechanism in the primary visual cortex, the response of each local receptive field can be considered as a pattern; therefore, in this paper, the orientation that each pixel presents is regarded as the fundamental element of a pattern, and the pattern complexity is calculated as the diversity of the orientation in a local region. Finally, considering both pattern complexity and luminance contrast, a novel spatial masking estimation function is deduced, and an improved JND estimation model is built. Experimental results on comparing with the latest JND models demonstrate the effectiveness of the proposed model, which performs highly consistent with the human perception. The source code of the proposed model is publicly available at http://web.xidian.edu.cn/wjj/en/index.html.

Contrast Gain Control in Plaid Pattern Detection.

A plaid is a combination of two gratings whose orientations are orthogonal to each other with the same or similar contrasts. We used plaid patterns as stimuli to investigate the mechanisms underlying the detection of a plaid to understand how the visual system combines information from orientation-selective channels. We used a masking paradigm in which an observer was required to detect a target (either a spiral or a plaid) superimposed on a pedestal. We measured the target threshold versus pedestal contrast (TvC) functions at 7 pedestal contrasts for various target-pedestal combinations with a temporal 2AFC paradigm and a staircase procedure. All TvC functions, except the one with an orthogonal spiral pedestal, showed a dipper shape, although the position of the dip and the slope varied across conditions. The result can be explained by a multiple-mechanism divisive inhibition model, which contains several orientation-selective mechanisms. The response of each mechanism is the excitation of a linear filter divided by a broadband inhibitory input. The threshold is determined by a nonlinear combination of the responses of those mechanisms. Alternative models with mechanism(s) specific for plaid did not provide a better description of the data. Thus, a plaid pattern is mediated by a combination of orientation-selective mechanisms. An early plaid-specific mechanism is not necessary for plaid detection.

Mechanisms of Orientation Selectivity in the Primary Visual Cortex.

The mechanisms underlying the emergence of orientation selectivity in the visual cortex have been, and continue to be, the subjects of intense scrutiny. Orientation selectivity reflects a dramatic change in the representation of the visual world: Whereas afferent thalamic neurons are generally orientation insensitive, neurons in the primary visual cortex (V1) are extremely sensitive to stimulus orientation. This profound change in the receptive field structure along the visual pathway has positioned V1 as a model system for studying the circuitry that underlies neural computations across the neocortex. The neocortex is characterized anatomically by the relative uniformity of its circuitry despite its role in processing distinct signals from region to region. A combination of physiological, anatomical, and theoretical studies has shed some light on the circuitry components necessary for generating orientation selectivity in V1. This targeted effort has led to critical insights, as well as controversies, concerning how neural circuits in the neocortex perform computations.

Effects of adaptation on orientation tuning in excitatory and inhibitory neurons in macaque V1 and V2

Adaptation modifies sensory pathways to adjust to the characteristics of the recently-encountered visual world. However, even for neuronal properties as fundamental as orientation, much is unknown about adaptation-induced alterations in tuning. To delineate the influence of adaptation on neural and network properties, we investigated its effect in functionally distinct cell categories (excitatory vs inhibitory) in two hierarchically-related brain regions, V1 and V2, of the primate visual pathway. We performed multitetrode recordings from 90 neurons in V1 and V2 of 6 macaques using drifting sinusoidal gratings, after adaptation to a preferred and a non-preferred orientation for brief (0.4 s) and prolonged (40 s) exposures. We found a wide range of tuning changes induced by adaptation. In layer 2/3, orientation selectivity either increased or decreased, and the tuning curve peak moved either towards (attractive) or away from (repulsive) the adapter. In contrast, Layer 4 neurons typically broadened their tuning after brief adaptation and narrowed their tuning after prolonged adaptation. In V1, findings were similar for inhibitory and excitatory neurons (as distinguished by their extracellular action potential shape). However, in V2, prolonged adaptation broadened the tuning of inhibitory cells, but narrowed the tuning of excitatory cells. Thus, there is a wide variety of tuning changes induced by adaptation in both V1 and V2, and dynamics of adaptation are distinct in excitatory and inhibitory neurons within V2. Notably, these findings have implications for the circuit mechanisms of orientation selectivity: attractive shifts do not occur in purely feedforward models, or in recurrent models in which inhibitory neurons are untuned. In sum, tuned inhibitory neurons contribute not only to orientation tuning, but also to how this tuning adapts – and hence, play a critical role in the neural representations that ultimately influence perception Meeting abstract presented at VSS 2016

Tilt aftereffect following adaptation to translational Glass patterns.

Glass patterns (GPs) consist of randomly distributed dot pairs (dipoles) whose orientations are determined by specific geometric transforms. We assessed whether adaptation to stationary oriented translational GPs suppresses the activity of orientation selective detectors producing a tilt aftereffect (TAE). The results showed that adaptation to GPs produces a TAE similar to that reported in previous studies, though reduced in amplitude. This suggests the involvement of orientation selective mechanisms. We also measured the interocular transfer (IOT) of the GP-induced TAE and found an almost complete IOT, indicating the involvement of orientation selective and binocularly driven units. In additional experiments, we assessed the role of attention in TAE from GPs. The results showed that distraction during adaptation similarly modulates the TAE after adapting to both GPs and gratings. Moreover, in the case of GPs, distraction is likely to interfere with the adaptation process rather than with the spatial summation of local dipoles. We conclude that TAE from GPs possibly relies on visual processing levels in which the global orientation of GPs has been encoded by neurons that are mostly binocularly driven, orientation selective and whose adaptation-related neural activity is strongly modulated by attention.

Synaptic Mechanisms Generating Orientation Selectivity in the ON Pathway of the Rabbit Retina.

A core goal for visual neuroscientists is to understand how neural circuits at each stage of the visual system extract and encode features from the visual scene. This study documents a novel type of orientation-selective ganglion cell in the retina and shows that the receptive field structure is remarkably similar to that of simple cells in primary visual cortex. However, the data indicate that, unlike in the cortex, orientation selectivity in the retina depends on the activity of inhibitory interneurons. The results further reveal the physiological basis for feature detection in the visual system, elucidate the synaptic mechanisms that generate orientation selectivity at an early stage of visual processing, and illustrate a novel role for NMDA receptors in retinal processing.

Orientation selectivity based visual pattern for reduced-reference image quality assessment

Image quality assessment (IQA) is in great demand for high quality image selection in the big data era. The challenge of reduced-reference (RR) IQA is how to use limited data to effectively represent the visual content of an image in the context of IQA. Research on neuroscience indicates that the human visual system (HVS) exhibits obvious orientation selectivity (OS) mechanism for visual content extraction. Inspired by this, an OS based visual pattern (OSVP) is proposed to extract visual content for RR IQA in this paper. The OS arises from the arrangement of the excitatory and inhibitory interactions among connected cortical neurons in a local receptive field. According to the OS mechanism, the similarity of preferred orientations between two nearby pixels is first analyzed. Then, the orientation similarities of pixels in a local neighborhood are arranged, and the OSVP is built for visual information representation. With the help of OSVP, the visual content of an image is extracted and mapped into a histogram. By calculating the changes between the two histograms of reference and distorted images, a quality score is produced. Experimental results on five public databases demonstrate that the proposed RR IQA method has performance consistent with the human perception under a small amount of reference data (only 9 values).